The location depends upon the type and nature of your business. If you are into manufacturing, you need to keep in mind the location to take of the logistics and then there are service-based businesses which operate remotely which does not need a specific location. Businesses should always cater to the needs of the geographical location you are launching the business. This helps to strengthen your business model which in turn will help you expand your business to other geographical location with ease.
Also "It’s hard to start a company from the ground up without the support of others who have been there, There are many mistakes to be avoided or lessons to be learned from seasoned entrepreneurs."
One way that startup founders can utilize other entrepreneurs in their area is to trade tips on finding investors. Since they’re already near the people looking to fund new businesses, founders can easily meet with them.
A big issue for startup founders is that they might not have the capital to afford office space in expensive areas. The only way to overcome is to do the best with what you have and concentrate on networking and go to events so the that the tips traded are not missed by you.