You're going to have to be together at some point, and if you aren't 100% sure that will work EARLY, you could be doomed. You need the brainstorming back-and-forth for new ideas and to see new market challenges. You also aren't considering the time when you need to hire more employees, all of which you want corresponding with you and giving you new ideas.
It's also easier if you all are together when times gets tough and motivation wains.
It's also easier if you all are together when times gets tough and motivation wains.