Is there some known principle that governs this, something that could be named along the lines of Parkinson's Law or the Peter Principle?
Anecdotally I noticed that even with 5 people you usually have someone starting to act as adhoc part-time manager, and this role quickly become too much for a part time responsibility as the team grows.
you don't need to create a middle management structure from the get go, just have a chain of command.
Any project with more than one team already has a middle management structure of sorts - you've got the CEO / CTO, then the team leaders (scrum masters, or just whoever's the loudest). There's your middle management layer already.