Actually, that's probably the most naive of the romantic notions of doing a startup. First of all, the things I described apply to businesses, not only VC-backed startups. Just because your company is small doesn't mean you're suddenly immune from talking to customers and dealing with bureaucracy. If anything, you spend more time talking to your customers when the base is small, because everything is so much more precarious.
Second, the number of "lifestyle businesses" that exist as single-founder-tech-guy-and-a-computer are so vanishingly small as to be unicorns. The people who claim to do that are either secretly making most of their money doing consultancy (which is fine, but it ain't a 4-hour work week), or they're in such a bizarre niche that they've built a knowledge and/or distribution moat around their business that is virtually impenetrable, through years of backbreaking work. Your example of "app store" developers is particularly on-point: it's notoriously rare and difficult to find a product that has a sustainable business model in that marketplace.
Law of the universe: if you want something, there are millions of other people out there who want it just as much as you do. Any business opportunity that is simple enough to be achievable by a relatively unskilled single-founder-with-a-computer, yet lucrative enough to support that founder indefinitely with a four-hour work-week, will invariably attract competition from the undifferentiated masses who don't want to work, either. So you're either competing with those people (hard) for a small pie, or you're trying to build a growth business in a large market (hard), or you're failing (hard), or you're getting a good job (hard). There's no free lunch.
There is the problem of telling whether there's a market for a given thing before you invest six months or a year building it, so I'm not advocating sinking a whole year without shipping, but there's definitely merit to starting with something that can't be built as a rails app.
Small businesses, at least in the UK, do not involve that much bureaucracy. You hire an accountant to deal with that.
And, seriously, you don't need to put back-breaking effort in for years to create a (non-consulting) lifestyle business. I used to work with a developer who coded a niche social network on the train to work each morning. The site now makes him far more than he ever did working as an employee.
There are plenty of other niches where developers can make good money. Competition does not work like you think - it is nowhere near as intense as you seem to think and, regardless, there is room for more than one player even in a niche market, if all you are aiming for is a lifestyle business.
Sure, there is no free lunch, but it ain't anywhere near as hard as you are trying to make out. Other readers, don't let this guy put you off setting up on your own.
The "person" described here in the parent to your first reply:
"Or startups in general. You're competing against the awesome developer doing their own thing."
With regards to that even "awesome" people make foolish choices all the time. It's up to the person trying to recruit the described person to give them, in plain terms, what's involved in doing it all yourself. My guess is that they don't do a particularly good job at this. Nor does the entire startup community either other than to make generalizations about difficulty and how most ideas don't work. But we're talking about the same community that thinks it actually makes sense to pass on Harvard acceptance and do YC.
Most lifestyle business are full time jobs, just not ones where you have a chance exiting with 20 mil in the bank.
There's nothing stopping people bootstrapping their freelance business into small digital agency if they want to expand.