So then how would a novice user get apps the first time they boot up their phone? They would have to know where to get apps from. That seems like an easy way for users to end up downloading a bunch of malware because they think it's the official Apple or Google app store when it's just a random website.
Having a built in app store has huge advantages in security and usability for the end user.
Also I believe it had a positive effect on Firefox gaining users, no one can claim that was a bad thing.
"Find Spotify on [list of ten app stores here]." One of the advantages for users and developers is that if you want an app on iOS (and for the most part) Android, you know exactly where to find it.
It's by no means a dealbreaker, but something to consider.