OpenStack was doomed from the start, but the reasons were subtly different than the difficulties of integrating software from multiple vendors.
However difficult OpenStack was to get (and keep) running, it would have been worth it if, once you (or anyone else) got an OpenStack instance up and running, a developer could migrate their app (from or to any other OpenStack cloud) with no code or configuration changes.
That was never really possible, since every OpenStack-based cloud provider insisted on adding their own special sauce to the developer experience. That was, for most of them, the whole point of participating in OpenStack: Sharing the cost of developing the code necessary for building a public or private cloud, but locking in their customers just as firmly as AWS was doing.
As a result, none of those individual cloud providers ever got big enough to give AWS serious competition, and there wasn't never a realistic portability story that could give them collective weight in the market.
The simplest way to ensure developed and deployed applications and services were truly (and trivially) portable between the different OpenStack cloud providers would have been to commit OpenStack to API-compatibility with AWS (which would also have given AWS customers a clear migration path to OpenStack), but this suggestion was rejected outright at the outset of the project.
Having real portability between different OpenStack-based cloud providers and AWS would mean that competition would largely have been on price (and quality/reliability) rather than features, both lowering AWS' margins and growing the market faster than Amazon could capture, as well as enabling higher-level businesses such as marketplaces with dynamic spot-pricing and rapid migration of jobs between providers.
Unfortunately, that isn't how it shook out. None of the cloud companies behind OpenStack wanted to compete with AWS on price instead of features, which meant they stuck their heads in the sand, kept their attractive margins, but effectively ceded the bulk of the market to Amazon, keeping prices high enough that the market grew only about as fast as Amazon/MS/Google could collectively add capacity (roughly maintaining AWS marketshare), rather than the hockey-stick growth that would have happened if everyone could have got in on the act (like web-hosting did in the 90s).