Not all software that people want to run can be efficiently parallelized. There's still a place for larger caches / fatter cores instead of blowing up core count.
In traditional web serving tasks and particularly for commercial cloud providers (usually renting out VM time), this isn't the case, and so I'm sure Ampere will be successful in that space.
Almost all service middleware is capable of running tasks in multiple threads so when serving lots of users that's a very easy way to get parallelism.
If you're just copying data around (like in an HTTP API implementation) you're mostly I/O bound anyway and adding more compute won't help that much so there's no return on investment.
Why not? We hit the single thread wall more than a decade ago. Why isn’t more software parallelized by now?