Usually long term initiative take a lot of domain knowledge and a specific set of connections. When something new becomes more important, they risk losing their personal position since they may not be the best person in terms of knowledge and connection to carry out the new direction. Instead they try to use their accumulated power to keep their direction supported even when it's not in the interest of the business for as long as they can.
To address this, perhaps there's merit in rewarding timely exits and somehow punishing people that have dragged things out. However the most that companies can do is usually just fire someone. And at that point they have probably sucked the company dry of the value they can personally extract.
It would be awfully surprising if virtually all companies had the same problems due to smart people suddenly behaving irrationally in the same way. It's much less surprising that smart people behave rationally in their own interests, but that those interests regularly fail to align with overall company interests.
And I think the pattern you describe is even mirrored on an institutional level. People are quick to mock Kodak for ignoring the rise of digital cameras, but they tend to undervalue the amount of risk that pivot would involve. If you're a world leader in product X, and that gets replaced by product Y, pivoting destroys the value of your expertise and puts you at risk. It's unpopular but perhaps sensible to settle on winding down profitably (the institutional version of a timely exit) instead of pivoting.