This side-steps the question of why these smartphones, with more computing power than desktop computers 15 years ago, can't use new mainline kernels, while desktop computers 15 years ago, and desktop computers today, can.
It's few different factors, like: Smartphones are harder to reverse-engineer, because of the smaller parts and greater integration and fewer common interconnects. Smartphones are more diverse, and each SoC "architecture" is significantly changed or replaced after just a year or two.
There's never enough time and mindshare for third-parties or the vendor itself to write high-quality base system drivers for these things. There's just crappy vendor "BSP" code that's never good enough to maintain support for multiple generations in the same code tree. It's always "fork and add messy hacks until it works". There's never enough time and motivation to make any of this support code good enough to be reused.
Some kernel maintainers have tried for some time to convince "enterprise" linux distros to keep up with newer kernels, and apply more of their testing and validation effort to newer kernels, instead of just maintaining one old kernel on their own for most of a decade. This enables more sharing of this effort between multiple distros and other users of the linux kernel. To some extent they have succeeded. But where they have failed, at least the "enterprise" distros did maintain the old kernels they distribute.
But these smartphone SoC vendors don't even do that. They really give zero fucks about software quality. It seems to work, ship it, move on to the next iteration because it has to make it to market in X months to keep up with the competition. At that rate, third-parties can't even come close to keeping up and justifying the investment of effort in reverse-engineering (which has been done for a lot of desktop hardware).
Because Linux doesn't maintain stable kernel API/ABIs, which requires either a shim to the more stable driver (a "midlayer") and ongoing development that's unlikely to get mainlined or a shim to a binary blob for the numerous hardware developers who don't want to open-source details of their hardware.
Linus is also incapable of dealing with these companies in a way that they care about (see the middle finger to nVidia) so it's unlikely there's going to be any real progress. This is a fact: no company wants to spend resources chasing after every new kernel development if it doesn't specifically suit their needs. Windows solved this by having stable APIs that only rarely change. OS X solved this by making it basically unnecessary -- most of the hardware that needs drivers is either built-in or works over something generic like USB -- and even then has nice interfaces for drivers that are well documented and don't change that often.
Linux attempts to have the freedom of the former without any kernel developer burden, and the outcome is that no HW vendor cares much to contribute upstream since it requires a lot of time and effort that's mostly just a cost
Linux has built-in support for way more hardware than OS X and full support for generic USB devices (and then some, with "quirks tables" for many standard-but-glitchy devices) so it's hard to say it's worse than OS X in that respect.
You'll also find that there are devices that had decent drivers for Windows XP when they were released, but never had Vista or Windows 7 or later drivers made. Those devices worked well on linux a couple years after they worked on windows with reverse-engineered drivers, and then (ironically) kept working for a decade or more, whereas the windows drivers did not. This includes a variety of printers, sound cards, non-standard usb devices, and more.
It's definitely true that many hardware companies which have modeled their driver development and support around the windows model can't handle the linux model at all. But linux has other priorities than to cater to them, and has done better than windows in server and embedded use-cases as a result.
It's few different factors, like: Smartphones are harder to reverse-engineer, because of the smaller parts and greater integration and fewer common interconnects. Smartphones are more diverse, and each SoC "architecture" is significantly changed or replaced after just a year or two.
There's never enough time and mindshare for third-parties or the vendor itself to write high-quality base system drivers for these things. There's just crappy vendor "BSP" code that's never good enough to maintain support for multiple generations in the same code tree. It's always "fork and add messy hacks until it works". There's never enough time and motivation to make any of this support code good enough to be reused.
Some kernel maintainers have tried for some time to convince "enterprise" linux distros to keep up with newer kernels, and apply more of their testing and validation effort to newer kernels, instead of just maintaining one old kernel on their own for most of a decade. This enables more sharing of this effort between multiple distros and other users of the linux kernel. To some extent they have succeeded. But where they have failed, at least the "enterprise" distros did maintain the old kernels they distribute.
But these smartphone SoC vendors don't even do that. They really give zero fucks about software quality. It seems to work, ship it, move on to the next iteration because it has to make it to market in X months to keep up with the competition. At that rate, third-parties can't even come close to keeping up and justifying the investment of effort in reverse-engineering (which has been done for a lot of desktop hardware).