In my (limited) experience, the only reason to use backports is because you have closed source kernel modules which you can't update (that of course ends up covering most Android phones, and many SOCs)
Distros standardize on version not because it is more stable but because tooling (which might include 3rd party modules for the kernel) can then rely to work on that version without recompile.
If you don't have that constraint yeah, not much reason.
The vast majority of the time fixes like this that are being backported are straightforward fixes for bugs (security or otherwise) that require very little manual conflict resolution, especially if the fix is just being backported one or two kernel releases. The developer can often just cherry-pick the commit into a few recent release branches and most of the time git will just automatically do the merge correctly, or if there is a manual merge conflict it's something really simple. In fact, if there's a complicated merge conflict often the change won't be backported at all unless the bug is actually serious enough to warrant X hours of someone's time to do it and get code review etc. Most of the time this process works correctly, but obviously there's room for error and mistakes can happen.
There's a tradeoff between the risk of running an older kernel that has known bugs, upgrading to the latest new kernel which has bug fixes but may introduce new bugs, and getting backports for known bugs to your known working kernel. Most of the time the last option is reasonable but it definitely depends on your use case and what you're optimizing for.
