One thing the OP didn’t mention is that the JIT is not enabled by default. They found that it has performance improvements for only long and complex queries yet. [1] But it can be enabled by a simple configuration command: jit = on.
It's expected to only have benefits for longer queries. The problem is primarily that the current logic what is likely to be a long query isn't perfect, so can trigger in the wrong moments. Since integrating the feature we'd added an item in the "pre-release item list" [1], to decide whether to enable/disable - but we definitely wanted it enabled during most of the beta test period. This wasn't really a last-minute "whaaaaaaa" thing...
[1]: https://www.phoronix.com/scan.php?page=news_item&px=PostgreS...