One of the great successes of Intel has definitely been establishing their processors as the default optimization status quo. Beyond the scheduler, compilers also optimize (by default) for Intel instruction latencies and pipelines.
One way to remedy this would be to ship more distros with march=native as the default. You only need to specify the architecture when you are building binaries for distribution (which mostly happens on CI servers these days). This would result in better perf on not only AMD chips, but also newer Intel chips.
I wonder if it is worth it to select a few core libraries like glibc, and get the distro to install multiple variants, each optimized for a different CPU.
For x64, you don't need to distribute multiple binaries. You can have one binary with variants of a function each optimized for a particular microarch, and resolve which variant to use at runtime using the CPUID instruction. gcc already does this for you with function multiversioning, and glibc makes use of it for its string routines.
Intels Clear Linux does this IIRC. Wins most comparisons on Phoronix. Works on AMD CPUs too.
(edit) Even though i'm a sucker for efficiency/performance, i prefer Tumbleweed myself. Since i like KDE, whose integration is as good as it gets on SUSE but left a lot to be desired on Clear which ships with GNOME by default. Both are rolling releases. SUSE has automatic btrfs snapshots on updates for easy rollback. But Clear has the performance edge.
Recently (in the last year or so), the platform ABI for x86-64 has been augmented with 4 levels of variants: base x86-64 (i.e., up to SSE2), and then assume up to SSE4.2, assume up to AVX2, and then a base AVX-512 level. (Not mentioned in each of these levels is the random other instructions, such as BMI/BMI2 that get sprinkled throughout.)
Imo, the solution is to stop using C/C++ for high performance applications. Today's computers are diverse enough that distributing compiled binaries leaves a ton of performance on the table.
What would you like you use instead? C/C++ code which targets generic x86_64 is a few percent slower than march=native but that still leaves it way faster than most other languages.
Intel P state drivers and linux schedulers are working great on my recent generation CPU. Power saving is important to me so I run PowerSave governor with turbo boost enabled, for most of my work load it stays at lowest P states with <50 deg C core Temp and during demanding tasks it scales up without delays.
It's unfortunate that AMD's scaling is not par with intel on Linux, I didn't know this earlier.
I don't think it's even slightly true that compilers default to Intel-specific optimizations. All the Linux compilers default to k8-generic, an obsolete AMD platform.
One way to remedy this would be to ship more distros with march=native as the default. You only need to specify the architecture when you are building binaries for distribution (which mostly happens on CI servers these days). This would result in better perf on not only AMD chips, but also newer Intel chips.