You can get ~3.4GHz average sustained all-core speed on a 3990x (64-core, nominal 280W TDP). This is with an off-the-shelf AIO cooler.
Note: top-end air coolers are often competitive with AIOs, and can be had for $80-$100.
If you're buying a several thousand dollar CPU, dropping even $500 (much higher than you'd need for closed loop liquid or high-end air cooling) on cooling doesn't seem unreasonable.
250W is not an absurd figure to shove in a server CPU. If you're buying an 80-core CPU, you're not going to be skimping on the cooling solution.
Especially given the target market for Ampere is cloud providers, you can expect these to be racked in enclosures that provide sufficient cooling for their operational needs.