CPU purchase cost matters too. The electricity cost of one 100W CPU might be around 500 USD per year in a datacenter. Assuming 5 years useful lifetime that's 2500 USD per CPU. But cheaper CPUs can easily cost 1000 USD less, so even if they're 20% less efficient they may still come out ahead.
A 100 watt CPU in any modern data center would be at an incremental cost of well under $40.
Data centers get cheap bulk power, cooling is mostly passive now (evaporative) and backup power systems are also a lot cheaper now compared to the compute cost than in the past.
