On: Study identifies weaknesses in how AI systems are evaluated

[Havoc]

Id hope anyone using LLMs in production is testing them against their use directly.

Benchmarks make for a good first pass though to figure out which ones to test