On: Artificial Analysis Intelligence Index v4.2

[__natty__]

I wonder if Artificial Analysis could be influenced by certain model companies. Looking at the changelog [0], they updated a few times after new models appeared, so US models progression was much higher than that of other vendors (like when they updated the algorithm after Kimi K3 versus Opus 4.7, so Kimi dropped in the rankings). Or maybe thats just coincidence.

[0] https://artificialanalysis.ai/changelog

[dist-epoch → __natty__]

That would be the fastest way for them to completely torch their company.

The only thing they are selling and why people look at them is trust that they do honest evaluations.

[marmarama → dist-epoch]

People still buy into "research" from Gartner and Forrester despite it being an open secret for decades that they are pay-to-play.

These kinds of "research" companies are there to validate people's preconceived ideas and purchasing decisions rather than being genuinely unbiased. And they are very useful for that, both for consumers and for marketers.