On: Claude Opus 5.5 Intelligence, Performance and Price Analysis (Max)

[tedsanders]

How many tasks were in this dataset?

If it's a single task where it dropped from 50 to 25, it could be random variation (not saying it is, but it could be). If it's the mean over hundreds of tasks, that suggests a problem with either the eval code/harness or our API.

[breckenedge → tedsanders]

Just about 90 PRs in that dataset to review.

[tedsanders → breckenedge]

Is it 90 independent tasks, or a single task with 90 pieces?

(it matters if they are independent or dependent)