Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Are these novel coding tasks, or could they have ended up in the training set?

Does the number/frequency of safeguard stops affect the measured speed?

 help



Private benchmark from private repos so yeah, novel.

I think I included the Opus 4.8-backoff runs in the speed number and they were on average faster than the actual Fable 5.1 runs, so the speed result might be even bigger if I excluded the Opus 4.8 runs.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: