No, an absolute massive amount of people do. In fact they have been doing exactl...

refulgentis 39 days ago | parent | context | favorite | on: 30% drop in O1-preview accuracy when Putnam proble...

No, an absolute massive amount of people do. In fact they have been doing exactly as you recommend, because as you note, it's obvious and required for a basic proper evaluation.