SWE bench has gold code patches and the passing test suite patch for after the github issue was completed. While you may argue over the style of the code produced by the model there is a known good passing state for the model to achieve. For now that's the closest representation of a real world problem solve in a controlled repeatable benchmark we have.
