But that exactly how human mathematicians do things. They upload their research to preprint services like arXiv as they await it to be peer reviewed and accepted into a journal. Why is it okay for mathematicians to publish preprint papers, but when OpenAI does it, it's irresponsible?
The difference is that human mathematicians wouldn't post it online, claim they have achieved some proof, and then check after publication and announcing the results to the world. You would always check your work first, then publish it. It's not about preprint vs peer-reviewed. That of course is normal practice, it's the high-profile claims that are being made that are the problem here. They just blindly published results produced by the LLM, with 0 due diligence.
Yes they would, in many fields it is (was?) normal to post a preprint and leave it up for the next year while the handful of other people working on the topic digest it and agree on whether it’s right or not, before even submitting to a journal. Sometimes the others would find an isssue in an argument and you hopefully manage to fix it or potentially retract / not submit to a journal.
You may want to read beyond the first part of a sentence. Yes, preprints have always been a thing. The claims and publicity around the potential findings are handled differently here. That's where the issue is.
What would’ve made you not complain about OpenAI’s papers?
They had over a hundred papers, they basically did a GitHub dump and a pretty bare blog post that mentions they have a retraction policy. Should they have done it anonymously? Is the blog post the problem? Is it that it’s on GitHub? Or what?
They should have posted the ones they actually confirmed. They don't have any time pressure to release hundreds of new proofs, they could do 10 a week for all I care, as long as they make sure everything is actually sound and properly checked. You know, do the due diligence with the results. Not only would that prevent them from having to do retractions, it also wouldn't be any less impressive. Just not as much shock marketing value.
It's really not that hard to understand. My issue is NOT with the fact that they publish results. It's purely about how it's done and what the misleading claims are that come along with them.
This goes back to the pre-print question then, which you just dismissed
> in many fields it is (was?) normal to post a preprint and leave it up for the next year while the handful of other people working on the topic digest it and agree on whether it’s right or not, before even submitting to a journal. Sometimes the others would find an isssue in an argument and you hopefully manage to fix it or potentially retract / not submit to a journal.
A pre-print has a chance to be incorrect. Pre-prints do sometimes end up incorrect and need adjustments or even retraction. It sounds like you define "confirming it before you publish" as being 100% certain it's correct. Nobody is 100% sure their pre-print is correct. If it was 100% correct and never retracted there'd be no need for this process of sharing and digesting it before actually submitting. You do your best and maybe someone else thinks about it in a new light you missed and it's suddenly wrong.
Is the rate of retraction and/or correction going to be worse for OpenAI than it would be in normal circumstances for a normal human author? Only time will tell, it's only been a few days.
But all these complaints really just look like pulling at whatever straw to deny what's happening.
They could've instead hired a select group of mathematicians to spend then next 6 months checking these works, trying to understand and refine them, build on them etc... You and the others here would have 100% complained. Even more bitterly. And then you would've perhaps been right: it would have looked like there's a select in-group of mathematicians that get to see and work on the new breakthroughs while the rest of the field is kept in the dark. Instead of now where it's public, anyone can do that. Maybe that's what will happen in the next release. It's not gonna be good though.
ok, that said there has alaways been a bunch of "maybe theorems" it's was a thing even in paul erdos books i think. now we have "maybe proofs". still better than "no proofs"
See DDOS. That's exactly how humans access a web page. Why is it okay for humans to access a webpage, but when a bot swarm does it, it's irresponsible?
That's an analogy among many others, but the point is that OpenAI should use their tools responsibly. If they have 700 potential ground-breaking but unproven results, they should share it in a way that they do not get free (possibly unwarranted) publicity for it.
