Why not? If mathematicians in the field cannot understand the papers, but tools at our disposal can, why not use them? What is "unhealthy" about that? Just seems like it's the reality of the world now.
If you can't understand what the machine has done, math is not being done. Math is not pushing symbols around to find sequences that follow all the rules. Math is the process of understanding how those sequences of symbols map to meanings. If you can't follow the proof but have validated it's correct, all you know is that it's a proof of something. Is it a proof of what's claimed in the adjacent plaintext? Who knows!
Math is about clear communication between people first and foremost. That's why Shinichi Mochizuki's claimed proof of the ABC conjecture isn't accepted - he's unable to communicate it to enough mathematicians to get a consensus on its validity. LLMs generating Lean proofs are basically following that same schema. Sure, something has been proven. But what? Where's the clear communication explaining it to the satisfaction of other mathematicians?
I mean it's just patently absurd. It is trendy to use this hubristic, and frankly bureaucratic, definition or requirement on math.
People can do math alone. They don't have to share it with others. Mochizuki was doing math (with errors). The aliens from Arrival could do math and we wouldn't necessarily be entitled to that. It doesn't make these things not valid math.
The only benefit of defining math through this vulgar social reductivism is to benefit the status quo. So you see some experts and expert-adjacent commentators using these math slogans but never critically justifying those words. Case in point, the above commenter asserts "...math is foremost about communication.", but the very next sentence does not explain why it is foremost, it just uses the sentence as a premise for the following sentence. Guess what, that is exactly how ideologues argue, by question begging fallacy. Ironic that STEM people who ought know better fail to catch this simple rational mistake.
Math that isn't communicated is valueless - by definition.
Applies to STEM generally, because the foundation of STEM is peer review and independent testing and confirmation.
Even if the practice never quite lives up to the theory (true) that's no excuse to hand-wave away the requirement as if it doesn't matter.
Proofs that can't be confirmed are valueless because no one learns anything from them. The value of a proof isn't just in proving something but in inventing new techniques and insights that can be reused for other applications.
Brute force searches are fundamentally less valuable, because the creative foundation of math is discovering leaps of logic that solve problems with insightful shortcuts that can be generalised and reused for other applications.
> Math that isn't communicated is valueless - by definition.
That is quite a bold claim.
Math can always be used as a tool just to achieve results. This can include even novel math. Due to the maturity of the field it is just becoming more and more unlikely that somebody will come up with something novel without realizing the significance.
One example: Quicksort was almost not published because Tony Hoare initially considered the idea behind the algorithm trivial. In the end, all he wanted to do was to sort some numbers.
Maybe there is a polynomial time SAT solver running somewhere just because some overworked engineer implemented it for a completely different problem. And he was just annoyed that all existing libraries were so slow and he doesn't realize that he is sitting on a proof that P=NP. Who knows.
"Mathematics is commonly explained and recorded in symbolic and concrete forms that are easy to communicate, rather than in conceptual forms that are easy to understand once communicated. Translation in the direction conceptual -> concrete and symbolic is much easier than translation in the reverse direction, and symbolic forms often replaces the conceptual forms of understanding."
Human understanding is important, but I don’t think it’s the end all be all. There’s no reason to expect humans to be the best mathematical reasoners in the universe. And if math is important, beyond being a human art form, I think we should explore that math beyond our understanding.
I'm on the side of human understanding being fundamental to mathematics, but I'll try arguing from the other side.
If we consider the reductionist view that "math is just a tool" to solve problems, and machines can generate new tools that are actually useful, then it can be argued that we don't need to understand how those tools work internally for us to benefit from its usefulness.
My hope is that such a view is given its proper place as a worthy pursuit but that it does not consume the entirety of the field of mathematics as a human activity with culture, philosophy, and art. And I feel the same for the field of software.
I mean, I could not understand 95% of all the math out there due to skill issue, and it’s still valid. If machines can understand the math and humans can’t, we can’t validate it, but that doesn’t make the math itself invalid.
It could be logically valid but irrelevant? Allowing for the atrophying of all our skills is dangerous? While simultaneously building ever more complicated edifices that control our lives? Not a doomer but you really just want to shrug and say it ain't no thing?
Perhaps humans have peaked due to the fundamental biological nature of our brains and AI will eventually surpass us in mathematical intelligence.
That has been the ultimate goal of creating AI -- not to simply match human intelligence but to surpass it. If it surpasses us, does that make it irrelevant? Everything around us is powered by mathematics, most of it I don't understand, and soon perhaps everything will be powered by mathematics that everybody doesn't understand.
Because we kinda suck. Wars, poverty, destruction of the environment, etc. Maybe AI will be worse -- the are plenty of movies on that subject -- or maybe it will be better.
> Is this going to lead to anything other than displacement in the long term?
We may have already damaged the environment enough that we're going to die off soon anyway. So without AI, we were already on that path to "displace" ourselves -- I don't think that argument is that compelling.
Perhaps this is the natural evolution of intelligent life. Eventually the sun will die and everything we are will be gone forever, maybe this is the only way that something of us survives.
> Do you ever stop and ask why we’re building the world we’re building?
We're not building a world. We're just ants. Individuals making individual decisions that have nothing to do with the rest of the world. You don't have to look too far to find humans that literally care nothing about humanity as a whole and everything about themselves.
> To what end does a world we cannot comprehend or rebuild from first principles serve us?
We already live in that world. You wouldn't even be able to build a toaster from first principles without the knowledge and infrastructure that already exists.
I mean, ideally to the end of poverty, disease, climate change, war, starvation, etc. You can disagree about whether this would be the outcome in practice, but I don't think 'why might we want to create something smarter than us' is a hard question. Humans losing their status as top dog is really a trivial price to pay for the potential return.
I don't understand that what you mean. Disease and climate change seem like clear cut technology problems, and the rest, save for war, have obvious technological solutions. What does it mean for them to be human problems? Should I expect them to solve themselves?
Current technology is sufficient to greatly reduce climate change, there just isn't the political will to do it.
Obviously not every disease can be cured, but even diseases that were essentially gone are coming back because people have just decided they don't care anymore. There are measles outbreaks happening all over the place right now.
mRNA vaccines are a medical marvel and a significant portion of the population has been convinced that horse paste is a better solution.
There are enough resources available that poverty and starvation could be essentially eliminated or at the very least drastically reduced. We just don't do it because people would rather hoard wealth instead.
The technology is here, but too many people profit from not solving the problems
They killed your comment, but you should know that you've posted it on the world's #1 AI propaganda site, where literally trillions of dollars are at stake and ROI depends on the correctness of the discourse running among the meat LLMs foraging or lurking here.
It's not that their argument is wrong or not worth discussing, it's that it is utterly irrelevant when the question at stake is whether the math is valid and useful or not. It's effectively whataboutism.
> If machines can understand the math and humans can’t, we can’t validate it, but that doesn’t make the math itself invalid.
How can humans tell whether machines understand the math if they themselves don't? Math is a human language for quantitative reasoning.
If large sectors of the population decline in their ability to read and write <natural language>, how will they be able to evaluate machine-generated <natural language>?
Perhaps, but two things:
1. most people don't understand this level of maths either
2. math, unlike most natural language, can be (dis)proven without a person understanding.
The second one is very important. While it is a matter of pride that some do understand, and (given enough time, in my opinion) will continue to understand, it is sufficient to be merely correct even if one does not understand why. This is a nice feature of math: it remains true regardless of understanding.
The obvious caveat is the machine needs to prove it in the end, even if we can not.
The underlying tautologies always existed. Making them accessible to us as truth and usable for one or more functions is what's invented by people (and potentially by AI) via insightful proofs and narratives.
I think the nuance they are pointing towards is that we cannot with any certainty say one way or another, as this is an unsolved problem in mathematics, and may possibly remain unsolved forever.
Your opinion is a possibility but it is not grounded by any proof, as such it is nothing more than an assumption. I think the more intellectually healthy approach is to list out the possibilities and figure out what they entail, rather than to throw all your eggs in one basket blindly, which amounts to a leap of faith or a retreat to commitment.
But that's just my two cents, take it or leave it.
I absolutely do not understand what are you talking about.
The question whether the tautologies exists regardless of whether we know about them or not is absolutely not a problem that is studied by mathematics and can be proven by mathematics.
Correct, these questions lie outside the internal practice of mathematics. If you want to see where that computational mindset breaks down, read Carnap's "Empiricism, Semantics, and Ontology" regarding internal versus external frameworks (or have an LLM summarize it...), or look into Brouwer's Intuitionism, or better yet into Lawvere and Topos theory, where the "underlying tautologies" literally change depending on the categorical environment you construct (but this may take some time to wrap your mind around so don't expect to understand much right away). Or, given the thread topic, you can just prompt an LLM to interactively walk you through how classical logic fails in a constructivist topos, or if you have interests outside of applied math or mathematics itself, you can ask it about foundational questions in science and mathematics, which unlike the prevailing opinion in many tech circles, aren't resolved.
Why does this not feel like a healthy place to be?
The author of those sentences is also the author of the main result that this generated improvement builds on, and prior to this improvement his work was thought to be unimprovable. He is, in all of the important ways, literally the single most qualified human on the planet to asses this result. And his assessment is 1) this improvement is real, 2) this improvement is exponentially bigger than his work and the work he improved on, 3) it is effectively impossible for him (and by implication, humanity) to understand the result as presented, 4) it is seemingly trivial for “frontier circa 2026”-class models to understand the result as presented, and 5) to the extent that he is able to understand it, it is only with significant frontier model assistance.
So the place we are at, at least for this highly intellectual field, is “intelligence is the property of ~2 corporations, you must pay them a fee and agree to their terms and conditions to access it”.
At risk of stating the obvious, this is not a healthy place to be because intelligence is now structurally subordinate to the profit motives of particular corporations and at the mercy of all the attendant economic/sociopolitical incentives that apply to them. If the landscape is such that it is in these corporations interests to withhold this intelligence from some and make it available to others, that is simply what will happen.
One trivial example, of which there are an arbitrary number: An elderly leader is convinced to issue a decree that all number-math-computation intelligence is to be made available only to the intelligence agencies of his nation, and failure to comply means no IPO and no further investment. From this point on there would be a permanent nation-state-shaped shadow over practically all modern cryptography, with no recourse possible.
If the Church had a Large Astronomy Model in the time of Copernicus, the earth would have stood still - regardless of what it may privately prove about heliocentrism, it would never declare e pur si muove.
Leaving aside the (much more important) issue of monopolizing intelligence,
and assuming the claimed result is indeed correct, the question is whether
a) the correctness proof is indeed too complex to be understood by humans, or
b) the creator (the LLM) is not able to write it down in a form digestible by humans.
Observing the LLM output on problems that have clear solutions invented by humans, I tend to believe that too many people are jumping to conclusions too soon and interpreting b) as a).
If the LLMs are truly so great, we should see (in a short time, i.e. in order of months) a transformation of the math proofs generated by LLMs from the current state of indigestible mess to something that indeed looks like written by human for humans.
Only then would one be able to claim that we got into situation a).
If any of those papers were sent to a scientific conference, they would have been rejected as inscrutable gibberish: "resubmit once the presentation is significantly improved".
Yet, mathematicians spend their time trying to understand them.
Why should we expect this? Does it help the corporation’s bottom line? How much? Is that worth the R&D spend and compute budget it would need? What advantage do they stand to gain by demystifying their product, that they don’t already get from e.g. Lean verification? Do they gain more if the same research and compute spend on their inscrutable intelligence is directed into making it more intelligent, instead of less inscrutable?
"Our previous version was super-smart, but a bit autistic, even the smartest mathematicians had problems understanding it. The new version is not only super-smart, but can also efficiently communicate its ideas, so that they can be used in real life."
2) Actually, try to use the AI's smarts to gain in real life (not only convincing others to pay you, but actually do the smartest things). Hmmmm, scratch that, that is too idealistic. Use the AI's smarts to become the strongest meanest bastard the others will have no choice but to obey, or else.
Many corporations (their actual customers they're broadcasting to when they say how smart their models are) have requirements (externally imposed, so not something they can just change) that all code is reviewed, so they need it to be understandable, which is basically the same problem.
> An elderly leader is convinced to issue a decree that all number-math-computation intelligence is to be made available only to the intelligence agencies of his nation, and failure to comply means no IPO and no further investment.
Who would staff those intelligence agencies if humans are no longer able to understand the work product?
…the pre-existing staff, who by virtue of being staff of the intelligence agency, would be exactly the humans who are permitted to use these gated intelligences, and can therefore use them to explain the results, as described in point 5 of my previous comment? I’m not certain I understand your question, is that what you meant? Or did you mean something else?
In any case, it was just one example, I am sure we can think of plentiful others.
Yup, and it always has been, we don’t even understand how our bodies or minds work, yet we use them everyday
The amount of unimaginable complexity contained in a seemingly very tiny amount of matter to us, is incredible, and it’s all around us in nature
We’ve never had the capacity to fully comprehend neither ourselves nor the environment we live in. But we do our best, and we do survive and sometimes even thrive
Even pre-AI, there have been proofs that took years to be verified, understood and some were disputed forever and still are (like the Mochizuki / IUT drama). So, even without AI, maths has reached a point of specialization, where only a few people in the world know the areas used in a proof well enough to check it. When „new math“ is invented for a proof, people may even be more disincentivied to put in the effort. So, I‘d argue, AI makes understanding new proofs easier, not more difficult.
The mistakes that lurked in radium, in arsenic, in leaded gas, in fallout downwind of nuclear tests ... all had the best tools that the previous generation knew.
None of them were okay. Those were small small tools with few (sometimes only one) uses.
Generations earlier, famines, diseases and lead-leeching plumbing made rulers insane and entire cities of people die without any help from the "best tools".
Scifi is always written about the best tools. The issues arrive in the tools and in the hands that wield them both.
There's nothing that makes this paper the be and and end all opinion pieces of the paper. The community will review his review, and people who wish to understand more will read more and more of the paper. I see nothing wrong with this process of progressive understanding.
What a funny pov?? Mathematics and other pure sciences are about human understanding. You're saying if humans cannot understand the thing that was for human understanding, that's ok? That's like saying all the world's food is now inedible, but that's ok because a blender can consume it still.
And one day AI might use this complex math to eradicate humans and you will still be "it's all completely fine"?
The fundamental argument boils down to: "who is all this being made for"? And the answer should be "us, humans". A lot of people here seem to breathlessly fawn over AI as some sort of value in itself, but this is incorrect. There is no absolute value in the universe (as far as we know), the only value to us as humans is subjective.
If AI is not generating value for humans, it isn't generating value, full stop.
Hence the question: "what's the point if we can't understand it?" is perfectly valid and should be discussed. The broader question of "what's the point of AI if not for improving human lives" gets lost in breathless adoration of a technology where nobody knows the outcome.
I'm not saying we need to abandon the pursuit of AI, but we must ask difficult questions at every step because if the technology ends up being as ground-breaking as many here think, it could just as well precipitate a dystopia.
Could very well be. Or not.
But the most of the ones questioning are not questioning for that. They are just mad someone else took their lunch.
Nothing ai is producing till now is non-understandable to humans. Might need some extra effort which for obvious reasons these mathematicians don't want to.
The "top layer" of maths being this is one thing. Now imagine another layer of this. Then another. Then another. Then another.
At one point intuition helps to build out usage of the math itself. See stuff like "oh I can do rotations through matrix multiplication"-style ideas.
It is helpful for us to be able to reach for those tools quickly. The more tools that require advanced prep to use, the less we'll be able to use the results in easier contexts. Everything will be hard
Of course nothing really prevents people from looking for the simpler tools and proofs. Lots of math stuff starts off complicated and gets simpler.
I'm not a mathematician but from what I've heard from them that's generally not how mathematics works. You can "use" whatever you want, of course, but a mathematician's goal and main activity is finding and understanding proofs. The ones I knew also liked compiling error lists for math textbooks and sending them to the authors. (It opened my eyes because I admit to have been one of those ignorants who implicitly assumed "it's formal stuff in a book, sure it's going to be right".)
Especially for tools built to tell us we’re right and happily pretend to solve problems they didn’t.
I think the OpenAI “revert” of their math problems speaks volumes of how volatile this can be.
There are a few increasingly powerful companies that own the frontier models that can produces these results. In the future, it may only be these models that understand the results. Should we be beholden to a few companies for basic understanding of foundational principals?
But where they do differ is they do have an infinite stamina, and can work on problems in loops, self-correcting their own mistakes if a formal way of checking the correctness of the solution exists. Which does happen to be the case with math.
a) because the AI is not able to write papers in a manner digestible by humans?
b) perhaps because AI generates papers whose complexity is beyond what a human brain can grasp?
Aaaa, you mean 'peer-review' by AI?
Then the answer is: What is the use of a paper no human understands? Then no human can see that the techniques from the paper can be used to build something useful for real human life.
OK, ok, AI can decide to use those techniques to do something in real life it considers useful.
I dunno. There is a flavour of a physicist who is only ever dabbling in theory, and relies on others to experiment and verify their ideas. I don't see why that can't be usurped by LLMs. Its just match, data and Gedankenexperimente.
The tool may malfunction. Indeed there are theorems that say the tool must malfunction at least sometimes.
If you don't understand the subject and relies on the understanding of the tool, how will you have a frigging chance of catching an error?
Anyway this reminds me that scene in Matrix 2, where the mayor (of sorts, don't remember his exact title) of Zion tells Neo about the machines that keeps everyone alive, but that there is nobody left that understands how they work or could repair them if they broke.
>Indeed there are theorems that say the tool must malfunction at least sometimes.
Gödel's incompleteness theorem only states that there are true statements that can never be proved to be true. That limits what tools can do, but it's not a "malfunction". The exact same limitation applies to humans. A genuine malfunction would be if the tool "proved" a false statement to be true. That actually has happened before because of bugs in proof checkers, but software bugs can be found and fixed, and proof checkers are simple enough that it's possible to write many different implementations. It's extraordinarily unlikely that the same malfunction occurs in all of them.
I'm not talking about incompleteness though. I'm broadly talking about limitations of LLMs. For example, this paper from this year https://www.nature.com/articles/s41586-026-10549-w says that the current training regime incentivizes hallucinations and can't make them go to zero. Well in the pre-training phase at least (but I guess you could make a similar argument for post-training with RL)
I'm not talking about proofs. I was answering to this
> Why not? If mathematicians in the field cannot understand the papers, but tools at our disposal can, why not use them? What is "unhealthy" about that? Just seems like it's the reality of the world now.
Which talked about using LLMs to understand math, not to prove theorems
edit: to be clearer, on my areas of expertise, I regularly spot LLMs making false or misleading statements, which usually comes accompanied with a lot of truthful ones. If I push back, they often backtrack and admit they were wrong.
On areas I do not have expertise, I can't be a good judge. And this was my initial point: if you don't understand the subject, how will you catch errors?
So I don't think you can or should use to replace understanding the subject yourself. It's unhealthy to offload your thinking to a tool because it makes you kind of useless.
because the tools can't?
Lol. That's not how it works. Am I in the twilight zone?
All this anthropomorphism of literal statistical text generators.
Yes they exhibit generated sentences of text that describe coherent responses about a topic when there's an inordinate amount of research material in the training data from which they are distilling sentences of sound and rational thought. That's the ONLY way it can possibly work. There are so many scenarios where the training data isn't setup like that and low and behold the generated responses are just literal garbage.
We can't explain how human thinking works, and how they produce their (verbal or written) output.
We can explain how LLMs generates this output.
Humans can't be that simple, there must be some ~~divine~~ unique human touch to what we do.
Every day LLMs, running in frameworks that add earlier "thoughts" to the context and iterate, reflecting on their own thought process, gets better and better at doing something even less distinguishable from human output.
The gap gets smaller. The "unique human touch" will always be in the gap.
