The AI angle is weird, and a red herring. I’ll happily give 100 to 1 odds that something as abstract as topos theory isn’t useful to machine learning. As far as I know, the full weight of topos theory isn’t even needed for algebraic geometry.
The story of Grothendieck is a tragedy about a generational genius, not unlike Godel’s story. It’s deep and far reaching enough to stand on its own without AI hype making it appear more relevant.
Wolfram is serious about his software, but, to my mind, has been closer to a crackpot scientist than a real researcher the last few decades. Him backing something up diminishes my subjective probability that this is something "serious"
I was sure this was going to be a case of spraying jargon at investors, but they've actually produced an interesting paper in this direction (in collaboration with some DeepMind people): https://arxiv.org/abs/2402.15332
this paper is not good: it's easy to describe things in a categorical language (which this paper does), but not as easy to draw insights from that framework (which this paper does not do)
Classic application of abstract math in something sexy at the time. I once saw a paper describing a trading strategy using stochastic calculus. Turns out it boiled down to buying when price went under some indicator variable, selling when above another.
Stochastic calculus is the only available tool for proving things about continuous-time stochastic processes. There aren't any alternatives, save guessing at criteria and backtesting them.
Yes, for sure useful for the appropriate mathematics. My point is, the trading strategy was a simple heuristic wrapped into overly complicated definitions and proofs. The complicated mathematics added exactly nothing to the application.
Yes, it was something like that. But "desirable" here means something very different for mathematicians and traders actually applying the strategy (I.e., they don't care at all, and neither does anyone else working in finance).
Huge difference between stochastic (processes, ODEs, PDEs, etc) and category theory. One makes money every day and the other is only good for writing papers.
It is rarely within reach to draw new insights from applied category theory, in particular because of the Yoneda lemma and the greater familiarity of sets and functions, and also because as algebraic objects categories have very few properties.
Bare categories do not give much insight in pure mathematics either, it's just a common language; interesting things are categories with lots of extra structure like toposes, derived categories, infinity-categories, and so on.
Ehh, I’m an ML scientist with a PhD in category theory and I really don’t see it going anywhere. The comparison to geometric deep learning is especially misplaced, because I think you’ll find that Bronstein’s school sets things us as a group action, etc, but then does a lot of really hard math to actually tease out properties of how information flows through a gnn. Here they just do the Applied Category Thing and say they’ve drawn a picture so QED.
Would you be willing to elaborate a bit on that? So my understanding here is that they are using monad algebras to model the kinds of constraints you might care about (symmetry invariance/translation invariance/etc), and then by instantiating these generic monads over the category of vector spaces and fiddling through the diagrams, you recover the usual constraints on the weights in your neural net. I don't think it's supposed to work around the fact that you have to do a lot of hard math in that step, but it gives you a blueprint for doing it that generalises symmetry constraints. So you could come up with some interesting idea for a NN layer/block and use this schematic to direct your derivation of the corresponding constraints on your neural net, I guess?
So it seems valuable to me in that respect, especially if they can achieve what they want (logical inference-rule-invariant NN blocks, PL semantics invariant stuff, etc).
Anyway, I'm interested in your perspective/objections! (if it's technical that's fine too, I have a lot of maths background)
Oh, I mean, the most obvious problem with that idea is there’s no way to ensure that whenever you update your weights that they will still satisfy those constraints.
This idea has been well studied in mathematical physics, going back to Poincaré, where you work with Lie groups and Lie group actions on your action space. The reason this works, however, is you get a Lie algebra/Lie algebra action that more-or-less behaves like the tangent bundle so the same basic theory around optimization works.
The main problem is they’re generalizing in the wrong direction. Everything still works when you move to Lie groupoids/Lie algebroids. You still get something like a tangent bundle, so ideas like gradient descent or Euler-Lagrange equations still make sense. Thats not the case with a generic monad - in fact the authors don’t seem to acknowledge the fact that there is some work to do regarding compatibility between the monad and derivative to ensure that gradient-based optimization will still make sense.
So, basically, anyone who is familiar with the basics of optimization on manifolds or Lie groups will immediately recognize this approach as hopelessly naive. All they’ve _really_ managed to do is draw some diagrams and say “wouldn’t it be cool if these things were preserved by gradient descent.”
Ah, I see, so they've basically found a nice way to express the easy stuff (finding the constraints) but the devil's in the optimisation, of course.
Thanks for taking the time to reply. Coincidentally, I'm doing a project with Arnold's book on CM at the moment, so that all makes perfect sense to me.
Right, basically the generalization goes: Lie groups -> Lie group actions -> Lie groupoids. This is not a new observation (in fact Arnold’s fluid mechanics can be rephrased using lie groupoids https://tspace.library.utoronto.ca/bitstream/1807/91859/1/Fu..., and big names like Alan Weinstein have worked in that area). I don’t think the authors actually understand that story, so they very naively went groups -> group actions -> monads. If it went that way someone in mathematical physics or optimization would have stumbled onto other concrete examples. But they haven’t, because it doesn’t.
> As far as I know, the full weight of topos theory isn’t even needed for algebraic geometry
The original topos theory developed by Grothendieck and his collaborators in the 60s is quite pragmatic and served to define cohomology theories for varieties over finite fields. Later other people, coming from mathematical logic, distilled "elementary" axioms from there and developed another kind of topos theory that is pretty much divorced from algebraic geometry.
Why not both? Crazy people can be smart. As for mystic delirium well that's our modern sensibilities talking. Hell there are people today who believe in Marian apparitions.
‘ And there is growing academic and corporate attention to how Grothendieckian concepts could be practically applied for technological ends. Chinese telecoms giant Huawei believes his esoteric concept of the topos could be key to building the next generation of AI, and has hired Fields medal-winner Laurent Lafforgue to explore this subject. But Grothendieck’s motivations were not worldly ones, as his former colleague Pierre Cartier understood. “Even in his mathematical milieu, he wasn’t quite a member of the family,” writes Cartier. “He pursued a kind of monologue, or rather a dialogue with mathematics and God, which to him were one and the same.”
Never heard of him before, RIP but this reads like the beginning of neal stephenson novel… interesting
Thought it was important to state this, as Grothendieck also wrote rather esoteric texts, Récoltes et Semailles as well as La Clé des Songes, which haven't been translated or even published.
Suppose he were both. Then do we need to distinguish between them? If so, would it be possible to make that distinction? If not, can we afford to need to make it? This applies to all incomprehensibly gifted persons.
The root cause for his departure from mathematics was his categorical refusal to let the military-industrial complex invest in his research at Bures-sur-Yvette.
- I know the story. He was a man viscerally opposed to militarism. He once said "he'd rather be shot than wear a uniform." One day a letter arrived at IHES in Bures, where the army's scientific services, called DRET at the time (Direction of Research and Technical Studies), now DGA (Delegation for Armament Applications), offered a grant of four thousand francs (650 euros). When he came across this paper, he flushed red, saying "No way we're accepting a penny from these people!" His colleagues tried to change his mind: "Listen, Alexandre, don't be so rigid. It'll pay for photocopies..."
- And then?
- He said, "It's not difficult, we'll put it to a vote. The IHES scientific council will decide whether or not to accept money from the soldiery. But if you accept this grant, I solemnly warn you: you'll have my resignation in your hands within the minute that follows."
- And what happened?
- They didn't take his threat seriously. The vote took place and the four thousand francs were accepted by a majority of one vote. His face then turned grey, hard as marble. He took a letterhead paper and simply wrote: "I have the honor to tender my resignation" then handed it to the council members and turned on his heel. The next day he didn't show up at his office, nor the day after. Paperwork began to pile up. There were letters from all over the world.
- He was a Fields Medal winner.
- His reputation was such that he attracted the greatest mathematicians on the planet to the Institute. For everyone, he was the beacon of Algebraic Geometry, illuminating the entire planet with all its light. At first, people thought it was depression or a disappearance. At IHES, he occupied an official apartment. After a week, they ended up calling a locksmith to open the door. The apartment was empty. Masses of his papers were found in a trash can. He had thrown everything away, his notes, his books, his reports, his correspondence.
- Incredible! ...
- Wait, weeks and months passed without anyone knowing where he had gone. You can't imagine the panic at the Institute. Scientists started calling from all corners of the world. They had to answer and admit that he had resigned. People wanted to know why he had acted this way, under what circumstances this had happened, where he had gone, what he was doing now. The most unbelievable rumors were circulating. At one point, they thought he had committed suicide, but as some people had met him, they had to face the facts: he was apparently still alive. We have a letter from him dated two years after his resignation from IHES addressed to a company providing organic fertilizers, where he complains that these do not meet the specified standards. It was indeed his signature and, it must be said, his style.
- And since then?
- Since then, nothing. The world's greatest mathematician simply vanished one fine day. He simply let the scientific community know that he wanted nothing more to do with this environment. He announced, through a letter he addressed to one of his former students, his decision to withdraw completely. As people had finally located him in this small village near Carpentras where he had rented a small farm, they hoped to flush him out by offering him a new prize, the Crafoord Foundation prize. This must have been in the early eighties. The amount was around forty million francs.
The background of this particular case is interesting, but the larger point is that what we call "genius" is a matter of priorities. From some angle, every gifted person appears mad.
Fantastic! In this money driven world, he had the courage to stand up for what he believed in. Also, I seriously doubt he was considered crazy, that was the war time propaganda trying to break him down and make an example of him, so that others wouldn't stand up. It's the same thing we see today with the coercion and censorship that plaques social and old school media.
I feel like it was more like his experiences from WW2 formed a very rigid world view about militaries. Arguing for pacificism is difficult as the Ukraine war rages on.
thank you for the great story
reminds me of another great mathematician
harvard Phd in math
youngest ever professor, UC berkeley
decided to leave it all behind and live innawoods like this man
oh and he also mailed bombs to people involved with tech (this man's equivalent to the military)
and he would have gotten away with it too if it weren't for that meddling sister in law! I guess he wanted to have his cake and eat it too.
That's correct! The text is presented on the source web page as a story, and it does not claim to be factual.
Alexander Grothendieck was indeed awarded the Crafoord Prize, which he rejected. (It was never worth 40 mil francs as the translation above claims. The original French put it at "40 briques" = 400,000, currency not specified, which is much closer to the more accurate 800K SEK ~ 800K FFr that he would have received. The fact that the full amount, 1,6M SEK, would be split between him and Pierre Deligne, whom Grothendieck had denounced, might have contributed to his decision.) Grothendieck's rejection letter was remarkably lucid and articulate: https://www.fermentmagazine.org/quest88.
"Although fifty-seven is not prime, it is jokingly known as the Grothendieck prime after a legend according to which the mathematician Alexander Grothendieck supposedly gave it as an example of a particular prime number." [1]
“ he rarely made use of specific equations to grasp at mathematical truths, instead intuiting the broader conceptual structure around them to make them surrender their solutions all at once.”
Something that caught my attention recalling Arthur Schopenhauer’s philosophy on the need for conceptual or apriori knowledge based proofs than empirical or derived in math.
The stories of geniuses suffering from depression and other mental illnesses sure make remarkably interesting reads. It’s a pity he didn’t get psychiatric help, this could have been a boring story of an aging scientist taking care of his plants.
Always infuriating to see that people always focus on the his pre-70s (hardcore math) or post-80s period (borderline mysticism ramp up). In the 70s he was most politically active and _definitely_ not delirious in any sense of the word and in fact according to Leila Schneps this is one of the few periods of his life he described as happy, the "sunday of his life" [1]. I translated the '72 CERN talk, its baffling how relevant it is, to this day [2].
The story of Grothendieck is a tragedy about a generational genius, not unlike Godel’s story. It’s deep and far reaching enough to stand on its own without AI hype making it appear more relevant.