It models our language, which is a flawed and imperfect way of describing the world. So far, it seems like a lot of these discoveries are "filling in the gaps" between the things we've written down and the things they imply (if you have the memory to think them through).
The story about OpenAI's Navier-Stokes solution is a good example of what I mean. I don't think it would have been possible without computer assistance because that proof is long and complicated. I'm also not sure that it would have been possible without a human proposing a new approach to the problem, because by all accounts that's exactly what led to the absurd amount of spending that OpenAI did to solve the issue.
I feel like that at least implies that there's some room left for humans in the new world.
You sound like someone out of date on AI. A lot of people think they can't reason, when they clearly do that well enough now to replace a lot of people whose job it is to do that. The reasoning will continue to get better. They don't make intuitive jumps yet, and they lack good creativity, but all that will come before long as well.
But stop thinking that just because they were initially simpler solutions that it stayed that way. We are in the singularity now!
I assure you I'm not out of date, but the reasoning these models do is obviously not yet sufficient to deliver groundbreaking proofs entirely on their own.
It took a human being to see the beauty of an alternate approach and purpose it, and only then could OpenAI spend millions of dollars to squeeze out a result.
I am very doubtful that we will see models capable of making intuitive leaps anytime soon, or being "creative." Neither of those are straightforward objectives to train against.
Math is really very different in an important way: its abstract to everyone. Inferring relationships through a world model is far more impressive to me. Hinton has a clip where he mentioned how impressed he was that an early ChatGPT could draw the parallel between compost heaps and nuclear reactors - totally different forces involved but still chain reactions that begin when a sufficient mass is packed into an enclosed space.
The story about OpenAI's Navier-Stokes solution is a good example of what I mean. I don't think it would have been possible without computer assistance because that proof is long and complicated. I'm also not sure that it would have been possible without a human proposing a new approach to the problem, because by all accounts that's exactly what led to the absurd amount of spending that OpenAI did to solve the issue.
I feel like that at least implies that there's some room left for humans in the new world.