> If the models can do this, they're almost certainly good
I would argue that "good at math" was a short hand for "good at X" because mathematicians were historically good at engaging with very complex ideas, distilling them and coming up with precise and concise answers they could validate by themselves.
Given that AI solutions are described as "psychedelical" and they rely on the outside source to validate the result, I don't think the same logic could apply to them.
I don't see anything stopping the models from getting to that point, though. So far we've gone from the models not being able to do math at all, to improving on progressively more difficult problems, to claims of "it's just a counterexample", to now producing constructive proofs and new mathematics on the hardest problems we currently have. All within less than two years.
Being able to do a task well and explain what you're doing in an organized way are fairly orthogonal, even in humans. I don't see why the skills the models currently have wouldn't transfer, even if the results look psychedelic. The next step would obviously be to improve trust by improving the way in which results are communicated.
I would argue that "good at math" was a short hand for "good at X" because mathematicians were historically good at engaging with very complex ideas, distilling them and coming up with precise and concise answers they could validate by themselves.
Given that AI solutions are described as "psychedelical" and they rely on the outside source to validate the result, I don't think the same logic could apply to them.