> The noise is suspicious because it's the difference between brute forcing and cognition. A human proof won't just be logically correct, it will be cognitively distilled and coherent. It may still take years to understand it, but the logical flow will be straightforward, not obfuscated.
This is basically how all AI approaches appear to work. They solve the problem you give them (in many cases) but in a very over-complicated way.
It's like they can't step back from the problem and realise that they need to simplify to make it work better. Nope, just keep hammering more code/proofs against the problem and eventually you'll hit the goal.
> It's like they can't step back from the problem and realise that they need to simplify to make it work better. Nope, just keep hammering more code/proofs against the problem and eventually you'll hit the goal.
There is nothing wrong with that approach if it accomplishes the goal faster, i.e. you can work at the speed of an AI.
> There is nothing wrong with that approach if it accomplishes the goal faster, i.e. you can work at the speed of an AI.
Sure, and god knows I've done the same myself when under time pressure (so similar to an AI in RL training, just get the goal).
The part that's lacking (that humans are much better at right now) is the meta-cognition around this works, but it's far more complicated than it needs to be.
Potentially because LLMs have larger "working memory" than humans, they can avoid this for a while, but once it blows up the context they fail in much worse ways than a human would.
Hmmm, this feels like a good metaphor, I need to write about this at some point.
This is basically how all AI approaches appear to work. They solve the problem you give them (in many cases) but in a very over-complicated way.
It's like they can't step back from the problem and realise that they need to simplify to make it work better. Nope, just keep hammering more code/proofs against the problem and eventually you'll hit the goal.
RL has a lot to answer for, I guess.