On: Can LLMs invent better ways to train LLMs?

[mdp2021]

> Can LLMs invent anything

Can they propose a working novelty:

-- after deep thought about idea soundness, probably not at this stage

-- through cycles of trials, not knowing exactly why - probably yes

After all, your hammer needs not intelligence.

[steve1977 → mdp2021]

But my hammer is totally useless without me as a human using it and telling it exactly what to do.

[mdp2021 → steve1977]

Yes, exactly. Tools perform without "knowing" the purpose. Unintelligent yet effective.

So, "perform a selection over the enumerated combinations in the solutions space" works without the process being further sophisticated. It works as much as it can - as a preparation of data until the stage in which intelligence is required.

We have been doing it since a while; simulated annealing, genetic algorithms... Dumb hammers in a way, encoding an action from an intelligent operator, and providing an effective aid when under intelligent control.