It’s real weird to see people argue that LLM output is no different than random gibberish and then handwave over the fact that it’s clearly not with terms like “training”, as if a steam of random garbage is trainable.
I quite literally created and productized predictive linguistics and behavioral vectors at Google.
If you had stopped to consider what I explained; you’d understand that it’s the process of turning random garbage into increasingly acceptable outputs.
Ie training the monkeys.
The insight you are missing is the rule of networked scale. It turns out that any reactive node scaled enough can form sophisticated predictive system given reward over a training topography, even if it starts out at garbage or is literally made of monkeys.
So it is garbage. And you can turn garbage into semi-intelligence.
A human child is born with no ability to speak intelligibly. All they can do is babble. Through years of training they gain the ability to speak intelligibly and communicate in advanced ways.
The act of successful training means it’s not garbage anymore.
> So it is garbage.
This statement is ultimately meaningless and I continue to find it weird that someone who works in this space would support this view. If you fundamentally change the nature of a thing, it’s no longer that original thing. Is tan HDD still random garbage after you fill it with family photos just because that’s how it starts?
If you start with a fire hose of literal sewage and install a series of filters culminating in a reverse osmosis step that pours clean drinking water out, the product is not shit even if the original input was.
I don’t believe that you can’t understand the distinction between “at one point this was garbage” and “at the present time this is still garbage”. You’re clearly smarter than that.
Except it’s not at all the same process. The fact that LLM are non deterministic is not the same as churning out random garbage.