On: LLMs Are Still Just Predicting the Next Word. So Why Are They So Good Now?

[tudorhn]

Author here.

Would you dispute that they're much better than two years ago? I, personally, am quite impressed at how quickly they improved. It's not yet at the level for independent decision-making, but if the rate of improvement holds, I wouldn't be too surprised if we got there in the near future.

[jqpabc123 → tudorhn]

Would you dispute that they're much better than two years ago?

I will readily admit that the guesswork has improved.

Will you admit that guesswork remains the foundation of what they do?

Guesswork will always be suspect.

Removing it will require a fundamental redesign and may not be achievable within the constraints of current hardware and economics.

My guess is any significant breakthrough will likely involve quantum computing and renewable energy.

[tudorhn → jqpabc123]

That's literally the thesis of the article.

Even though guesswork remains the foundation, the improved training can help discover patterns, backtrack, etc. Basically, explore the problem space by "guessing" through it

[jqpabc123 → tudorhn]

Basically, explore the problem space by "guessing" through it

This is where we disagree.

"Guessing" (aka trial and error) is not a "legally" responsible way to make decisions in many cases.

[tudorhn → jqpabc123]

Right, I think I understand.

You are specifically talking about independent decision-making.

I have been focusing on "problem-solving".

Tbh, I have no opinion on the legal framework. I'm sure most decisions will be supervised, at least for a while.

[jqpabc123 → tudorhn]

The technology is being marketed as providing "intelligence" --- not trial and error problem solving --- where "error" is a likely result.

I wouldn't be surprised if the tool vendors aren't eventually forced to rename it and/or limit it's use in their terms of service due to legal liability.

A good analogy is Tesla's "Full Self Driving" which was renamed to "Full Self Driving Supervised" --- an oxymoron.