On: Fable and the end of the free lunch

[shawnb576]

Because LLMs don't understand anything. That's the tech. They can only predict what they have been trained with and fail daily at the most basic tasks. Granted they can do amazing things, no question there. But they are not "smart".

For example, it seems that even at Fable scale, simple concepts like the passage of time or (gasp) timezones elude them. I live in UTC+10 and with any RFC8339 data LLMs are constantly confused - is it Sunday the 10th or Sunday the 9th, etc. I have tried many solutions for this and every time it finds a way to get it wrong.

[penteract → shawnb576]

To me it sounds like you're repeating what gp said about the lack of online learning. Do you think that's insurmountble?

Getting confused about timezones does not place LLMs behind that many humans. (But doing so repeatedly does highlight the lack of online learning).

[dboreham → penteract]

I see this thread as progress because now there's three people saying this (seemed like it was just me for a year or two).

[josephg → shawnb576]

> Because LLMs don't understand anything.

How do you square that then? They can do amazing things, but they're also not smart? Do you think its possible to solve Erdos problems without any "smarts"? Can you do it without even understanding mathematics?

I find it very hard to hold the idea that LLMs don't understand anything. They can explain concepts, translate them, simplify them and implement them in code. From the outside, LLMs seem to understands most concepts better than most humans do. Do you understand anything? Couldn't I make the same argument? How would you prove that you understand what a for loop is, or that you know what calculus is? I assume you'd demonstrate your knowledge by using a for loop in a program, or explain calculus back to me. But LLMs can do that too.

> For example, it seems that even at Fable scale, simple concepts like the passage of time or (gasp) timezones elude them.

Funny example, because lots of human struggle with this too. The number of meetings I've had with people in the US! "Lets meet on thursday morning australia time!". Only, they actually meant thursday night US time, which is friday morning australia time. "Oooh that's so weird! Its the next day for you!". ...... Yes, I know.

I think LLMs are just a different kind of intelligence than humans. They're better at some things than us, and worse than others. They can find latent security vulnerabilities in the linux kernel, but struggle to count the Rs in strawberry. They're not as smart as humans in many ways. But we're not as smart as LLMs in plenty of ways too. I didn't find those linux bugs.