I've listened to hours of Hinton and Yan LaCun on their tacit debate on this. And over and over I come away with Hinton explaining mathematically and conceptually whats going on in LLMs and why it's a useful simulation of intelligence and will continue to grow, also while the world backs him up.
While Yan LaCun's arguments are just 'it doesn't work that way' and 'thats not intelligence' and at point literally quoting stories of failed LLM tests that are patentently untrue today.
I dunno I find the whole thing REALLY wierd... beyond the explanation that LaCun still adheres to a really outdated form of saying we must teach AI logic explicitly for it to be intelligent.
I think when he uses the term he means a good simulation of what we would call consciousness -- so yes I agree. It can pay attention at many layers of abstraction and create from it.
He said:
"I think multimodal chatbots are already having subjective experiences."
You changed this in your paraphrasing to something along the lines of "I think multimodal chatbots are already good simulations of subjective experiences".
I don't think the meaning is the same. A simulation of the weather wouldn't be expected to blow my house down. It isn't interchangable with the word weather.
I also don't disagree with that. Attention layers can alter vectors derived from learned/read/stored experiences either by others or itself, then alter its vectors to output new relative responses based on its unique and subjective experiences. I think it a consciousness that can be snapped in and out of existence -- but yes... in a way...so can humankind.
> And over and over I come away with Hinton explaining mathematically and conceptually whats going on in LLMs
Every time I see Hinton talking about LLMs he's just anthropomorphizing whatever 'mathematics' is going on there. He's a great researcher but tbh I think he's a really silly guy
In his lex interview he repeatedly stated that GPT4 failed basic reasoning tests that if you sat down and ran, like the monte test with modified elements, it can pass just fine. It can reason on new applications of older observations.
This BASIC misunderstanding of GPT was repeated constantly to a degree that made me question if he even understood them at all.
Is this something that changed much between versions of GPT-4?
Do you think Yan meant that GPT-4 failed with some special private test set in the same style (e.g: to avoid contaminated tests)?
I couldn't identify a "monte test with modified elements", do you mean the "Monty Hall problem"? (Not sure that's a basic reasoning test, but it's the only thing I could think of). I didn't see that in the transcript either though: https://lexfridman.com/yann-lecun-3-transcript
Or do you mean that by adding some Monte Carlo magic to the test's completion sampling that it can pass just fine?
For the Lex interview I only managed to catch the first parts about image processing, which did seem perhaps a bit dated. I mean to watch the podcast soon though.
Thanks for your thoughts, sorry I didn't get it yet!
While Yan LaCun's arguments are just 'it doesn't work that way' and 'thats not intelligence' and at point literally quoting stories of failed LLM tests that are patentently untrue today.
I dunno I find the whole thing REALLY wierd... beyond the explanation that LaCun still adheres to a really outdated form of saying we must teach AI logic explicitly for it to be intelligent.