On: Visualizing Attention, a Transformer's Heart [video]

[Demlolomot]

If learning in real life over 5-20 years shows the same result as a LLM being trained by billions of tokens, than yes it can be compared.

And there are a lot of people out there who do not a lot of reasoning.

After all optical illusions exist, our brain generalizes.

The same thing happens with words like the riddle about the doctor operating on a child were we discover that the doctor is actually a female.

And while llms only use text, we can already see how multimodal models become better, architecture gets better and hardware too.

[mjburgess → Demlolomot]

I don't know what your motivation in comparison is; mine is science, ie., explanation.

I'm not interested that your best friend emits the same words in the same order as an LLM; i'm more interested that he does so because he enjoys you company whereas the LLM does not.

Engineer's overstep their mission when they assume that because you can substitute one thing for another, and sell a product in doing so, that this is informative. It isnt. I'm not interested in whether you can replace the sky for a skybox and have no one notice -- who cares? What might fool an ape is everything, and what that matters for science is nothing.

[Demlolomot → mjburgess]

My thinking is highly influenced by brain research.

We don't just talk about a LLM we talk about a neuronal network architecture.

There is a direct link to us (neural networks)