On: Something Big Is (Not) Happening

[MattGrommes]

This is what I've been saying. We're not so much learning that LLMs are intelligent, we're learning that a lot of what we think of as intelligence is actually just pattern matching.

[red75prime → MattGrommes]

But the real intelligence exists and it is beyond all those trinkets. Right? Yeah, this is a common cope.

There's no solid evidence that the brain uses quantum computations. And that's the only definite thing (besides 'magic') that can solidly place the brain out of reach for now.

[alansaber → red75prime]

To frame the human brain itself as a startup, i'm not convinced by "latent quantum effects" at all as the moat for human sypanses.

There's something else that we are missing there- probably at the convergence of multiple biochemical effects (like ion transport).

[red75prime → alansaber]

Evolution has scaled the brain to hundreds of trillions of synapses (parameters) and hundreds of billions of neurons (processing nodes). Current models have around 1/1000 of that number of parameters. The top data centers operate at around 1 exaflop (on par with the brain). Inference uses significantly less than that.

I’ll wait until the effective number of parameters in ML models is comparable to that of the brain. Then it will be clear whether the brain has something up its sleeve.