This is a video of a talk from Gerald Sussman in 2011 at the Strange Loop conference. Abstract below.
Though we have been building and programming computing machines for about 60 years and have learned a great deal about composition and abstraction, we have just begun to scratch the surface.
A mammalian neuron takes about ten milliseconds to respond to a stimulus. A driver can respond to a visual stimulus in a few hundred milliseconds, and decide an action, such as making a turn. So the computational depth of this behavior is only a few tens of steps. We don't know how to make such a machine, and we wouldn't know how to program it.
The human genome -- the information required to build a human from a single, undifferentiated eukariotic cell -- is about 1GB. The instructions to build a mammal are written in very dense code, and the program is extremely flexible. Only small patches to the human genome are required to build a cow or a dog rather than a human. Bigger patches result in a frog or a snake. We don't have any idea how to make a description of such a complex machine that is both dense and flexible.
New design principles and new linguistic support are needed. I will address this issue and show some ideas that can perhaps get us to the next phase of engineering design.
Gerald Sussman
Massachusetts Institute of Technology
And yet the computer is so much better in many areas than a human.
It so weird to compare the two. Why would you even want that? We have already plenty of brains on earth. Who sais, that once a computer works like a brain.. it doesn't also come with it's downsides?
There are lot's of problems in compute.. making it more brain like, more linguistic isn't going to solve that.
I believe the whole notion of "a computer as an assistant", is doing more damage than good.
My takeaway is that Sussman is looking at his field after a lifetime of contribution. He's looking for signposts on where we can go next. He's not valuing one over the other, but looking at the brain and saying, "Here's something that exists that's operating on problems in a way we can't begin to approach using current software/hardware techniques". He sees some hints available to us in the noise reduction things he talks about at the end. In that way, the talk is more him challenging the audience to question our approaches and implied constraints rather than him trying to present us with an answer. It's one of my favorite videos (the way too loud HI at the beginning, notwithstanding).
Well an assistant is traditionally the one who eventually replaces the master, so that notion is not conceptually wrong.
Do people still think there will be any humans left alive in a few hundred years? We've gone as far as we could as biological constructs, switching to a more adaptable architecture is just the next step forward. Our bodies are overfitted for living on this one specific planet which is greatly limiting the real goal of life: to expand to new areas and replicate.
Ok, I'll spell out what I meant. One of the possible scenarios where, centuries after GAI was created, there would still be humans (recognizable to us as such) alive, is the same as the reason why horses and pigs and cows are alive today - because they're bred as inputs to some industrial process.
Well, in human history there have been many people bred as inputs to an industrial process. Today most people of conscience look on those situations (past and present) in horror. I would hope that humans someday learn to never allow that to happen to any of us again.
Dude, already happening, it never stopped, it isn't 'horror' it's 'efficiency'. There is no such thing as 'people of conscience'. It will always happen, because, human nature.
Alright there will always be some that stubbornly cling on, but I would bet that most entities that could be considered intelligent beings won't be human by that point, just out of practicality.
Stupid flesh and blood humans with their finite needs! Let's build cyborgs with infinite information processing capability so their needs can never be saturated!
F1 drivers can instinctively respond to a minor bump on a curve and make a decision in probably ~100 milliseconds, so the lower bound is just 10 computational steps.
A more complex decision such as swerving to avoid a wheel that just came off the car in front, probably takes around ~500 milliseconds, or 50 computational steps.
Considering that modern computers need billions of computational steps to start a calculator app and draw it on the desktop, we have a long way to go.
Though there has been regression on this point, a 1990 NeXTStation Color could do the same with ~50 million cycles on a 1120×832 display. (on 12 MB of RAM and 1.5 MB of VRAM.)
I think the trick is, our computational steps are massively parallel. Take the visual system. By the time you notice - consciously - the pothole you're about to drive into, it's already processed into high-level concepts.
Our visual senses aren't camera feeds - the image we are aware of isn't raw input from the eyes, but rather it's generated out of some combination of raw inputs, low-level hacks to hide limited FOV and discontinuities caused by saccades, and higher-level interpretations based on how we feel, what seems to fit the context based on our prior experiences, etc.
I think it makes sense to assume that this process has many extra off-ramps that can trigger high-level reactions while bypassing conscious awareness. I.e. imagine that half-way through the post-processing stack, the relevant parts of the brain are somewhat convinced you're looking at a fast-moving object about to hit you in the face. There's a point at which it makes sense for the brain to make your body start a dodging move, even if it may be a misprediction, rather than waste a few dozen milliseconds to make sure, at the risk of getting a ball (or a rock, or a fist) up your nose.
This is an extremely important point. "10 computational steps" is obviously underestimated. Each neuron can communicate with 10s to 100s of additional neurons, so if you're talking about 10 ms per step 100ms being 10 steps you are missing the fact that each step can be an exponential increase in the amount of computation involved. This is one of the main reasons I believe we are nowhere near human or even mouse level AGI. Many of the simple neuron count metrics do not include the synapse counts or the incredible amount of parallel information that is processed. We have network interfaces with 100-400 Gb throughput whereas the communications between synapses represents exabits of throughput between computational units. We are very far from AGI just on a fundamental processing scale. Not to mention we know very little about the algorithms involved.
I would almost agree with you on the AGI angle, but my beliefs are still recovering from the shock of how we kind of randomly stumbled into solving multiple classes of seemingly intractable cognitive problems, just by feeding a program half the Internet and letting it position pieces of words in a hundred thousand dimensional vector space.
Human brain has to do a lot of work processing and integrating all the senses we use, which could cover at least for part of the neural capacity you mention, but I feel that where thinking is concerned, we may have accidentally solved the core problems. I somewhat expect that we'll look back and see that current LLMs were to human minds what the ol' Model T is to Tesla Model S - vastly different in levels of advancement, but also fundamentally the same thing.
> the shock of how we kind of randomly stumbled into solving multiple classes of seemingly intractable cognitive problems...
Another excellent point. The idea that we may stumble onto massive computational leaps based on imprecise data processing requirements is terrifying. I do think we are nowhere near where we will need to be with respect to computation and, more important, information throughput. But the fact that we "oops"-ed into such a strong statistical model of conversation is deeply unsettling. I do believe GPT based language models will be part of an AGI.
Part of the reason I think LLMs are a step in the right direction is that it's too big of a coincidence. We don't know how our own brains work, but we have good reasons to believe that, whatever the "magic" trick is, it can't be too complex, because evolution had to "oops" into it too.
Now, it would be one hell of a coincidence if the model we "oops-ed" into with LLMs, and the model evolution "oops-ed" into, were completely unrelated approaches to the same problem.
If there are a billion different ways to solve this problem, it would seem much more of a coincidence if with LLMs we've "oops-ed" into an approach just like the one inside our heads. It would seem more likely that they're unrelated approaches.
Right. However, I don't see the reason to believe there is a billion ways to solve the fundamentals. And even if, they're not all equal here: we're considering solutions that are very simple, provide huge leverage, and are continuous.
For our own brains, this is because evolution is the OG incremental learner - it just doesn't do leaps of faith, nor does it invest resources. Incremental evolutionary advancements must quickly stack into something improving survival rates, or else they get selected out of the gene pool. This strongly suggests that brains (human and animal alike) are based on a design that scaled easily and continuously - one that started simple and could be improved step by step, with each step being simple and conferring some non-zero survival advantage.
For our AI work, LLMs are exactly this: they derive from simple models that were iterated on over the past few decades, in small steps. They are still structurally simple as programs - but they've hit the right structure that allowed them to make qualitative jumps in capabilities from simple scaling - more parameters, more memory, more compute, more training time, more input data. This may be just my own perception, but I feel that LLMs are exactly the kind of model that an evolutionary process would be able to develop incrementally.
With the above in mind, plus the fact that LLMs are trained on the output of our brains (and their output is rated by our brains), makes me believe it's highly likely that, with language models, we've stumbled on the very same fundamental method of implementing thinking that is core to how our own brains work.
> I think the trick is, our computational steps are massively parallel.
Yes, one trick is that. Our computers have to serialize everything. We don't know how to build or program massively parallel computers with distributed memory. But the GP already touched on that point.
That feels like an odd example. Going on a google for speeds, that means they are responding to a bump after they have traveled 30 feet past it. Impressive, to be sure, but I'm not clear what the learning is there. More engineering went into the vehicles so that, by design, many other things happened before the driver is aware of it.
Seems like fractals are the difference maker here because if you don’t use a fractal you have to program the whole thing whereas if you use fractals (embryology) then your code can be massively recursive. Alternative splicing and epigenetics (holy shit, I can’t believe the iPhone autocorrect doesn’t know the word “epigenetics” in ‘23) also multiply the flexibility of the genome. Also, phosphorylation: proteins are not Boolean switches, they have arbitrary numbers of activation states. Ternary logic is already way more dense than Boolean so when you go even further like into 6-ary, 9-ary logics you’re a lot more free to program stuff. Also the network motifs are a lot simpler than first appear. The metabolic map looks like spaghetti on the wall but each point is composed of reusable blocks like feedforward and feedback etc (see Uri Alon’s book and “Wetware” book and Synthetic Biology: A primer for more detail)
I’m biased as a molecular biologist but genetics and systems biology are the most incredible thing. Natural nanotech
When I was a kid in the 80's there were a number of gross/gore/horror trading cards for kids - Garbage Pail Kids being the prime example. One of the sets I had was set in some far flung future, I can barely remember the series but part of the theme were living, biological machines. The only card I can clearly remember featured a flying prisoner transport where the prisoners were seated in the ships cavernous stomach and "if upset during transport could accidentally digest the prisoners." (Honorable mention to HL2 featuring "synthtech" - the drop ship, strider and gun ships were living biological machines.)
This stuck in my head as the idea of designing machines which grow like plants or raised like animals was an incredibly amazing and interesting concept. But as I got older and educated the idea them seemed as far flung as the cards themselves. Though reading you post makes me think, maybe they aren't so far flung after all. Though I still fell that a creature which grows steel or titanium skeletons is still quite firmly in the realm of sci-fi.
> So the computational depth of this behavior is only a few tens of steps.
One of the primary "criticisms" of pre-LLM machine learning was that it is inefficient compared to, say, a human, because it requires a plethora of examples to classify something just to match the performance of a human who has only seen a few examples.
That argument no longer holds the same value since the discovery of few-shot learning features of LLMs (though they still have a long way to go). Which makes me think: maybe the reason behind human brain's apparent "efficiency" is the complexity that hides beneath the billions of neurons we have.
Disclaimer: I am an amateur and probably have only a litte idea of what I'm talking about.
> A driver can respond to a visual stimulus in a few hundred milliseconds, and decide an action, such as making a turn. So the computational depth of this behavior is only a few tens of steps. We don't know how to make such a machine, and we wouldn't know how to program it.
IMHO, that makes no sense. Assuming we have a really large state machine with all many possible situations in a state of driving on a straight highway with a few vehicles with more or less constant relative velocity, determining if we need to make a turn is definitely doable in a few computation steps.
We attempted the symbolic route of AI programming and it didn’t work because it is explosive in computation time and exhaustive to model in code.
We can’t program our AI “models” now because they are based on chains of probabilities. Therefore they can run in real time. They don’t have any real logic that we specifically programmed, only what the model was able to infer and then encode.
Perhaps there should be a marriage between these two attempts of AI.
Hum... The numerical route for AI necessarily has the same complexity in computation time. Those two have equivalent semantics. If you can get a result with one, you can get the same result with the other, it can just differ on how easy it is to program.
By the way, symbolic AI can learn too, you don't need to model things in code.
I was talking about attempts such as expert systems. We attempted to symbolically encode things as sums of their parts in plain terms, e.g. a bicycle is two wheels, an axle, a chain, some peddles, etc. but in more detail than that. The problem is the level of detail required for general knowledge: firstly it is hard to codify, secondly as it grows computation becomes exponentially more expensive.
We as humans have the intelligence to simply see for fractions of a second and then just know — we can know the answers to math problems and how a person is feeling. The symbolic AI programming was an attempt to codify what symbols we are seeing but they are so diverse and nuanced it became apparent this was an ineffective scientific route.
Yeah, that's because you decided it should be a hand-coded expert system. If you review the set of hand-coded neural networks, you will discover they even more bloated, and have an extreme low ceiling on usefulness.
Besides, keep in mind that the currently useful NNs are stuff that requires dozens of GB of memory to run.
Anyway, training a NN is also an attempt to codify what symbols we see on the world.
Not really a fair comparison--250 ms is reaction time. A deliberate conscious response would be much longer. "Reaction times can reflect habits rather than computations"[0].
I don't get how this is impressive or relevant on its own. Sure, it sounds small in face of the final result : a full-grown human. But this 1GB program is born out of and expressed into the full OS that is the physical world.
Though we have been building and programming computing machines for about 60 years and have learned a great deal about composition and abstraction, we have just begun to scratch the surface.
A mammalian neuron takes about ten milliseconds to respond to a stimulus. A driver can respond to a visual stimulus in a few hundred milliseconds, and decide an action, such as making a turn. So the computational depth of this behavior is only a few tens of steps. We don't know how to make such a machine, and we wouldn't know how to program it.
The human genome -- the information required to build a human from a single, undifferentiated eukariotic cell -- is about 1GB. The instructions to build a mammal are written in very dense code, and the program is extremely flexible. Only small patches to the human genome are required to build a cow or a dog rather than a human. Bigger patches result in a frog or a snake. We don't have any idea how to make a description of such a complex machine that is both dense and flexible.
New design principles and new linguistic support are needed. I will address this issue and show some ideas that can perhaps get us to the next phase of engineering design.
Gerald Sussman Massachusetts Institute of Technology