That quoted sentence argues that we do not have AGI now. I have no counterargument against that. We do not have AGI now. That sentence on the other hand fails to argue it is impossible to develop AGI.
Someone before the invention of the aeroplane could have said:
Our only known good model for flying is birds and insects. It took billions of years of the universe churning at random to accidentally generate birds and insects. We have no clue how to replicate that scale.
And yet we know that it's not impossible to create flying machines.
Flight and consciousness are not in any way compareable concepts. We literally cannot perceive the constraints of the latter's underlying physical system.
Sometimes I feel computer scientists choose to misunderstand physicists because it would make them feel stupid if they did understand.
>Flight and consciousness are not in any way compareable concepts.
For pete's sake, it's an analogy, not a direct comparison, and it is perfectly valid as such when interpreted with due charity.
You can say "Brains are complicated. They took time and evolution. That sure is hard. See how hard it is?" The same can be said of flight at a certain level of abstraction as a valid analogy, which can be charitably interpreted as such without the need for claiming anyone is purposely choosing to misunderstand physics.
The fact that the examples of brains and of flight given to us by nature sure seem complicated doesn't establish as a matter of principle that their salient properties can't be modeled in machines, and that's the real thing that's at stake. Disputing that requires a different kind of argument than saying "gosh it sure is complicated", and that's what the analogy is pointing out.
This is interesting. Is aeroplane flight akin to bird or insect flight? Rolling down the tarmac, peering out the window, the planes look more like elongated fish bodies than soft bird bodies, or compact insect bodies. Our planes rather swim in the air than fly in it, I think.
Our flight is some other kind of thing (whatever we uncovered the model-able, salient properties of flight to be). Computer consciousness might similarly be some other kind of thing. And that’d be ok.
But can they be equated? Only at some abstraction level. A plane is obviously not a bird or an insect or a fish. Aeroplane flight is not bird or insect flight either, nor is it swimming. But it is safe travel through the air, from one earth-bound destination to another.
Someone before the invention of the aeroplane could have said: Our only known good model for flying is birds and insects. It took billions of years of the universe churning at random to accidentally generate birds and insects. We have no clue how to replicate that scale.
And yet we know that it's not impossible to create flying machines.