I've given this a skim, and it seems to be very much "of its time" (the notes were written in 2006, but only published in this paper in 2021). It basically takes as an axiom that in order to be conscious you need to be interacting with the physical environment.
But the thinking has moved on since then, and we can view the internet as an alternative "environment" in which a conscious entity could exist.
Instead of existing primarily in the physical world and using clumsy tools to access the internet, conscious artifacts will exist primarily in the internet and will use clumsy tools to access the physical world.
I think there is something to the “interaction with an environment” paradigm for consciousness. It’s not clear to me that interaction with a much, much more parameter constrained environment such as textual data will lead to consciousness in a way that we can recognise it.
I posit that robotic embodiments of AI will lead to something we can recognise as “consciousness” in a more tractable way than other environments that are wholly alien to human experience.
Nonphysical environments may also pose a risk for the “stealth” development of consciousness since we may not easily recognise the presence of a consciousness developed in an alien environment. We can intuit that a dog or cat is “conscious” easily, but could we detect that consciousness if we were limited to only linguistic communications with those subjects?
So physicality, I think, is probably not a prerequisite of consciousness… but it may be a safer route, and also may result in more human-interpretable versions of consciousness.
>I posit that robotic embodiments of AI will lead to something we can recognise as “consciousness” in a more tractable way than other environments that are wholly alien to human experience.
But sitting with your eyes closed and just thinking about something is also familiar to human experience. Of course you still have a body and some extent of awareness of it all the same, but it is conceivable that that's not essential to the experience of being conscious of solving a sudoku in your head (or whatever).
Maybe it's all a moot point though, since no matter what any artifact must somehow be embedded in the physical world for us to say that it exists at all..
[1] Any metaphysical framework really, including science as it is practiced/experienced, as opposed to the technical scriptures, guidelines, and intentions. For example, see how the Rationalist community has framed the phenomenon:
One other difference for a person sitting and thinking with their eyes closed is that they have not been doing this all their lives, and have the memories and other mental abilities that have come with interacting physically with the world. Even so (or perhaps because of it?) most people do not cope well with extended sensory deprivation.
The experience of someone who has had only very limited sensory experience of the world might tell us more about this, and in that regard, there was an article not so long ago here on HN about Helen Keller’s memories of what her life seemed like before Ann Sullivan began attempting to communicate with her. My recollection is that she reported having little sense of herself in the world until that time.
> I think there is something to the “interaction with an environment” paradigm for consciousness. It’s not clear to me that interaction with a much, much more parameter constrained environment such as textual data will lead to consciousness in a way that we can recognise it.
PS Incidentally, that's why I find approaches such as the article's - or, more recently Joint Embedding Predictive Architecture (JEPA) - to be interesting. At some point we'll need architectural progress if we really want to move forward.-
This, for example, makes for a very interesting read, and might be on to something:
Incorrect. If I say “ think of a famous actor”. (Please do it benefit reading on..) l, aha you actually look back at the process and study it, you’re not actually choosing what you choose. It’s a mere illusion. The famous actor has in fact just come into your mid on its own. There was really no choice.
The fallacy of composition is an informal fallacy that arises when one infers that something is true of the whole (all forms and manifestations of thinking) from the fact that it is true of some part (your single example) of the whole.
Also: the truth value of your theory is subject to your theory itself.
I knew somebody would bring free will into this :) I should've said "you decide to think a thought (for some definition of "decide" according to your free will stance) ...". I mostly agree with you, but I decided to compartmentalize it out for the sake of this particular discussion, as I don't think it matters - a machine with these properties would be as conscious as a human.
A corollary to this is that you didn't arrive at this conclusion as a result of analyzing the possible perspectives and choosing the best one. Similar to what you said, you didn’t choose to believe this, the veracity of this conclusion just came into your mind on its own. So why should we, or you for that matter, take this claim seriously?
This isn’t what everybody else means when they use the term “embodiment”. It refers to sensing and actuating in the world (or a simulation!) in realtime.
It’s explicitly referring to the opposite of a “brain in a jar” scenario.
It is explicitly compatible with the formal definition of embodiment, which is roughly: select an action from the action set, observe results of the action taken, update internal state, in a loop.
In the same way that a sandboxed HTML page is "not executing machine code on the CPU".
Of course that's how everything works if you drill down deep enough, but if your direct connection is to the internet, rather than to your arms and legs, I think it's fair to say that you exist on the internet rather than in the physical world.
I think the term is "reductionist". It is reductionist to say that because the internet exists in the physical world, existing on the internet is no different from existing in the physical world.
There is no "the Internet". There are computers, and wires, and silicon and electrons and photons, and there are cells and nerves and brains, but there is no "the Internet" in the real world.
> your direct connection is to the internet
You've still got a brain? A brain in a jar wired to a computer network is still in the physical world.
You have to figure out your metaphysics before you can talk about this sort of thing. What are you?
(I would say that you are subjective awareness itself, but that's useless from the POV (no joke intended) of someone trying to build a machine that can be a conscious being, or trying to transcend physical reality.)
Of course software runs in the real world from a substrate level.
But from a systems perspective it just runs on a CPU, because it operates by an assembly language that the CPU provides and you don’t need to know if the CPU is transistors, vacuum tubes, optical switches to understand the software operations.
Likewise, you don’t need to know where the CPU is. What kind of system is producing its energy, etc.
So there is a different level of operation between the physical substrate (or many alternative substrates) and the software.
This is very obvious in computer games. The game provides an environment that is independent of what kind of computer you are running it on (given enough speed to be indistinguishable).
Likewise, the Internet has rules that are implemented with physical things, it independent of the exact physical things. Who made your router? Most of the time the answer is “who cares?” as long as it correctly follows the networking rules of a router.
The essence of the question is the definition of "you" in your sentences.
Who or what is "you"?
(In fact that question is the short wide path to enlightenment, if "you" are into that sort of thing. Cf. https://en.wikipedia.org/wiki/Self-enquiry It's metaphysical dynamite.)
If you don't nail down that fundamental ground (who or what are you?) then there's no point talking about it.
What is the "you" that has "a systems perspective"? What is the "you" that is going to live in the imaginary computer universe you are postulating?
Because it sounds like you think you are imaginary.
> Likewise, you don’t need to know where the CPU is.
But it still is in space? It still exists? It's not imaginary?
> The game provides an environment that is independent of what kind of computer you are running it on (given enough speed to be indistinguishable).
The computer still has to have a monitor? A controller with switches and buttons?
> the Internet [is] independent of the exact physical things.
No my friend it is exactly those physical things and nothing else. There is no "the Internet" there is only physical things and subjective awareness. As far as we know "subjective awareness" whatever it is only occurs in brains, physical brains.
You are saying that "subjective awareness" can exist (whatever "exist" means) without any hardware at all. I don't think you (or anyone else) has any evidence that that's true.
> No my friend it is exactly those physical things and nothing else. There is no "the Internet" there is only physical things and subjective awareness.
So if an AI model is controlling a character that walks around and interacts with human-controlled characters on a Minecraft server, according to you none of those exist, it is all just wires and electric currents? Then by the same logic, "you" also don't exist, it is all just neurons activating and making your muscles move.
If I write "I am alive" on a piece of paper does that make it a living being?
If I carve Aleph on the brow of the Golum does it become a man?
How big does the pile of switches have to be before it changes from "what" to "who"?
Look the point I'm making is that before one can speak meaningfully of uploading consciousness into the Internet one has to specify one's metaphysics.
I have my own beliefs, and I'd love to yammer on about them if asked (fair warning!) but that's beside the point. Without figuring out what "you" is it's meaningless to talk about "you" living in "the Internet".
Seems that nobody will ever satisfy your metaphysical expectations.
I am content with the old-fashioned Turing test: if my interlocutor can communicate reasonably intelligently, I am happy to treat him/her/it as an intelligent being.
> Wow, it seems I was not able to satisfy your sky high metaphysical expectations either.
You didn't even try. Your argument, if it can be called that, seems to amount to "what if metaphysics doesn't matter?" which is lazy and pointless. Essentially you're just saying "nah uh" without actually presenting any sort of evidence or deep thought.
> You are saying that "subjective awareness" can exist (whatever "exist" means) without any hardware at all. I don't think you (or anyone else) has any evidence that that's true.
A software program running exists. Its rules exist. It requires hardware to run on, but the details of the hardware, beyond accepting the instruction set the software uses, are irrelevant.
Checkers is a game with rules that are independent of the materials that make up its playing peices.
Minds as systems run, exist, on physical things. But they can be independent of any substrate in that any substrate that implements the rules they need to operate will support a mind. That exists.
You can't put "a software program" into a wheelbarrow. You can print out code and put that in a wheelbarrow, or write it to a disc or USB key or whatever, and each of those things exist, but they are not the program. Patterns in real things exist. The pattern of the bits is real (even in the brain) but the "a software program" is imaginary: it "exists" only in your mind.
Bucky Fuller had a great demo: he would take a length of rope that was made of three smaller lengths of rope spliced together, jute, cotton, and silk. He would tie a loose granny knot in the jute end, hook his finger into the loop, and draw the knot along the length through the jute, into the cotton, then into the silk and then off the end. As with all great demos there is the inner and the outer meaning.
The outer meaning should be obvious: self-referential pattern (the knot) has or can have it's own stable existence in a proper medium (the rope). Both exist, the knot and the rope, but in very different ways.
Are you familiar with Meta-II? The same software program has been translated to many different languages, each translation exists (can be printed out) but none of them are the Meta-II software program. That is just a concept in a human mind.
> the details of the hardware ... are irrelevant.
But it still has to exist? The one detail that matters is that the hardware exists.
> Checkers is a game with rules that are independent of the materials that make up its playing peices.
Checkers doesn't exist. It's just an idea. If all the people who know how to play checkers were to somehow vanish, so would checkers, even if the boards and pieces and rule books remained.
(While I'm at it, I'd like to point out that Tuesdays don't exist either. Just as an aside.)
> Minds as systems run, exist, on physical things.
If those physical things are brains, yes, as far as we know. At least it sure seems that way.
> But they can be independent of any substrate in that any substrate that implements the rules they need to operate will support a mind.
How do you know?
That is an interesting speculation but one that has no evidence to support it.
Even if it's so, and we can build machines intricate enough to carry the knot from our meat into silicon, there is no reason to expect that our subjective awareness would go with it. Mind is not self. We can build models of our minds that act like us, just as Babbage had the Silver Lady, an automaton that danced like a ballerina, but they will not be alive anymore than she.
To me it sounds like you are saying that you believe that knot is mind and can be drawn into non-biological media, I'm saying that "you" are not mind, so even if this is possible and you can achieve it, you will only have succeeded in making a sculpture of yourself which can have no more life or being than stone.
sec. 3.5: "Edelman thought that action selection and the sequence of actions was a first step towards language. He was a strong proponent of the idea that language was rooted in action and motor control."
I thought it was understood that vocal language grew out of sign language used by hominid hunting groups?
It seems to me that "action selection and the sequence of actions" reaches it's zenith in pre-linguistic animals in hunting. It would be the most complex sequence of actions any animal attempts? (Building a nest, say, is complicated but static: the nest materials are not actively attempting to avoid being made into nests. Mating behaviors are instinctive and stereotypical. Only hunting is open ended?)
Mating choices and acceptance are both a negotiation and a competition.
So mating strategies are open ended in complexity as different mating roles each adapt to adaptions in the other role, and in competitors with the same role.
The plethora of absolutely odd mating practices in the natural world demonstrates that. Almost any aspect of an animal and its behaviors can be co-opted into being a mating factor, at which point selection takes that characteristic to crazy places completely unrelated to its original form and function.
All those things are answered well enough by evolved programs ("instinct"), it's
only hunting (and later war) where you get, yeah, adversarial training.
Hunting in groups requires intelligence and communication to coordinate actions over time and space. With humans the idea is that we started with hand gestures and then added vocalizations. (I have no idea where I got that, nor how mainstream it is with anthropologists or anything. I hope I'm not just repeating "just so" stories...)
Very. Very interesting. The relationship between language and/or physical activities in a physical environment (ergo through some sort of physical embodiment) is interesting. As some would have it, essential.-
But the thinking has moved on since then, and we can view the internet as an alternative "environment" in which a conscious entity could exist.
Instead of existing primarily in the physical world and using clumsy tools to access the internet, conscious artifacts will exist primarily in the internet and will use clumsy tools to access the physical world.