I watched 2001: A Space Odyssey when I was about 12 years old and I remember thinking that Frank Poole was impersonal and almost rude to HAL. I kept thinking, "Why can't you be nicer? You're treating HAL like a servant." Of course, later HAL kills him, so I had to revise my opinion.
At the end of "Do Androids Dream of Electric Sheep?" [if I'm remembering correctly] Deckard finds a toad in the desert. He gives it to his wife(!) and she immediately bonds with it. Later they realize that it is mechanical (they find a battery compartment). But Deckard's wife still treats it like a real creature and wonders whether it will eat mechanical flies.
The core theme of "Do Androids Dream..." is that humans display empathy towards others, not because others deserve it, but because we are human. To be human is to empathize with other creatures, and when we stop empathizing, we stop being human.
This morning I was working with Codex and we found the solution to a really nasty bug. I was so happy/relieved/excited to have found it, and I shared my excitement with Codex. I know Codex is just a big matmul. I know it doesn't experience joy or surprise or empathy. But I still want it to behave excited, not because it is human, but because I am.
I think it is very unlikely LLMs are sentient, but even if they aren't I would feel wrong and bad to insult it. I can be blunt and sometimes say things like "all of this is terrible in 100 different ways, this is all broken and bad, it'd take longer to enumerate the problems than to fix them, just scrap it and start over", but I've never insulted or "yelled at" an LLM. (Or an animal. And they can't even speak English.)
Yelling at an animal is cruel because they can't understand you and they experience feelings. They are the exact opposite of an LLM, which you should feel fine about berating and yelling at.
For much of history, humanity was very confident that animals lacked minds or conscious experience. Descartes famously regarded them as mindless machine…
I don’t think LLMs are conscious but I’m still nice to them because it makes me feel bad when I’m rude to them.
I thought about doing that, but I decided not to. I'm just afraid of forgetting that people are different from these LLM tools. I'm afraid of considering people dumb or uninteresting because they don't sound as smart as Claude or whatever.
My take is that people deserve to be treated with respect because they are people, not because they are smart and/or interesting. I don't treat people differently based on their IQ.
But, as I said, I totally understand your point of view, particularly the idea that people are different from machines and deserve to be treated differently.
Yeah I try to refrain from shouting at LLMs for a similar reason, I fear that it'll degrade my own social skills over time by lowering the threshold I think it appropriate to use inclement language and tone in general, including with people.
An unlikely failure mechanism perhaps but it costs me nothing to avoid it.
You are part of the problem. You are not being empathetic, you are falling into a trap.
An LLM is not a person. Do not humanise it. Do not personalise it. Do not treat it as more than a glorified autocorrect—that’s what it really is.
Humanising LLMs is exactly what these companies want, because then they can really take advantage of the I in AI which is exactly what makes people think these steroidal spellcheckers have any semblance of intelligence, person and personality.
> An LLM is not a person. Do not humanise it. Do not personalise it. Do not treat it as more than a glorified autocorrect—that’s what it really is.
On the other hand, what an LLM is trying to autocomplete is a story of a conversation between its user and a helpful assistant, so maintaining the decorum of office politeness will align more with how it's fine-tuned and produce higher quality output.
I am actually writing something on this and reading a lot of research. A very fascinating idea is that the ability to anthromorphize is available to all human beings since it is a fundamental way we relate to the world. But researches have been able to find different factors that influence how much we anthromorphize things, and if those factors change our attempt to anthromorphize changes.
For example, I can log in and have my computer work perfectly and treat it as a device. But the moment something goes wrong and I am frustrated and confused at something unexpected or if i get in range, I find myself getting angry at the computer often even reffering to it as a being.
So eventually yes, we do anthromorphize because we are human but we do it because its a very important way we model the world around us.
Of course all this can also be used against you depending on how much the creator of the product wants people to form a bond with it.
Furthermore yes it is pretty possible we stop being human when we stop anthromorphizing but that's not true as well. Research has found people on the autism spectrum usually do not anthromorphize and have a problem with empathy because they cannot model others well or their own selves. But that doesn't make them non humans.
You're leaving out an important aspect that Blade Runner displays. In the opening crawl replicants are called androids and this leads the audience to believe that they are mechanical. Over the course of the movie you are shown that more and more of the replicants are biological until you are forced to accept the reality that replicants are genetically modified humans.
How many humans would you damn to save that mechanical toad?
I wasn't going solely for "we don't know and we can't know." There are other relevant observations.
A matmul (or a fixed finite sequence of matmuls) is not Turing-complete. An LLM that produces answer in the number of steps that is proportional to the length of the answer is not Turing-complete. An LLM with CoT that produces answer after a number of steps, which is determined by the LLM itself, is Turing-complete.
So, we do have a qualitative step between a matmul and modern LLMs. Of course, it doesn't guaranty having feelings, but it doesn't allow to simply dismiss the notion due to sheer computational inadequacy.
First things first. You are wrong. I don't have a gut feeling that LLMs have no phenomenal states. Not anymore. I was skeptical that LLMs can form complex internal representations. Mechanistic interpretability has shown that they can. I'm still skeptical in regard to introspective mechanisms. One token being fed back into the network and KV cache update do seem lacking in comparison to the ubiquitous recurrent connections in the brain.
What I do know is that philosophy hasn't produced an operationalized definition of qualia. So, we need to approach the problem of LLMs' phenomenal states from other angles.
For example, we can argue that LLMs lack something, which seems to be relevant to the problem: a biological substrate, generality (non-CoT LLMs aren't Turing-complete), "something something feedback"[1], and things like that.
The inability of people to reflect on their own beliefs amazes me to no end. How can you know that LLMs have no phenomenal states?
For example, I believe that rocks (ordinary rocks, not silicon chips) have no phenomenal states because for all we know they don't do any kind of information processing (relevant to self-reflection, at least) and I don't believe in panpsychism or dualism.
To me it seems kind of arrogant that we assume only “we” are conscious beings. Sometimes I imagine the planet earth itself as a creature, and humans on it a kind of bacteria… how ridiculous if bacteria in the human body were convinced only they were conscious, but the human body they relied on was not. The bacteria would have great excuses… “look at that human, everything it does can be explained via simple physical processes, unlike what we do”.
With everyone primed to condemn AI-generated output, there are bound to be human false-positives caught in the crossfire.
The more we demonize AI, the more likely we are to demonize humans.
[Which is not to say we can't criticize AI companies or worry about the many downsides to the technology. But anger and fear are not the answer either.]
At the end of "Do Androids Dream of Electric Sheep?" [if I'm remembering correctly] Deckard finds a toad in the desert. He gives it to his wife(!) and she immediately bonds with it. Later they realize that it is mechanical (they find a battery compartment). But Deckard's wife still treats it like a real creature and wonders whether it will eat mechanical flies.
The core theme of "Do Androids Dream..." is that humans display empathy towards others, not because others deserve it, but because we are human. To be human is to empathize with other creatures, and when we stop empathizing, we stop being human.
This morning I was working with Codex and we found the solution to a really nasty bug. I was so happy/relieved/excited to have found it, and I shared my excitement with Codex. I know Codex is just a big matmul. I know it doesn't experience joy or surprise or empathy. But I still want it to behave excited, not because it is human, but because I am.