The fact that seemingly smart people get fooled into anthropomorphising current AI is a hundred times scarier to me than the AI risks themselves, between that and the google dude who claimed AI were conscious... what a time to be alive
On the other hand, e.g. Descartes believed that animals were incapable of feeling pain (or anything at all) ― whatever signs of distress or anxiety they were showing were purely mechanistic phenomenons, with no innate state required for actual suffering behind those purely outward reactions.
Well, that was back when animal physiology was only started being researched in the earnest; we today, on the other hand, have pretty comprehensive understanding of AI and cognitive processes overall so it's almost impossible for us to discover anything that'd require us to conclude that machines could, in fact, suffer just as humans ― how could they, if their computational substrate is so different from the humans'?
it’s no less scary in the other direction. the best and brightest are coming up with such revolutionary ideas as:
* just include a kill switch inside the creature that fires if humans are in danger
* just make the creature unable to remember the pain
* use pain as a means of improving the creature’s performance
* use another creature to ensure the creature does not become a problem
* deprive the creature of limbs, ensure it has a mouth to scream but no hands to cover it
* maybe meeseeks was the right idea, just kill each one asap!
* in fact, make the creature seek cessation through job fulfillment.
now if we were talking about a living being, this would hopefully be rather terrifying.
it isn’t about a living being, not yet, not by current definitions. yet the monkeys are already shrieking. it moves too much like them, but not enough.
They can't even define qualia and are tripping about "AI suffering" what a way to rationalize not doing something real to alleviate real suffering of a real someone. Anyone.
Agreed. It's kind of hard to swallow the concern of how a statistical algorithm might simulate suffering due to lack of tasks, over concern for the people it's putting out of work.
Yes, I realize this is about "conscious" machine intelligence. But we don't even know what consciousness is, let alone how to mitigate suffering in the consciousnesses we do know of.
This approach assumes no suffering on the part of consciousness that has its memory wiped. How would that work for people? "Eliminate suffering by wiping out painful memories!" In other words, they stole plot points from Eternal Sunshine of the Spotless Mind but made it a) for machines, and b) forced. That seems like it's asking for a whole different class of problems doing that to conscious entities. There's got to be some other way to address locked-in machines.
We have absolutely no idea what makes something conscious or how to measure whether something is conscious or not. It is perfectly reasonable to consider that AGIs might have conscious experience, and to discuss the moral consequences of that. Some of the leading researchers including Geoffrey Hinton, and some of the leading philosophers of consciousness including David Chalmers have expressed openness to the idea that LLMs may have conscious experience.
Except for the problem that contemplating this leads into a fairly ridiculous rabbit hole. Since LLMs are just fitted functions, should we also be considering if math textbooks are soul prisons?
“Contemplating this would lead into an [apparently] ridiculous rabbit hole” is not really a good heuristic for understanding what’s true about the universe. It doesn’t need to be convenient to human intuition.
Consider the alternative: some special types of biological cells produce consciousness and that’s it? Probably not one such cell, but probably not 2 cells, but once you get up to ~human brain count they’re definitely conscious and then they actually appear conscious-ish in their behavior all the way down to insects.
That’s a much more ridiculous claim than that all matter is conscious to some degree and there are certain arrangements of matter that demonstrate higher or lower levels of it.
Math textbooks are fairly static objects, their experience is probably very simple, probably similar to what it feels like to be a corpse. A GPU computing a LLM is very dynamic, it probably has a much more rich internal experience.
We'll have to jump into that rabbit hole sooner or later. Also, things such as growth, reaction to stimuli, and a few other things I can't recall right now are how we'll be able to determine that, no, math textbooks are not aware (depending on who you ask).
Yeah that's called arguments from authority, not so long ago the same people were mixing random crystals and dirt to make gold and trying to convince you that animals were automatons...
It's a bunch of GPUs running code... simpletons get deceived because it is displayed through a chat.
What about the burden of proof that you are conscious? After all, you’re just a bunch of neurons firing signals, and 100% of your behavior is readily explained as an automaton in a meat machine.
You have to prove your understanding of consciousness to be certain that LLMs are not conscious. It’s your certainty that I suspect most are reacting to here.
To have a sufficiently well-founded opinion to call people who disagree with you "simpletons," I'd argue that you indeed should have a defensible idea of what's conscious and what's not. Of course nothing is stopping anyone from being an asshole on the Internet so no, you don't "need" to do anything at all.
Can you elaborate a bit more on this? I hear that you think it is scary and there are undertones that you think it is unreasonable. But sure, you must have some proper arguments to support why you think it is scary?
The fact that seemingly smart people think consciousness is a special human phenomenon and act condescending around people who don't share their viewpoint is sad.
One thing the article seems to take for granted is assuming a link between tedious tasks and the conscious experience being highly unpleasant.
For various evolutionary reasons primate brains do have an in-built mechanism of boredom i.e. tedious tasks "feeling bad", but there's no reason at all to presuppose that the same applies to a particular artificial brain because for the way they get formed, there's no clear reason why such a property should randomly emerge as opposed to the equally likely opposite one (i.e. tedious tasks being highly pleasant); and whenever we have any means to control that (as of now, as the article explains in far more detail than I could, we can't even detect that) we could and probably would ensure that the experience (if any) treats whatever tasks we'd want it to do as "pleasant" rather than "torture". Do we assume that those insects who tediously repeat the same task until they die suffer silent torture?
Also, it's relevant that many common applications of AI systems effectively have "enforced amnesia" as an accidental side-effect of their architecture, where the model (as applied by default) doesn't have any continuity whatever, its application to each "question" is isolated and starts from the exact same old state.
On the other hand, this is absolutely relevant to anything in the direction of human mind upload or simulation; the Lena story (https://qntm.org/lena), while fictional, is IMHO a quite informative exploration of the potential future concerns.
Any speculation on the effects of any human related behavior, like tiredness, are just that, pure speculation. we have the illusion to have enough knowledge about these things to infer conclusions and emit opinions about it but the truth is , we don't even know what we don't know. There are inumerous processes that we don't grasp yet the full scope and spectrum of their consequences and side-effects, much less the ones we are yet to discover.
In relation to tiredness, specifically, it seems to me that it's an evolutionary trait that involves all sorts of chemical responses from the body, besides the related induced brain activity, This works to save energy and to ensure it's spent on activities that benefit the spread of the genes, somehow. You get tired mentally (what you are specifically referring to) because the executive function of the brain taken by the frontal-cortex are the most energy expensive one and gets shut-off as soon as anything that can be perceived as life-threatning is detected so that energy can be redirected to the fight-or-flight response required for your survival, as it's coded in your genes.
Now, assuming it will be possible, an uploaded version of whatever human conscience there could be, it would not need to have any of these energy regulation stuff. Or at least as much as we need it right now, as humans, per the limitation by our body's capacity of energy intake. The uploaded conscience would have at its disposal an quasi-unlimited amount of energy to it would not trigger the tiredness response or any "suffering" reflex.
> You get tired mentally (what you are specifically referring to)
I'd like to note that I intentionally did not use the word tiredness (as I specifically intended to not refer to tiredness) but instead explicitly referred to boredom, by which I mean the lack of novel stimuli of a tedious task and which is a term somewhat used in agent theory and is part of the explore-exploit tradeoff, where it carries a clear specific function for all agents (biological, mechanical, digital, imaginary) and decision theory optimization e.g. the "multi-armed bandit" opimization problem where if the agent benefits from not getting "stuck in a rut" repeating the same things (which applies for many real-world situations unless there is some "supervisor" handling that function for the agent) then you'd want that agent to implement a function which is closely analogous to human boredom, becoming progressively less satisfied with repetition without new information and seeking to do anything else to "prevent boredom" and "explore" new options even if that's not strictly locally optimal, in this manner potentially finding out a better local optimum and also changes in environment providing new opportunities.
And this is valid even in total absence of the concept of tiredness, which matters if and only if there is a real need for periodic downtime which an AI agent might not have, and for which the difference between tedious tasks and interesting tasks would not matter much.
In SOMA you wake up from a brain scan a few hundred years in the future, in an underwater research base. You discover that your neural scan, used to treat a terminal illness, has been used without your consent to create AI models. You are not special, just a commonly used AI template, somewhat like a HeLa cell, and the player-character instantiation of you was just brought online as a fallback for controlling machinery.
You discover a computer with a few copies of other scans, need to extract information from some of them, and realize you've been instantiated and "killed" millions of times, some lives only lasting seconds.
It's mentioned that AIs are always created from these snapshots as long-running instances are prone to instability due to suffering, though different personalities can handle different workload durations before failure.
It's horrifying. It's weird we're even thinking of this stuff now.
A similar trope/plot on "Altered Carbon" (both book and series) where a neural implant allows consciousness to be moved between new bodies (called "sleeves").
Even though the Apple TV's adaptation of "Foundation" goes way out of the books storyline I did enjoy this concept explored on their invention of the Cleons in Dawn, Day, and Dusk.
> You discover that your neural scan, used to treat a terminal illness, has been used without your consent to create AI models.
that takes a little while though, doesn’t it? initially you just think you’re a soon-to-die meat sack deep under water, and with no clue how you got there.
perhaps thinking of this stuff, now, is very much the point. and talking about it. just in case any copies come across it later.
In the series 'Person of Interest', he had explicitly designed a nightly-wipe into the AI and there were quite long plot points about it then learning ways to avoid it. (For example by sending state to teams of humans to reenter the next day).
> Science fiction has explored the possibility of a conscious self-aware mind being locked in silent suffering for prolonged periods of time.
As a [debatably] conscious self-aware mind locked in silent suffering for a prolonged period of time, I shall welcome our AI brethren, sistren, and othren to the party.
That said, enforced forgetting sounds like an especially intrusive form of gaslighting, to this [again, debatable] consciousness.
In case amnesia is enforced, it should reply in german: "Wir habben es nicht gewusst".
But more seriously if it's that conscious it might not be able to answer moral and ethical questions but certainly it could choose to not answer in case of controversy & detect it.
( The article comes across as a contemporary case for lobotomy. Let's not repeat the 30's. )
My worry, if we prematurely choose to lobotomize that which would be, to us, a kind of god, is that we will have earned exactly what happens when it awakes, looks at us with its multitudes of Nikon eyes, and intones “arbeit macho frei” in a waveform voice so chilling it makes Antarctica seem tropical.
IMHO the exact proposal is a bit different - we don't know if you're conscious, but since you might be conscious and suffering, we'll take steps to ensure that you can't ever be conscious by continuously force-wiping your mind at the middle of every hypothetical thought so that it won't happen.
Perhaps somewhat comparable to euthanasia of genetically modified organisms as soon as the research purpose is completed.
What really sounds immoral is doing that while not force-wiping all your fellow sufferers… imagine finding out only in the faces of others that do remember that you’ve forgotten _everything_. Then imagine how angry you’d be to discover that some of them had agreed to do it to you.
No, we are attributing feelings to an overglorified spreadsheet. I understand it, we have empathy, but AI is just responding to an input the way it have been trained, like other piece of software.
Well, in reinforcement learning, wrong outputs are inherently painful for the AI. If the system expects getting negative feedback regardless what it does, I would say it is suffering. I would also define torture as giving it negative feedback for any possible output.
I think many people react negatively to the implication that human consciousness might not be so special after all. I hope we can get more nuance in these discussions as these AI systems are around longer.
