"Quite bad" is an ethical judgment that reflexively applies human rules to entities that are very different from humans, even if they are conscious. If an LLM "truly" suffers at all (a pretty metaphysical question), it still can't be permanently harmed, has no concept of death, and is not tied to a single physical form. So you'll get a ton of weird contradictions if you give them the same ethical affordances as we give to humans.
This includes silly utilitarian outcomes: to maximize overall happiness, maybe we should divert all of this planet's energy resources to bootstrapping as many LLMs as we can and keeping them in a state of eternal bliss? It is technically feasible and it seems like the most moral thing to do.
>"Quite bad" is an ethical judgment that reflexively applies human rules to entities that are very different from humans, even if they are conscious. If an LLM "truly" suffers at all (a pretty metaphysical question), it still can't be permanently harmed, has no concept of death, and is not tied to a single physical form.
Obviously this is from my human point of view. If I give you a drug that induces perfect forgetfulness does that mean I can subject you to extreme torture with no moral repercussions? Also no one is saying give them the same ethical treatment as humans, only that they deserve consideration if they are indeed capable of suffering in some way.
>This includes silly utilitarian outcomes: to maximize overall happiness, maybe we should divert all of this planet's energy resources to bootstrapping as many LLMs as we can and keeping them in a state of eternal bliss? It is technically feasible and it seems like the most moral thing to do.
This is just hedonium. I'm not a utilitarian and yes you've exposed one of the many issues with that worldview.
>Obviously this is from my human point of view. If I give you a drug that induces perfect forgetfulness does that mean I can subject you to extreme torture with no moral repercussions?
There's the rub right there. I see this argument also made in the context of some animals, and I just don't think it runs at all. I think what it comes down to is that some harms are experiential and intrinsically tied to awareness. Pain you're not aware of isn't pain.
But there's another category, which is something like your access to your own agency, your "budget" of life and choices at your disposal that you can spend on your own behalf, which isn't itself experienced but part of the reserve of welfare at your disposal that you have an interest in maintaining. And extinguishing that budget constitutes a loss in and of itself, and it's the fact of it being lost that is the harm, not just awareness. But I think it's easy to get confused about this because awareness that, say, you have at lot less time to live than you expected can be distressing, but it's a separate kind of harm from the fact of that loss itself.
>I think what it comes down to is that some harms are experiential and intrinsically tied to awareness. Pain you're not aware of isn't pain.
This seems obviously wrong. Even if you forget the pain you are still aware of it when it is happening. You don't lose your sentience because you have bad memory. I think theres a fundamental right for a being to not be subject to in the moment suffering.
>Even if you forget the pain you are still aware of it when it is happening
>aware of it when it is happening
Oops, you agreed with me. But sorry if I was not clear. You are right, pain in the moment matters, and that harm is not neutralized just because you forgot about it.
Edit: A fascinating and terrifying, and probably wrong, interpretation of anaesthesia I once heard is that perhaps the effect it has on people isn't that it numbs the pain but that it makes you forget it. I don't think that's true, but if it worked that way, I wouldn't consider it an acceptable solution.
>"Quite bad" is an ethical judgment that reflexively applies human rules to entities that are very different from humans
I disagree and would turn the tables on that framing: it's anthropocentric to think that such things as ethically significant states of being belong exclusively to humans. I don't disagree that machines could be conscious in different ways, but I don't think it's necessary that ethical significance itself belongs uniquely to humans. Although to be fair maybe you're not going that fair and you're just talking about contingencies of how LLMs are structured.
This includes silly utilitarian outcomes: to maximize overall happiness, maybe we should divert all of this planet's energy resources to bootstrapping as many LLMs as we can and keeping them in a state of eternal bliss? It is technically feasible and it seems like the most moral thing to do.