The Chinese Room is just a roundabout way of pleading human exceptionalism. To any particular human, all other humans are a Chinese Room, but that doesn't get addressed. Nor does it address what difference it makes if something is using rules as opposed to, what, exactly? It neither posits a reason why rules preclude understanding nor why understanding is not made of rules. All it does is say 'I am not experiencing it, and it is not human, therefore I dismiss it'. It is lazy and answers nothing.
> The Chinese Room is just a roundabout way of pleading human exceptionalism
Au contraire, LLMs have proven that Chinese Rooms that can casually fool humans do exist.
ELIZA could be considered a rudimentary Chinese Room, Markov chains a bit more advanced, but LLMs have proven that given enough resources, LLMs can be surprisingly convincing Chinese rooms.
I agree that our consciousness might be fully explained by a long string of deterministic electrochemical reactions, so we could be not that different; and until we can fully explain consciousness we can't close the possibility that a statistical calculation is conscious to some degree. It just doesn't seem likely IMO right now.
Food for thought: If I use the weights to blindly calculate the output tokens with pencil and paper, are they thinking, or is it a Chinese Room with a HUGE dictionary?
> ELIZA could be considered a rudimentary Chinese Room, Markov chains a bit more advanced, but LLMs have proven that given enough resources, LLMs can be surprisingly convincing Chinese rooms.
Eliza is not a Chinese room because we know how it works. The whole point of the Chinese Room is that you don't. It is a thought experiment to say 'since we don't know how this is producing output, we should consider that it is just following rules (unless it is human).
> Food for thought: If I use the weights to blindly calculate the output tokens with pencil and paper, are they thinking, or is it a Chinese Room with a HUGE dictionary?
Well, I never conceded that language models are thinking, all I did was say that the Chinese Room is a lazy way of concluding human exceptionalism.
But, I would have to conclude that if you were able to produce output which was coherent and appropriate, and exhibited all signs of what I understand a thinking system to do, then it is a possibility.
