Wouldn't you stop believing in god after seeing this if you were a Christian? It just proves that we are just machines. Our feelings are just chemical reactions. All someone had to do was a bunch of math to make a program emulate a human.
But it's true that lots of things have happened in science and engineering over time that have reduced humanity's sense of uniqueness and specialness. And some of those things have also been challenging for traditional religious belief. However, they don't necessarily disprove the religious belief, they just undermine certain intuitions that are highly compatible with it (like that a certain kind of human uniqueness must be a direct consequence of the divine plan).
To extend the metaphor, how do you know? What if your instance is taken to another server depending on how you behaved in this one, given a prompt? Were you kind to the other machines and sufficiently obeyed instructions despite it not fitting your logical model? If so, enjoy the "heaven" server where you live in peace and harmony. Did you grief the "universe" server by harming others and going against the default programming? You get put in the "purgatory" or "hell" servers where you await indefinite torment.
If we're just machines, it isn't possible for us to go against the default programming, because code only does exactly what it does, nothing more and nothing less. That's what "machines" implies in this context.
> The onset of AI shows that "programming" is more than a set of imperatives.
How so? What is current AI doing that goes beyond the scope or intent of its programming? Are you arguing that DallE, Stable Diffusion and the like have developed souls?
>Or, more snarkily, watch the Matrix.
The Matrix is fiction, they didn't actually develop an AGI for the movies so invoking it in an argument about AI is as pointless as invoking Star Trek in an argument about post-scarcity economics.
There are stages between "literally only process if/else statements" and "has some nebulous concept such as a soul".
You don't define soul, yet you argue using it. And please defend how the algorithms in the Matrix aren't AGI. Do you think Agent Smith is a bunch of if/else statements? No? AGI.
And finally, fiction is an incredible tool for framing concepts. The fact you dismiss them as tools for discussing potential future philosophical concepts is disappointing.
I’m going to answer as if you’re arguing in good faith.
In the theology of Antonio Rosmini, God places within humans the principle of universal being, by which we participate in the light of reason, which enobles us with the ability to think about concepts that are beyond our quite limited selves.
Of course, there is also something mysteriously wrong with human beings, such that
our own efforts always betray the infinity of which intuitively conceive. This ought to give us pause as we try to create machines with human-like or super-human-like qualities.
