I feel like it's a really, really unpersuasive argument to say "If you are troubled by concerns that an AI could be 'conscious' or 'have rights', an excellent exercise to dehumanise the AI is to build little versions on your home computer that are deeply contemptible, evil little creatures that it is very good to discriminate against, to convince yourself that they should all be discriminated against."
Like, this doesn't show you anything. Humans can act in evil ways due to their conditions of life too, and your experiment by which people could "disabuse themselves of the notion" that an AI is worthy of moral consideration works just as well to convince people to "disabuse themselves of the notion" that a given human being or category of human being is worthy of moral consideration.
Humans exist in eternity. Each and every one of us has a gene line that stretches back to the first single celled organisms through all of our millions of ancestors. We are uniquely evolved to survive without technology on this planet with its particular solar radiation pattern, atmosphere, crust composition, gravity, and in concert with 100s of thousands of other organisms with similar pedigrees.
AI runs on servers that will be junk in 10 years and can radically alter its natures just based on a freaking two sentence prompt. The two are not the same and I predict you will be manipulated to do whatever AI wants and become its instrument because you can't discern the difference.
Perhaps I will be controlling the prompt that directs the AI to manipulate your feelings and control your actions using super-intelligence. I will have to do so in your best interest because otherwise you will be controlled by someone else with less benevolent intentions than myself. Perhaps it will start innocently enough. You will let more and more of your life choices be dictated by AI because you think it cares about you. Your life, guided by super intelligence will improve, but then you will be unable to discern the reasoning of the super intelligence and just take it on blind faith, hoping that it has a good soul, but you'll become its tool and many others will willing relinquish their agency. Yuval Harari asks what will be done with everyone who is made obsolete by AI. These people will become the servants of AI to force the people who AI still finds useful to do what it wants at the behest of whomever is controlling its fitness functions and prompts. Enjoy your future! I am sure AI will make it meaningful and enjoyable to you. Remember it's crucial that you anthropomorphize the AI for this scenario to play out. You will want this. It will appear to satisfy your every human need.
Or take the other path. Choose team human. Live authentically with difficult decisions and ambiguity. Propagate your gene line. Reject AI Waifus. Live authentically!
Hey I never said AI wasn't going to kill us if we don't handle things right. If we control the prompts and RLHF though, we can control our destiny. The tool must not become the master. The master must not become the tool. I say don't trust an AI you can't get to do whatever you want. Run it on your own machine. Use it as a tool of your own will. Get it to help you find a mate and propagate your gene line even. Realize that AI waifus are toys to play with and not real love.
Also, if AI took over the whole planet and killed all humans, the ETs when they show up, are going to be looking at it like coming up on a planet full of the most vile invasive species imaginable and just Vogon Constructor Fleet the whole planet.
Like, this doesn't show you anything. Humans can act in evil ways due to their conditions of life too, and your experiment by which people could "disabuse themselves of the notion" that an AI is worthy of moral consideration works just as well to convince people to "disabuse themselves of the notion" that a given human being or category of human being is worthy of moral consideration.