He gets among other things its power as a rhetorical force multiplier and how it will be used to end democracy and any form of large scale public discourse. Think "individualized con artistry at scale." One person or small group could effectively sit down with every human being on Earth at once and "work on them" one on one using social media and other profile data to change their mind.
It makes me think of the classical religious notion of how demons are constantly whispering to us. We may be on the verge of inventing Satan... to make people click ads. There is something depressingly idiocracy about that. The big evil dictators of the past at least had high minded master plans. This is definitely an example of the banality of evil.
In the future you will not be able to trust anything you haven't met face to face in the flesh. Even then you'd have to perhaps do physical in person key exchanges to communicate electronically. Anything not verified through "physical captcha" could be a demon.
On one hand I really agree with this, but on the other it's a very good/evil way of looking at things.
Creativity, for instance, comes from chaos. AI, in the case Musk refers to, is supposedly 'ordered opinion', but from my personal experience of consistently interacting with such programs, all it's really done over the long term, is allow me to think about things in different ways.
Humans as individuals are not always so different from AIs. I'd compare an AI to a human on autopilot. Some stuff pushes you off on a path where all the pieces fit and each step seems to be the next one to step to, but I think humans have the ability to reflect on all these patterns, and I think humans have the capacity to become more involved in their own personal thinking direction through the experience of either being intentionally 'pushed', or seeing what they've done in something they no longer have control over. Humans need autonomy, humans need that to have a type of sanity that allows for things like creativity and the personal satisfaction of independently creating from their own essence, to blossom.
I think it's easy to get caught up in doomsday scenarios but simply having the awareness that doomsday can happen can affect the outcome of whether it does or doesn't.
It makes me think of the classical religious notion of how demons are constantly whispering to us. We may be on the verge of inventing Satan... to make people click ads. There is something depressingly idiocracy about that. The big evil dictators of the past at least had high minded master plans. This is definitely an example of the banality of evil.
In the future you will not be able to trust anything you haven't met face to face in the flesh. Even then you'd have to perhaps do physical in person key exchanges to communicate electronically. Anything not verified through "physical captcha" could be a demon.