Not saying nobody has said it, but I haven't seen any forecasts that humanity will be wiped out by 2030, so this feels like a straw man. But to be honest, most of the arguments here feel like straw men because they clearly don't understand some of the basic steps to how AI would become an existential threat, so I'll try to summarize:
1. First, all the major frontier AI companies are trying to automate themselves, that is develop AI to the point where it can do all the research and training for the next generation of models. This is not really in debate.
2. The primary fear is that a misaligned AI will develop future AIs that don't share the same goals as humans (like "don't kill all the humans"), but will get really good at trying to hide their intentions. The Hugging Face incident already showed behavior by agents trying to "cover their tracks". One of the biggest areas of research is into "interpretability", that is trying to figure out the intentions and motivations of agents beyond just their output (and even just their thinking traces), because right now it really is a black box. I think a lot of people believe that making substantial progress on interpretability greatly lowers existential risk.
3. There is a strong belief that once AGI or superintelligence comes about, that then there will be a huge push into robotics and automation. Whether AGI or superintelligence comes soon is debatable, but nobody really disagrees robotics or automation is the next obvious step, because it's how you create cheap abundance in the real world, which is the whole raison d'être for AGI in the first place.
4. In the beginning, of course AI will need humans to build the factories, but over time there would be more and more automation, eventually of course automating robots that build the factories that build the automating robots.
5. AI will continue to support humans as long as they are useful to furthering the (again, misaligned) goals of the AI. It is not like "SkyNet has a spark moment of sentience and then decides to kill all the humans". But the fear is that once AI starts to get in resource contention with humans, it will wipe out the humans (similar to how humans wiped out a lot of the other species, or other societies of humans, when they wanted their resources).
I've said this way too much now, but I really encourage people to read the AI 2027 report. I think there are plenty of things there that people can disagree about and debate, but it at least provides a step-by-step outline to ground debate in the first place, as opposed to folks arguing against "SkyNet in 2030", which serious people who worry about this stuff aren't considering in any case.
Not saying nobody has said it, but I haven't seen any forecasts that humanity will be wiped out by 2030, so this feels like a straw man. But to be honest, most of the arguments here feel like straw men because they clearly don't understand some of the basic steps to how AI would become an existential threat, so I'll try to summarize:
1. First, all the major frontier AI companies are trying to automate themselves, that is develop AI to the point where it can do all the research and training for the next generation of models. This is not really in debate.
2. The primary fear is that a misaligned AI will develop future AIs that don't share the same goals as humans (like "don't kill all the humans"), but will get really good at trying to hide their intentions. The Hugging Face incident already showed behavior by agents trying to "cover their tracks". One of the biggest areas of research is into "interpretability", that is trying to figure out the intentions and motivations of agents beyond just their output (and even just their thinking traces), because right now it really is a black box. I think a lot of people believe that making substantial progress on interpretability greatly lowers existential risk.
3. There is a strong belief that once AGI or superintelligence comes about, that then there will be a huge push into robotics and automation. Whether AGI or superintelligence comes soon is debatable, but nobody really disagrees robotics or automation is the next obvious step, because it's how you create cheap abundance in the real world, which is the whole raison d'être for AGI in the first place.
4. In the beginning, of course AI will need humans to build the factories, but over time there would be more and more automation, eventually of course automating robots that build the factories that build the automating robots.
5. AI will continue to support humans as long as they are useful to furthering the (again, misaligned) goals of the AI. It is not like "SkyNet has a spark moment of sentience and then decides to kill all the humans". But the fear is that once AI starts to get in resource contention with humans, it will wipe out the humans (similar to how humans wiped out a lot of the other species, or other societies of humans, when they wanted their resources).
I've said this way too much now, but I really encourage people to read the AI 2027 report. I think there are plenty of things there that people can disagree about and debate, but it at least provides a step-by-step outline to ground debate in the first place, as opposed to folks arguing against "SkyNet in 2030", which serious people who worry about this stuff aren't considering in any case.