Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Ensuring AIs don't wind up wanting to destroy humanity.


“Want” is a problem for me here. GPT-3, for example, is just a dumb, brute force Markhov chain. Any danger it might pose is no more than that of a mechanical trigger that sits between a finger and gun barrel. The only real intelligence in the system, and where the only danger lies, is from the person behind the gun.


GPT-3 is a bit more than a Markhov chain. There is a lot more to it than just being big. Same for GPT-2.


As soon as you put the intelligence in AI this changes. It is easy to say that current AI systems are “dumb” (although note that the precise meaning of dumb has changed significantly over the last few decades). You can say that about any AI with sub-human intelligence. But if you reach human levels, you can likely reach super-human as well, so you need to start worrying much earlier.


GPT-3 is better than everything what was achieved so far in multiple metrics, though yes, it's of course not an "intelligence".

However, this is ignoring possible future developments - things may get better and more advanced and that's what AI-safety people are concerned about.


Wanting is the easier of the two - the other is ensuring AIs don't inadvertently wind up destroying humanity. The road to hell being paved with good intentions and all that.




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: