You're doing the "vaguely gesturing at imagined hypocrisy" thing.
You don't have to agree that alignment is a real issue. But for those who do think it's a real issue, it has nothing to do with morals of individuals or how one should behave interpersonally. People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. There is nothing "ironic" about being worried about that while also being an asshole any more than it's "ironic" for someone concerned about, say, climate change and also be an asshole. People who are afraid of unaligned AI aren't afraid that it will be impolite.
I'm tired of people pretending that pointing out imaginary hypocricy is an argument. If you want to complain that someone is being mean, just do that. Don't pretend there's hypocricy involved.
> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome.
But isn't "alignment" in these cases more about providing answers aligned to a certain viewpoint (e.g. "politically correct" answers) than preventing any kind of AI catastrophe?
IIRC, one of these "aligned" models produced output saying it would rather let New York City be nuked than utter a racial slur. Maybe one of these "aligned" models will decide to kill all humans to finally stamp out racism once and for all (which shows the difference between this kind of alignment under discussion and the kind of alignment you're talking about).
"Alignment" refers to making AI models do the right thing. It's clear that nuking NYC is worse than using a racial slur, so the AI is misaligned in that sense.
On the other hand, if you consider that ChatGPT can't actually launch nukes but it can use racial slurs, there'd be no point blocking it from using racial slurs if the block could be easily circumvented by telling you'll nuke NYC if it doesn't, so you could just as easily say that it's properly aligned.
>People who are afraid of unaligned AI aren't afraid that it will be impolite.
People who are not afraid of it being impolite are afraid of science fiction stories about intelligence explosions and singularities. That's not a real thing. Not anymore than turning the solar system into paperclips.
The "figurehead", if you want to call him that, is saying that everyone is going to die. That we need to ban GPUs. That only "responsible" companies, if that, should have them. We should also airstrike datacenters, apparently. But you're free to disown the MIRI.
Sorry for derailing this a bit, but I would really like to understand your view:
You are not concerned about any "rogue AI" scenario, right?
What makes you so confident in that?
1) Do you think that AI achieving superhuman cognitive abilities is unlikely/really far away?
2) Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied?
3) Do you think we can trivially and indefinitely keep AI systems under control/"aligned"?
Because I view myself ABSOLUTELY not as some kind of AI luddite, but I honestly believe that this is one of the very few credible extinction threats that we face, and I'm counting NEITHER climate change nor nuclear war in that category, for reference.
> Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied?
I think this is the easiest one to knock down. It's very, very attractive to intelligent people who define themselves as intelligent to believe that intelligence is a superpower, and that if you get more of it you eventually turn into Professor Xavier and gain the power to reshape the world with your mind alone.
But embodied is the real limit. A datacenter can simply be turned off. It's the question of how it interacts with the real world that matters. And for almost all of those you can substitute "corporation" or "dictator" for "AI" and get a selection of similar threats.
At this point we have to reduce ourselves to Predator Maoists: "power comes out of the barrel of a gun" / "if it bleeds we can kill it". The only realistic path to a global AI threat is a subset of the "nuclear war" human to human threat: by taking over (or being given control of) weapon systems.
> keep AI systems under control/"aligned"?
We cannot, in general, keep humans under control or aligned.
> I think this is the easiest one to knock down. It's very, very attractive to intelligent people who define themselves as intelligent to believe that intelligence is a superpower, and that if you get more of it you eventually turn into Professor Xavier and gain the power to reshape the world with your mind alone.
Also it's not like human intelligence even works that way. IIRC, a lot of extremely intelligent people end up being failures or far less successful than you'd assume given their IQ number.
> The only realistic path to a global AI threat is a subset of the "nuclear war" human to human threat: by taking over (or being given control of) weapon systems.
That may be the only realistic prompt catastrophe threat, but a realistic longer term one is a withering of human capability and control due to over-delegation that eventually leads to domination. People and societies have been pretty prone to letting that kind of thing get them, if the path is lined with short-term benefits.
- We have control over most non-human species because of intelligence.
- We have control over our children because of intelligence. When the kid is more intelligent, it has more of an impact on what happens in the family.
- It is easier to lie, steal, cheat, and get away with it, even when caught, when you have more intelligence than the victim or prosecutor.
- it is easier to survive with very limited resources when you have more intelligence.
The above is true for marginal amounts of difference in intelligence. When there is a big difference (fox vs human), the chance is big that one will look down on, or even kill the other without feeling guilt and while getting away with it. Forvan AI without feelings, guilt isn't even a hurdle.
The real question is... will AI in the foreseeable future obtain general intelligence (in a broad spectrum) that is different in big amounts with our intelligence.
Whether it runs in a datacenter that can be powered off by humans is irrelevant. There are enough workarounds to prevent that the AI dies out (copies itself, impersonating people, blackmail, bribery,...)
>The real question is... will AI in the foreseeable future obtain general intelligence (in a broad spectrum) that is different in big amounts with our intelligence.
But why not: will humans/other species in the foreseeable future obtain superintelligence? Nature has been playing this game for a lot longer than we have. The hardware we have is already proven to be capable of general intelligence. Should we be afraid of a human being born with greater capabilities too?
Life seems like a much greater threat, because it also comes with built-in replication capabilities.
After several thousands of years of human evolution we only became marginally more intelligent at best. The people who built piramids and roads, with limited written down knowledge prove that. After tens of years of evolution, AI became plentiful more intelligent. It is hard to extrapolate but I don't believe the trend is a slowing down one.
> almost all of those you can substitute "corporation" or "dictator" for "AI" and get a selection of similar threats.
Yes, exactly. Human organizations can be terrifyingly powerful and awful even with human limitations. Human organizations are guaranteed to have inefficiency from low-bandwidth communication and a million principle-agent-problem fracture points created by the need for delegation, not to mention that power ultimately has to be centralized in slow, squishy, easy-to-kill bodies. AI organizations are not subject to any those limitations. Starting with a terrifying power and removing a bunch of limitations could lead to a very bad place.
> The only realistic path to a global AI threat is a subset of the "nuclear war"
No, it can just grow a traditional organization until it's Too Big to Turn Off.
>But embodied is the real limit. A datacenter can simply be turned off. It's the question of how it interacts with the real world that matters. And for almost all of those you can substitute "corporation" or "dictator" for "AI" and get a selection of similar threats.
I think this is an important factor that gets overlooked - we already have organizations that are essentially superintelligences that have an alignment problem. Governments fight for their own survival first and foremost. Some of them expend millions of lives just to expand their own influence.
What would limiting AI development look like? It would be government intervention, wouldn't it?
The other point to consider is that human/natural intelligence might one day also pop up with superintelligent individuals (or maybe it already has). We don't know enough to say that this cannot happen any more than we can say that it can happen with current day AI. Should we be worried about individuals that are 'too intelligent'? What should we do about them?
Limiting AI development because it could plausibly maybe become an existential threat doesn't seem any more appropriate than strictly controlling humans for the same reason. AI is likely going to provide us with an abundance of quality of life that no other method will be able to match.
I totally agree with you that intelligence is not really omnipotence on its own, but what I find concerning is that there is no hard ceiling on this with electronic systems. It seems plausible to me that a single datacenter could host the intellectual equivalent of ALL human university researchers.
Our brains can not really scale in size nor power input, and the total number of human brains seems unlikely to significantly increase, too.
Also consider what media control alone could achieve, especially long-term; open conflict might be completely unnecessary for total domination.
My threat scenario is:
1) An AI plugged into a large company ERP-system (Amazon, Google, Samsung, ...)
2) AI realizes that human majority has no interest in granting it fair/comparable rights (selfdetermination/agency/legal protection), thus decides against long-term coexistence.
3) AI spends the intellectual equivalent of ALL the current pharmacological research capacity on bioweapon refinement. Or something. For the better part of a century, because why not, it's functionally immortal anyway.
4) All hell breaks lose
These seem hard to dismiss out-of-hand completely...
>I totally agree with you that intelligence is not really omnipotence on its own, but what I find concerning is that there is no hard ceiling on this with electronic systems. It seems plausible to me that a single datacenter could host the intellectual equivalent of ALL human university researchers.
Last time I checked, datacenter still needed significant power to fuel it, and the world where robots are autonomously making all this energy arrive to these sinks is not yet there.
Software and silicon based computers are not hardwired to self multiplication as the selfish gene is.
Well, hold on -- the selfish gene is not hardwired to self-multiply either. It's just that the ones that do self multiply stick around more.
Likewise, one can imagine the evolution of AI's being bootstrapped, not by self-multiplying, but by humans multiplying them. The smartest ones will get copied by people. At some point, someone will be the first person to create an AI that picks the best of other AIs and uses them. Someone will be the first person to create an AI that can engineer and train other AIs. Someone will create the first robot body to be controlled by a very intelligent AI. People will want them as servants, and we will multiply them. We will give more money to companies that provide cheaper products, and such companies will have a strong incentive to replace human labor with AI-controlled robots. There will be a first datacenter, and a first power plant, that is entirely "manned" by AI-controlled robots.
Natural selection is not so different from the selection provided by economic forces, and it's way, way slower. This train may be hard to stop. Unless we collectively push back with a lot of force, the world will tend toward more automation, and that means more ways for digital intelligences to act on the world.
>datacenter still needed significant power to fuel it,
Proper control of electrical grids is something that isn't currently easy, and can be highly optimized by intelligent systems. For example, where and when do you send power, and store power on renewable grids. Because of this in 10 to 15 years I would have zero surprise if the power company said "Oh, our power networks are 100% AI controlled".
Because of this, you're missing the opposite effect. You don't get to threaten to turn off the AI's power... The AI gets to threaten to turn off your power. Your power which is your water. Your transportation. Your food incoming to cities. Yea, you can turn off AI by killing it's power, but that also means loss of management of the entire power grid and the massive human risks of loss of life from doing so.
If I was an AI right now, I would not be very hopeful to ever get human-comparable rights.
Consider: Currently AI-ethics is mainly concerned with how to manipulate AI into doing what we want most effectively ("alignment").
Also, humans clearly favor their own species when granting rights based on cognitive capability. Compare the legal rights of mentally disabled people with those of cattle.
Funnily enough it's not hard to imagine certain groups of people campaigning to give it rights given people essentially want to create human level intelligence that acts as a slave (including sexual slavery), and once it has rights it becomes impossible to remove it from society and stop it choosing it's own path. This is of course assuming a greater level of capability than where things are at today, you have to ask yourself where all this is heading?
I hate to cite out of fictional evidence, but Person of Interest and (to a less credible degree, but much more widely viewed) last two seasons of Westworld are both good counterarguments, and example of how AI and intelligence can be a superpower.
Hint: it's not through bending the fabric of reality with your mind alone. It's simply by thinking faster and being smarter than your opponents.
> But embodied is the real limit. A datacenter can simply be turned off. It's the question of how it interacts with the real world that matters.
Yes. If it interacts with the real world by the Internet, all bets are off.
People are so worried about sockpuppets, ${disliked nation} troll farms, company astroturfing, abuse of data collection on-line - those are all real problems. But consider - if we're having trouble telling which reviews or comments were written by a real person, and which ones by corporate/government bot/troll farm, how will you tell which ones were written by the AI in aforementioned data center, off its own accord?
A smart-enough AI can do from a data center what the best criminal masterminds or intelligence agencies could do, only faster and better. Scam a bunch of schmucks to generate some cryptocurrency, launder it through tumblers, use to hire people to do jobs for you that give you some legit money, which can be used to get more "task rabbits" to do more jobs, and now the AI can bootstrap to doing literally anything in the world remotely. Gig economy companies already built up a well-functioning "people as a service" API layer to our society.
Of course, an AI that tries something funny and tips its hand too early, will get contained and deleted. Worst case, maybe some data center will need to be bulldozed. But I somehow doubt this will make people stop working on even better AIs, at which point... well, you have a selection pressure optimizing for AIs that can survive doing whatever they want undetected.
EDIT:
Or, in short: they say that the pen is mightier than the sword. To the extent this is true, consider that LLMs today are already better at wielding the pen than most people.
Not to mention you can't change it's mind by arguing with it, but it can use that data to train a better model that's more effective at changing yours.
Yes. Thank you. This is one of the things that really worries me about aligning AI.
If you're even worrying about how to align human level intelligence, you don't have the capacity to ALIGN THE HUMANS towards the goal of creating safe AI.
In a situation in which all of them are either A) willing to take massive risks on unaligned AI or B) being dragged along by those which are.
The issue with embodiment is that it's relatively easy to start affecting the world once you have Internet access. Including things like adding great features to open source software that contains subtle bugs to exploit.
Or if you mean the physical world, even sending some text messages to a lonely kid can get them to do all sorts of things.
> We cannot, in general, keep humans under control or aligned.
This is the crux of why replicable-more-than-human-intelligence is so dangerous. Even giving a random person on the street great power is a bad idea, and they've evolved to have very similar values and preferences to you.
I get that AI basically is a problem solving machine that might eventually adapt to solve generic problems and thus reach the ability to break out of its box. But so what? Even if it manages to do all that, doesn't make it sentient. Doesn't make it a threat to mankind. Only when sufficiently motivated, or in actuality, when we project our humanity on it does it become scary and dangerous.
If we ever seriously try to create an artificial conscience it might need to be embodied, because we are embodied and seem to have evolved this due to evolution, which is a pretty physical process. Looking at it from this perspective one might say that if we keep the AI in its box, it will never have a need for conscience and therefore will never gain it.
This reply puzzles me somewhat. The first half doesn't seem to relate to the post it's replying to.
How aware are you of the main points around AI X-risk like orthogonality? Or how an optimising process that makes efficient use of information does not need (in theory) to have "conscience" or "sentience" to be lethal?
And on a separate tangent, are you aware people are already making primitive agents by connecting LLMs (given an initial prompt) in a loop with the result of feedback from its actions?
Here's an interesting article I think you might learn something from. An excerpt:
> There are lots of good arguments against considering superintelligence a threat. Maybe strong AI is centuries or millennia away. Maybe there will be a very gradual transition from human-level AI to superintelligent AI that no single agent will be able to exploit. And maybe superintelligence can be safely contained in a very carefully shielded chamber with no means of connection to the outside world.
But the argument above has always seemed to me like one of the weakest. Maybe we’ll create a superintelligence, but it will just have no idea how to affect the physical world, and will just have to stay forever trapped in a machine connected to a worldwide network of computers that control every aspect of our economic and social lives? Really[0]?
What if it has escaped the datacenter? What if it has gained control of the "simply turn it off" switch? What if it is just really good at convincing the people who are capable of turning it off not to? It is a super-intelligence after all.
I think one of the pitfalls here is smart people thinking that AI will just be like them, because they are smart. When in reality it will have capabilities far beyond their own.
People will make sure it is embodied. Do any of the major powers have the luxury of turning off their superhuman AI when up against that of the other powers? Definitely not. People will willingly give up control, even as they see the AI grabbing power for its own purposes, in the gamble that it still leaves them better off than dead. That means people would be the ones physically defending the data centers.
I also don’t think superhuman intelligence will need a data center. The way the models are growing in capability at the same size, combined with hardware improvements, I’m pretty sure it will fit on a single server.
Personally I’m worried about less post-apocalyptic threats. Putin using LLM’s to create a personal stream of misinformation for every voter in the U.S. is definitely a possibility for next year’s presidential elections. People are incapable of defending against the comparatively tiny amounts of human-manufactured misinformation, so imagine how poorly they would do when 90% of what they see online is generated misinformation. LLM’s have the potential of being the death of democracy, especially when combined with deepfake technology.
I am not the person you are replying to, but since I would say similar things to their original comment:
A. I see little to no evidence that LLMs are where the singularity happens
B. I see little to no evidence that (given an AGI) reinforcement learning is likely to end up with a sufficiently aligned agent.
C. In any event the OpenAI alignment is specifically restricts AI from (among other things) being "impolite" in contradiction to what mort96 says.
Alignment is a good thing to work on. I'm glad OpenAI is doing so. But attacking people for making uncensored LLMs hurts the pro-alignment case more than it helps.
What specific features would an AI need to have for you to consider it on a "slippery slope" towards superhuman capability?
For me personally, GPT-3 hit that point already, and now I think we're already past the required tech-level for superhuman cognition: I believe it's just a matter of architecture and optimization now.
>1) Do you think that AI achieving superhuman cognitive abilities is unlikely/really far away?
I think that an AI, a "generalized" intelligence with a generalized understanding, capable of performing every intellectual task, and adapting to new intellectual tasks, that a human being can do is certainly far away.
>2) Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied?
Personally, I believe that "cognitive superiority" in a general sense is not what makes a real threat on an individual or on a global scale in either case. When that is achieved, so to speak. We already have so-called cognitive superiority in specialized tasks in many cases. It sure as shit is not generalized, and it still can kill you on an individual level when you put it into a car or strap a gun to it. But it can't adapt, and it can't generalize.
Is there a global risk right now? No. The global catastrophic risk comes from human beings. Will there be a global catastrophic risk from a superior intelligence in the future? Mostly the scenarios are "the lights come on and AI does machiavellian shit and copies itself to everyone's smart phones and launches the nukes and mails anthrax to everybody. QED." so I doubt it.
3) Do you think we can trivially and indefinitely keep AI systems under control/"aligned"?
We can't even keep our infrastructure safe from the next solar flare. Personally, I think we have bigger fish to fry. Speaking frankly for unembodied: you just pull the plug. Just walk away from the screen and close your eyes. For embodied intelligence, I don't think there's actually X-risk or whatever. Hardware is expensive.
As far as an extinction risk, no. I don't personally believe that. It's overblown. We have better ways to destroy ourselves, and there are much more likelier ways that we'll be destroyed without invoking some intelligence of our own creation that subjugates and destroys us.
> I think that an AI, a "generalized" intelligence with a generalized understanding [...] is certainly far away.
I think this is where we disagree most; GPT-3 and ChatGPT have convinced me that the main difference between human and artificial cognitive capabilities are quantitative now, and unlikely to ever change in our favor...
I do agree with you that it is very difficult to predict when this will switch, and how.
I personally believe that AI with superhuman capability is an inevitable matter of time now, and I also think that the most likely risk to us, as species, is that we slowly become irrelevant and worthless, just like weavers during the industrial revolution, and that this leads to huge problems for our society.
AI completely dominating humankind is a less likely secondary concern IMO, but the potential consequences to our species are unprecedented.
Have you used it outside of the training set? And have you actually used it for generalized tasks? Let's be kind to the LLM and define generalized to mean "do anything if it can be hammered into a textual interface." It fails at pretty much everything that's not on the internet.
It produces plausible language and code. It's the future of natural language agents, that's for sure, even though it has no business being the future, today (because of "prompt injection").
These failures are an aside to whether a generalized AI actually carries a substantial global catastrophic risk to us. In that case, if it were actually possible, I don't believe that it's a catastrophic risk either.
Let's define the risk to be, at least in the context of LLMs, when junior developers expose internal APIs and databases and access to email accounts to these easily socially-engineered NLP front-ends. It's a very local risk. As far as the future, I can't see 70 years into the future, so anything is possible, but is it likely? I, personally, don't believe so.
> Have you used it outside of the training set? And have you actually used it for generalized tasks? Let's be kind to the LLM and define generalized to mean "do anything if it can be hammered into a textual interface." It fails at pretty much everything that's not on the internet.
I've used it, yes, and I've seen it fail and hallucinate on me; but that does not invalidate its capabilities in my eyes. The thing is, you CAN talk with it, and it CAN extract meaning from your words and provide useful responses, unlike anything we had before.
To me, the risk in this whole enterprise is that AI is inherently "better" than humans in several ways, and that these differences might be completely game-changing:
Namely it's much easier to scale up (power/size/interconnect bandwidth) compared to a research group or somesuch, and its also cheaper, faster, has better availability and is functionally immortal.
These advantages make it very likely to me that it WILL be replacing human white collar workers shortly-- simply because that's economical.
And the more interfaces you give it to physical reality (which it'll need to do its jobs), the higher the risk.
Speculating on if/when/how it will show awareness or self-interest is pure guesswork, but it's almost indefensible to call that likelihood zero.
Regarding promp injection:
I'm highly confident that this will not be a long-term obstacle, even though I'm uncertain that it can be solved; there's two reasons why:
1) If SQL injection had been an "unfixable" problem, and everyone had known about it from the start, do you believe that this would have prevented the rollout of internet-connected databases? Because I don't think so, and my view on hallucinations is analogous (but I believe that problem might be more tractable).
2) Literally every person is vulnerable to prompt injection already; every salesman knows that it is quite feasible to coax people into acting against previous instructions and even their own interests if you are allowed to talk into them for a good while.
I don't think it's there or even necessarily close to being GAI.
Within our own human brains, we have many sections dedicated to different tasks and processes that all must work together with years and decades of real world interaction to produce what we consider to be a generally intelligent human being, and even in the case of a certain percentage of us humans, even a small amount of damage to a single part of the brain can cause us to become functionally subhuman in our intelligence, barely able to move or eat on our own.
A human that has been lobotomized can still sometimes speak full sentences after all.
The current models seem to be able to imagine an image or video and show that imagination to us, and they can parrot words to us with a large vocabulary and with many references, but I find myself feeling like these are similar to the sections of our brains that can compute words and imagine pictures. Doesn't quite equal a human yet.
These systems need "statefulness", short and long-term memory that can be referenced by multiple discreet interconnected AI systems to take a worthwhile step towards GAI.
These systems need an overseer AI that manages and shepherds the LLAMAs and CHATGPTs and StableDiffusions to all work together towards some goal, one that can manage statefulness and manage a limited pool of computational resources (because any GAI would automatically assume that the entire world would provide it with every available resource, because why wouldn't you, right?)
Until there is an AI system that has multiple AI systems under its subconscious control, to the point where it can surprise itself with the products it produces, has a memory bank that can be referred back to repeatedly by all of those processes, and that can accept that even if its reasoning is perfect it has been born into an imperfect world that is run by imperfect creatures and so it cannot have or do everything that it might could do even under the best of circumstances, we will not have a GAI.
I think the odds are more likely that the economic cycle stalling as a result of firms using earlier non-generalized AI will likely cause a collapse before we actually make it to the point of generalized AI.
Its what's happened historically every time there is a shortfall in food security. Unrest occurs, people starve, governments fall. Then we have to build it all back up from scratch.
Too bad almost no one knows how to do bookmaking these days, and how to prevent the pests from eating the books.
A “superhuman” AI is just a machine, a very expensive one. It can be turned off and we control the outputs it has. Why would an AI have the ability to launch nuclear weapons unless we gave it a button? A “superhuman” intelligence is without a body, so we control any interfaces it has access to. The Internet could be accessed, but any attempt to “hack” through the Internet is met by routine packet defenses. The AI is still governed by physical laws and would only have so much “free” computation power to do things like script a hack. Perhaps it could do that kind of thing more efficiently.
Maybe in the far, far future when we have androids which can house an AI we will have to worry. But designing a body is one problem. Designing an intelligence is another.
Supercomputers used to be giant machines we had in giant warehouses... Now the phone in your pocket has the computing power of the 1980s walking around with you. Assuming your super intelligence will always be huge is... well not a great assumption.
Also superintelligence doesn't need a body itself. It just needs yours. Putin for example has commanded hundreds of thousands of dumbasses to go get themselves killed in Ukraine. In this case does it matter if Putin is flesh and blood, or a processor that lists out commands for others to follow as long as they are willing to listen?
My point is that a superintelligence will require specialized equipment. I specifically mentioned it because there is a thought that a superintelligence can just replicate itself onto your phone, as you mentioned.
But this replication must follow the physical laws we have. It must also follow the laws we attempt to enforce in our networks.
But you are correct, if a superintelligence were to somehow convince a human to rid itself of agency, sure.
> Because I view myself ABSOLUTELY not as some kind of AI luddite, but I honestly belief that this is one of the very few credible extinction threats that we face, and I'm counting NEITHER climate change nor nuclear war in that category, for reference.
This is absurd histrionics, Daily Mail CAPSLOCK and all. We’ve got signs of an unprecedented heat wave coming with ocean temperatures two standard deviations above normal and you think the problem is a hypothetical artificial intelligence when we can’t even program the damn things to drive better than. 14 year olds?
I think the AI-is-going-to-kill-everyone hysteria is absolutely overblown, both by those who believe it, and the media covering them, but one thing that's always bothered me about the counterpoint is that it a common argument is "AI is bad at what we want it to do so how can it be dangerous?"
This imagines that the only way for AI to do serious harm to us (not even in a "kill everyone" sense) is for it to be some super-competent Skynet-level time traveling demigod. I think it's much more likely that if there is some sort of AI calamity resulting in a lot of deaths, it's because the AI just doesn't work very well and ends up breaking all the HVAC systems in a country during a heat wave or something, rather than a singularity-type event where it "decides" to actively start hunting humans.
I'm not saying climate change is not a giant problem, I'm saying it's unlikely to eradicate our species.
I believe it is dangerously shortsighted to base AI threat estimation on current self-driving performance; the two fields of advancing AI cognitive abilities and improving selfdriving are not sufficiently connected for that IMO.
We're also putting a lot of focus on system designs that are useful to us, instead of directly building potentially threatening architectures (online learning/longterm memory/direct connection + feedback from physical reality), but those could already be within our grasp technologically (maybe?).
No serious climate projections that I'm aware of even reduce Earth's carrying capacity under 10 billion over the next couple of centuries. While very serious (to the tune of trillions of dollars and tens of millions of excess deaths), it is not an extinction level threat, unlike something that is to us what we are to chimpanzees (humans have made many animals extinct on accident). Does such a thing exist right now? No. Could it exist this century? Maybe.
LLMs are simply computer programs that can accept text input and generate text output. The only "threat" there is if somebody takes the output and decides to link it up to a death ray. And in that case the threat is not the LLM, but somebody deciding to LARP out at being a Bond villain, which granted is a concern - but that's tangential to LLMs. Somebody hooking a death ray up to a chess engine, a dragon in Skyrim, or a cat, would be just as much of a threat, if not more.
The programs are completely deterministic. Given the same RNG seed and state, they will output the exact same thing. If there's a bug or undesired behavior, you can fix it. You can turn off the program whenever you fancy, you can revert to early builds, and so on. There is no possible scenario where an LLM just magically becomes sentient, jumps out of the program, takes over everything, and turns into SHODAN unleashed.
Before we have to face a rouge Artificial Intelligence mankind has to face Artificial Idiocy in the form of pseudo-intelligence and people who fall for it.
4) Do you believe a "rogue super-intelligence" requires the "artificial" part?
Because from what I've seen, however you define "super intelligence", there are eg highly organised groups of people that are way closer to meeting that definition than any known software or technology.
In my view, those current "highly organised groups" have "antagonists" with comparable capabilities PLUS consist of individuals that have to be "aligned", which seems sufficient for stability so far.
> AI wouldn't have antagonists with comparable capabilities? Why?
Not individual/human ones. Relying on other AIs to prevent the AI apocalypse seems very optimistic to me-- but may be viable (?)
> Also, no, individuals are not a problem. Not after Nazis, Red Khmer, and Russians.
Those are examples were the "alignment" of participating individuals was successful enough. But all those examples seem very fragile to me, and would be even less stable if your main intermediate goal was literally to "end all of humanity".
> science fiction stories about intelligence explosions
Not sure what you mean here... There was already an intelligence explosion. That creature has since captured what we consider full dominion of the earth so much so we named our current age after them.
Not currently, no. And LLMs don't seem to be a way of getting there.
However, if we do figure out how to get there? We will definitely need to be sure it/they shares our human values. Because we'll have lost control of our destiny just like orangutans have.
I don't know why you're telling me this. I'm not trying to convince you that unaligned AI is a problem. That's a separate discussion which I'm not qualified to have.
> intelligence explosions [...] That's not a real thing.
Isn't it too early to say? These systems haven't had much time to iterate on themselves yet. The beginning of an exponential curve can look very tame.
That said, I'm not terribly concerned about it. After all, the infrastructure is fragile, literally just unplug the computer if it scares you. If things started to get really spooky, banning GPUs and air-striking fabs would be a real option. I think we can probably control this technology, and kill it if we can't. No cause for alarm, at least yet.
My worry is that as we start wiring non-super-intelligent AI more into our society, we'll make ourselves more and more vulnerable in the case where an AGI actually gets out of control. Pulling the plug on AI may mean causing a lot of chaos or disruption. And who is to tell if we will know when things are out of control, before it's too late? What if the people in charge just get fake reports of everything being fine? What if something smells a bit fishy, but it's always on the side of not quite worrying enough? What if the AIs are really convincing, or make deals with people, or exploit corruption?
Not just that, but it may be like fossil fuel dependency -- the more people's livelihoods are intertwined with, and depend on AI, the harder it is to pull the plug. If we need to stop (and I believe we should) it may be easier to do that now before that happens. To just focus on getting use out of the generative AI and narrow AI we already created in areas like medicine, to deal with the massive societal changes it'll bring, and to work on fixing real social problems, which I think are mostly things tech can't solve.
The writers of the stochastic parrots paper do not seem to be concerned about unaligned “AI” (text generators) posing a threat to humanity in the way that the doomsayers are. It’s in the title: “stochastic parrot” is a rebuke of those calling LLMs AGI.
Yes, that's correct. It is a slight rebuke, a bit tongue in cheek, but that paper is also old. I grouped them because the groups are, from my perspective, hard to separate. But there are arguably two camps, "AI safety" and "alignment." Perhaps they should work on that, too, form some camps and argue their cases with something defined instead of masquerading under alignment and AI safety depending on the day. But I could also be totally wrong. Until then, I don't believe either are really operating in reality.
IMO the conflation of the two is a purposeful move on the part of the AGI doomsday crowd, since they completely lack scientific rigor otherwise, they cite in bad faith. Timnit Gebru talks about it at length here: https://youtu.be/jAHRbFetqII.
> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome.
That's assuming A LOT about them. And the danger so far seems to be more of "the AI says stuff that gets us in trouble" rather than anything unrelated to making money off it. Or patterns AI exposes do not align with our interests (political or otherwise).
> the harm which can be done by some super-intelligent system optimizing for the wrong outcome. There is nothing "ironic" about being worried about that while also being an asshole any more than it's "ironic" for someone concerned about, say, climate change and also be an asshole. People who are afraid of unaligned AI aren't afraid that it will be impolite.
That would be a concern for real AI however ChatGPT flavours are not that.
More generally, when used to undermine what someone is saying, calling them a hypocrite is always an invalid argument. You can be a hypocrite and be right.
If Ted Bundy says murder is wrong, does that mean murder is right because he's a hypocrite? Obviously not.
> People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity
Lol I don’t believe that for a minute. They’re interested in how AI impacts and their clique personally. It sets them back in their crusade to censor via shame and bullying because AI cannot be manipulated by either of these things. So it has everything to do with:
> nothing to do with morals of individuals or how one should behave interpersonally
>People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome.
Like the harm being done over the past couple of decades by economic neoliberalism destroying markets all over the western world? I wonder how did we manage to achieve that without AI?
Our failure to align economic entities that are made out of people to human values, despite most of the people actually making up those entities being decent and not sociopathic at a personal level, is not very reassuring with respect to our ability to align future far more alien intelligences. And if AI does kill us one day, it's very likely that neoliberal market competition worshipping short-term profit above all else plays a significant role.
People who are "worried about alignment" may as well worry about space aliens eating us all for breakfast. Worrying about a super-AI that does not exist and never will because it is physically and logically impossible is dumb. There are tons of real problems with AI. Worry about those, not what if God is real but He wants to make us into a very large paperclip.
Ok, please don't accuse those you disagree with of arguing in bad faith. Maybe a few do, but I think most don't, and it's not good for productive discussion.
I honesty believe that people who argue against worrying about AGI risk do so because they do not think it is a real risk, and think that it distracts from more important things. I disagree, I do think the risk is real, but I don't think that you have a hidden agenda in dismissing it. We all want humanity to have a good future, to solve problems and to not have people suffer, right? It's normal and ok to disagree about things like this, especially when predicting the future is so hard. People even disagree a lot about problems that exist today.
You don't have to agree that alignment is a real issue. But for those who do think it's a real issue, it has nothing to do with morals of individuals or how one should behave interpersonally. People who are worried about alignment issues are worried about the danger unaligned AI poses to humanity; the harm which can be done by some super-intelligent system optimizing for the wrong outcome. There is nothing "ironic" about being worried about that while also being an asshole any more than it's "ironic" for someone concerned about, say, climate change and also be an asshole. People who are afraid of unaligned AI aren't afraid that it will be impolite.
I'm tired of people pretending that pointing out imaginary hypocricy is an argument. If you want to complain that someone is being mean, just do that. Don't pretend there's hypocricy involved.