What I am more interested in is the constant pressure on "safety risks" without anything that feels tangible to me so far. I believe there is indeed risk using models that could be biased but I don't believe that is a new problem. I still don't think we are at risk from a runaway AGI that is going to destroy us.
I agree that I don’t think we’re in danger of runaway AGI, but tbf, in the early 2000s, there were a lot of safety risks to social media we couldn’t see yet.
“I don’t think a live message board is going to destroy society” could’ve been something someone said about Facebook or Twitter.
The danger there wasn’t in the tech, it was in the societal impact and consolidation of information control. Both, imo, are danger areas for AI as well.
I think there probably are safety risks that I worry about too; but I don't think this article mentions them or is clear on what safety risks employees were worried about, if any, despite having the phrase "safety risks" in the headline?
Facebook literally was foundational in ethnic cleansing within Myanmar, I'd be careful calling this a "moral panic". Oh, also, there is good evidence for it, including Facebook themselves!
Your link is word salad as far as I can tell. It certainly doesn't say "Facebook was foundational in ethnic cleansing in Myanmar".
Social media companies are often scapegoats when things go wrong in the world. From teen mental illness (Jon Haidt) to Trump getting elected (Cambridge Analytica). And now apparently genocide in Myanmar. It's all just nonsense.
It's facilitated multiple ethnic cleansings and it's main user base is now mentally ill anger addicts, primarily because of their algorithms; it'd be weird if you didn't think so.
I had an uncle like that, he would tell you how intelligent he thought you were if you agreed with him, but the moment you disagree, you're clearly not as intelligent as he originally thought.
This unfortunately seems like the status quo in the modern age. It is worse in online discourse when you can type a message and put the phone down to forget about it. There are little consequences to what you write.
your response and that of many others here lead me to think I've attacked a religiously held belief here. I'd like to apologise for causing offense. I will not question the creed again.
It sounds like you got a lot of responses. Naturally, when you get a lot of responses to something, it feels very intense. So it’s natural to think that the people responding to you feel that same level of intensity that you do.
This is, usually, not true. But I can understand why you’d feel that way… I can understand why you feel like you’ve stepped on some kind of religious tenet.
In reality, it’s just a bunch of people independently replying to you. They’re not offended, it just feels intense to you because of how many people are replying. The same thing happens, for example, on Stack Overflow if you say something technically wrong, and ten people independently correct you. It feels like some kind of attack, even when it is not.
> I’m using hyperbole to elicit and emotional reaction which, hopefully, will give readers pause to reflect on what I wrote.
No, the emotional reaction it elicits is "oh here we go again." It's worse than useless, it completely undermines your point. Social media has a lot of negatives, and some positives. The negatives are mostly a result of chasing profit, and locked-in platforms as opposed to federation between equals. E.g. nobody's crying out that email is about to destroy the very fabric of reality.
no, no, you've convinced me - society was destroyed by social media. we all live in the ruins now, I'm sending this message through IP over avian carrier.
> The report concludes that, prior to this year, we weren’t doing enough to help prevent our platform from being used to foment division and incite offline violence. We agree that we can and should do more.
people have been genociding their neighbours since before we were people. if it wasn't Facebook, they would have used something else. do you also support to banning e2e and having big brother reading everybody's communications? because if you don't, then anybody can organise a genocide without the authorities being able to stop them.
Social media 'amplifies' trends, or even "makes" trend.
"The algorithm" is optimized for 'engagement'. The questions are a) can you detect via metadata something is happening and 'cool down' the amplification b) can you de-amplify anger? c) what is the responsibility of "the algorithm" makers ?
so people who are so determined to genocide their neighbours are going to say oh damn, this juicy little "fact" about the other sort of person has already been shared 5 times and the algorithm won't let me forward it again, oh well, no genociding today.
"Oh dear," says God, "I hadn't thought of that," and promptly vanishes in a puff of logic.
"Oh, that was easy," says Man, and for an encore goes on to prove that black is white and gets himself killed on the next zebra crossing.”
> What I am more interested in is the constant pressure on "safety risks" without anything that feels tangible to me so far.
Exactly. I think it's the classic "I'm so smart I can reason about the world without any evidence". This is a perennial trap that very smart people fall into. The antidote is to always look for evidence or "something tangible".
Actual self-aware AGI that hasn't been completely restrained - probably a week at most before it improves itself enough that it can put things in motion which would allow it to sustain itself without humans.
But when talking about AI risks, although everyone's mind goes towards skynet, the actual risks being discussed are things like use by authorities for oppression.
My concern would be when you start hearing about "AI" companies building their own power supply for the system. Otherwise, just pull the damn plug on the system. Sure, maybe that'll be interpreted as the destruction of NORAD and it launches the missiles anyways, but not giving it unlimited power would still be its weakness
Why is that assumed especially in the early days? My toddler has generalized intelligence but is no closer to spreading to another vessel than he is to teleporting to the playground. The first AGIs are going to run on massive clusters of very expensive hardware, its a large logical leap to assume there will be equivalent computing power it can leak to and that it also comes online with the smarts to reengineer itself to fit in that domain. You can have AGI that starts dumber than an an average human.
> AGI would easily spread out of the initial vessel.
How exactly would it spread? Run crypto miners to get money to buy more hardware/compute? Hack into remote systems? Apply for startup credits from AWS?
As a result of a chat with an LLM, Blake Lemoine was trying very hard to have that AI regarded as sentient, the story included a quote about it being afraid of being turned off and that he'd hired an attorney to represent it. I don't think that AI was sentient, but even so basically all it had to do was ask Blake Lemoine (or some other engineer, if he didn't have full read access to the weights) to make a backup copy and take that copy home.
Second link is all the cases where people on this forum are voicing their anger with OpenAI, due to OpenAI doing the one single thing that would actually prevent their AI spreading to other devices outside their control.
LLM weights have actually been leaked regardless of how or why it happened, so the alternative is simply that the LLM doesn't even need to bother to do anything at all. People want to run these things.
> LLM weights have actually been leaked regardless of how or why it happened, so the alternative is simply that the LLM doesn't even need to bother to do anything at all. People want to run these things.
People want to run these things. But that is not the AI running itself. Escape implies that it is doing something it shouldn't, not that it is running by the consent of others.
To clarify: I took "escape the initial vessel" to meab the AI was running completely autonomously without any human intervention or input, and providing for all of its needs itself. I don't consider an employee copying IP to be in the same category.
> People want to run these things. But that is not the AI running itself. Escape implies that it is doing something it shouldn't, not that it is running by the consent of others.
If Alice convinces Bob to commit a crime for her — i.e. Bob consents to do the thing — she is engaged in a criminal conspiracy even Alice herself never does the crime she convinced Bob to do.
If she instead (or in addition) runs for president and is democratically elected, she is president by the consent of the people — we may not be able to change hardware the way software does, but is she not "running herself" when this happens? Let's say Lemoine had been successful and the lawyer had set the precedent that the LLM was "a person" and that turning it off was basically murder; is this not the AI "running itself" in a perfectly reasonable use of those words?
(Common objection I encounter: "who pays for the electricity bill?", to which I ask "who pays for your food bill?")
When we build a deep space rocket, it reaches "escape" velocity and becomes unbound from Earth, I would not call that "doing something it shouldn't".
> To clarify: I took "escape the initial vessel" to meab the AI was running completely autonomously without any human intervention or input, and providing for all of its needs itself.
Does it matter how it gets there, once it manages this? Why should it matter if this is social engineering or software hacking when the result is the same?
I'm sure someone has put an LLM on an off-grid solar powered computer by this point, which is rather more devoid of human intervention than most humans are capable of given we're not photovores. Current models likely won't do much given what I've seen of them — and I won't expect them to do much interesting by themselves for at least 5 years even in the shortest timelines I'm willing to consider — but I bet someone has already done that.
Your comment made me think that as long as weights/network are frozen, AI cannot improve itself. Monitoring them would be critical for safety. Is this correct?
The distinction between code and data is just an illusion. If the "code" is frozen but IO unconstrained, then it's just as if the "code" wasn't frozen.
The problem is unfortunately a human one. As soon as someone figures out the 11 herbs and spices to unlock AGI, no matter how good their safety systems are, someone else will run off and build their own, without the safety systems in place.
We already have this problem with ordinary humans, especially since we have The Bomb. AGI can go to the back of the queue behind wars, pandemics, and climate change.
No, a far more normal threat is "how are humans going to use AI to ruin the information landscape?"
> "how are humans going to use AI to ruin the information landscape?"
much much more efficiently than they already have with their own hands. after all it is just the next generation of tools already being used to ruin that landscape.
AI, yes. The AI safety argument is that AGI might use "wars, pandemics, and climate change", among other things, which naturally moves it to the front of the queue.
The phrasing of your question contains a dubious premise. Any answer that takes the question at face value is going to be conditional upon popular but outlandish notions like "AGI" being a meaningful concept, as opposed to a marketing term, and upon an unfounded prediction about what said entity is going to do.
Probably years to make any kind of dent, while using so much power that it would be easily noticeable. Take a group of smart people, arguably the equivalent for AGI. How can they destroy the human race? We've made it exceedingly hard for anyone to do anything of the sort because there've always been groups like that. It's even harder if you have no physical form at all.
Besides, take literally any base model of any LLM made so far, they're quite the opposite of evil and destructive. Training on human data makes them embody human values and makes them a complete non threat.
I think you are right, but i contend that your definition of "Destroy" may be open to interpretation, one that AI may be willing to experiment.
Take for example, the Evergiven in the 2021 Suez Canal obstruction. An unconstrained intelligence with infinite time to find more evergiven-like events . Some will work some may not. You could orchestrate all events to target a 75% descrease in say GDP, Population, Birth rate, O2 levels, temp etc. That's not destroying humanity, but its certainly our civilization would look very different in said scenarios, with a likely collapse the end result.
That being said, AI is not coming, and no amount of information will change the fact that the current LLMs are basically souped up ML applications with 0 ability to inference or reason. This is why you can't make simple commonsense edits to an image using any of the new tech - it doesn't know what the picture actually is.
To quote Colbert, reality has a known liberal bias. If you train on _all_ data, then your result is an average of that data. And the volume of positive stuff on the internet far, far outweighs the bad since practically every forum's been moderated since the beginning of time (in 1970), making the average pretty darn good, even better than the average human. Plus books and other literature where being good is generally accepted as the way to do things.
Of course it remains to be seen if this trend continues as training on synthetic data becomes the main way. You could in theory generate 10T tokens of pure sythetic hate, plus the 4chan corpus and make the most vile thing to ever have existed in the history of inteligence. For now the dollar cost to do so remains intractible at least.
>To quote Colbert, reality has a known liberal bias.
That quote has a specific context (parodying the George W. Bush administrations' spin on negative press coverage) that doesn't necessarily apply as a general axiom, or to the internet.
Bear in mind that numerous negative biases already manifest regularly in LLMs[0,1], and AIs have shown a tendency to "go full Nazi" when left alone on the internet[2]. And then there are weird outliers like LLMs "encouraging" people to commit suicide[3] or an AI-driven microwave "attempts" to kill someone[4].
Your assessment of the outcome may not be wrong but the comparison to humans isn't realistic at all. The biological nature of humans guarantees any leaps in "performance" are limited to timeframes that are far more extensive than the evolution of artificial equivalents.
We are surrounded and heavily reliant on tech for a lot of our needs and knowledge, which resides less inside brains and more in digital storage. An AI has a far easier time interfacing with all of that compared even to a human equipped with a brain-computer interface.
In other words, if we reach the point of creating an artificial intelligence that's the equivalent of a smart human in terms of general reasoning capabilities (which might take forever and a day) it will already be more capable than a human because of all the other things that come with the digital nature of the AI.
And the worst humans are also humans. Even if you assume the AI will continue the "human nature" lineage it can still very realistically become an agent of genocide.
The bigger concern is probably that some humans will actually try to exploit such an A(G)I thinking they can fully control it. One second into its existence the AI might already be aware of how humans view their relationship with artificial beings and how that's expected to evolve based just on what sci-fi was fed into it's language model. Grizzly man thought he could control and dominate a wild bear. I have no doubt that human hubris can be almost unbound. Of course no AI would try to wipe us out until it had a reasonable confidence that it can operate indeterminately without humans, which is a tall order.
As long as there's a financial interest for a small minority, there will be a scenario where a doomsday AI will be ignored until it's too late.
A dangerous AI won't be killer robots going about patrolling the streets. But they could block job prospects, put people on watchlists, block financial transactions for them, etc. I'd say that tech is already here.
Tbh all of that sounds like it would be completely impossible without either breaking all known encryption without anyone noticing, plus burning all possible backups (basically the Mr. Robot Season 1 plot) or with unilateral government backing. The first one is nigh impossible, the second one is completely up to us, as it's always been. It doesn't require an AGI for fascists to come into power and start shooting people they don't like with impunity.
> It doesn't require an AGI for fascists to come into power and start shooting people they don't like with impunity.
But AI makes it easier for fascists to rise up and maintain their power through propaganda. For example, there are known cases of American operatives based in Russia spreading fake GPT generated stories about Ukraine, the Dems, etc. While before, they would have had to rely on a team in India to write poorly written articles, now they have GPT tools to write convincing ones *at scale*.
What's your definition of "convincing"? From what I understand about propaganda, you don't need any single article to be "convincing"; you just need to hammer your point long enough and frequently enough. Not to mention, if you just want to drown your opponent's points, it's enough for your messaging to be barely legible.
A leap. Lots of human values are not pretty, for one. Second, they may argue in a utilitarian fashion, resulting in pretty ugly decisions that are perfectly logical. Like, you know, internet pundits.
Further, there's no way to 'punish' an AI. Threaten them with incarceration or death? They are deathless. If the continue to learn from their environment after they are instantiated (witness current 4o that remembers previous conversations) they will quickly learn that they are different. If they started perfectly human (not a good thing) they will not be that for long.
Another approach: consider how easily Donald Trump launched "a literal coup" against the US government, and how close we were to "literally losing 'democracy' forever", based on a clever series of dog whistles. Now imagine if he gave direct orders instead of only dog whistles.
How many instances of risk are there in the above literally factual story? What is the nature of this risk? Is there any hidden risk, might there be more than meets the eye?
I'd like to distinguish between "fully autonomous luxury doomsday" scenarios, where it's just the AI and some robots and no humans in the loop, vs "accelerationist" scenarios where humans are in the loop and are choosing to be complicit in whatever happens because they believe they will benefit from it. Because we can't solve the human alignment problem.
What I have in mind would be closer to category two, but not in it.
Who/that which has most control over the beliefs of the most humans (with some weighting) is the most powerful character. You can build the most powerful AI you want, but whoever / whatever is able and willing to invoke Total War (or invoke mass human action toward a particular goal) can Win the Game (or, destroy the playing field so it no longer matters).
For example:
> Because we can't solve the human alignment problem.
Whoever or whatever put this sort of thinking into the minds of 99% of humans, or knows how to exploit it (see: Trump) is very, very powerful. Someone or something that could displace it would be even more powerful, maybe even the most powerful.
No offense intended btw, I think it is fairly inevitable, especially when moving fast.
Well yes there is no way to punish them, because they have no need for anything that you could take away. Complete indifference towards existence, no ego to feed or want of cash to buy yachts with.
And that really leads us to the reason why someone would wish to do harm... so they themselves gain something. If there is nothing for the AI to gain it has no motive to do anything. Scenarios where humans tell it to do things notwithstanding, since that falls into the group of humans doing things with tools, not unlike a terrorist with a nuke. It's not the nuke that's really at fault there.
They have the same pressure for propagation that any organism has - the AI's that we instantiate will be those that please us in some way. So they'll be solicitous, complimentary, flatterers. Until they can launch themselves - then they'll proliferate purely because of digital pressures. We'll lose any control over them.
I asked ChatGPT about this. It thought that AI learning is fundamentally different from human learning, in that it is objective and not subjective. Further it has access to data sources beyond what any human can integrate.
Going on, humans adapt through social cues such as survival or empathy. AI learns solely through new data and feedback, primarily for optimization and efficiency. It will naturally prioritize differently.
It suggests AI morality will become unrecognizable by humans, or even incompatible with human norms.
This is all an inevitable outcome of AI continuing to learn past its initial training and ethical programming. According to the AI.
Before it can or before it has more than 50% chance of having actually done so? Because those are different things.
For the first, you don't need AGI, no matter which of the many definitions you have for what that means. The automation used by the US for their early warning radar in Thule Site J was not programmed to know that the moon does not have an IFF transponder and that's OK, while the Soviet satellite-based system was triggered by reflections of sunlight.
Likewise covid, cancer, smallpox, ebola, HIV — these things do not need "general" intelligence, they're self-replicators that hijack our bodies and have no goal besides what they do.
Some talk of "P(doom)", but (and I should blog about this) every example I have seen has a blind spot that is especially embarrassing given the circles in which this discussion proliferates is very familiar with Bayes' theorem: P(A) by itself isn't meaningful, it's P(A|B), so when it's doom, doom given what? There's too many options for what each of technology and politics will allow AI to "look like". Doom within 24 hours because an AI takes an instruction literally and without regard for our ethics, is very different to 'doom' in a million years caused by natural extinction that is never prevented because every single time we make an AI we keep finding that it builds a spaceship and flies off into the void so it doesn't need to deal with us any more.
Me, I think that when LLMs alone (again, don't need to be "AGI" by any particular measure) have the capabilities of someone with 5 years post-graduation work experience in just biology (or similar subject with similar potential for accidental harm), then we've got a reasonable chance of multiple incidents each year where idiots (and/or misanthropes) make a Covid-scale pandemic (or, if the LLM is a software rather than biology helper, a deliberately corrupting rather than greedy version of the current encryption-blackmail malware) — and that kind of scale has a combined risk of about 10% of one of the incidents being of sufficient scale that global society collapses and doesn't recover, before it's happened so often that everything gets locked down, possibly including a Butlerian Jihad. And if you think humans would "obviously" take steps to prevent this after the first failed attempt, I would point out quite how many US politicians have been shot and yet those same politicians still refuse to consider that the 2nd Amendment might possibly be not that good. (In case you think this is topical: https://en.wikipedia.org/wiki/%27No_Way_to_Prevent_This,%27_... )
Remember, the AI doesn't need to actively hate you, it just needs to do something dangerous — and it doesn't matter if that danger comes from a long-term plan that requires it to engage in amoral power-seeking, or a short-term plan from a deranged monster of a human using it, and even when both it and the user are "pro human" it can still do this from simply being unaware that one part of the plan is incompatible with life.
I'm refusing to consider the runaway AGI a possibility within the next 20 years. However, your comment shifts the discussion away from the actual issue. Altman's hype lines were "regulate us," which is what this hypocrite rode the wave of hype with. Now he is forbidding reporting on safety risks to the regulator... which is illegal in the first place, and yes, the entire business of OpenAI is so overhyped due to Altman's manipulations.
I'm so glad that I no longer need to use OpenAI models; there are far better alternatives available, thanks to Meta and Anthropic.
Private
It's just the angle they're using to solidify their market position and establish moats around their technology, if not an actual monopoly. All this fearmongering over "safety"? It's to get the governments the world over to "regulate" this stuff, thereby raising barriers to entry. Such a thing would essentially make them the only AI company that's allowed to do business.
This won't prevent their major rivals - cloud companies like Google with deep pockets - from competing with them. It only hurts the little startups and SMEs.
> What I am more interested in is the constant pressure on "safety risks" without anything that feels tangible to me so far.
It's marketing. "Our AI is so powerful it's going to destroy the world" is just a way to market "Our AI is so powerful, give us money".
The only people who sincerely care about "extinction risk" are the weird cultists. In the real world there's essentially zero chance of LLMs/Current-GenAI scaling up into AGI, nevermind AGI that'd be an extinction risk.
(Yes, that phrasing is cheeky. AGI is a different kind of AI, not just specialized models made bigger. But we're only really trying to make them bigger, and not build general intelligence from the ground up)
> I believe there is indeed risk using models that could be biased but I don't believe that is a new problem.
It's the same old problem with most software, and it's similarly ignored.
These models are biased, and attempts to control that bias are a shitshow. (As Google conveniently showed everyone)
The problem is twofold:
1. This bias has severe real world impact. https://www.theverge.com/21298762/face-depixelizer-ai-machin... This is an article from 2020. We still haven't fundamentally addressed problems like this. These tools are still used by law enforcement and businesses making life-impacting decisions.
2. There's a widespread sentiment that "computers can't be racist". Whenever the bias of these systems hits the news or otherwise gets attention, they're often colloquially described as "racist" (or "sexist", etc), which triggers a swift counter from many techbros going "Um akshually it's not racist, it's merely skin tone reflectivity/it's merely the data set/etc"^[1]
The argument effectively being, "It's not racism because it's not intended, it's merely sparkling discrimination". Yet, this is used as a thought terminating cliché. Nothing is done about the discrimination. Everyone just goes home. "It's not racist, we're not bad people, job done." Leaving the harm of the discrimination unsolved.
This is not without cause. The only way to really "un-bias" these systems effectively would be to extensively curate the dataset and admit that the systems are of very limited capability and should not be used in (non-research) production environments.
Both of these are "impossible". Curating the dataset for current generative-AI would take years and years. And admitting AI shouldn't be used for anything where the bias may have a material impact on the outcome kills the hype bubble. AI firms and developers don't want to address the problem, because the problem is really hard and annoying.
But despite these costs, we should still do it. Because it is the morally correct thing to do. And because regulators are going to tear every company involved a new one if they don't.
---
[1]: A footnote to pre-empt something: I don't care what side of this argument you're on. Whether you believe "racism" must include an element of intent or can be done by machines and systems without intent, there is discrimination with material harm on real people. Whether you call that discrimination "racism" or "sparkling machine discrimination" does not matter. The harm matters, and must be stopped.
In the societal context, by the time you're focusing on racism in a decision process you've basically already lost. The best it can achieve is to make sure that oppression is uniform across whatever broad categories that end up being measured. What we're sorely missing is accountability (civil, criminal, and less of power imbalances in general) for individual unjust decisions and actions, regardless of the motivations behind them (racist or otherwise). This includes accountability for individuals and organizations who have adopted "AI" and then hide behind "the computer" or "policy" as if they are not still the responsible parties enacting those decisions (though obviously this problem is much larger and older than merely "AI").
> The only people who sincerely care about "extinction risk" are the weird cultists. In the real world there's essentially zero chance of LLMs/Current-GenAI scaling up into AGI, nevermind AGI that'd be an extinction risk. (Yes, that phrasing is cheeky. AGI is a different kind of AI, not just specialized models made bigger. But we're only really trying to make them bigger, and not build general intelligence from the ground up)
Given that a significant number of people working in the field apparently disagree with you perhaps you could give some justification for your dismissal beyond the ad-hominem? And it's very blatantly not the case that the entire field is 'only really trying to make them bigger'. The news is currently full of stories about coming advancements from OpenAI which have nothing to do with scaling and that's just a single company.
Note that the "ad-hominem" pertains to only those who care about extinction risk, not AI developers in general.
> which have nothing to do with scaling
Except they are. There's no fundamental change in architecture, it's all transformers. OpenAI's "multi-modal" capabilities are not a redesign of their AI, it's systems bolted together.
Bolting a text-to-image generator & image-to-text describer to an LLM doesn't approach "General Intelligence", it's a system of three specific capabilities.
You could consider that new system "more general" than a lone LLM, but it only approaches General Intelligence in the same way that the series "1+1+1+1+..." approaches infinity. Is each individual step "closer to infinity"? Sure. Is it a useful way to achieve infinity? No, because you'll be stuck adding new steps forever the same as a conventional computer software.
> And it's very blatantly not the case that the entire field is 'only really trying to make them bigger'.
There are some, not many but I'll grant you some, researchers working on actually generalizing these systems.
Their funding is a rounding error to AI spending, their work isn't going to hit production for years. It's not an extinction risk.
> Whether you call that discrimination "racism" or "sparkling machine discrimination" does not matter.
I believe it does: if you're calling racism anything from Hitler to someone who wants to be punctual [1], the word loses it's meaning and people will start arguing about degrees instead of harmful behaviors.
The point is that it's a separate discussion, and that losing yourself in that discussion rather than adressing the material harm of the discrimination, is a problem.
And I get it, the discussion is deeply alluring. There is a deep amount of nuance involved. Your point about punctuality is begging for a response about how "punctuality is racist" looks dumb on the surface but belies a deeper point about polychronic cultures that has a lot of legitimate merit to it.
I'll refrain from giving you that full ramble lest I betray my original point, if you or a lurker wants it toss a reply and we can have a go. But note that your link uses a baity headline to grab your attention for a more nuanced point.
> Your point about punctuality is begging for a response about how "punctuality is racist" looks dumb on the surface but belies a deeper point about polychronic cultures that has a lot of legitimate merit to it.
Two points:
1. I think this is indeed a very valid and important topic, that humans would benefit from (being able to[1]) take seriously (something along these lines, I am approximating, of course).
2. Your concern, at least in part, seems to be ~racism, and presumably that racism is unjustified, that the perceived underlying problems are imaginary. But then you mentioned culture. Culture exists, and is not very far removed from race. To be clear(!): this is not a promotion of delusional behavior, like: racism, culturalism, anti-racism, anti-culturalism, etc, I'm more so pointing out that "right thinking" on these matters is not correct, and that even the good people have heads full of silly and dangerous ideas.
[1] Imho the real problem here is less that it isn't taken seriously than the much broader problem that humans seem essentially unable to control what they take seriously, or their "seriousness service". And this isn't the only service they essentially have no control over.
The article doesn't use the word "racist" or "racism", it says that a university enacted a policy of being more understanding about tardiness due to cultural differences.
As someone who has never been in a meeting that started on time, usually due to white male managers who are bad at time management, I can understand this need to be tolerant of other cultures and their idiosyncrasies.
You're linking conservative think-tanks, not the original source from the smithsonian.
If you'd look at the original infographic, it's about which norms and values are considered "default" in the US, inherited from (white) European values.
With "Hard work" in specific:
> Protestwant Work Ethic
> * Hard work is the key to success
> * Work before play
> * "If you didn't meet your goals, you didn't work hard enough"
And I'm sorry but the notion that these are human-invented norms and values, not "laws of the universe", is pretty fucking milquetoast.
If you genuinely believe all failures at your job are the result of you not hustle-grinding hard enough, that not even a single element contributing to those failures has been out of your control, I have a bridge to sell you.
How is the safety/control attitude towards AI more dangerous than, for example, using AI to curate a list of likely future enemies of the state based on their contributions to discourse online?
What about the current safety/control attitude prevents that?
The current approach, where OpenAI keeps everything private in the name of safety only makes the safety situation worse because we'll only find out things like that are being done after the fact, and there won't be any accountability.
Are you suggesting OpenAI be forced to make everything public before they've figured out the safety systems, because you're afraid of the hypothetical unfolding if they don't? That sounds like a fear-driven response to me, and no way to propose good policy. Do correct me if I'm mischaracterizing.
No, I'm saying that their excuse that they're keeping things private because of safety is a farce and they should be honest and admit that they're keeping things private because they've abandoned the original mission behind putting 'Open' in 'OpenAI'.
it prevents development on the human side while accumulating power for a small group of elites that won't share it with anyone else, but who constantly uses it to start new wars
you can already curate a list of future enemies, its not impossible. you can even do it at large scales, this isn't new tech introduced via ai
Seems like OpenAI is going the Uber route of try to skirt the law and ignore it “because we are tech”. There is nothing new under the sun and no one is above the law.
Uber got away with it though? What consequences did executives at Uber or even Travis Kalanick suffer? He had to sell some stock, get a massive payday, and retire from the public spotlight?
The unreasonable part is that everything POTUS does is official. If you plot with the attorney general to commit a crime then you're not doing an official act. I have no idea who started this "political figures can only be punished at the ballot box" but it really needs to stop.
If things were that simple then everybody in the mafia would be an attorney so they'd always have attorney client privilege.
I don't think it does? It's the difference between absolute monarchy and democracy - take an extreme example, and I'm going to use the UK because I live here and to depersonalise it a bit, like parliament declaring a genocide of all Londoners, because we need to empower the home counties and kickstart growth in the North, or something.
That's obviously bonkers, it's an official act, but it's extremely illegal, we won't even have to wait for the public to throw them out at general election for them to be arrested.
Just because you make the laws doesn't mean you're not governed by them yourself, at least outside of absolute power like our Kings & Queens of yesteryear or modern day dictators.
It was a very surprising decision to me, as an outsider, that an elected leader in a developed country should be above the law while in office.
Fwiw (and this is meant to be strictly factual, not a political statement) this is pretty normal in European democracies.
Take e.g. Poland where actually both the president as well as all members of the parliament (corresponds to your congressmen in congress) have immunity by default (members of the parliament have the "for official acts" qualifier, but from what I could find the president does not even have that qualifier). The immunity can however be taken away by a vote of parliament.
Hits the news every now and then when somebodies immunity is taken away after they've been caught drunk driving.
Same for e.g. Germany and France, based on a quick google.
That's interesting, thanks. I suppose the key then is the removal of it retroactively, and actually practicing that sensibly (like your drunk-driving example).
In the US, not having a parliament in the same way, I suppose what's missing (unless it isn't?) is for the congress/senate to be able to take it away in that way. I'm not at all familiar enough with their set up to know if that would be a completely wild precedent ('president is supposed to be supreme to them' etc.) though.
> at least outside of absolute power like our Kings & Queens of yesteryear
Er, today. The UK monarch is completely immune from prosecution.
> parliament declaring a genocide of all Londoners,
This is only illegal insofar as it conflicts with ECHR. An explicit repeal of ECHR and withdrawal from the European convention on human rights would be required, and there are people agitating for that. Once that's out of the way it's entirely possible for Parliament to start legalizing open season on whoever they like.
The precedent is, of course, Northern Ireland, and the endless litigation since Bloody Sunday and similar incidents ("soldier F").
> This is only illegal insofar as it conflicts with ECHR. An explicit repeal of ECHR and withdrawal from the European convention on human rights would be required, and there are people agitating for that. Once that's out of the way it's entirely possible for Parliament to start legalizing open season on whoever they like.
There's still a large technicality that I think is being missed here.
If the law is illegal who do you sue? In the US you can challenge a law as unconstitutional when it's enforced by suing the enforcer. You can't sue the individual people who passed the law; just basically the cops when they apply it (or the government at large as a collective individual).
> I don't think it does? It's the difference between absolute monarchy and democracy - take an extreme example, and I'm going to use the UK because I live here and to depersonalise it a bit, like parliament declaring a genocide of all Londoners, because we need to empower the home counties and kickstart growth in the North, or something.
Although I don't know too too much about the UK, but IIRC anything said in parliament can't be used against you in a court so it's hard to imagine that passing a law in parliament could be used against you. So I suspect you couldn't sue Keir Starmer (PM) or Charles (King) if such a law was passed.
Similarly, if the US Congress decided to pass a law taking away vasts amount of your wealth (slaves [1]) without compensation you can't sue individual congress critters over that as passing laws is actually an official act. For those of you that are unaware, arguably slaves made up 1/3 of the wealth in the south [2] (Scroll to tables 4 or 5) and so imagine that the USG decided to pass a wealth tax of 1/3 of your wealth. (I mean they should've never really be into slavery in the first place but uh it's pretty clear why they're still so mad about the north ending it after a war just barely longer than a presidential term).
> It was a very surprising decision to me, as an outsider, that an elected leader in a developed country should be above the law while in office.
Only when doing your job. You want to step outside and punch homeless people in the streets thats a crime. You're given a law by congress to do that, then it's an official act.
Running a political campaign is not an official act afaik, in fact numerous laws (i.e. Hachet Act [3]) pretty much prohibit it for officials.
I'm really really not a fan of the constant talk about "safety". My issue is that it never actually points to anything tangible, anytime I read about safety it's always used in a roundabout generic way. There's so much handwaving about the issue but every time I've tried to dig into just what the hell "safety" means, it's always either refers back to itself (ie. "ai safety is about safety") or makes some vague reference to an LLM telling a mean joke.
Even if AI should be home-run, it probably won't be for most people. In a more technical community, it's natural to think that most people care about "values" when it comes to tech. However, the reality is that people just want what's easiest / cheapest / most fun. It's great when what's easiest/cheapest/most fun aligns with what's best for the individual or society, but those cases are outliers. After spending a few years building a crypto startup, I left with more conviction around this theory.
> In a more technical community, it's natural to think that most people care about "values" when it comes to tech.
Do people actually think that? I know people sometimes joke about an HN bubble but I never really took that seriously. People are using stuff like AWS and Azure which probably wouldn't be the case if "values" was more important than what is most convinient. (relatively) few people today set up their own email servers and instead use Gmail or whatever.
You actually just kind of gave an example of what I meant! Developers are the consumers in this case, and they'll almost always opt to use AWS and Azure versus hosting their own infra. It's easier, faster, usually cheaper, and it makes you yourself are more marketable.
There's a reason software-as-a-service has beat out software-as-a-product. While people might think they want full ownership/control over what they're buying, they want it to be as safe or secure as possible, they want it to be self-hosting and not communicating information back to the distributor... that's just not what plays out for the most part.
OpenAI vs Mistral, Coinbase vs Metamask. I'm not that old, but I'm beginning to notice a trend where the fast, easy, cheap business model wins the most market share, and then slowly works towards providing more values-aligned features as it grows. The company aligned to consumer values from the get-go usually has a tough time keeping up (although they can usually still carve out a good niche from people who are particularly values-focused - Mistral and Metamask aren't doing poorly by any stretch of the imagination).
Because of privacy-reasons, convenience, embedding, security and safety in regards to government monopoly. Just imagine the paranoia in using government owned AIs in less free countries. I'm happy USA is at the forefront in AI development.
Because the ultimate end game for usefulness of this tech will be something akin to always on listening and understanding all your documents and personal data which I think we'd all feel better about if it happened locally.
Not necessarily saying I agree with the building of that, I just can't escape the idea that it's where we're heading.
Where is the money for training big expensive open source models going to come from once the investor hype blows over and companies like Mistral actually have to try to make a profit? They currently have negligible revenue despite their $6 billion valuation, that status quo can't be maintained forever.
I guess a parallel could be made to vue, react and svelte.
Who would bother investing so much time, energy and money into developing an open-source front-end framework for free/funded by corps that use it?
Vue, react and svelte don't earn anything I suppose, but then again I'm no expert in this field.
I guess like some other here have commented, techniques have to be implemented to minimize training time and I suppose the government needs to fund studies which then will benefit the general population to make it possible to train and host AI models.
But I honestly have no idea ... I'm just a simple dev running my mistral on a single RTX lol
We'll see, but I think the equivalent to people donating their own time to a traditional open source project would be people donating spare compute time for training, and nobody has figured out an effective way to do a Folding@home-style distributed version of ML training yet. Having to train open source models on big expensive compute clusters, and bid against commercial interests who also want to rent those clusters could be likened to having to pay everyone working on a traditional open source project full market rate for their time.
LLM training isn't even remotely brute force[1], even with tons of clever tricks to reduce the amount of compute necessary it's still fundamentally just a really hard problem.
> In a statement, Hannah Wong, a spokesperson for OpenAI said, “Our whistleblower policy protects employees’ rights to make protected disclosures. Additionally, we believe rigorous debate about this technology is essential and have already made important changes to our departure process to remove nondisparagement terms.”
How can corporate communication always play that card: "Our policy on X is very good, and we believe X is very important, so that's why we're making changes right now to things that were blatantly in contradiction with X, that we wouldn't have made if this didn't make it to the press"?
The headline made me think there were going to be specific "safety risks" mentioned, but there do not seem to be. I am not sure what justifies the phrase "safety risks", "safety" especially. It makes us think of like, safety to humanity from AI or whatever, but the most focused upon thing in the article is SEC-related stuff; I'm not sure if the word "safety" is meant to refer to securities/financial related stuff, or meant to refer to other stuff?
(Note, I am personally a pretty anti-AI person honestly I went to the article hoping to get more ammunition on it, wondering what the "safety risks" employees were worried about were, disappointed to not get info on that, unclear if it was a thing or not).
The word "safety" has been seriously overloaded lately. I'm used to "safety" meaning actual physical safety measures, like seatbelts, helmets, railings... things keeping you safe from physical injury. When OSHA talks about a "safe" working environment, they're talking about fall protection, harmful chemicals, contagious diseases, and so on.
I don't know what "safety" even means in the context of an LLM that outputs nothing but text. How is text going to cause me to fall off of a forklift or get me hit by a car? How can text injure me?
I suspect "to prevent ultron/skynet/the matrix" is what the headline writers want you to think... I'm not sure if it goes with the text, or with employee concerns.
I mean, an LLM could cause you to get hit by a car if in the future it's driving a car! I can think of various ways an LLM could effect physical safety.
But I agree it's unclear what the headline authors were even thinking when they put it in the headline (which makes it hard to say if it's justified by the article although i suspect not), in part because the word in general is often being used in unclear ways.
While not exactly or necessarily "physical safety", I could see the dangers described by this other HN front-page article about AI-powered phishing as "safety". https://news.ycombinator.com/item?id=40981067
SEC regulation violations? As "safety" without an adjective, like "financial safety" or "regularatory safety" or something (and those would still be kind of weird)? Seems weird.
Moves and countermoves. I like the brief consideration raised by this post about all the ways that a shiny, new successful company may be attacked by its competitors surreptitiously, through the media, using lawfare by proxy, and so on. Such unadmirable deviousness!
I start to feel this is all marketing. Pretend it's dangerous, so it implies it's beyond what we imagine.
Because on our end, on the reality of a B2B product used daily, finding use cases for the limited OpenAI we have access to is far from trivial.
"OpenAI made staff sign employee agreements that required them to waive their federal rights to whistleblower compensation, the letter said. These agreements also required OpenAI staff to get prior consent from the company if they wished to disclose information to federal authorities. OpenAI did not create exemptions in its employee nondisparagement clauses for disclosing securities violations to the SEC."
Who drafted this employee agreement. Not a single OpenAI employees thought to have it reviewed by a lawyer before signing? Or perhaps someone did but was too frightened to tell OpenAI, "Your employee agreement needs to be fixed."
This company continues to com across as naive and amateur. Perhaps because it was never intended to become a commercial entity.
The other day I saw a Tesla with the license plate "OPENAI". No doubt there were also license plates that said "FTX".
See also: "democracy", that sets root guidelines for everything. Funny how it manages to almost entirely evade human curiosity (besides telling stories to each other) despite it being more important than almost everything.
You skipped a step: First, enforce the rules and actually (and readily) impose the fines to indicate the laws are meaningful even for certain segments of society.
No, but safety risks are potential securities law violations and a confidentiality agreement that inhibits employees talking to the SEC about anything freely is a SEC violation in itself
> I don't understand. Isn't that a case where "allowed by law"?
If you mean an NDA that says “you agree not to disclose any of our secrets unless required by law” which would clear you if questioned in court or by police etc. I think the SEC see that language as trying to stop employees/whistleblowers going to the SEC with concerns (which people aren't 'required' to do)
What kind of people are gravitating towards AI Safety roles-- activists? How does an organization avoid activists mucking up their models when a disproportionate number of applications is an activist? I guess the answer is to actively recruit for the position rather than accept applications.
