It's amusingly ironic how closely those scenarios resemble the clickbait journalism described in this piece. We have a large, opaque system (the news industry) intended to be helpful (by informing us of important events) but which, due to broken reinforcement mechanism (optimizing for clicks), ends up harming us (with alarmist and inaccurate articles).
Maciej Cegłowski has some really good essays and talks (unlike most programmers who do talks, in his case they're kind of the same, especially if you use the transcripts) that touch on this. They are similar, but have enough variety of angles to be worth going through:
Oh my god, it's even a pay-per-click maximiser. That is just too good.
It's not just the media; this is a problem inherent to capitalism (at least the type of relatively unregulated capitalism as currently practiced in the USA). Charles Stross discussed this at his recent talk at 34c3. The corporation is a "paperclip maximizer" that optimize for "profit", and they already took over and enslaved us.
 among other high-level vision ideas
 (the part I'm referring to starts at ~13:05) https://media.ccc.de/v/34c3-9270-dude_you_broke_the_future
>The automobile industry in isolation isn't a pure paperclip maximizer. But if you look at it in conjunction with the fossil fuel industries, the road-construction industry, the accident insurance industry, and so on, you begin to see the outline of a paperclip maximizing ecosystem that invades far-flung lands and grinds up and kills around one and a quarter million people per year—that's the global death toll from automobile accidents according to the world health organization: it rivals the first world war on an ongoing basis—as side-effects of its drive to sell you a new car.
> I’ll tell you the commercial they’d like to do, if they could, and I guarantee you, if they could, they’d do this, right here. Here’s the woman’s face, beautiful. Camera pulls back, naked breasts. Camera pulls back, she’s totally naked. Legs apart. Two fingers, right here and it just says, “Drink Coke.” Now I don’t know the connection here, but goddamn if Coke isn’t on my shopping list that week.
-- Bill Hicks
Oh yeah, it's always "us as society" when things get pushed by actual individuals working at specific companies with whatever means they can grab, to whomever they can push it on, side-effects and consequences down the road be damned. "We as a society" decided Colin Powell should lie his butt off in his presentation about Iraq before the UN.
Never mind car crashes, if the people who get killed in wars of aggression saw more than the raw fact of them having being tortured and/or murdered, if they just saw the profits for select individuals, and the way faceless masses cheer "liberty and convenience!", they'd be more than cool with it, that's obvious.
A society is made up of acting and thinking individuals, and referring to decisions made "as a society" is not really from that realm. Start with your own position, or with why you don't have one in favor of such abstractions that hold no water.
>Anything that good hackers would find interesting. That includes more than hacking and startups. If you had to reduce it to a sentence, the answer might be: anything that gratifies one's intellectual curiosity.
>Off-Topic: Most stories about politics, or crime, or sports, unless they're evidence of some interesting new phenomenon. Videos of pratfalls or disasters, or cute animal pictures. If they'd cover it on TV news, it's probably off-topic.
Do to this "maximizer function", including comment guidelines, many people refrain from adding noise to subjects they don't have much to say about. HN has lots of readers on each story who do not comment, because they have nothing to say. Cf Reddit, where having nothing to say has never stopped anyone.
Do you think HN's guidelines, moderation, and culture, have any unintended side-effects?
EDIT: The votes on this are weird :) First it got voted up then down. To be clear, I don't have any specific expectation of what your response might be! It's a totally open-ended question. I'm just interested in your thoughts,
mundo, for the reason I cited (that I think you make a subtle and interesting observation). Really no agenda here! :)
I actually think HN is fairly immune to clickbait, and a big outlier in terms of quality of discourse, which is why I'm here. Unfortunately, I think the reasons for this tend to be unique to HN's weird status (guerilla marketing for vc firm? rich guy's vanity project?) and not very transferable. For example, I think the excellent moderators are a big part of the reason this place is so good, and I also think that most people comport themselves better here because they know luminaries in their chosen industry might be reading, but "just hire paid moderators and then get famous people to join" is not very actionable advice for anyone who is not already themselves wealthy and respected.
Another analogy that has been on my mind the blind men and the elephant parable, where no single blind man can have a full picture of what the elephant is.
> Analyzing the reward function and changing the parameters of an experiment is NOT the same as “unplugging” or “shutting down AI”. If that were the case, every AI researcher has been “shutting down AI” every time they kill a job on a machine.
How long until this is sensationalized as "Brave Scientists Kept Skynet at Bay for Decades"?
story evaluate ai wrong i i
no this story ai ai bad bad wrong wrong i i i
next story ai ai ai bad bad bad bad wrong i i i
Or better still, the AI trying to understand human language by modelling how many scary adjectives will be packed into the next iteration of the story and how many more clicks it will garner as a result.
EMACS used to have a command called M-x psychoanalyze-pinhead, which generated a conversation between Eliza (M-x doctor) and Zippy the Pinhead.
Along with pointing the finger at "media sensationalism" I think we should also implicate the strong fear/desire we have towards achieving a strong AI. If anything looks like it could belong to a strong AI, holy crap, could this be the start of a strong AI??? I'd draw a parallel at how eager we are to hear that there might possibly be some sign of life on another planet. Perhaps this sort of confirmation bias comes from our sense of loneliness as a species.
Exactly! However, there is one thing common between those clickbaity articles and the language they claim AI bots invented: "Nonsense"!
The root of the problem is that most tech writers aren't actually technical experts at all, let alone engineers with a degree in the field they're covering. So while they love to look down their noses at the mainstream media's clumsy coverage of scientific and medical advancements, these same "tech journalists" can routinely be found making the same kinds of inaccurate and outright misleading claims.
Part of our new project Skynet Today - Accessible and informed coverage of the latest AI hype and panic.