I think this is mostly fair, but I think it could also be helpful to have AI generated "auto answers" that come with a warning. Like when you are typing in a question SE already tries to figure out if your question has been asked so there isn't a double posting. But what might also be nice is a stochastic answer that says something like "We have generated a possible answer, but be ware that this may not be accurate or to our community standards. If the answer is acceptable please accept and if it is not then reject and we will post your question." I'm sure someone could come up with a better phrasing. This would also de-incentivize people from posting AI generated answers and can make SE adapt to the changing environment rather than reject it outright (we do expect LLMs to get better, even if they are always stochastic parrots).
Would this not be a better "middle" ground? Thoughts?
I think this could be dangerous because a generated answer might be 100% wrong and look correct, yet the question asker might accept it nonetheless. The more this happens, the worse the entire site becomes. I tend to agree with the concept of preventing as much AI as possible, simply because it is much harder to distinguish wrong AI answers than organic ones. Just my opinion though.
You could just make the AI answer private to the asker. That way there's hopefully minimal harm to the repository of answers, it just stops containing questions that are easy enough for the AI to handle.
From the perspective of AI Safety (may get important soon), it is perhaps a very bad idea to allow AI agents to suggest execution/injection of arbitrary code into random places with a review from a single human that needed help from Stack Overflow.
[And don’t believe ChatGPT that claims it is an only a Language Model. It is not. It is a RL Agent trained with PPO.]
One of the first steps of a runaway AI attempting to secure its presence would likely be inference/GPU/TPU compute access. And code injection is a vector. There are multiple other vectors.
But, perhaps considering AI as an adversary is a bad idea. And alignment along the “Love is all you need” lines is the actual solution. Tricky problem…
I was laughing last night when I realized that even if you want to make an AI that only knows and cares about maximizing paperclip output, you also need to teach it to love all humans unconditionally to prevent the apocalypse.
That doesn't solve the problem that GPT-based AIs don't "know" when they're wrong. They will happily spit out authoritative-looking nonsense when presented with questions they don't know how to answer -- and the people asking the questions may not be able to recognize those answers as wrong.
Where scale really comes into play and gets scary is when bots can vote on other bots. Right now, most wrong human answers are pretty rapidly downvoted or corrected. But the supply of humans to make incorrect posts is limited and relatively balanced by the supply of humans to downvote them. Subtle errors in AI posts could become so widespread that it's impossible to counter them effectively.
They'd probably present it differently from human answers.
If people really want to be careless they can get AI generated code from copilot or chatgpt on their own already, I don't think this would be worse than that.
I've banned this account because of the Hitlerian (really?!*) URL in the profile. You can't propagate that stuff on HN. We'd ban an account for doing that in comments, and a profile is no different.
It's a pity, because you've also posted good comments and I think the proportion of good comments has been getting better over time, which is great, but that doesn't make things like the above ok.
Also, you have a history of using this site for ideological battle and we don't want that here—it's not what this site is for, and destroys what it is for.
If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html.
* Edit: on closer look, I can't tell if it might have been a bad joke instead.
Just as someone who's helped people deal with nuanced reasons why exact SQL queries optimize better or worse under various scenarios, a hundred or more times on S.O., I think the reason they turn for human advice is that they place more value in an answer to their specific question that can't be generated by AI.
I think this is a great idea, especially if the AI answer is tagged as such on the site. That way, future AI researchers will be able to avoid pulling those answers into their datasets and creating a feedback loop or whatever.
Think about Copilot taking code from GitHub. If you start letting it contribute potentially shaky code back to GitHub, it's going to learn more incorrect patterns on its next harvest. It's a feedback loop. Anyone who wants accurate AI answers would not want the AI itself to contribute to the learning pool. The same is true for S.E. and ChatGPT. ChatGPT doesn't understand why something works or doesn't; it just infers the next word or sentence from the previous language. It's not factually accurate, but it draws from material that contains facts. If it's allowed to contribute back to its source data, it gets worse as a model.
> Think about Copilot taking code from GitHub. If you start letting it contribute potentially shaky code back to GitHub, it's going to learn more incorrect patterns on its next harvest. It's a feedback loop.
ChatGPT (and presumably soon CoPilot) learn via * Reinforcement Learning from Human Feedback*[1] on top of the raw language model. This takes low numbers of high quality examples to improve the quality, and avoids the "junk training data" entropy problem you identify.
The OpenAI Codex text-davinci-002 and text-davinci-003 models have been trained for code generation using this human feedback process[2].
>> ChatGPT (and presumably soon CoPilot) learn via * Reinforcement Learning from Human Feedback*
Technically, they aren't trained that way, their opposite recognition AI is trained that way. That's less than ideal when their original set of training data aren't drawn from their own output, only from a pool of humans influenced by their output. It becomes suicidal to the AI when it begins consuming its own output through what it considers to be trusted channels.
[edit] The ancient coder phrase "garbage in, garbage out" has never been more significant than now in the context of what's fed into neural nets and ML algos. I think it's basically silly hubris sprouting from a lack of underpinning technical understanding when people assert these models will train themselves without much more extreme means of assessing their output than can be provided by a few minimum wage workers somewhere.
> Technically, they aren't trained that way, their opposite recognition AI is trained that way.
No, they are trained that way. These aren't GANs or anything resembling that. There is a reinforcement model that is build from expert human guidance which is then used to fine-tune the LM.
From the HuggingFace explainer page I linked above: [there are] three core steps:
1. Pretraining a language model (LM),
2. gathering data and training a reward model, and
3. fine-tuning the LM with reinforcement learning.
Read https://arxiv.org/abs/2203.02155 if you want all the details (although the HF explainer is easier to follow). From the abstract of that paper:
> Starting with a set of labeler-written prompts and prompts submitted through the OpenAI API, we collect a dataset of labeler demonstrations of the desired model behavior, which we use to fine-tune GPT-3 using supervised learning. We then collect a dataset of rankings of model outputs, which we use to further fine-tune this supervised model using reinforcement learning from human feedback.
Human evaluators can filter out the (for now) intermediate bad results. All of our AI models and products will climb a quality gradient year by year to the point where this will be increasingly less necessary.
To your other point, you can use machine data to bootstrap a "real" model. My team has done this to incredible effect.
You may be right in the long run, but it's definitely a gradient and the capacity for AI answers to be pasted thoughtlessly far outweighs the human capacity to correct it on a site that already creaks under the weight of insufficiently answered questions. I think if you ran such a site, you'd want to stay a little behind the curve of intermediate false but truthy-sounding responses.
What makes you so sure of it?
LLMs are just the novelty of the moment, no empirical evidence yet indicates that it's the direction the industry is going to take, it could just be just another fad like VR or web3.
The only real difference is those answers won't show in Google search. I imagine at some point someone will build an SEO spam site that generates both questions and answers to those questions to flood google with pages for every single thing you might search.
Maybe you could get some kind of browser extension malware that scrapes the things people are searching to use as input.
> The only real difference is those answers won't show in Google search
Give it a month before Google Search is overrun with AI-generated SEO spam (and a year before the next generation of AI has been trained on the aforementioned spam).
One wonders what the overall impact of such systems will be on the internet as a whole. Most (all?) people don't really view things like movie/game/product reviews in the same as they did not that long ago, because there's a greater perception that those reviews are now less than authentic. So they have less value. What happens when internet dialogue itself is seen as unreliable as even being an indicator of "authentic" opinion?
The obvious prediction is that the vast majority of sites will try to ban software generated text (while probably covertly allowing some for motivations from increasing user engagement to fulfilling government "requests"), but another equally obvious prediction is that such software will gradually become much more accessible, including being able to compile/tweak it at home, which means any effort to put the genie back in the bottle is certainly doomed to failure.
We may be living through the end of the era of being able to believe that a blurb, like this one, was actually written by some human somewhere. The implications of this seem nearly as impossible to imagine, as going back 30 years ago to trying to imagine the implications of us being able to post and exchange text/data with each other on a global network.
Interesting times we are living through, seemingly as always.
Before the machines being able to generate cr*p that looks authentic at glance, the assumption was that the screen we look at had human generated patterns. Since this is no longer the case, it's nowhere nearly interesting.
It actually had been losing value since these patters were generated for profit by army of people but with the arrival of the convincing AI generated text, the process has been greatly accelerated.
ChatGPT is very cool when it generates output upon my request, similar technology used for pretending being written by people is toxic. The problem with these AI is that they look convincing but they don't know what they are talking about(like the worst kind of a person). This is not only problem because often its not accurate(people are wrong too all the time) but because it doesn't contain the core reason we interact with people: To change their life and opinions or make them do something. AI can pretend to be different person but all AI out here is the same machine trained on the same things so it's like talking to the same annoying person everywhere all the time. AI can become interesting once it's trained individually like a human AND our input becomes very influential on them(for example if we can get an AI trained Cairo be interested in French literature by telling is something fascinating about it).
I guess the internet 2.0 will die off in spam and content will be created and shared in well moderated communities that can guarantee high ratio of people. WhatsApp or Telegram groups are very popular these days.
I wonder if this kind of ban will become common in most forums.
I like to farm karma as a hobby, and part of my work involves harvesting a lot of the most upvoted comments in discussions and using them as training data to generate new comments that have high potential for upvotes. Eventually this AI can be deployed to build up new accounts that have high karma.
This is a good thing. StackExchange no doubt provides a very large and useful resource for AI training on programming knowledge. Putting answers generated by AI into StackExchange would degrade the training and be quite useless. A separate resource is required for something like that.
>While the AI that generated it could be attributed, the nature of AI is that it breaks down existing writing and reconstructs it, so it's not that simple to name a source.
This is a very weird and muddled description of what a language model does, let alone "AI" in general.
I can see their plagerism angle. After a lifetime of training to not post someone else’s work off as my own, it feels strange copying output of ChatGPT without sourcing it. Though I know ghostwriting and email templates are a thing so I should just get over it
It's a silly rule. As someone mentioned in an answer there, what happens if someone copy-pastes an answer from a site he found on the Internet, and that site happens to have its content generated by AI?
It almost surely is. The "fist, finally", the vague tips, the list of non controversial statements that support both viewpoints. The verbosity and lack of "character".
It's always a shame when content is banned based on who wrote it (or didn't write it), rather than the actual content.
I have a sneaking suspicion that in a few years time, sites that explicitly ban AI content will either reverse their decision or become a thing of the past. AI tools are very quickly becoming accessible to the masses and that lets the masses create more and/or higher-quality content -- and, IMO, that's a very good thing.
But, obviously, established sites always struggle when they suddenly receive a large influx of new users/content, especially when they're at odds with (or completely oblivious to) the societal "norms" already established on those sites.
It's getting quite hard to use basic search to find product review sites or software tutorials that aren't generated (usually badly) by some form of ML. If anything, it seems that people are turning away from search engines that uprank those. Until machine-written articles can be as nuanced and factually accurate (or at least as intriguingly opinionated for weird reasons) as something written by a human with hands-on experience of a product or software, I think they're no different from link mills in the pre-AI era. Which means search engines and sites like S.O. will have an increasing, not decreasing, financial interest in penalizing them.
> It's always a shame when content is banned based on who wrote it (or didn't write it), rather than the actual content.
Yes in theory. In practise in this case specifically the problem is often that it is very difficult to judge the quality of the actual content in and of itself. This is already a problem for stackoverflow, and you will see many highly-upvoted incorrect answers to questions with correct answers languishing. Having AI-generated plausible answers to everything would likely further muddy the waters.
This is sort of bullshit. It's like saying "Witches are banned!" Now for my next trick, purchase my 100% Guaranteed Witch Detector! All Manner of Witches, Covens and Goblins Detected At No Extra Charge - Dial 555-Dewitch Now for Limited Time Complementary Offer. Hurry Only While Stocks Last.
It's just another "StackExchange Mod Tool/Policy" with which to oppress the curious innocent masses. YoUr QuEsTiON is UnCLeAR--said the SPhynX. Then they press the button and you fall through the trap door into Jabba-the-Hutt's underthrone dungeon, of "closed as poorly worded / likely witchery" questions. Ugh...
Will the evil domino of "No Bots Allowed" fall at HN next? "You sounds like a bot. Off with yer head!"
According to dang's post at the top of https://news.ycombinator.com/item?id=33945628, "...HN has never allowed bots or generated responses..." so the domino wasn't standing in the first place.
I know and I think that's bad. I asked dang if we could add the shallow lazy (but all-too common) dismissal, "Sounds like it was written by ChatGPT" to the list of banned behaviors. Did not hear back. :)
I actually think sites like HN would thrive in the AI era. There's 0 incentive to "farm Karma" here using AI. There's no point in being here except having a human conversation.
I mean, unless somebody wants to manipulate people by training AI to spread crypto misinformation, but with the quality of ChatGPT at this point, it's going to be obvious and have the opposite effect.
I think you underestimate the detrimental effect a large quantity of low quality comments have on a forum.
You saw this maybe 5-10 years ago when the "50c army" was at its height spamming specific topics. The good comments were just impossible to find because of all the other junk in there.
Maybe, but I think you all underestimate the extent to which this was already going on en masse before ChatGPT was released to the public. Who gets to access all the juicy new tech before it filters down to the masses? Who indeed? ;) Not saying OpenAI was involved, but this sort of high-level "conversational" tech has been available for a long time in secret.
https://news.ycombinator.com/item?id=33882298