"The researchers hypothesize that GPT-style generative models, and similar types of language generation frameworks, have been used to produce much of the text in the flagged papers; however, the way that a generative model abstracts its sources makes this difficult to prove, and the chief evidence lies in a common-sense evaluation of poor and unnecessary synonyms, and a meticulous examination of the submission’s logical coherence."
This sounds extremely unlikely.
First, scientific fraudsters are typically extremely lazy; they do not use fancy methods like writing generative models of fake data to ensure the numbers, say, add up to 100%, much less neural nets. (Do we believe that someone would go to the effort of setting up GPT to generate fake papers and then be satisfied with results like 'profound neural organization'? Easier to write such garbage by hand!)
Second, the given examples are so bad I'm insulted on behalf of GPT: 'profound neural organization' looks like a word-by-word calque using a thesaurus or particularly antiquated machine translation (in a "the vodka is good, but the meat is rotten" way), no contemporary AI involved; GPT would not generate such garbage.
Third, if GPT was generating these papers, it would be easy to detect by looking for top-k signatures, using GPT-targeted classifiers (they get >90% accuracy), or looking for stylistic tics of GPT; that they don't provide any specific GPT-like aspects of the text suggests there are none.
"Do we believe that someone would go to the effort of setting up GPT to generate fake papers..."
I'd say it depends on what their desired outcome is.
If they know that other people are using AI to assess the literature about a particular topic, then the content only needs to be good enough to pass that analytical AI's sniff test and bias the corpus in the way that the owner wants it to.
If the owner is lazy, perhaps that affords some time against this kind of pollution. But the costs and level of effort involved will continue to reduce.
The examples of "failed attempts at creative synonyms for known phrases in the machine learning sector" seem the output of translating the text multiple times (Language1->Language2->Language1), more than using a language model.
Why would GPT-3 or another language model use "profound neural organization", which is a non-existing term that it has never seen during training?
Here in Argentina we use a lot of loaned words from English for technical terms, sometimes I don't know which is the official translation to Spanish. But in Spain, they like to translate everything, so sometimes reading technical stuff in es-es is difficult. And if you blindly translate from es-es to English, you may get surprising results.
I sometimes write an early draft of a paragraph in es-ar, and then autotranslate it to English. The autotranlation handles the easy part, like word for word substitution and reorder noun-adjective to adjective-noun, but it needs a lot of further fixes to get a good translation. Technical words are sometimes messed, so it's important to double check them. Also, errors that are small typos in Spanish sometimes are translated to very things, that also cause weird results. (For example "cOsa"->"thing", but "cAsa"->"house".)
ant colony: ‘subterranean insect (state | province | area | region | settlement)’
It's so clinical and literal. It reminds me a lot of the way Overwatch phrases things in Half-Life 2. For example, when you use a swarm of antlions (giant alien insects) to stage a prison break, Overwatch says
"Overwatch acknowledges critical exogen breach. Airwatch augmentation force dispatched and inbound. Hold for reinforcement"
"airwatch augmentation" sounds much more ominous than "air support".
I would be interested to see what GPT-3 could come up with if explicity tasked to do this type of re-phrasing. What happens if you ask it to pretend to be an alien intelligence?
> I would be interested to see what GPT-3 could come up with if explicity tasked to do this type of re-phrasing.
First, this is definitely not GPT-3, see my other comment. However, as one would expect, GPT-3 is capable of all sorts of fascinating 'rewriting' tasks. GPT-2 could do summarization via 'tldr' but GPT-3 can so so much more!
I haven't tried but I would expect the literary style parody prompts to work just as well if you dropped in "alien robot" or something like that, giving you text style transfer.
Which uses whatever AI the big translation services use.
What's surprising is this is many years old on the web but I'm not sure it's been written up?
Massive 'news' blogs use it. Most stories will get a translation spam version.
Basically it allows copying without obvious copyright issues and might get great SEO. Not sure how on top of it Google is, some articles are saying Google can't detect it.
Even if Google is on top, Bing mightn't be able to find Google translator spam without Googles help/breaking Google's TOS
we could just move that information onto a site where scientists have to verify their identity, we could also build a science blockchain and verify identities that way as well
This sounds extremely unlikely.
First, scientific fraudsters are typically extremely lazy; they do not use fancy methods like writing generative models of fake data to ensure the numbers, say, add up to 100%, much less neural nets. (Do we believe that someone would go to the effort of setting up GPT to generate fake papers and then be satisfied with results like 'profound neural organization'? Easier to write such garbage by hand!)
Second, the given examples are so bad I'm insulted on behalf of GPT: 'profound neural organization' looks like a word-by-word calque using a thesaurus or particularly antiquated machine translation (in a "the vodka is good, but the meat is rotten" way), no contemporary AI involved; GPT would not generate such garbage.
Third, if GPT was generating these papers, it would be easy to detect by looking for top-k signatures, using GPT-targeted classifiers (they get >90% accuracy), or looking for stylistic tics of GPT; that they don't provide any specific GPT-like aspects of the text suggests there are none.