
Darpa Wants to Build a BS Detector for Science - mcone
https://www.wired.com/story/darpa-bs-detector-science/
======
pmilot
In the fourth paragraph, Dr. Adam Russell (the interviewee) is quoted as
saying: "I wouldn’t characterize it [as a bullshit detector], and I think it’s
important not to". Then the author titles his article "DARPA WANTS TO BUILD A
BS DETECTOR FOR SCIENCE".

That's not just weasel-y journalism, it's downright disrespectful to the
person you're interviewing. If I was in Russell's shoes, I'd be insulted.

~~~
epistasis
In my experience it's completely par for the course in science journalism. At
least his contrary quote was included!

Assuming that he actually said that quote or something similar to it. It's
really hard for me to trust quotations in articles after what I've seen.

------
Animats
The intelligence community works fairly hard on this, with only moderate
success. But they have active opponents trying to fool them.

One of the CIA's checks is to ask of information that confirms other
information, "is this from a different source, or is this from the same source
via another path"? That's a problem that could be addressed on the web, and is
helpful at detecting "fake news" and hoaxes. A graph of which stories are
derived from what other stories, and when, gives a good sense of where
something came from.

Google is terrible at finding provenance. They're likely to list the popular
clickbait site, even if it's a scraper site, rather than the actual source.
Sites which are listed as "news" get special treatment, but Google doesn't
find the original news story, rather than its indications. Google crawls the
web so frequently that it could get a handle on provenance from timestamps
alone.

------
GFischer
Clickbait title, but the issue of reproducibility on social science studies is
well worth investigating.

What the article wants is something that I'd say couldn't and shouldn't be
done (in terrible writing: _" some sort of machine with, like, a slot for
feeding in journal articles. And two lights on the front: red and green. Ping
or bzzzt._)

What the investigators say they're working on goes from good suggestions (
_pre-registering a research plan (to ward off accusations of P hacking) and
making full sets of data and the code used to analyze it available_ ) to bad
ones ( _Social networks_ ) - if you're an outsider like Perelman you're out.

------
hyperion2010
Building a bullshit detector is easy. Building one with a low false positive
rate is hard. A low false positive rate is also extremely important because in
many fields truly novel research findings are often dismissed and it takes a
generation for them to become accepted. See for example quasicrystals. Given
how hard making new discoveries is, and the fact that the peer review process
is already stacked against new discoveries (as opposed to follow ons), I don't
think this is a good idea for many fields. On the other hand, if you know an
experimental methodology is flawed, then you can pretty much throw out all
that research since the scientist didn't do the right experiment or they
didn't do it with enough power (social sciences I"m looking at you).

------
amelius
Why not use crowdsourcing, and add some inputs to the detector? That might
make it more reliable. The crowdsourcing could be limited to peers, which
could increase the reliability further.

~~~
alexschiller
There have been some efforts. For example, there was a betting market on the
psychology reproducibility project, see
[http://www.pnas.org/content/112/50/15343.full](http://www.pnas.org/content/112/50/15343.full)

------
sgt101
Gedankenexperiment: construct such a machine and then create an adversarial
network that attempts to fool it.

What do we do with the examples that come from the adversary?

~~~
TeMPOraL
Slight variation: how about an adversarial network attempting to fool
scientific publishers directly?

~~~
sgt101
I reviewed a paper for a little local conference last week that I think was
probably machine generated, or the person who wrote it was on some very heavy
medication. There was a similar one last year which I also rejected. I suspect
that the team is trying to get them accepted so as to debunk the conference,
which is small, but I like to think reputable - perhaps this is a widespread
effort though?

------
milesward
Read "Truth Machine" by Halperin. Might have broader than intended
consequences.

~~~
joe_the_user
Oddly enough, here's a situation where headline's idea sounds better than
what's discussed in the article.

The idea of a "confidence detector" seems both Orwellian and AI-complete.

But a "bullshit recognizer" seems simpler and practical. A device the
recognizes contradictions in text, conclusions unrelated to evidence or
similar "egregious" things could be useful.

It might even be a trainable thing. You have N papers with M passage-
combinations that show the claims of said paper are bullshit and develop a
finder/recognizer. The problem could wind-up not that different than sending
Watson at Jeopardy.

The drawback, of course, is the thing would have to be taken no more seriously
than a grammar checker - it just serves to point reviewers at potential
problems rather than serving as proof of a problem.

~~~
drunkenmonkey
Wouldn't, for example, set theory, be registered by such a machine as bs,
owing to Russell's Paradox?

~~~
joe_the_user
Well, if the mechanism was smart enough to flag naive Cantor/Frege set theory
as contradictory by formulating Russell's Paradox, the thing would be pretty
smart indeed. Naturally, you know Russell's Paradox only applies in that naive
case rather than Zermelo-Fraenkel or Von Neumann–Bernays–Gödel theory.

However, I'd imagine the mechanism as having more modest goals - flagging the
kind of language that's used to shore up papers with little logical basis,
notice when hypotheses aren't used for conclusion or when conclusions are
hypotheses, that sort of thing.

------
newman8r
This is interesting. I'd think about submitting a proposal for my own projects
in progress if it actually made sense to do so. Anyone with experience in that
area - is it worth the time for a single dev to submit their proposals?

~~~
nl
It's not entirely out of the question.

Some of the other *ARPAs have "proposer days" where you can go and ask
questions.

~~~
newman8r
thanks, I am looking into these. here's one link for anyone else interested
[https://www.iarpa.gov/index.php/working-with-
iarpa/proposers...](https://www.iarpa.gov/index.php/working-with-
iarpa/proposers-day)

specifically, this one looks interesting, with a deadline in the next couple
of weeks [https://www.iarpa.gov/index.php/research-programs/amon-
hen/a...](https://www.iarpa.gov/index.php/research-programs/amon-hen/amon-hen-
baa)

