

The Wise Way to Crowdsource a Manhunt - scholia
http://www.newyorker.com/online/blogs/comment/2013/04/reddit-tsarnaev-marathon-bombers-wisdom-of-crowds.html

======
zeteo
So the "wisdom of the crowd" turns out to work best when it's not allowed to
be a crowd. Taking independent votes from non-communicating individuals is in
line with Surowiecki's original example (estimating the weight of an ox), but
far from the common understanding of a "crowd" [1] or from Reddit's core
mechanisms. These latter are more about emotional involvement and early
consensus (both of which seem to be harmful for the accuracy of conclusions).

Infrastructure operating close to Surowiecki's specifications does exist:
Mechanical Turk. I'm surprised he's not mentioning it. After some initial
hype, its actual uses have turned out to be far less glamorous and
revolutionary than originally hoped for.

[1] <http://en.wikipedia.org/wiki/Crowd#Terminology>

------
joekrill
Everyone is quick to blame _only_ Reddit -- and they certainly deserve a great
deal of it, collectively. But I think we should look at laying part of the
blame on the professional investigators themselves, too. They clearly knew who
they were looking for long before they released any useable information, and I
think they should have taken more initiative to explicitly eliminate Reddit
suspects they knew were not involved. They could have easily issued a brief
statement definitely stating they cleared Sunil Tripath, for example, without
releasing information on the identities of the actual bombers. Same goes for
the two innocent guys that were featured on the front page of the NY Post.

The investigators did a terrific job, and I don't want to put down anything
they did. And maybe I'm being naive and this would be a slippery slope. But I
still think some of these specific false accusations could have been mitigated
very easily.

~~~
ProblemFactory
I don't think that's a workable solution.

The professionals had identified the brothers as suspects - but only as very
probable suspects. They could still have been wrong. Publicly definitely
clearing someone who might later turn out to be the actual bomber or an
associate would have enormous political (and perhaps legal) consequences.

I believe that any form of crowdsourced investigation is reasonable only if
there just isn't enough professional interest or resources to pursue a case.
For example finding your stolen laptop. If there are thousands of FBI agents
working on a priority case, with additional private data feeds from security
cameras, telephone logs and wiretaps, public "help" is of no use beyond
stroking the ego of the participants.

------
kanamekun
The Snapshot Serengeti project is trying to do something along the lines of
NASA Clickworkers: to use groups of amateurs to "identify the thousands of
animal photographs collected from 225 motion-activated cameras in the nature
preserve": [http://connect.dpreview.com/post/3745415348/sanpshot-
serenge...](http://connect.dpreview.com/post/3745415348/sanpshot-serengeti-
crowdsourcing-zoology)

It's a lot of fun and it feels great to contribute to a noble project. Plus
it's surprisingly addictive: <http://www.snapshotserengeti.org/>

~~~
incision
Very nice, hadn't heard of that one. I've spent some time on SETILive [1]
which does something very similar for signal classification and found it to be
almost meditative.

It's not quite the same thing, but also I get the same kick out of foldit [2]
and eterna [3].

1: <http://setilive.org/>

2: <http://fold.it/portal/>

3: <http://eterna.cmu.edu/web/>

------
nsns
I think it is worth mentioning that once the bombers were identified, Reddit
did an amazing job covering the manhunt in real-time, I think it was a first,
and a great achievement. With live reporting, pictures, videos and police
scanner streams, it almost felt like actually being in Boston, and a new form
of journalism.

~~~
Androsynth
reality tv + vigilantism = a new form of journalism?

im being glib, but there are some serious ethical journalism questions that
deserve to be brought up. First of all, does every step of the investigation
need to be broadcast? Is this reality tv (in internet form) or actual
journalism? Can rampant vigilantism on the internet be stopped? Can real
journalism prevail in the face of the instant-gratification ppl can get on
reddit? Do people want real journalism?

The redditors are basically using the scorched-earth method of journalism,
with no regard to who they accuse. Theyre causing considerably more harm than
help, which is typical of vigilantism.

An interesting question is this: why is having reporters in vietnam a good
thing while redditors following every move of the bombing investigation a bad
thing (considered so by some of us at least)? (please dont answer that with a
smug, poorly thought-out answer. sometimes its better to think things through
and not just go straight to whipping out the epeen)

~~~
darkarmani
> The redditors are basically using the scorched-earth method of journalism,
> with no regard to who they accuse. Theyre causing considerably more harm
> than help, which is typical of vigilantism.

What? The difference is that redditors don't claim to be journalists, so why
would anyone put any weight in their accusations? No one was lynched by
redditors (or in danger of being lynched), so let's not call this vigilantism.

~~~
justin66
> No one was lynched by redditors (or in danger of being lynched)

Anyone who thinks that Salah Barhoun's life wasn't in danger for a few hours
there just hasn't thought things through. This country is littered with small
people with guns who would love to assassinate the newest boogeyman.

~~~
analog
This is absolutely true. But he was only put in danger once the Fox News
affiliate broadcast the image (purportedly from the FBI) of Salah and his
friend.

~~~
justin66
Fox (and, indeed, at least one other Rupert Murdock property: the New York
Post) didn't help, but if I were in Salah's place I'd be most concerned about
the ar15.com nutjobs who were trolling him:
<https://news.ycombinator.com/item?id=5573281>

In any case, no number of intermediaries would absolve "the subreddit" of
responsibility if something went tragically wrong as result of their actions.
Mob psychology and the diffusion of responsibility certainly makes people feel
like it would.

------
podperson
The first key point of this article was that the footage showing the actual
bombers wasn't released until the FBI identified them as suspects, therefore
the "crowd" was on a wild goose chase.

Also pointed out in the article, having the "crowd" incorrectly identify a
suspect isn't the big problem, the police are perfectly capable of persecuting
the wrong people without the help of a crowd.

------
brown9-2
A lot of this article's advice refers to Reddit as a singular entity, as if
Reddit The Company was behind the online investigation:

 _For example, in a situation similar to the marathon bombings, Reddit could
simply post the available images without comment (save, perhaps, for time
stamps and where the pictures were taken), and allow users to sift through
them, looking for both incriminatory and exculpatory evidence (like, say,
people carrying black backpacks near the bomb sites, or still carrying them
once the bombs had already gone off), while creating a voting system that
would let people independently vote on which images they thought were worth
further investigation. Once a critical mass of users had perused the images,
Reddit could aggregate the results, and pass them on to the authorities. (The
results of the voting wouldn’t even have to be made public, if people were
concerned about false accusations.) This would take advantage of the crowd’s
ability to filter through lots of images quickly, while also preserving the
diversity and independence that make crowds smart. The point of this wouldn’t
be to compete with, let alone replace, the work of official investigators, but
rather to provide, as it were, another set of eyes, one that might see
something that investigators missed.

If Reddit were looking for a model to follow, it could use NASA’s Clickworkers
experiment..._

For any of this to work, "Reddit" would need to be an organization with a top-
down command and control hierarchy, not an online forum where any user can
create a new subforum at any time.

In this idealized world of Reddit investigations, who is overseeing the crowd-
sourcing and handling the aggregation?

------
danso
This was one of the most insightful, thoughtful pieces on crowdsourcing that
I've ever read. Reddit's an easy target to bash, but the OP is right that the
expectations, on both sides, were too high given the disparity of information
between the crowdsource and traditional investigators. Until there's a
situation in which both groups have roughly the same access to available
information, there's not much point in arguing which way is more effective.

This, of course, is a different argument than whether crowdsourcing, in its
pursuit, ends up causing too much collateral damage. The voting scheme
proposed by the OP seems like a possible way to mitigate the herd
mentality...but I also agree with the OP that the feeling of community is what
makes Reddit so engaging, for better or worse.

~~~
maxerickson
I wasn't watching terribly closely, but it was a bit disappointing to see
people circling backpacks, rather than say, establishing a set of
photos/videos that were in close proximity to the bombs, in both time and
location.

Much of the time, the context people were putting around photos they were
linking was observably false (that is, contradicted by glancing at the
photos). Stuff like saying a backpack was gone, when a strap was clearly
visible...

A WaPo article [1] about the bombings mentions that negative consequences from
internet speculation factored into the decision to release the photos of the
suspects.

The potential to do some good certainly seems to be there, but I'm not sure
the efforts to do things in a careful and organized manner will ever be able
to keep up with the excited speculation. Of course, the careful and organized
effort may still useful if it can be separated from the excited speculation.

[1] [http://www.washingtonpost.com/world/national-
security/inside...](http://www.washingtonpost.com/world/national-
security/inside-the-investigation-of-the-boston-marathon-
bombing/2013/04/20/19d8c322-a8ff-11e2-b029-8fb7e977ef71_story.html)

~~~
stordoff
>Much of the time, the context people were putting around photos they were
linking was observably false

The most obvious case I saw of that was the number of times photos like
this[1] were posted (and others of the same people without one of the
backpacks) as 'proof' of their involvement, despite one of the photos being
from after the explosions[2]

[1] <http://i.imgur.com/GZgmb3L.jpg> [2] (warning: shows bomb site. Not too
graphic, but some may find it unpleasant) <http://i.imgur.com/iVzG7MK.jpg>

------
kstenerud
I for one am glad that the Reddit effort turned into the fiasco it did.

Criminal investigation is a profession. Asking a bunch of amateurs to come up
with a solution is akin to asking a bunch of amateurs to build an air traffic
control system. Sure, there might be a few people in the crowd who can
actually do it, but most people can't. Worse, they're likely to suffer from
the Dunning-Kruger effect. And with everyone given equal voice regardless of
ability, and a popularity driven forum, you have a recipe for disaster every
time. The only difference between this and Salem is the crowd size.

The FBI played it right by not releasing anything important and letting the
public have its little playtime. It's unfortunate that this caused some
innocent people to be flagged by the mob, but I think that given the
circumstances of our more connected lives, this was the best play to make in a
bad situation. The mob would move regardless, so why not slow then down by
sending them chasing their tails for a bit while the real professionals got
down to business?

~~~
darkarmani
> I for one am glad that the Reddit effort turned into the fiasco it did.

I don't know if i really believe that it was that much of a fiasco. All the
photos i saw where pictures of people with backpacks near the area of the
bombings. That definitely gave the FBI leads.

~~~
kstenerud
Meanwhile every person singled out by the mob was quickly alerted to that
fact. A big part of professionalism (and law enforcement in particular) is
knowing when to be discreet.

------
sp332
The "clickworkers" comment at the bottom reminds me of
<https://www.zooniverse.org/> which is still ongoing. You can sign up, and
they quickly train you to sort and catalogue various kinds of images including
moon craters, weather patterns, galaxies, etc.

------
Vivtek
This is a thoughtful article, except for the basic premise that if Reddit
wants to be useful at crowdsourcing of this nature, it needs to be something
other than Reddit.

Reddit is a social forum. It allowed people to feel part of things, and that's
great. Expecting your hammer to be good at driving screws, though? That's just
doomed to failure.

Moral of this story for me: just because you have a crowd _doesn't mean you're
crowdsourcing_. Use tools appropriate to the task and you might actually get
results.

Sure didn't stop the national media - who actually get paid to know better -
from blowing Reddit crowdsourcing up as some kind of magical deus ex machina,
though.

~~~
pseut
That was the last paragraph of TFA:

 _The problem from Reddit’s perspective, of course, is that this method of
sleuthing would be far less exciting for users, and would probably generate
less traffic, than its current free-for-all approach. The point of the “find-
the-bombers” subthread, after all, wasn’t just to find the bombers—it was also
to connect and talk with others, and to feel like you were part of a virtual
community._...

Your last two sentences are points made by the article.

~~~
Vivtek
No, I think these are afterthoughts. The point of the article as I read it is
that Reddit needs to change, that these effects are only due to its current
design. But I think a saner takeaway is that Reddit is simply the wrong tool
for crowdsourcing.

------
sodomizer
"this method of sleuthing would be far less exciting for users, and would
probably generate less traffic, than its current free-for-all approach."

When people do things for their own entertainment, they're always thoughtless.

When they do them out of deep conviction that they're contributing something
positive, as in the open source movement, they are more effective.

Ironically, terribly and not-funny-at-all, the Boston bombers were working
with the latter mindset. This enabled them to put a city on standstill and
terrify a nation using bombs made from household items and black powder
(normally considered too slow to be useful as a modern weapon).

------
macspoofing
I do have to say that for an internet witch hunt, this one was very
restrained.

------
salman89
I thought about this too, ideally you would want to use crowdsourcing to just
gather evidence and let professionals who are also liable to sift through the
evidence. Another angle to work is to allow the crowd to analyze the evidence
but not make that analysis public. Problem boils down to incentive - if your
contributions are not made public and there is no attribution, is the
motivation still there? 4chan is probably more interesting to look at in this
regard, since all posts are anonymous.

~~~
prof_hobart
Some of this could potentially be solved by having the crowdsourcing
contributions filtered/moderated by the professionals (e.g. not getting posted
publicly until a pro has read it, and "voting" done or at least swayed, by the
pros).

The question is whether the time spent doing this moderation would be better
spent actually doing the investigations themselves.

------
juskrey
Rubbish in == rubbish out. Crowdsourcing or not.

Exposure to bad photos and news is not the same as exposure to crime scene
(and police data).

------
corresation
In the not so distant future, visualizing and "walking through" the past --
ala the movie Deja Vu (but minus the wormholes) -- will become a reality. It
was the lack of even a rudimentary version of this that made the Reddit
sleuthing so hopeless, with most media having zero context, location, or even
time taken, that metadata not available.

But imagine photo synth/streetview with a time slider: Thousands of pictures
and overlaid videos time synchronized and placed into a scene. Such a tool
could be incredible for such investigations.

~~~
hornbaker
Google will almost certainly add that functionality to Street View in the
future. The coming ubiquity of always-on Glass-type cameras will turn a
significant fraction of the public into walking surveillance cameras.

