

The 2011 DARPA Shredder Challenge - jarcane
https://medium.com/backchannel/how-a-lone-hacker-shredded-the-myth-of-crowdsourcing-d9d0534f1731

======
otaviogood
I was on the team that won the shredder challenge. I talked to one of the UCSD
people afterwards and he practically convinced me that my team did it. After a
while I realized that it was so damn easy to sabotage their effort that I
didn't really care if my team did it because that's what happens when you
invite the whole internet to your party. Now I guess they managed to clear us
of that. :) Unfortunately, when this Adam guy wrecked their program, he wrote
them a letter which simultaneously implied that he was our team and the group
"Anonymous" at the same time. Some people at DARPA seemed to believe that
claim (even though I used my real name to sign up) and so DARPA was very
hesitant to announce that we had won the competition. It was a fun competition
though.

~~~
blakecaldwell
I worked with Keith at the time. Wasn't it true that you guys figured out that
the yellow paper had a regular blue dot pattern watermark, and that you were
able to use them to help solve that one? If I recall correctly, DARPA didn't
know about that.

~~~
otaviogood
Yes, one of the keys to reconstructing the harder puzzles was that we found a
yellow dot pattern on the paper. At the time, I was reading HN and I saw a
headline that said something about the government tracking photocopies using
little yellow dots. DARPA had photocopied all the docs before shredding them,
so they all had a very high-res repeating dot pattern on them. We made our
program snap the pieces to the dot pattern. With that in place, the puzzles
came together relatively quickly. Some people at DARPA knew about the concept
of little yellow dots, but didn't think much of it. I don't blame them. The
shredded pieces were so small that it would be hard to imagine any secret
pattern helping with the reconstruction. But I guess it helped enough. :)

------
steven
Interesting response from Harper Reed, who led Obama's 2012 online campaign.

[https://medium.com/@harper/crowdsourcing-isnt-
broken-5681da9...](https://medium.com/@harper/crowdsourcing-isnt-
broken-5681da92b109)

~~~
z3t4
WOW Thanks. There are some very good advice in there. Do you know any more
articles in this area? Or how to manage a community in general!?

~~~
JamesSwift
The Harper Reed article reminded me of this article [1] from a while back
about how StackOverflow handles similar issues. Pretty similar, but maybe you
will get something additional from it.

[1] - [http://blog.codinghorror.com/suspension-ban-or-
hellban/](http://blog.codinghorror.com/suspension-ban-or-hellban/)

------
crimsonalucard
If you think about it, corporations are essentially crowd sourcing machines
with sophisticated recruiting, hierarchies, regulations and rewards. One could
say that corporations are on the far right of the spectrum while the contest
itself with minimal rules, regulations and vetting for participants is at the
far left.

Which is more efficient Corporations or crowd sourcing? Perhaps the sweet-spot
is somewhere in the middle. Just add some rules and structure for vetting the
players or distinguishing the saboteurs and maybe it might work!

------
noobiemcfoob
"Game theorists have found that systems where individuals can build up a good
reputation, are (probably) not as prone to devastating attacks from within.

But wily humans are good at finding their way around even the most secure
digital systems."

/me eyes HN's karma system suspiciously...

------
codingdave
So a crowdsourcing platform that knew it had security holes, but decided to
risk it anyway... got trolled.

Interesting, and a good story, but it hardly invalidates the concept of
crowdsourcing.

~~~
mturmon
The article does highlight that there is an interesting game-theoretic issue
here. Given a crowd of N people, with such-and-such dynamics, is the damage
that 1 person can do bounded or unbounded? Is it better to control the number
of bad actors, or to restrict the actions any one actor may take?

I'm un-aware of results in this area, but as online communities become more
important, and bots become more sophisticated, the stakes will raise. Having
some community models, analysis tools, or heuristics, would be very
interesting.

------
Aardwolf
Does anyone know why a _high end_ shredder would shred the paper in such
regular pieces, and of such size that there are still letters visible on each
piece?

~~~
klenwell
Here's the real-world case involving the East German Stasi that I remember
first hearing about years before this article:

[http://www.npr.org/2012/10/08/162369606/piecing-together-
the...](http://www.npr.org/2012/10/08/162369606/piecing-together-the-worlds-
largest-jigsaw-puzzle)

Why such regular pieces?

"Petter says the shredding machines were under such strain they eventually
burned out... Panic-stricken, the Stasi's agents resorted to ripping up files
with their bare hands."

~~~
dalke
Or as another example, quoting from
[http://en.wikipedia.org/wiki/Paper_shredder](http://en.wikipedia.org/wiki/Paper_shredder)
:

> After the Iranian Revolution and the takeover of the U.S. embassy in Tehran
> in 1979, Iranians enlisted local carpet weavers who reconstructed the pieces
> by hand. The recovered documents would be later released by the Iranian
> regime in a series of books called "Documents from the US espionage Den".[8]
> The US government subsequently improved its shredding techniques by adding
> pulverizing, pulping, and chemical decomposition protocols.

~~~
eru
What about burning the shreds?

~~~
saraid216
It's a good addition to any suite of destruction, but creating any kind of
fire is a higher amount of risk for collateral damage than the other methods.
Plus, unless the combustion is thorough, you may still be able to discern
information off the paper.

~~~
eru
Yes, I meant burn after shredding. If you have time, composting (after
shredding) should work well.

At the very least, you can mix your shreds and soak them in water.

~~~
sukilot
That's pulping.

------
willvarfar
I wonder if self-organisation can help?

If people can reenforce matches, and people who match alike are probable to
see the same board, then the trolls will most match the other trolls and be
invisible to the serious competitors?

I've played with collaborative filtering for sites like HN and I wonder if the
same approach might sort trolls from the serious in these kinds of things?

[http://williamedwardscoder.tumblr.com/post/15581427232/self-...](http://williamedwardscoder.tumblr.com/post/15581427232/self-
organizing-reddit) <\- my blog on collaborative filtering and trolls

~~~
eru
A bit like pull-requests in git?

------
lifeformed
Interesting article but a completely hyperbolic title and paragraph headers.
"The end of crowdsourcing?"

~~~
dang
The best way to complain about a title on HN is to suggest a better title. If
you're right, we'll probably use it.

Edit: We've changed the HN title to a phrase taken (mostly) from the article's
subtitle. If anyone suggests a better one, we can change it again.

The article itself is clearly good and on-topic, so let's not fret about the
title too much.

~~~
jonaldomo
I find it interesting that you do this. I think that the title is a part of
the original work. After all the author did put energy into selecting the
title, just as he selected the words to use for the content of the article. I
suppose the person that submits the link might not be the author, which is
probably what happens the most. I'm still torn.

~~~
dang
Respect for the original content is a high value at Hacker News, but so is
intellectual substance. These two values sometimes conflict, especially when
it comes to titles. When they do, intellectual substance wins.

It has to. Otherwise threads would get sucked into the black hole of arguing
about titles. In that world we would probably be wading through angry
denunciations and counter-denunciations of this one, for example, instead of
pure HN gold like
[https://news.ycombinator.com/item?id=9025587](https://news.ycombinator.com/item?id=9025587).

