
Dropship — successor to torrents? - herbatnic
http://forwardfeed.pl/index.php/2011/04/24/dropship-successor-to-torrents-eng/
======
schrototo
This isn't even remotely similar to bittorrent, it's more akin to rapidshare
et al. You're completely at the whim of a 3rd party, Dropbox. And I'm pretty
sure something like this would violate whatever contract you agree to when
signing up.

So no, not a successor to torrents.

~~~
dbuxton
I'm pretty sure I've seen stuff suggesting that Dropbox occasionally purges
copyrighted files from its system (which is made super-easy by the hash
fingerprinting system that it uses to deduplicate storage) so I agree that
this is not likely to meet most people's use case for torrents; i.e. stealing
copyrighted music and films.

I wonder why the github repo has been taken down.

~~~
wladimir
Arash (the CTO) asked me to, in a really civil way. So I decided to respect
his wish and take down the repository.

Myself, I really regarded dropship as a nice feature. As Dropbox had
implemented the great idea of putting all humanity's data in one big hash-
addressable vat, sharing is a logical extension. If you would cache the
popular blocks locally (dropbox already does this in a way with LAN P2P),
global data distribution would be pretty much a solved problem.

Obviously, this affects legal and illegal files in the same way. It's really a
shame that people are still so obsessed with the illegal applications, that
they become blinded to how useful this is for legal ones.

~~~
wolfparade
I think that's a very poor business choice by Arash. Third party developers
need freedom.

~~~
DrJokepu
They don't want their brand associated with piracy. According to the
developer, they have resolved the issue in a civil way. I don't see a problem
here.

------
tlrobinson
From the README, in case it wasn't obvious:

 _"These utilities make use of the deduplication scheme of Dropbox__ to allow
for "teleporting" files into your Dropbox account given only a list of hashes,
provided of course that the files already exist on their servers. This enables
arbitrary, anonymous transfers of files between Dropbox accounts."_

Between this and the minor information leakage issue I suspect Dropbox will be
making changes to their deduplication scheme.

A simple way to fix both of these issues is to require each user to upload the
complete file once, regardless of whether Dropbox already has it stored.
Deduplication in storage and per-user uploading is still possible.

Also interesting to note is the Github repo for this has been deleted. Tarball
of the source is still available.

~~~
limmeau
Napkin-cryptographic way how Dropbox could fix this while still getting full
deduplication: currently, when the client discovers that a file has been added
locally, it sends hashes of 4MB blocks, and the server considers the file
added.

Additional measure at that point: the server could challenge the client to
provide the values of bytes at a couple of arbitrarily chosen byte offsets of
the original file. (Could precompute that, provided the queries don't repeat
often).

~~~
alex1
What would stop pirates from querying each other (maybe on some P2P network)
for those random bytes?

Client A wants the file that Client B has so when Dropbox asks Client A for
some random offset, Client A asks Client B in the background and relays the
result to Dropbox.

It really depends on how far pirates would be willing to go.

~~~
limmeau
Of course, Dropbox can't prevent people from sharing content out of band. But
if Client A and Client B are offering arbitrary byte ranges to complete
strangers, they are effectively playing BitTorrent again.

~~~
a3nm
Yes, but they are only exchanging a constant amount of information to fool the
server challenge, whereas we could hope to do better if the server builds
challenges which use information that he knows the client has.

For some reason, this inspired me to write a blog post:
<http://a3nm.net/blog/deduplication_attacks.html> and
<http://news.ycombinator.com/item?id=2489594>

------
Stuk
From "How does it work?" in the Readme:

    
    
        Dropbox its deduplication scheme works by breaking files into blocks. 
        Each of these blocks is hashed with the SHA256__
        algorithm and represented by the digest. Only blocks that are not yet
        known are uploaded to the server when syncing.
    
        By using the same API as the native client, Dropship pretends to sync a
        file to the dropbox folder without actually having the contents. This bluff
        succeeds because the only proof needed server-side is the hash of each 4MB block
        of the file, which is known. The server then adds the file metadata to the folder,
        which is, as usual, propagated to all clients. These will then start downloading
        the file.
    

It looks like the Github repo was deleted a few hours ago, but the direct
download link still works.

~~~
lachyg
I still don't get it. Anyone willing to explain?

~~~
saulrh
Dropbox avoids having to store multiple copies of huge files by detecting
duplicated files, storing only one copy, and letting every user that stores
the file download from that one copy. Dropship exploits this system for
filesharing by lying to the Dropbox servers and saying that it already owns a
copy of the file.

For example, Person A wants to distribute a copy of a CD or something. They
upload the file to Dropbox normally. They then use Dropship to create
something describing that file, which they then publish. Persons B and C
download that descriptor and feed it to Dropship, which tricks Dropbox into
thinking that they also own copies of the file. Dropbox then lets Person B and
Person C download the file that Person A wanted to distribute, and mission
accomplished.

It's all very clever. I like it.

------
steipete
Here's a mirror: <https://github.com/steipete/dropship>

~~~
driverdan
And here's a tar mirror on Dropbox: [http://dl.dropbox.com/u/7562426/laanwj-
dropship-464e1c4.tar....](http://dl.dropbox.com/u/7562426/laanwj-
dropship-464e1c4.tar.gz)

~~~
drdaeman
> Restricted Content

> This file is no longer available. For additional information contact Dropbox
> Support.

So, Dropbox has censorship? Ni-i-ice.

~~~
driverdan
Dear Dropbox User:

We have received a notification under the Digital Millennium Copyright Act
("DMCA") from Dropbox that the following material is claimed to be infringing.

/Public/laanwj-dropship-464e1c4.tar.gz

Accordingly, pursuant to Section 512(c)(1)(C) of DMCA, we have removed or
disabled access to the material that is claimed to be infringing or to be the
subject of infringing activity.

\-----------------

This is BULLSHIT! Dropbox is censoring this because they don't want it to get
out there. What will they censor next?

~~~
drdaeman
Wow. Either Dropbox have some copyright issues with Dropship's code, or this
is just a blatant misuse of DMCA to take down the content. Won't speculate,
but I personally suspect the latter.

Wladimir did release the software under FOSS Expat ("MIT") license so he can't
really take it back. It's now up to good will of others.

While I understand that this may put Dropbox in unfortunate situation, such
methods to take down the problematic piece of software somehow feel wrong.

------
orofino
forgive what is possibly a very ignorant question, but are there security
concerns here? I understand that the key space is immensely huge and that for
any file over 4MB in size it would be virtually impossible to guess, but what
is to stop someone from just trying hashes for fun to see if they get
interesting files?

Like I said for file over 4MB it seems fine, guessing sequential hashes would
be all but impossible. I assume the realistic solution is just to encrypt my
files (preferably in a truecrypt volume over 4MB in size) if I'm truly
concerned.

On a side note, it would be interesting to see if this could be modified to
tell me how unique my overall file set is.

~~~
power78
I see your point here. I hope non-public files are protected from Dropbox's
deduplication

~~~
valindar
They aren't. A colleague copied a whole bunch of documentation from his
private Dropbox onto my computer; when I then copied it into my Dropbox it
took around half a minute to sync and it was a couple hundred MB.

------
Jarred
I think the real successor to torrents was actually its predecessor, and
that's binary usenet files. Download speeds are bottleknecked at your own
downstream, most providers have SSL support for encryption of everything you
download, and there's a plethora of content. People don't really know about it
though

~~~
kin
absolutely. instead of leaving a movie download overnight i leave it for a
shower. i hope it never hits mainstream, especially since it's subscription
based

------
EGreg
It's centralized, not distributed. You are at the whim of dropbox. This is not
a successor to torrents at all.

------
antimatter15
It's a novel exploit of deduplication, but I don't see how it's practically
any better than moving a file into the /Public directory and handing them a
URL.

~~~
Pahalial
Dropbox has bandwidth limits on /Public URLs, particularly low for free
accounts. This wholly circumvents that, and I suspect that's the real reason
Arash asked for a takedown, not so much the loose association with piracy.

~~~
dr_win
Dropbox could do the same for hashes. For example internet cannot download
more than 400MB from a hash (4MB) per day.

------
Aissen
At last. I'm surprised nobody thought of implementing this earlier. I thought
about it, but this being a direct attack on Dropbox, I don't see much value in
it. Apart from being unethical, it will only force Dropbox to either remove
this very useful feature, or implement a challenge-like system which will
render this useless. This will be short-lived code if it spreads.

In fact, I think this "feature" is one of the (many) reasons why Dropbox
doesn't have an opensource client. And it isn't exposed it in its so-called
"API".

Edit: I just saw that they killed the feature:
<http://news.ycombinator.com/item?id=2483053>

------
joshbaptiste
Indeed, great idea/hack but the only issue I can foresee is Dropship becoming
very popular for illegal file duplication, bringing forth the attention of the
RIAA/publishers etc.. causing legal headaches for our beloved Dropbox.

------
gbraad
A lot to do about this posting: [http://razorfast.com/2011/04/25/dropbox-
attempts-to-kill-ope...](http://razorfast.com/2011/04/25/dropbox-attempts-to-
kill-open-source-project/)

------
power78
So who is game for setting up a repository site of json hash files?

------
JeremyBanks
Does anybody have another mirror?

