

Build An Opensource Dropbox Clone - tommynazareth
http://fak3r.com/2009/09/14/howto-build-your-own-open-source-dropbox-clone/

======
telemachos
This is an off-topic short rant. Vote as you like (obvious enough), but skip
reading this if you want on-topic discussion.

Shoot me if this is how the web is heading. (Recent browsing suggests maybe it
is...) We get not only the insufferable "bottom bar" (is there a standard name
for those yet?), but also a "top bar", a hover "Click Here to Share" box and a
pop-up "Learn More, Instantly" box (not to mention the general clutter and
overdense design).

Man, that's foul.

~~~
w1ntermute
Add the following to your hosts file:

    
    
      # Web toolbars/widgets
      127.0.0.1       wibiya.com
      127.0.0.1       cdn.wibiya.com
      127.0.0.1       apture.com
      127.0.0.1       www.apture.com
      127.0.0.1       cim.meebo.com
    

In this case, the top toolbar is from Apture and the bottom from Wibiya.

~~~
sprout
Or simply install NoScript, and worry about whitelists, not blacklists.

I can get at the content there with no trouble, and don't have to worry about
any of those toolbars.

~~~
jpr
NoScript is a symptom, not a cure. I haven't yet figured out a good name for
the disease though.

~~~
pbhjpbhj
> _NoScript is a symptom, not a cure. I haven't yet figured out a good name
> for the disease though._

Web3.0?

------
dododo
lsyncd does the brunt of the work. it uses inotify, which is a facility of the
linux kernel, to tell which files should be copied. but i think there is a
problem.

inotify is not guaranteed to see every file system access. in fact, it often
misses them when the file system is busy because there is an upper bound upon
the number of inotify events that can be queued, set in
/proc/sys/fs/inotify/max_queued_events.

i.e., you could get data loss using this mechanism alone. you need to also run
rsync every so often, i think.

the lsyncd guys are aware of this: <http://code.google.com/p/lsyncd/>

(maybe this is how dropbox works under linux too, if so it presumably has the
same problem..)

~~~
brlewis
I would guess Dropbox has this same problem on Linux, though I can't say with
certainty. What I can say with certainty is that they're thinking about the
problem.

As is often the case, you learn a lot about a company through its jobs page.
Their latest challenge for applicants looks very much like this problem.

------
tommynazareth
I don't plan on ditching Dropbox, but I'm definitely interested in having a
similar sort of automatic backup with diffs that I can host myself. This looks
like it might be an option.

Does anyone else handle this in a different way?

~~~
aidanf
I've used Unison (<http://www.cis.upenn.edu/~bcpierce/unison/>) to get a
dropbox type setup running across several machines.

~~~
larelli
I've thought about that as well. Could you give some details on the topology
used? Did you run unison in automatic mode via cron?

~~~
durin42
I use unison for my code repositories (I use dropbox for other things) and I
don't bother with it in cron. It's been fast enough in my experience that I
don't mind waiting, and it sometimes (rarely) requires manual conflict
resolution.

------
gvb
What about using unison <http://www.cis.upenn.edu/~bcpierce/unison/> instead
of lsyncd? It seems like it would be more resilient. It does look like unison
would require a cron job or human trigger where lsyncd is triggered by a
file/directory change. On the plus side, it seems like unison would work
better in an offline mode.

~~~
res0nat0r
I've tried to use this on syncing my mp3 directory between two boxes and it
seemed to be _horribly_ slow. It needs to stat each file for a second or two
before it even begins to sync, this takes way too long.

~~~
dagar

      -fastcheck
        do fast (and slightly unsafe) update detection on windows

~~~
res0nat0r
I think I remember doing that and it didn't seem to help, I was using my
Ubuntu box if that matters.

------
mikeyg
Zimbra's open source edition's WebDAV sharing works well once you get the URLs
figured out. You get a lot more fine grained permissions. It's cross platform,
there are shared folders, etc.. if that's what you're looking for. You can
make folders public to share with others, and so on. With bandwidth as fat as
it is these days DAV's heavyness doesn't feel so heavy anymore. Am I the only
one that shares these sentiments? Did I mention it works on MacOS 10.6, Linux,
and Windows 7?

If you're using Dropbox to back up some of these other solutions might make
more sense. Crafting up a script using hard links and rsync can give you daily
snapshots of any filesystem at only the storage cost of the delta (which can
be easily turned into a cron job) .. here's a decent resource for that:

<http://www.mikerubel.org/computers/rsync_snapshots/>

------
please
except that it misses 90% of what dropbox does. like multi client sync, cross
platform, shared folders, web interface and so on

~~~
fluidcruft
except that it covers 99% of what I actually use dropbox for.

------
res0nat0r
I've looked at this project before, it looks like a nice implementation of
rsync with inotify, but it seems to be just for a typical use case for rsync,
ie, push your local changes to a remote host.

I think if you try to use this like a real dropbox, ie changes occurring on
multiple hosts at or around the same time you are going to run in to issues.
Also does lysnc handle deleting data on the remote end and propagting those
changes out (rsync --del). Also I'm wondering how it will handle conflicts and
open files (which it says isn't recommended). I see the .dropbox dir using
some type of uuid's or sha1 sigs so I'm sure its doing something more
sophisticated to keep things in sync. I've noticed my daemon has died on my
linux box every once in a while, and it somehow tracks multiple changes from
multiple hosts and replays those transactions correctly and everything rolls
up fine.

------
xxpor
I don't understand why dropbox doesn't open source their client. Their
business isn't built off of their technology, which, while nice, isn't exactly
novel. They make their money from selling storage. Open sourcing their client
doesn't change the fundamental fact people need the space to store stuff.

~~~
edanm
Showing it doesn't necessarily hurt Dropbox isn't enough. You have to show
that it actually _helps_ Dropbox to open-source the client. Otherwise, why
should they?

~~~
xxpor
2 things that I can think of immediately

1\. Would increase adoption on Linux clients 2\. Increased security (the whole
security though obscurity doesn't work thing)

~~~
edanm
Well, I don't think the Linux market is big enough to make it a compelling
argument (but I really don't know; I could be dead wrong).

As for security, I'm not sure how big a problem Dropbox has with security. And
I would't open-source a client just to get better security for it, but maybe
that's just me.

Open sourcing the client is betting on the fact that the client doesn't
matter. Maybe it doesn't matter, but maybe it does. More importantly, maybe we
think it doesn't matter right now, but we'll find out that we were wrong in 2
years, after it's too late.

~~~
michaelbuckbee
Linux as a desktop client is potentially too small a market to address, I'm
not sure the same could be said about the Linux server market.

~~~
edanm
Good point, I hadn't thought of that.

Are people using Dropbox on servers, though? A lot of the benefits of Dropbox
don't strike me as helping for servers, for the most part. I don't have too
much experience, but it seems to me servers are one place you have to
_actively_ manage your backups, and can't rely on the "it just works"
philosophy of Dropbox.

~~~
brlewis
Not many are using Dropbox on servers yet, but if their users upload files,
they should.

My itty bitty part-time startup has a better uploader than Flickr, thanks to
Dropbox on the server. Users share a folder with the server and copy photos
into it that they want uploaded. They can be choosing photos to upload even
with no connection, e.g. traveling with a laptop.

~~~
edanm
That's a great idea. I wonder how many people are using Dropbox in this way,
i.e. as a framework for sharing things with the server.

~~~
brlewis
I think there's an audio site out there using Dropbox this way. Other than
that I haven't heard of it. Maybe when their API includes folder-sharing
functionality more people will do it. Right now I manually accept the shared
folder request.

------
dochtman
I thought <http://www.sparkleshare.org/> was more interesting.

------
khingebjerg
Old news. <http://news.ycombinator.com/item?id=1196421>

