

Build your own private, encrypted, open-source Dropbox clone - Tho85
https://gist.github.com/Tho85/6045429

======
moe
Or, you know, just use Dropbox. And add encryption on top with these two
commands:

    
    
       mkdir ~/Dropbox/_ ~/Dropsafe
       encfs ~/Dropbox/_ ~/Dropsafe
    

Yes, it's as easy as that. On OSX you'll have to 'brew install encfs' first.

~~~
edvinbesic
Interesting, does this allow for incremental updates while encrypted?

~~~
e12e
Encfs is pretty opaque -- afaik modifying a file will at least change an
entire block, possibly the entire file. Two identical files will look
completely different (per file IVs) -- so no deduplicaton is possible.

But adding and removing files works as expected, so adding a few files to the
encfs system, should translate to the same number of files added to the
underlying folder.

~~~
Tho85
EncFS allows to deactivate some of these features at the price of reduced
security. The EncFS manpage [1] has a good explanation of all the tunables. If
you want to use Boxcryptor, you're even advised to deactivate most of these
features [2].

[1] [http://linux.die.net/man/1/encfs](http://linux.die.net/man/1/encfs)

[2]
[https://boxcryptor.desk.com/customer/portal/articles/565934](https://boxcryptor.desk.com/customer/portal/articles/565934)

------
sehrope
Nice guide. One nitpick:

> Add a user for our project and give him a decent password:

Better off not giving the encbox user a password at all and only allow SSH key
based login. You can already login to the primary/root account and sudo/su to
setup the encbox user and copy SSH keys.

------
pavs
I am using Owncloud, super easy to install[1]. I have a non-us/eu vps storage
solution, despite reading others having problem with it, its working great for
me.

[1] [http://www.slashgeek.net/2013/05/16/host-your-own-dropbox-
li...](http://www.slashgeek.net/2013/05/16/host-your-own-dropbox-like-cloud-
backup-service-under-5-minutes/)

------
chakalakasp
I don't know why this doesn't end up in every thread, but Synology's
inexpensive NAS systems have a "be your own cloud" feature built in, with
corresponding iOS, Windows, Mac, Linux (I think) and android apps. All in a
little box you can keep in your house.

------
mikevm
I'd never heard of Backupsy before, looks very cool!

What is special about their VPS offering that makes it so cheap? I mean, from
what I can see for an extra $2/month they let you host websites and install
anything you want (except torrents, TOR or anything illegal).

I've been wanting to have an affordable VPS solution so I could be host my own
stuff, and have the freedom to experiment with various development tools, and
this might be a decent deal.

~~~
pliu
There is nothing particularly special about Backupsy as far as I can see. Just
cheap simple servers with a bunch of disks in them. This type of hardware is
very cheap. The software layer is equally simple - seems like just KVM and a
dashboard, nothing fancy. So then low cost, low price right. Good combo.

However as you can see from the SLA (99.9%), you do have to pay something in
the form of reduced redundancy and availability. If, for example, an HP RAID
card freaks out and all the VM's get corrupted on your blade, well you are out
of luck. That data is gone forever. It's a small risk, but it is a real one
(in fact with the P410 they are using, that's actually happened to me before
in production). There are lots of other things that can go wrong too.
Virtualization is not all rainbows and unicorns.

In the end I think it's still a good deal if you need a backup target with
phat storage in it. But if you are interested in just trying out stuff, I
think Digital Ocean is a better call since they have a higher SLA (99.99%) and
probably a better dashboard and it's around the same price. Backupsy, as the
name implies, is made for backups.

~~~
StavrosK
DigitalOcean is not really comparable, they only give you 20 GB of disk. I
can't use the 250 GB Backupsy gives you, but 100 GB for $5 or 50 for $3, I
would sign up yesterday.

~~~
Spittie
If you're fine with OpenVZ and being in the USA, RamNode
([http://www.ramnode.com/index.php](http://www.ramnode.com/index.php)) offer
50gb for 2$/month and 120gb for 7.50$/month (And it's ssd-cached space). You
can even use a coupon like "leb35' to get 35% discount forever.

Note: Not affiliate in any way with RamNode, just an happy client.

~~~
StavrosK
Wow, $30 a year for 90 GB, that's amazing, thank you. You've been using them
and you're satisfied? Basically, all I care about at the moment is that they
don't silently corrupt my data, as it's backups.

~~~
exhilaration
How did you get that price for that much disk space? I'm on the RamNode
website and can't find a plan that matches.

~~~
StavrosK
I went here:

[https://clientarea.ramnode.com/cart.php?gid=11](https://clientarea.ramnode.com/cart.php?gid=11)

And got the 256 RAM one, it has 90 GB space. Then I used the leb35 code the GP
gave me, and it comes out to $29.83 yearly.

------
tokenizerrr
Don't reuse the keys, just generate a different key on each client and add
them all to authorized_keys

~~~
Tho85
Good point, I fixed that.

------
StavrosK
After finding Backupsy (and RamNode, which a commenter suggested), I've found
the holy grail of backup solutions. I will write a simple backup script that
mounts a remote directory (through SSH or whatever), mounts an EncFS directory
on that and rdiff-backups files onto it.

This will give you encrypted, snapshotted backups using open-source systems,
that are better than Duplicity because you don't have to be creating full
backups every so often. If you'd be interested, you can subscribe to my
mailing list at [http://bit.ly/stavroslist](http://bit.ly/stavroslist) or
follow me on Twitter or something, but please post comments below so I know
what functionality to include.

~~~
buro9
I just use Tarsnap for this scenario:
[http://www.tarsnap.com/](http://www.tarsnap.com/)

What I am looking for is a folder that syncs, but is secure and open source.

BitTorrent Sync is the closest to how it should work, but is not open source:
[http://labs.bittorrent.com/experiments/sync.html](http://labs.bittorrent.com/experiments/sync.html)

~~~
StavrosK
Tarsnap is ideal, but it's around $27 a month, versus $3.

~~~
buro9
It's a question of how much you're backing up... my Tarsnap use and cost is
very low.

I retain off-site encrypted HDD images updated once a month of things like
media which constitute the vast majority of my data and do not need daily
snapshots.

~~~
StavrosK
Yeah, I want to back up my media as well (around 90 GB), but I want my backups
to just work. Tarsnap would have been fine if not for the cost, so a tool that
combines EncFS + SSH + rdiff-backup will be ideal for me.

------
Bjoern
How does this solution measure up to hosting your own Owncloud or alike? Any
upsides / downsides? I'm wondering because many non-technical people need a
dead simple solution for this if you want to substitute Dropbox.

~~~
wazoox
From what I've seen, Owncloud doesn't provide encryption (files are lying in
the clear on the server); furthermore I didn't dig much into the web app part,
but I wouldn't bet it to be extremely robust security-wise.

~~~
tomaac
That is not true. Owncloud provides both SSL and file encryption:

[http://doc.owncloud.org/server/5.0/admin_manual/configuratio...](http://doc.owncloud.org/server/5.0/admin_manual/configuration/configuration_encryption.html)

------
jlgaddis
> $ cat ~/.ssh/id_rsa.pub | ssh encbox@your.vps.com "mkdir ~/.ssh; cat >>
> ~/.ssh/authorized_keys"
    
    
        $ ssh-copy-id encbox@your.vps.com
    

Much easier, IMO.

~~~
falcolas
Doesn't exist on all machines which would be capable of using this - namely
OSX

~~~
jlgaddis
Good point. It is installed on my MBP but it looks like I installed it via
brew.

------
jfb
I applaud this, and other similar efforts (OwnCloud et al) but for me the win
with Dropbox is not system to system syncing (for which Unison or rsync works
Well Enough for me) but rather as a synchronization service for my mobile
devices. This is sadly enough an area where roll-your-own is not going to be
able to compete.

~~~
jancborchardt
Yes, it’s difficult but I don’t think »roll-your-own is not going to be able
to compete«.

(Disclaimer: I’m interaction designer on ownCloud.) With ownCloud Inc. we have
a company, customers and full-time employees. There’s no reason for it to not
be able to compete except time needed to catch up to par.

And also the code happens to be open source. In that respect it’s a challenge
because we don’t have the one centralized reference server where everyone has
an account. There are people running it on Ubuntu, Debian, CentOS, Windows
Server, …, Apache, Nginx, Lighttpd, …, with MySQL, SQLite, PostgreSQL, Oracle,
…, not to mention the different browsers people use it with (we support down
to IE8) and operating systems the clients use.

Anyway, I digress. ownCloud is also a synchronization service for desktop and
mobile devices. In fact we don’t have any system-to-system syncing built-in.

The mobile clients and especially the desktop client have gained a lot in
stability in the last few weeks – if you’ve tried before, I encourage you to
give it another shot. We’re also in the process of reworking the design for
the mobile apps. Overall simplification, better visual design. Let me know if
you have any specific feedback.

------
emerika
Are people looking at bitTorrent Sync?
([http://labs.bittorrent.com/experiments/sync.html](http://labs.bittorrent.com/experiments/sync.html))
I'm uing it on a laptop, a server and my android devices. It works great.
Seemless.

~~~
Joeboy
Closed source is a dealbreaker for lots of people.

~~~
rektide
They can keep the source for all I care, but it's hardly BitTorrent without an
open extensible spec.

------
VuongN
One question: how do you share files securely with others? Have you check out
our company's free products,
[http://ncryptedcloud.com](http://ncryptedcloud.com)? We secure your data
before it goes into Dropbox, allow securely sharing and many more features for
FREE to all our consumer users! We only charge for things like auditing beyond
certain amount of times, single sign-on integrations, enterprise stuff etc.
All we want to do is secure this whole cloud mess.

------
mikevm
A few notes on Backupsy from their website:

>Will you backup my Backup VPS?

> Unfortunately, no. Even though we use a RAID protected setup, there is still
> a slight chance of data loss due to RAID controller failure. For extreme
> redundancy you can order 2 backup VPS in different nodes and we can mirror
> them for you ("Configure it for me" addon should be purchased).

------
Osmium
This looks cool, but of course you can also use encfs directly with Dropbox
or, if you prefer a graphical interface, Boxcryptor[1] have a fork of encfs
specifically for cloud storage along with some platform-specific apps.

[1] [https://www.boxcryptor.com](https://www.boxcryptor.com)

~~~
Tho85
I used that combination before, it works really nice. But at some point I hit
the storage limit and had to come up with a self-hosted solution.

------
luisehk
I wonder if this is really usable like Dropbox. I tried owncloud which is
supposed to be the more stable alternative but it kept replacing new files
with old ones, sync took ages and security was weak. I really want to support
this kind of projects but they hold me from being productive, which I really
need right now.

~~~
Tho85
The underlying sync software (Unison) has been around for years now and is
regarded as stable. So syncing should work just fine, although you should keep
a backup of your files just in case.

You can also use Dropbox and Encbox together if you're unsure: Point your
Dropbox installation to ~/Encbox and have Dropbox sync your (then decrypted)
files. So you can be sure to have backups, file sharing features, etc. and see
if Encbox is stable enough for you.

~~~
aw3c2
I often have Unison not notice file changes until I repeatedly scan or
sometimes even until weeks later. If I new a better alternative I would not
use it.

~~~
Tho85
They use some kind of heuristic to make change detection faster, see their
documentation:
[http://www.cis.upenn.edu/~bcpierce/unison/download/releases/...](http://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-
manual.html#updates)

Maybe that's the root cause of your issues?

------
brymaster
This should be taken a step further and have a UI and web control panel just
like the real dropbox.

~~~
sehrope
Putting aside the work itself of creating a UI (this is just a simple install
guide), that's not possible without exposing the encfs keys to the remote
server. In this setup the client data is all encrypted _before_ it reaches the
server. The server is just a sync point for multiple clients.

You could have a UI (web or otherwise) on a separate client but it would have
nothing to do with the server. It would just be a client that provides a UI
interface to the decrypted encfs filesystem.

~~~
Tho85
Theoretically, it should be possible to have a web UI running on the server
itself. You just need a decent AES implementation in Javascript to do client-
side decoding of filenames and files. Any volunteers? :-)

Edit: Looks like someone is already working on it:
[http://stackoverflow.com/questions/10909500/use-encfs-
with-j...](http://stackoverflow.com/questions/10909500/use-encfs-with-
javascript)

~~~
colinsidoti
I've looked into this before and concluded it was possible, although I forget
exactly how.

I bookmarked [https://crypton.io/](https://crypton.io/) and
[http://peerjs.com/](http://peerjs.com/) . PeerJS is p2p WebRTC, which I
believe I chose because I wanted to do secure, real-time p2p file sharing.

That isn't 100% relevant for this, but downloading the file over some socket
into client-memory then decoding it there sounds like something webrtc could
be a good option for.

Crypton handles the crypto.

I think the biggest thing someone who tackles this needs to realize is that
perfecting the crypto doesn't matter 100%. If you could get something working
that's doing some kind of encryption/decryption without exposing anything to
the server, crypto experts will come in and help secure everything (see: mega)

Edit: I should mention that I didn't research this extensively, and better
libraries might be (and probably are) available.

~~~
sehrope
It's a really interesting idea (and sounds really fun to build) but a bad idea
from a security perspective. Javascript crypto in the browser just doesn't
make sense[1].

Summary of the link: Since you trust the server to serve the JS files, you
might as well trust it to do the decryption and present it all over SSL.

> I think the biggest thing someone who tackles this needs to realize is that
> perfecting the crypto doesn't matter 100%. If you could get something
> working that's doing some kind of encryption/decryption without exposing
> anything to the server, crypto experts will come in and help secure
> everything (see: mega)

This is an especially bad idea. Application that perform crypto should be
designed properly from the start. It's not something you can just patch on
later. Building crypto programs to learn/test/explore is fine but building a
product for actual usage and then expecting others to fix it after the fact is
a terrible approach.

[1]: [http://www.matasano.com/articles/javascript-
cryptography/](http://www.matasano.com/articles/javascript-cryptography/)

~~~
colinsidoti
Err I don't think this would stop me. But let me preface this by saying I
learn something new about security quite frequently, so if there's a major
premise I'm missing, I am both interested in learning about it _and_ not
completely surprised.

The main issue here is that your Javascript can potentially be owned by XSS or
other approaches. If you own the javascript, you can send plaintext passwords
or keys (which should have only been available to the client) back up to some
server.

Even with that possibility though, I still think using browser-based crypto
creates a more secure environment than the server storing unencrypted files.
The author makes a good point in saying that you download javascript on each
request, which makes it more susceptible to getting owned than a native-app,
but I think a successful attack with this approach will still be an order of
magnitude less severe than if an attacker owned a server full of unencrypted
data. If the data is unencrypted, an attacker that manages to own the server
has access to everyone's data. With it encrypted, the attacker will only get
data from users who are using the site while it's owned.

Isn't that still a better option?

~~~
sehrope
> Err I don't think this would stop me.

On the contrary go right ahead. Learning through doing is the way to go. I'm
just saying there's a big difference between testing/learning/prototyping and
releasing something that is claimed to be secure. Otherwise you end up with
Cryptocat[1].

> Even with that possibility though, I still think using browser-based crypto
> creates a more secure environment than the server storing unencrypted files.

If you're using any JS then you have to completely trust the server that is
serving it. At any point the server could replace "good.js" with "bad.js" and
the browser will happily execute it. Any encryption you perform on the client
side would be moot as "bad.js" could do whatever it wants with your plaintext
and your encryption keys.

> If the data is unencrypted, an attacker that manages to own the server has
> access to everyone's data. With it encrypted, the attacker will only get
> data from users who are using the site while it's owned.

Again you're trusting the server to not save your encryption keys though by
serving you "good.js" and not "bad.js"! All roads lead back to you trusting
the server to play nice.

Unless you use untrusted remote services as opaque object stores and handle
all encryption via secure code on the client side with proper signatures to
prevent remote tampering[2], you're trusting the server not to compromise you.

[1]:
[http://en.wikipedia.org/wiki/Cryptocat#Security_concerns](http://en.wikipedia.org/wiki/Cryptocat#Security_concerns)

[2]: Tarsnap ([https://www.tarsnap.com/](https://www.tarsnap.com/)) is a
perfect example of a client not trusting the server. The tarsnap service
stores it's data on S3 but first all data is encrypted client side on your own
machine. From the tarsnap service's perspective it's just storing a bunch of
random binary blobs and that's all your client sends/recieves.
Encryption/decryption happens locally on your machine.

~~~
e12e
Additionally, it is almost impossible to verify the security of a system that
uses encryption via javascript in the browser. The algorithms aren't that hard
to verify, but making sure that keys are handled securely, and that you don't
open yourself to timing and side channel attacks is _really_ hard with such a
big stack of things competing for scheduling, managing memory etc.

So there are two problems:

1) You get code from the server and execute it -- it might not be the code you
think it -- it might not be the code you got yesterday (and there might be
third party code injected, if there is an XSS vector -- or you know, your
browser doesn't check to see if the servers ssl certificate has been
revoked...).

2) Even if you run the code locally (say distribute it as html+js+css "app" in
a zip file signed with gpg) -- it is still running on top of a pretty rickity
rack of technologies, your javascript vm, the browser dom, a mess of ui
toolkits and c/c++ routines that manage memory -- and has been seen again, and
again -- it is _really_ hard to _actually_ implement crypto in such a way that
there are no side channel attacks, no timing attacks and no information leaks.

Essentially 1) running random code isn't secure, and 2) implementing good
crypto is hard. Really hard.

The only real solution is to get a good crypto-api into html5/browsers, but
even then things aren't "safe". Say you could:

    
    
        plaintext = api.crypto.decrypt(AES-256-CBC, \
                      ciphertext, key)
    

Now, what happens with plaintext? Is it written to swap? Cahces? If you use
that to show an image in the browser, is the image cached along with other
files?

------
happywolf
If I remember correctly, ZFS has a 'push' feature that could be a good
candidate for this purpose

~~~
happywolf
Oops, should be ZFS send.

Reference Link:
[http://docs.oracle.com/cd/E19253-01/819-5461/gbinw/index.htm...](http://docs.oracle.com/cd/E19253-01/819-5461/gbinw/index.html)

------
highball-it
You have to trust the VPS provider.

~~~
Tho85
You don't have to if you use EncFS. All encryption is done client-side.

The only thing your VPS provider could do is delete your files, but Unison's
backup feature should protect you from losing your files (in a way).

~~~
bmslieght
Your VPS provider, can in effect, chroot in to your environment, so EncFS is
vunerable.

~~~
pyre
The EncFS is mounted locally (client-side), so the files aren't decrypted at
all on the server-side. You're just syncing fully encrypted files.

