Another Dropbox horror story: My colleague lost most of his Dropbox folder last week, and wasn't able to recover everything.
What happened: he had Dropbox synced to 2 computers, but one of them hadn't been used in a while. He turned on the old computer, and apparently, for some reason, Dropbox decided that the fact that many files weren't there was because they were deleted. Of course, the files weren't there because they hadn't been synced yet!
So Dropbox decided, when the old computer was turned on, to delete 10s of thousands of files that were in my colleague's account.
He reached out to them to help restore. It took several days (this happened on a weekend), but they did eventually restore it. Unfortunately, since it was difficult for them to pinpoint exactly what was the start of the event, some files were not recovered (I'm not sure the exact reason this was difficult to do).
I love Dropbox , but this was a serious blow to how much I trust them with my files.
 Just look at my comment history to see - I've raved about them several times on HN.
I've read through a lot of this thread, and I'd like to follow up with a few thoughts.
First of all, just to make it clear: Dropbox support has been GREAT. Though it took a few days because this happened on the weekend, Dropbox support did contact my colleague shortly after, and have made it clear that they will work hard to make sure everything that can be is restored. The problems with realising what is left to restore seem to be technical (again, I'm unsure of details). The support staff themselves have helped a lot. In addition, someone has reached out to me from Dropbox support because of this thread, to try and find the root cause of the problem - I hope we managed to help Dropbox find whatever it was that caused this to happen, so it doesn't happen to anyone else.
Another misconception I saw, about "Dropbox is not a backup solution":
My colleague and I both realize Dropbox is not for backup. The most important files were backed up, at least to some degree. Having said that, Dropbox is used by myself and many others as a semi-backup solution. My personal strategy is to backup the whole Dropbox folder to an external disk every few weeks, but rely on Dropbox for everything else. This is not a good backup solution, but realise that for most people, it's not "Dropbox vs. a real solution", it's "Dropbox vs. no solution at all".
I'll close with the same message I give to most people - Dropbox is brilliant, most of the time. It's not a great backup solution, but it's better than what most people do. If you're not using anything else, Dropbox is a life-changer in terms of ease-of-accessing-your-files-anywhere, and feeling secure that everything is probably backed up.
Totally agree on the "not a good backup solution". But we both (obviously) know the difference between redundancy and backup. My poor mum does not. So in the end, that's a problem, and i'm not bashing dropbox either (paid account and all, use it all the time) : their position as it stands, is a bit like the "nobody can presumably be ignorant of the law" article in french civil code (which was suppressed a few years ago), a position no commercial service whose customers are volatile should take for granted.
While I can't comment on this specific case, I do want to say there is a tremendous amount of rigor (and layers of protection) to ensure the end clients are resilient to misbehaving computers and do the right thing. For example, you shouldn't have to worry about the clock being accurate as it has no bearing on the final sync state.
Just to dive in a bit for the HN crowd: We base the changes that should be applied to your Dropbox on whether the client has successfully synced the previous version of the file. The server doesn't allow the client to apply changes unless the client is fully up to date. And in the case of deletions, the desktop client is only permitted to issue deletes to files that have been successfully created on disk and then later deleted.
We also have a proper support team to catch things that fall through the cracks, and if you have issues, you should reach out to them first (and as soon as you notice; their ability to fix things becomes harder the more changes you make to your account).
Again, not commenting on the above issue, but I did want to point out that we take the integrity of your data very, very seriously.
I've replied to this elsewhere, but let me reiterate here:
sean_lynch is not kidding about Dropbox's support staff. They are great, very helpful, and according to my colleague, are very serious about trying to restore everything. The issue he had was a software bug, and apparently some things are problematic to restore because of other technical reasons, but Dropbox has been very helpful.
And to make this clear - they were very helpful when this happened last week, before this thread - you don't need to write a blog post to get their help and attention.
Client rigor doesn't help with server failure this severe.
The guy lost everything because a temporary team membership was revoked. And now you're coming back to talk about your rigor, and how you catch things that fall through the cracks.
This really comes off, to me at least, like LinkedIn did when after their breach and it was revealed they weren't even salting passwords, they tried to brag about their security, as a way to step around owning up.
I think you should consider this tone very carefully.
Completely disagree. I found no problems with the tone here. Instead, you have an insider who shares a few implementation details to give us an idea about the depth at which they go to prevent data loss failures. I saw nothing in his post as "bragging".
The guy lost everything because a temporary team membership was revoked.
What would you like him to do, apologize profusely and robotically without knowing the details of this incident? Dropbox is a decent-sized organization and I am pretty sure they have guys already responsible for addressing specific issues like this. To expect every employee to know details of every issue seems unreasonable.
John - take a second to read the comment. He was providing feedback on a scenario in which a machine which had not been powered up in a long time apparently deleted files because they were not present. The Dropbox model does not allow this - you can only delete files that are already on your hard disk. This, is what he was trying to explain to the parent poster. His tone was entirely appropriate.
Shoot. I've been thinking of dropbox as my backup. Now I wonder what would qualify as backup considering I want it to be automated and preferably not require me to write to media like DVDs. (Time Machine seems to be doing the trick on the Mac side, although... having all the backup media connected to the machine it's backing up means it's susceptible to simultaneous deletion, as well not being offsite means it is susceptible to theft/fire.) On the Windows side, I've been backing up just "important" files with dropbox, which has the benefit of being offsite and automated. I once used Mozy, but when I needed to do a restore they had problems generating a recovery image for me. I think they eventually got access to all the files after a few days. That wasn't reassuring.
I use Dropbox for synching only. I don't consider it a backup. For backup, my solution is redundant:
I have a local Time Machine backup that is going continually. This allows me to quickly recover from most problems. However, if my house ever blew up, it wouldn't be sufficient, so in addition...
I have a Backblaze account. $5/month for everything on one computer, which is fine because Dropbox syncs my laptop to that computer. That backup runs continuously and keeps 30 days history (so if I delete a file and need to get it back, I can). Since I only care about disaster recovery here, if my house does blow up, they'll send me a hard drive with everything for around $100.
The combination means I feel pretty comfortable. I just hope my house doesn't blow up!
All my systems (Linux) use btrfs and I have cron jobs making regular snapshots including of the Dropbox folders, so even if horrible things happened I still would have access to the original files. I'd have to have Dropbox mess up server side and have something bad happen to all of my client machines.
Works fine for me. I use it on every system and have it on a desktop, a laptop, a server and a HTPC. It is used on SSD, HDD (SATA and USB), RAID 0, RAID 1 and over dmcrypt. The only place I do not use it is one filesystem for MongoDB, and it would probably work fine there too if I disabled COW.
The builtin checksums (and scrub) functionality are crucial to me. I had a drive start to fail with ext4 and the only reasonable way to do a scrub would be to take it offline which I calculated would take 23 hours (plus a huge amount of extra work to map bad sectors back to relevant files). At some point a failed sector had also lead to ext4 giving back zeroes when the containing file was copied. btrfs has two copies of metadata by default so an unfortunately placed bad sector is less likely to wipe out knowledge of entire trees or files (happened to me with ext4 as well when a directory disappeared).
The volume management is great too. It is really easy to add and remove devices/partitions without having to take systems down, change between raid levels etc. You can do the same thing with LVM (which I was using before) but it is a lot of fiddly work to get the right commands and options, plus deal with physical and logical. It is one command to add or remove a partition with btrfs and it is trivial to work out what the command is from 'btrfs --help'. (LVM requires several different commands to be run which I always had to lookup and often had a lot of difficulty with such as running RAID 0 with partitions of different sizes.) btrfs also only does operations on the actual used portions. eg if I change from RAID0 to RAID1 on a filesystem where I am using 1GB of 1TB space then it will only worry about that 1GB not the whole 1TB. LVM can't see into the filesystem to know what is actually used.
I have compression (LZO) turned on everywhere. You can't (currently) find out how effective it has been, but I do have a lot of data files that are highly compressible (eg CSV files, SQL dumps). It is more convenient to have them expanded than to teach every single program that accesses them how to decompress on the fly.
I used to use rsync and hard links to make snapshots. These completely hammered the machine when run causing large amounts of I/O since all metadata has to be scanned plus all changed/new files. Consequently I only made daily snapshots. With btrfs making snapshots is virtually instantaneous and is unaffected by how much data has changed, size of volume, I/O speeds etc. I now make snapshots hourly.
Ultimately I have things such that I will proactively find out about bad sectors and similar low level corruption, have snapshots to deal with issues over time, and have data replicated over machines (mostly via Dropbox and git/hg), both local and remote. I am not relying on the filesystem of any one machine to always be perfect.
> Also, since you seem to be knowledgeable about Linux, why do you use Dropbox at all, instead of just git or rsync or scp or whatever?
Because Dropbox just works. Many of the alternatives haven't figured that out yet, assuming they even support Linux. The importance of actually working can't be overstated. It also requires no administration or maintenance. Sync automatically happens when machines are on and there is appropriate network connectivity without requiring any baby sitting from me.
I use git/hg for source which is their sweet spot. Using something like rsync is a pain once you have more than two machines, and it requires a full blown system accessible offsite for an offsite copy which is yet more administration and maintenance.
git/rsync/scp aren't usable from mobile devices. I do things like put documents, ebooks, photos, and music into dropbox which makes them be present on everything, no messing around needed. Dropbox behaves very much like DVCS for that kind of content doing an N-way sync, having a history (you can get last 30 days by default, more if you pay more or use a team account).
Finally Dropbox allows collaboration. You can easily share files and directories. I can do a software build for Android, put the APK in a shared folder and a colleague can install the app on their device without hassle. Various tools run periodic reports and put their output in shared folders which makes it available to everyone even if they then happen to jump on a plane.
`rsync-for-android` is actually pretty good if rsync is what you actually want; android-only of course, and FolderSync has a slightly nicer UI-wrapper (using sftp directly, rather than actually rsync'ing) and "experimental" auto-push-on-change, so between the two of them my phone and tablet now "play well" with my existing linux-based photo workflow. (I agree dropbox has many advantages, and a simplicity that comes from not having features... but with rsync, I can keep "just the last 30 days" of my full photo archive on my tablet, nothing else gets me that particular kind of control.)
Can't speak for roger, but I find it is very useful for syncing across OS's, and sharing files with friends (though ever since my school went Google apps, Google Drive has been the go-to for this kind of thing).
Those are programs, not services. Dropbox is a great service (client + server) for syncing your files.
There is no other simple option that "just works". So my choice is Dropbox + encfs (to encrypt everything). But I also keep another local copy of the entire dropbox folder in case the sync fails and decides to delete everything.
I would take a look at the link in my comment, there is a neat way to using external drives for Mac and Linux. I'm sure there's a similar way to schedule it in Windows, and I wish the author had gone into that instead of his tongue in cheek bashing, but I don't know it off the top of my head.
Absolutely you need to test your restores, regularly!
I used to use Mozy but when I tried a trial restore it was taking forever and missing some files.
Also it inserts these .PART files into your system that can almost double the size of the folders you're trying to back up.
I looked at backblaze but it doesn't work with server computers.
So I'm now trying crashplan.
I'm really interested in the feature of crashplan which allows
automated backups to other computers. Only problem is that I've not been able to get this to work so far. Backing up to the cloud is working well though. Also its very competitively priced. I've yet to try a restore.
One trouble with testing restores is that my likely avenue in the event of a real catastrophic failure that required a real restore would be to have them send me the data on a hard disk, to the tune of about $200. Not something I want to try just to make sure it works. I could try a download restore, but this would entail downloading a couple hundred GB of data, and wouldn't really exercise the same path I'd use in a real scenario. Any thoughts on the right way to approach it? Doing occasional small restores (usually using Backblaze as a sort of really slow Dropbox, restoring to a different computer when I need a file I forgot to share) at least gives me some peace of mind that the system works at all, but you're right that it's not quite rigorous.
For servers, I've heard good things about Tarsnap, but never used it myself.
No, that's a tough one, especially with a back up the size you're doing.
Everytime you do a restore that's just confirmation that a particular backup was/was not sound.
Wouldn't it be neat if the back up program had functionality that allowed you to restore a random sample of your files ( and maybe then compare these to the originals ) and give you a likelihood that your entire backup is sound?
A sort of sampling quality control process.
I'm ok with them for "I accidentally deleted a file" type stuff, I would not suggest it for a full system backup. I'd suggest SuperDuper! for that if you're on a Mac; I don't have any useful suggestions for Windows or Linux users (sorry!). Of course, I use a full system disk image for "I need to be up and running at 95% capacity within an hour", and these offsite file services as, basically, long-distance undo, or for files I'd be unable to get back in some other form. Think tape storage: it doesn't need to available ultra-fast, but I need to have as many nines of certainty possible that there will be some way to get at those files again.
 though I'm switching off of them, not sure where to yet. There's some sort of issue in their client software that murders my home network, rendering the entire thing unusable while a backup is running. Their support was... less than helpful in trying to remedy the situation. For $5 a month and with the amount of stuff I was backing up, I can't blame them.
About the Network issues: something similar happened to me yesterday. I just moved to a new flat and my Internet connection went from 50MB/10MB to 10MB/1MB and if Backblaze starts a backup with the Automatic Throttle option on it just takes the whole upload bandwidth killing every other connection and rendering the connection unusable for the rest of the network.
I don't have any QoS set up on my home network, I found two fixes: running the backup manually before going to sleep (maybe I can create an Apple Script and just cron this) or turning the Automatic Throttle off and giving it the minimum available (for me) 20kbps.
Get Norton Ghost for your Windows setup. It works and is a no-brainer.
Once a month do a full image on both Windows and OSX systems. Save the images to another backup drive and take that backup to a safety deposit box at your bank.
If you want to be really paranoid, find a provider that offers fireproof storage for media. If you can't, go to your local storage facility and rent a small unit (maybe $30 to $75 per month, depending on where you live). Buy a fireproof safe and stick it in there. Once a month (or whatever) rotate drives through the safe.
You can also avoid the storage unit rental expense and locate a fire-proof safe at a trusted friend's or family members home or garage. Be creative.
In other words, creating a system that will virtually ensure the security of your data isn't hard or expensive --particularly when compared to the value of the data you are trying to protect.
This is wildly inefficient and over kill for most people. Imaging my hard drives would require terabytes of storage when in reality my unrecoverable data is small enough to back up to Amazon for less than a dollar a month.
You need to have context before you say something like that. Just ONE of our engineering machines probably takes two or three weeks to setup from scratch due to the amount of software and configuration it requires. It has some 600GB of valuable data, without counting the OS and applications installed.
So, in that context, taking the time to make multiple redundant disk images is invaluable and, yes, very efficient. If the system drive dies and no other hardware failure is present, you can be back up and running within an hour or two. That's worth money.
Yes, and we can always trust advertisements to be truthful....
If you accidentally delete a file in Dropbox on one computer, Dropbox will helpfully and immediately propagate that deletion to all of your other computers. It can be used for backup, but it's not very good at it.
It strikes me as unfair to say "sure the company advertises that they can perform function x, but trusting them to actually do that makes no sense." I don't have any objection to dropbox not being a good backup solution, but I do have an objection to them advertising themselves as a good backup solution when they are not.
Is it possible that the unused machine has the wrong date? Perhaps that confused the operative system file dates and Dropbox.
(I have an old machine that hardly use now that has problems with the motherboard battery and sometime it travels in time to a funny date like 2000 and all the security certificates are marked as wrong.)
Indeed. I once spent several hours debugging a desktop client application that relied on a correct system time.
Finally, we noticed the user's system time was set to several weeks in the future. "Oh yeah, that?" He showed us how he uses the system calendar to check dates in the future. When he was done, he'd click the "Ok" button, thereby setting the current date in Windows to some future day.
hmmm, I think it's fairly common practice amongst similar software (i.e. firefox, chrome, etc.) to use system time as a signal that you may be connecting to a server that has an outdated SSL certificate. while it's not perfect, it's a good first order security precaution to take (in addition to verifying the CA) to ensure you are talking to who you think you are.
iOS does it that way. My iPod touch always forgets date and time when it runs out of battery. After restart you can't do anything in the App Store or on https websites before you've set the correct date (it defaults to 1970 which is when the iPod was first introduced I believe).
If you know you're on a network -- and Dropbox does -- then I'd fetch the time from a time server somewhere, perhaps Dropbox itself, or time.nist.gov. Of course, then you're at the mercy of the user's hosts file, I suppose.
If the time is just used to check for certificate expiration, maybe nobody considers it worth worrying about. Which I guess is fine as far as it goes... but the other day I couldn't get Dropbox to do its thing when my clock was only a few hours off. It didn't seem likely that an SSL date check was involved in that case.
So the silver lining is still that they came through and restored the data, aside from the fact that the old and presumably really old Dropbox client on it went haywire. Yes, Dropbox is free and no I don't expect them to have a file retention guarantee that safeguards my stuff against nuclear holocaust, but I feel for you friend. In this case, storing files is all that Dropbox does. If my harddrive decided to do that intermittently, I'd purchase a new drive and probably mirrored them.
Why not come up with some sort of open-sync protocol where you could have your data mirrored by many Dropbox like services at once
I had this problem when I tried installing dropbox on my VPS.
I wanted to use dropbox to get a small 1gb backup off of the server, as my puny ADSL would take days to do it, but for whatever reason, after it installed, I noticed on my current PC, 100's of "whatever.file has been deleted" notifications from my dropbox tray icon, and I was nearly dying, thankfully they have a restore option on their site, so I was able to "undelete" the files and save myself from losing so much data...
Seems dropbox really does believe if a file isn't there, it's been deleted.
I recently recycled my old notebook and took out the hard drive before recycling. Just a few days ago, I noticed videos getting added to the automatic camera uploads folder. These were not from my camera and strangely were of women undressing taken secretly outside windows. When I went to dropbox.com, it seemed that these files were added from the notebook I recycled.
I reached out to Dropbox support but got no response yet.
Sorry about your colleague's experience, but from my understanding of how Dropbox works even with old clients this shouldn't happen. Are you sure your friend hadn't deleted the contents of his Dropbox folder on that system and then turned the client back on later? That would definitely explain this. It'd be great to have your friend weigh in.
Down-voted because I think that it is just dumb to have a Dropbox account and forgo local backup of the files on the service. Dropbox is not an extension of your mission-critical local storage and backup.
My approach for Dropbox on both Windows and OSX is to have the Dropbox folder on a drive that will never see development work, even if that means creating a separate partition for it. In most cases the system drive works fine (you do have separate system and data drives on your machines, right?).
This forces a COPY operation rather than a file MOVE if you drag-and-drop files into any Dropbox folder. Which means that everything in the Dropbox folder could be trashed tomorrow and nothing whatsoever would be lost.
On Windows you can also do this by dragging files while holding down the RMB and on OSX while holding down the Option key. This, however, is fraught with issues because it is easy to forget and move something in or out of Dropbox accidentally.
In addition to that, daily system and data backups to external media on EVERY SYSTEM in the office ensures that even if you took a sledge-hammer to any given machine all the data could be recovered to within a 24 hour boundary.
So, no, this is NOT a "Dropbox horror story" as far as I am concerned. At best this is an unfortunate "pilot error" and at worst this is a sign of incompetence.
Sorry, a little harsh there, but, as a student, I lost six months of heavy coding work once in college. That really hurt. And that was probably the best time and place to learn that lesson. That is all it took for me to be completely insane about having backups of backups on anything important. Now, may years later, you will not find any critical system in my office that is not backed-up at least once and usually twice.
These days there's almost no excuse for loosing your work. Don't go around blaming Dropbox, not their problem.
I am not associated with Dropbox in any way other than being just another user.
FEATURE REQUEST: It would be very nice if Dropbox client software could have an option to auto-magically COPY files in and out of Dropbox rather than allowing any files to be moved. The above-mentioned hack works fine but it'd be nice to not have to use it. Also, I'd venture to guess that most Dropbox users don't do this, which opens them to unintended loss.
It is in no way "pilot error". If the post you replied to is correct, the dropbox software performed an action it should never perform - deleting files the user did not tell the OS to delete. What's more, every dropbox plan I'm aware of keeps deleted files for 30 days, so if it wasn't possible to recover some of the files, dropbox failed not once but twice.
It's all very well blaming the user for not having multiple redundant backups, but that doesn't change the fact that deleting user data without explicitly being told to do so is one of the most egregious sins software can commit. It's not "pilot error" when the software performs an action that should never occur.
The "pilot error" is in not testing the solution before trusting it and in making an assumption that the data would be safe in an untested external system.
Let me put it into a financial context. I did a project that added up to some $800K in cost. Two developers over about a year. Do you think that for even a microsecond I would trust Dropbox as a backup mechanism without extensive qualification and testing? And, would I have that as my sole approach to backup?
And this isn't me coming down on Dropbox or suggesting that they are unreliable. I use Dropbox and I am very happy with the service. However, based on experience gotten the hard way, I do not use it for anything that is mission critical. The important stuff is backed-up locally, sometimes with redundant backups.
I don't have any sympathy for an engineer who uses a service and relies on it without the proper testing and qualification phase. Behaving that way is most-definitely, as I said, to be kind, "pilot error".
It is completely unfair to blame Dropbox for anything. This is like the idea of cutting-and-pasting code from the Internet and trusting it with a mission-critical aspect of your application without dissecting the code, testing it and fully qualifying it for what it is you need it to do.
There are tons of examples of this behavior, perhaps the most common one in the web development community is email validity verification. How many careless developers copy a regex like this to validate their email and don't think twice about it:
These things are all over the web. And most of them suck. Even some of the most impressive ones will lead you astray.
How many engineers (and, as a result, websites) fail to test and research and end-up with questionable solutions?
Would you blame the regex writer for this or the engineer who chose to use it without doing any testing at all?
It doesn't matter what the regex author claims or how authoritative the website featuring the regex might be, you have to test it before deploying it! If you don't test, deploy and then loose customers because it is a bad regex it is YOUR fault, not the author's. Blaming them is nonsensical.
How did you even get to talking about engineers and testing/qualification and $800K projects?
Dropbox is a consumer service, also useful for businesses. Nobody's trusting it with the nuclear launch codes. But no matter what, it should never, ever delete your files, unless you do it yourself. Period. And if it's ever unsure, it should ask you.
It's meant to be used with the primary copies of your files, not relegated to a special drive. If it deletes your files wrongly, the fault is clearly with Dropbox. It's marketed as an easy-to-use product you just install and use.
Well, let's just agree to disagree. Get a few more years under your belt and loose some valuable work for relying on such services without local backup and, yes, testing and qualification and then we can talk. Perhaps then you'll understand where I am coming from. There's nothing more painful than such a loss to make it clear that YOU are responsible for your data and nobody else is, no matter what the claims might say.
I cannot blame Dropbox if I loose data. Maybe the service has issues, but loosing my data is MY ISSUE, not theirs. The minute you trust service X with your data without full qualification and testing is the minute you lost your data. Don't blame them for it.
> Dropbox is a consumer service, also useful for businesses.
Same point. Same issues. If you put all of your documents and financial data on Dropbox (or service X) and don't take the steps to have local (and possibly redundant) backups you could easily be confused for a moron. Blaming service X for it is a cop-out. They had nothing to do with the total loss. Yes, they had everything to do with the partial loss.
It is my contention that it is your responsibility to fully qualify such services rather than using them blindly.
I use Dropbox extensively and have ZERO concerns about data loss.
Maybe experience has made me less trusting with what is important to me than a typical user? I could not imagine saving my kids pictures on service X without any other backups. That would be insane. And, if for any reason, service X looses my pictures I was the moron who allowed that to happen. They might suck, but it was my fault, not theirs that I lost EVERYTHING.
Maybe that's the real point I am trying to make: Loosing EVERYTHING is YOUR fault. Loosing WHAT IS STORED IN DROPBOX could be their fault. If you don't have backups for what you store in Dropbox (or service X) you are responsible for the TOTAL LOSS not service X.
> There's nothing more painful than such a loss to make it clear that YOU are responsible for your data and nobody else is, no matter what the claims might say.
These are the kind of lessons that for some reason don't transfer person-to-person like they should be only sink in once they've happened to you (at a guess, that's why you're being downvoted). Whether it is your corporate data worth $800K or your photographs of your first child hours after being born through the point that you realize that it is all gone data is priceless.
In the end it all boils down to the same thing: you can't outsource your responsibility on this one, and a single service holding your data still counts as a single point of failure.
Personal responsibility and the acceptance thereof is a major item on the checklist, anybody downvoting robomartin in this thread still has some learning to do on that front. Don't shoot the messenger if you don't like the message. Dropbox is not a backup, no matter what their marketing literature says and no matter how safe you think they are. In the end you and you alone are responsible for your data and its safety.
A back-up that you have not tried to restore might as well not be there.
>Personal responsibility and the acceptance thereof is a major item on the checklist
There are two separate things going on in this discussion.
You have a personal responsibility to keep your data safe. I don't think anyone is disputing that.
Independently, softwares and services which handle user-created files should never, ever delete them without a direct instruction from the user. Regardless of your backup procedures, it is completely unacceptable for this to happen. Even if I had a copy of my data in a fire-proof safe on every continent, I would be appalled if I found that a file syncing and backup service like Dropbox had deleted some of my files and was unable to restore them, because i) it's an unacceptable thing for any software to do, and ii) it completely contradicts all of their company messaging.
> Independently, softwares and services which handle user-created files should never, ever delete them without a direct instruction from the user.
We're 100% in agreement on that. But since you can never be sure that the software that you're using is bug free (and in the case of dropbox there is now at least one more datapoint that confirms that their software contained at least one bug) you have to more or less count on losing all your data.
Dropbox messaging is of course not going to highlight the fact that their software may contain bugs, it simply isn't in their best interest (and none of their competitors do so either).
Maybe they have a section to that effect in their fine print but even if they don't on hacker news we really should all know better. Unless there is someone here that only writes bug free software... In that case please send me your resume.
> Independently, softwares and services which handle user-created files should never, ever delete them without a direct instruction from the user.
Show me a business that is willing to offer you that guarantee and I'll show you a business that is out of business.
What if there's a fire? Or malicious destruction? Or massive hardware failure?
Anyhow, the only people who would down-vote some of the comments I am making are those who have never suffered the pain and agony of data loss. If you've ever been in those shoes you know, without an iota of doubt, that what I am saying is 100% on point.
There's nothing wrong with Dropbox. The problem is in how naively people are choosing to use the service.
Could they be better? Of course. Would I still have my own redundant backups and recommend that Dropbox users do the same and fully understand the service. Absolutely!
Personal responsibility is a bitch. It's much easier to blame someone else. Guess what? Blaming someone else for your failings will not make things right or bring back your data.
> Don't shoot the messenger if you don't like the message.
What if the message is liked but the messenger is behaving like a silly sausage?
Everyone knows that multiple backups are important. If I'd had multiple backups, and Dropbox had deleted items that I hadn't asked it to delete, I'd still be able to comment about that because data deletion is serious business.
This thread isn't (at least, shouldn't) be about data being deleted; it's about unpredictable behaviour from software. When that unpredictable behaviour includes deleting data that that's a valid serious concern even if everyone has great backup strategy.
> but the messenger is behaving like a silly sausage?
I think that's mostly a format issue, the underlying message is solid.
> Everyone knows that multiple backups are important.
My experience and your experience are apparently not the same in this respect. And I look at a lot of companies every year. The number of times the question 'do you do trial restores of your back-ups' is answered with either 'what backups' or 'no' is way larger than what you'd expect.
And those are companies, not individuals where I'd expect the situation to be much worse.
> If I'd had multiple backups, and Dropbox had deleted items that I hadn't asked it to delete, I'd still be able to comment about that because data deletion is serious business.
Yes, absolutely it definitely is. Dropbox dropped the ball here. The problem is that you can pretty much expect them to drop the ball on occasion, no large service has every been 100% without data loss. Amazon, google, dropbox, microsoft, they've all lost some customer data at some point.
It's the rule, not the exception. When you're dealing with data data loss is the thing you're working hard to prevent but it'll never be 100% perfect. It can't be. There will always be edge cases and the simpler you try to make it on the outside the more complex the actual software becomes. Complexity leads to bugs, bugs (can) lead to data loss.
> This thread isn't (at least, shouldn't) be about data being deleted; it's about unpredictable behaviour from software.
All software that I'm aware of contains bugs. The discipline and experience required to write bug-free software is universally claimed to be present in one industry only: aerospace/aviation. And even there they have bugs, just fewer of them at massive expense. Bugs are the norm, not the exception.
> When that unpredictable behaviour includes deleting data that that's a valid serious concern even if everyone has great backup strategy.
Yes, it is a valid concern. And the way to mitigate that concern is by looking it in the eye and saying 'I don't trust software'. Any software. Including dropbox.
He put in a huge amount of effort to derail the discussion. Pretend that where it says "wasn't able to recover everything" the post said "wasn't able to recover everything from Dropbox, and had to resort to getting external backups mailed to him". Same terrible problem on Dropbox's end, suddenly no need to waste pages arguing about 'responsibility'.
He brought out the part of the HN crowd that wants things to be on-topic and/or useful instead of user-blaming. :)
I'll quote the most relevant and definitely constructive/useful parts here you don't have to bother with the links if you don't want to. If you do, please slow down and read it all. Then think about the idea of loosing, I don't know, a year's worth of work and blaming someone else for it.
Here it is (reformatted because HN does not have a much-needed block-quote markup):
My approach for Dropbox on both Windows and OSX is to have the
Dropbox folder on a drive that will never see development work,
even if that means creating a separate partition for it.
In most cases the system drive works fine (you do have separate
system and data drives on your machines, right?).
This forces a COPY operation rather than a file MOVE if you
drag-and-drop files into any Dropbox folder. Which means that
everything in the Dropbox folder could be trashed tomorrow and
nothing whatsoever would be lost.
Do this TODAY and unless you have massive local loss of data you don't have to worry about Dropbox loosing any of your data at all. Not one bit. If that isn't constructive I don't know what it.
Then I also said:
FEATURE REQUEST: It would be very nice if Dropbox client software
could have an option to auto-magically COPY files in and out of
Dropbox rather than allowing any files to be moved.
The above-mentioned hack works fine but it'd be nice to not have to
use it. Also, I'd venture to guess that most Dropbox users don't do
this, which opens them to unintended loss.
In other words, if Dropbox could implement the suggested approach EVERYONE, engineer or not, would be spared of total data loss due to something going on at Dropbox. Local data loss, of course, is still your responsibility.
And then, trying to understand further, I asked this:
So, I'll say that overall I've been pretty constructive. If anyone adopts my approach of storing their Dropbox folder in an unused drive they are almost guaranteed that Dropbox can't loose their data. That is worth money.
Please point me to a link or blog post with your immediate solutions to this problem. I'd love to compare notes. Maybe there's a better approach. Always interested in learning. I am sure there are lots of people who, tonight, might be interested in adopting your approach to ensuring that accidental data loss by a remote service of any kind does not mean total data loss locally.
I read your posts. The constructive parts are minor compared to the derail, and you easily could have avoided the enormous argument. Also, do you really see no downside to the fact that you have basically reimplemented manual sync on top of dropbox? It throws out most of Dropbox's safety features just because they aren't perfect, and throws out half the ease of use.
It's OK. My data is safe. I suspect yours is as well.
Mark Twain: A man holding a cat by the tail learns something he can learn in no other way.
I love done that. I've held a number of cats by the tail over the years. And, he is right, sometimes you don't learn until you have claws painfully sunk into your skin, no-matter what anyone around you might say.
And, I love Dropbox. It's a great service. As far as I am concerned, there's nothing wrong with it. If you own your data Dropbox could go out of business tomorrow and you would not care.
> I don't have any sympathy for an engineer who uses a service and relies on it without the proper testing and qualification phase. Behaving that way is most-definitely, as I said, to be kind, "pilot error".
How the heck do you effectively test a service like Dropbox fully, without making it your full-time job (IT dept). Take the other comment on this article, where someone fired up an old machine and it wiped out his files. What test would have caught that?
> How the heck do you effectively test a service like Dropbox fully, without making it your full-time job (IT dept). Take the other comment on this article, where someone fired up an old machine and it wiped out his files. What test would have caught that?
And, if you can't reasonably expect to fully test it and verify it's reliability (of any service) then you DO NOT rely on it for critical data you cannot loose. Choosing to do so and then crying foul when you do loose data isn't engineering, it's, at best, wishful thinking.
Again, this does not mean that Dropbox --or any other service for that matter-- is bad. Not at all. It just means that you have to understand what you are walking into and, if you can't, or don't, then take steps to safeguard from the potential for external data loss, because you just don't know.
> At best this is an unfortunate "pilot error" and at worst this is a sign of incompetence.
I'm not sure knowledge of an obscure Windows-specific drag and drop quirk should be a requirement for using Dropbox safely.
> you do have separate system and data drives on your machines, right?
I'm not sure use of a partition table editor should be a requirement for using Dropbox safely. Most people use the operating system that came installed on their PC as-is.
> daily system and data backups to external media
Dropbox makes the following claims on their website:
"Your files are safe"
"Even if you accidentally spill a latte on your laptop, have no fear! You can relax knowing that Dropbox always has you covered, and none of your stuff will ever be lost."
"Even if your computer has a meltdown, your stuff is always safe in Dropbox and can be restored in a snap. Dropbox is like a time machine that lets you undo mistakes and even undelete files you accidentally trash. "
I think it's completely reasonable to expect Dropbox to be a reliable backup solution as-is.
> I'm not sure knowledge of an obscure Windows-specific drag and drop quirk should be a requirement for using Dropbox safely.
Not a "obscure Windows-specific drag and drop quirk". This is how the OS works. If you are a developer you need to know your OS.
> I'm not sure use of a partition table editor should be a requirement for using Dropbox safely.
It isn't, but if you are smart you'll take my advice and go implement it right now.
> Most people use the operating system that came installed on their PC as-is.
Developers are not "most people" and should certainly not behave as such. Your data is the only thing that is important. Your work product. Investing in separate drivers (not partitions, separate physical drives) on every development machine is invaluable. And, backing-them up with disk images on a daily basis is just as important. If the system drive goes out your data is intact. If the data drive goes out the system is intact. Recovery is an academic exercise. If both drives go out it is a little more of a pain in the ass, but at most you'll loose a day's worth of work, not a lifetime or a whole year's worth.
> Dropbox makes the following claims on their website:
Pardon the raw-ness: I don't give a shit of what anyone says or claims about any service. Neither should you or any serious developer. You, and only you are responsible for the safety of your data. You HAVE TO assume failures, not just locally, but also remotely. Hardware fails. Software fucks-up. Even if Dropbox (or name your service) guaranteed 500% redundancy I'd still have my own backups. It would be irresponsible to do otherwise.
Then there's the practicality of the whole thing. The machine I am using to type this has the following physical drives (not partitions on a single drive, these are independent pieces of hardware in the chassis):
Internal backup drive
External backup drive (USB)
The last four are backed-up daily with incremental backups to both backup drives.
The data drive alone has over 300GB of data right now. The Library and Development drives are about 150GB of data each. That's 600GB of valuable work in three drives probably representing millions of dollars of work-product and value.
So, Dropbox is backup, right? Well, uploading 600GB to your "backup" would take somewhere in the order of 100 days of continuous 24/7 upload with my current DSL connection (~600Kbps upload speed). That's a third of a year. Nope, it's not backup.
The OP never said their colleague was a developer. And Dropbox isn't specifically marketed towards developers. When you develop software aimed at the general public, which Dropbox does, you do have to worry about most people. And regardless of how irresponsible the user was or how knowledgable they were, it's still a screw-up on Dropbox's part. "You shouldn't have relied on this service" != "this service isn't doing anything wrong when it breaks." You should lock your doors at night, but the robber who just saunters in to steal your stuff is still a badguy.
Also, since this is HN, let's talk about how we build software, as in, put yourself in the position of the Dropbox developer and instead of just the end user. Most of us, I imagine, build software used by non-developers. I hope your attitude towards these things isn't "fuck it, if they were a competent developer like me, they'd have known better", and instead, "damn, I really need to go bulletproof my software so it doesn't delete people's data like that." In that context, there's plenty to criticize and learn from in Dropbox's mistakes here.
That's wonderful, except that this thread is about a developer being added and then removed from teams while working at a startup and this triggering data loss. This isn't about grandma storing her pictures of the kids. Grandma ain't gonna join "teams". That's for developers.
Did you read the article this thread is about?
You could, of course, take the approach that Dropbox should be there to protect everyone from themselves.
I have, in other comments, given the example of my own usage, which guarantees that no data can ever be lost by Dropbox. It's simple and it works. I've been constructive here. If you implement my approach you should be safe. Still, don't take my word for it, implement and verify. That's engineering.
I have also, in another comment, posted about a feature request that might help insure that this does not happen to casual users:
Dropbox, if possible, should modify their client software such that files can only be copied in and out of Dropbox folders and never moved. For the adventurous, they could make this an option that is set by default but can be disabled. A huge warning about the potential for data loss would accompany the act of disabling this feature.
In that regard, I've been as constructive as one could be in this thread.
In the end, you can't protect everyone from their own mistakes, incompetence or carelessness. And, no, I don't think everyone is stupid. We all make mistakes. I had a very painful data loss event in college, so I learned my lesson very early on. I have never lost valuable data since.
I am not going to blame a service for something that is the responsibility of the user. Engineers, in particular, have no excuse for data loss. A 2TB external hard drive is about $100. Please.
You could also argue that business users in this day and age cannot be computer illiterate. Being "literate" does not mean being able to open a browser and push a mouse around. If a business owner is not capable beyond that they ought to be smart enough to hire a qualified IT service to help them manage and secure their systems. If they do have in-house IT then data backup and security ought to be one of their top functions.
Private users are a little bit of a different story. There's a huge chunk of them that, today, could loose every digital asset they own and have no way to ever recover it. This is still a business opportunity for a startup somewhere. Services like Carbonite and others abound:
At one point you have to point your finger at the data's owner and say "You, and only you, are responsible for your data".
I can guarantee you that if you read the TOS of any online data backup service there's a clause there about the potential for accidental loss. There is no way in hell that an attorney would advise anyone not to have that clause there. And, no matter how good of an engineer you might be, there is no way you would risk it all to offer some kind of an absolute no-loss guarantee.
The test is simple: Would you, personally, be willing to be sued into absolute poverty for issuing that guarantee? Probably not.
So, we can sit here in righteous indignation for me daring to suggest that users are ultimately responsible for their data and lie to each other or accept the reality that the reality, developer or not.
Down-vote away, but I am right.
If we go back to the very original article that this thread is about, the programmer who joined the startup team seems to have not had any explicitly-created local backup of the data in question. He got lucky. He said "The client left all the files on my machines, so I didn’t lose any personal data - it wasn’t a catastrophic failure.". That's luck, that is not planning. So, he did OK, he talks about having to re-upload some 2GB of data to a new account. He was off and running after that. I can bet you that, unless he isn't very smart, he now has private local backup of his data in case something like this happens again. And that's the right way to handle it.
Don't trust anyone with your data unless you can independently verify the ruggedness and reliability of their solution. Period. If you did not do that, engineer or "civilian", total data loss is on you, not the service.
I added a "Development" drive as an experiment a few years ago. I'm still on the fence about it.
I own more than one business and we also do work for other businesses, so, generally speaking, the "Data" drive holds business data partitioned-off into appropriately-named directories.
I also have what I call "support" directories there. For example, I have a "SolidWorks Support" directory with models, macros, tools and other items that are there to support work with that software. The same is true of Photoshop, Altium Designer, Web Design and other segmentable tasks.
Then there's more mundane stuff, like "Digital Photography", "Personal", "Scans", "Temp", "Outlook Data", etc.
Projects started to kind of pollute the data drive in some form. Also, the directory hierarchy started to become a little crazy. For example:
D:\<company name>\Projects\<project name>\Design\Mechanical\<project work folder>\Suppliers
D:\<company name>\Clients\<client name\...
Even there were hardware, embedded and web projects with their own requirements. And, even though we use Git, I still have the habit of making full copies of directories every so often to freeze thing in time. With projects such as Solidworks mechanical designs or some electronics designs using source control can be painful, so, full backup copies of work directories works very well. Storage is cheap.
At one point I thought that I should segregate active development from finished or shipping product. In other words, make a distinction between "project" and "product". That's why I introduced the "Develpment" drive.
There's another reason. When doing FEA thermal and/or flow simulation for various projects the Data drive started to become clogged-up with simulation data. As you navigate through many iterations of design ideas and simulations you can easily fill-up gigabytes. I didn't think that this belonged anywhere near the "Data" drive.
Finally, some tools (Xilinx ISE back then) did not take kindly to paths with spaces. So I always had to have a separate folder for Xilinx projects which has always bothered me.
This "Development" drive now allows me to create a simple folder with a short path:
This "feels" simpler and it "feels" like the right way to do it.
I also have an "xampp-sites" folder on there for web projects. The projects can live in their own directories in the "Development" drive:
These have full source control, work folders and generally contain all the work resources for a project (for example a Photoshop work directory). The "xampp-sites" directory is setup as the testing server and only sees an upload of the files that would normally go to the real production server (easy to do with tools such as Dreamweaver --which is never used in design mode here!).
Once a "project" graduates to being a "product" it is zipped-up and copied or moved into a "Products" folder under the appropriate company directory in the "Data" drive. As an example, if we are talking about an iOS project it turns into a product once it is submitted to the app store and approved.
The jury is still out on whether or not this is a good idea or simply a reflection of being insanely anal about organizing data. I don't know. I am always open to new and interesting ideas on this front. I try to think of the "Development" drive as a dirty or work drive of sorts where I can mess with a lot of things without polluting the "Data" drive. I have this implemented in one machine and have not been compelled to do so in other machines yet. Still thinking about it.
>Well, uploading 600GB to your "backup" would take somewhere in the order of 100 days of continuous 24/7 upload with my current DSL connection (~600Kbps upload speed). That's a third of a year. Nope, it's not backup.
Apparently you've never heard of incremental backup? Dropbox may not be a backup but there are many services that backup via the internet. Assuming you don't change backup systems every month, 100 days for the initial sync is perfectly acceptable.
> Apparently you've never heard of incremental backup?
> Dropbox may not be a backup but there are many services that backup via the internet. Assuming you don't change backup systems every month, 100 days for the initial sync is perfectly acceptable.
Oh, please. Again.
If you don't have local backup, for a third of a year you have no backup whatsoever. Even after that, depending on the nature of your work, even incremental backups could have you unprepared for failure for days.
Not saying at all that remote backup is a bad idea. Not at all.
Remote backup BY ITSELF, without local high-speed backup that you control is a very bad idea.
The best pattern is to have single or redundant backup under your control (yup, use that "incremental" thing I didn't know about) and remote backup. Don't expect anyone backup destination to be 100% reliable, not even local backup. If your data is important you need multiple redundant backup, local and remote. Then you can sleep at night.
In my particular case, no, sorry if I gave that impression. I have not seen anything yet that would compel me to use remote backup as my primary backup destination. Potential failures aside, it's slow in both directions.
My local backup strategy is a collection of external --and these days inexpensive-- hard drives as well as a large rack-mount NAS RAID array. Each of about a dozen systems has it's own local external backup drive right on the desk next to the computer. Some have dual local backup drives. We are talking in the order of $100 for a couple of terabytes today. Then, a number of systems also backup to NAS. Every so often we rotate drives for longer term storage at a fireproof external location. It'd take a lot more than Dropbox or any service having a glitch for me to loose any data.
I really don't understand folks who don't, at the very least, have one external USB backup drive on their system. On OSX you have Time Machine which is ridiculously easy to use. On Windows you can spend a few bucks and get Norton Ghost and you are good to go. All-up, probably not more than $200 per system and maybe half an hour to set it up.
Do that plus my recommendation to host your Dropbox location on a dedicated partition in order to force a copy operation during drag-and-drop (both Windows and OSX) and you will not care less about anything that happens at Dropbox or any other service.
It's about engineering, not hoping for, a system to protect your data.
As far as remote backup is concerned. I'd be interested in a system that might allow me to send them encrypted disk images on physical media for backup while providing some online access to the same.
Even with incremental backup you have to do a full backup every so often. In the case of our Windows systems running Norton Ghost, they are setup to do full backups the first day of the month and incremental backups every day after that. It's dead-easy, reliable and works great. Saved my hide a number of times.
A full backup of about 600GB happens in --I think-- about three or four hours. That's the problem with remote backup, the same full image would require a third of a year on a typical DSL connection available in the US today. Actually, it could take twice as long, two thirds of a year, because you would have to interrupt your backup in order to get your bandwidth back for use during business hours. So, if it takes you nearly a whole year to backup this much data the whole thing is just-about useless as implemented. Your incremental backups are likely to take days and you can't even consider the idea of doing full images every thirty or sixty days. That's what's broken about the concept of remote backups without even looking at the issues with potential software bugs at the various providers that could lead to data loss.
A more usable system would be one that, as I said, would receive my full images in physical media to absorb into their storage arrays for both backup and remote access purposes. If you needed to recover a few files here and there you could easily do so over a decent DSL connection. Full recovery would require physical media being shipped to you at a greater cost. Every x number of days you'd send a new full image set and go incremental after that.
The game changer here will be if we ever get to 100Gb network connectivity to the home and office. That would change the landscape in amazing ways. You could talk to remote storage probably as fast as you talk to local storage. At that point in time, having multiple redundant and geographically separate remote backup locations might very well be the most sensible approach to an organization's backup strategy. Such a system could even talk to a locally installed "backup server" in order to make sure that if connectivity is compromised in some way you still have access to your organization's data during the blackout.
The topic of backup is conceptually very but becomes really complex when you consider the multiple potential points of failure and how to deal with them.
This is why I don't consider any issues at Dropbox to be serious. I obviously don't think of them as backup. And they can't convince me to think that way no matter what they do or say. This isn't to say that I think the service is bad. Not at all. It's because I've been around and I've seen too many failures (some of my own) that I take a very careful and guarded approach to my data. And that's healthy. I use Dropbox for team communications. I almost think of it as a really neat way to "FTP" stuff around. So, instead of setting-up my own FTP server and having to manage my users and storage I can use Dropbox. No data is ever moved to Dropbox. All data is copied to Dropbox. That means that the data remains locally stored and, more importantly, locally backed-up every night. So, through engineering, failures at Dropbox or anywhere in between my DSL connection and their data centers are of no consequence whatsoever.
A full backup of about 600GB happens in --I think-- about three or four hours. That's the problem with remote backup, the same full image would require a third of a year on a typical DSL connection available in the US today. Actually, it could take twice as long, two thirds of a year, because you would have to interrupt your backup in order to get your bandwidth back for use during business hours. So, if it takes you nearly a whole year to backup this much data the whole thing is just-about useless as implemented. Your incremental backups are likely to take days and you can't even consider the idea of doing full images every thirty or sixty days.
The thing is, you don't need independent full backups within a single online backup provider, and they would probably deduplicate your backups anyway. Incremental backups work just fine, and will always get your data backed up that night unless you are the kind of person that generates many gigabytes of content in a single day.
I use Dropbox for team communications. I almost think of it as a really neat way to "FTP" stuff around.
Yeah, I can understand that. I was considering replying to one of your other posts with the comparison, but I didn't want to start too many thread at once. The problem is that when you use dropbox as a better FTP, you lose out on a lot of the benefits of syncing. No longer can you go to a different computer and pick up right where you were, if you didn't happen to copy in the files again since the last change. If you're sharing files with a coworker you no longer have any idea where the most recent version is. You don't have a full list of file versions. And your FTP-method of dropbox doesn't really have anything to do with backups. You could move files into the dropbox folder and then set it to be backed up every five minutes if you wanted to.
Obviously there are a number of use patterns for Dropbox. I just don't use it as a "live" store. The whole concept scares me. As far as moving to another computer and having access to the latest files, I can always VNC into a machine or VPN into the network if I am outside.
I just couldn't bring myself to using it that way. Our files are our work product. There is no way I could consider having the only copies of the files connected to a service that could cause total loss of data. It is, in my world at least, an absolute non-starter.
With regards to the idea of sharing files with a coworker and not knowing where the latest files are, well, that's what Git is for, isn't it?
As a matter of principle I have a bad reaction to the idea of calling a data loss event "Another Dropbox horror story". And this has nothing to do with Dropbox or any other service. Dropbox is not responsible for your data. You are. If the data is important enough that total loss would be catastrophic, what are you doing placing all the eggs in one basket? It isn't a Dropbox horror story, it's a story about someone who didn't care enough about their data to safeguard it and then the blame is shifted towards whoever last held the data. That just ain't cool. I own my fuck-ups. It isn't fun, but I stopped blaming others for my crap a long time ago. In the end it works out better that way.
Does Dropbox have issues to fix? Sure. What piece of software doesn't. Knowing that software is imperfect is more of a reason to not, again, place your valuable eggs in one basket.
I have a feeling that there are a lot of young and inexperienced people surfing HN who have never lost anything of significance. They come across someone like me who calls bullshit when he sees it and is willing to stand for what is a solid position and they don't understand it. The only tool they have to express themselves is to mindlessly down-vote cargo-cult style. That's fine, with that approach they'll learn soon enough.
I don't know of anyone --not ONE person-- with say, twenty or thirty years of successful work in computing who would trust the only copy of their projects to any one service --no matter what they claim or how good they might be. Hell, most of these people, just like me, would not trust the only copy of their projects to any single device, local or not. Shit happens. And some of us have seen it happen and have had to clean-up the mess on more than one occasion. Eventually you learn.
I do love Dropbox, but they are not going to come and redo two years worth of work if something goes wrong. And that can happen. And, if that were to happen, I would not blame them. I'd blame myself for being a moron and not having my own backups. Maybe I'll figure out a way to use their active sync technology while still satisfying the requirement for solid redundant backups. Until then, I have enough problems with other stuff to have to worry about having only one copy of our projects stored anywhere, local or not.
Critical files or not, data loss is a 100% unacceptable bug. I really doubt you will be able to convince anyone that data loss is okay. It's not.
With that said, I think Dropbox is a pretty amazing service and I credit them for getting Google and Microsoft to wake up and make some similar services which are solid and different in some respects (Google Drive and SkyDrive).
Not saying anywhere that data loss is OK. Absolutely saying that data loss is a part of life, no matter what anyone says or chooses to promise.
So, you can travel two paths: One where you believe that service X can absolutely-positively not loose your data for whatever reason. Or, another, where you understand that your data can be lost at any time and for any reason by both remote service X or even your local $100 USB drive.
If you choose the first path, you are going to get stung sooner or later. And, it is my contention that blaming the service provider for your total data loss is nothing more than not wanting to admit the truth of the matter.
If you choose the second path, which could sound really paranoid but is actually very realistic, you take steps towards creating enough redundancy that a single point of failure isn't going to burn days, months or years of data. And, while no absolutes exist in this world, it doesn't take much in this day and age to have a system that will almost guarantee that your data is safe from most loss-inducing events.
I consider problems like data loss to be serious engineering problems. The difference is that I choose to look in the mirror first and blame myself first before pointing the finger at someone else.
> This forces a COPY operation rather than a file MOVE if you drag-and-drop files into any Dropbox folder. Which means that everything in the Dropbox folder could be trashed tomorrow and nothing whatsoever would be lost.
This removes the main point of Dropbox, that when you edit your files they're automatically continually re-uploaded. If you're only making copies into Dropbox, you might as well just mount an SFTP server and save a bunch of money.
Using anything this way (meaning, without testing and without knowing of what you might be exposing yourself to) is absolutely dumb.
The down-vote wasn't for posting about his friend's experience. The down-vote was about his friend and also for blaming Dropbox for something that was 100% his friend's responsibility.
Let's bring it down to a "skin in the game experience".
- Save-up $250K
- Invest all of it in a startup
- Hire a couple of programmers to help you out
- None of you setup any local backups
- You choose to rely on web service X for backups
- You don't test anything 'cause everyone is doing it this way
- You still don't have local backups
- A year later something goes wrong and you loose all of your work
- You just lost all of your time, money and the startup tanks
- You go online and blame service X for your loss
Clearly service X had nothing to do with the series of decisions and actions that led to your loss. Your loss was due to "pilot error" and plain-old home-grown incompetence. Nothing more, nothing less. Blaming someone else might feel good, but the reality is that a less-than-professional treatment of the matter is what caused the loss. Service X was just along for the ride.
I quit blaming others a long, long time ago. In my experience you can do forensics in almost all of these situations and identify someone who was either dumb, lazy or incompetent who ended-up giving you the gift of irrecoverable data loss.
Like I said in my original post, I was EXTREMELY lucky to have learned this lesson while in college. I lost six months of work of a project for the Physics department to a drive failure. Horribly painful. Ugly. I am so thankful for having learned that lesson in that context. It would have sucked to have learned it outside of academia and while working on projects where data loss could have resulted in significant financial loss.
My sentiment stands: With local storage being so plentiful and inexpensive these days there is no excuse for not having multiple redundant backups of anything that is important. You can even have a policy of shipping physical redundant backups to another storage location to mitigate the possibility of your local backups being compromised by something like a building-wide fire.
In another post I treat the other fundamental issue of backups over connectivity such as DSL.
This has NOTHING to do with Dropbox or not trusting it. As a developer, if you've been around computers for more than a microsecond the NECESSITY and VALUE of backups don't need to be explained. Dropbox is not backup, no matter what anyone says.
tl;dr: The scenario described by the OP is now less likely to occur. Since October, individual Dropbox users are encouraged to create separate accounts when invited to a Team and warned that Teams admins will have control over the account.
We want individual and Team Dropbox users to have the best possible experience. Some users want to migrate their personal accounts into a Team. Others are much happier with separate personal and Teams accounts. We’ve been working to make that choice much clearer for the account holder and the Teams admin.
If users do choose to merge their personal account with a Team, things become a bit tricky. Teams admins want better control of data within a Team, and users want easy access to their personal stuff, but it’s not possible for us to differentiate between Team data and personal stuff in the same account.
Here are some thoughts on the points raised by the OP:
* Better support for multiple accounts: Users can quickly switch between using personal and Teams accounts on the web today and we intend to make this better across our platforms.
* Improved messaging to Teams admins: We plan to provide better messaging to admins before disabling an account that was migrated in.
* Disabled accounts are not immediately deleted: We can work with Teams admins and users to sort out account issues and recover users’ files.
We’ve contacted the OP to help resolve his case and are sorry for any pain this caused!
> Teams admins want better control of data within a Team
Some admins do I'm sure. We had to cancel Teams because of this. :(
Let me explain: most people at Steamclock sync personal stuff in their Dropbox, and we want to also share a large folder. Upgrading everybody to Teams seemed like a good idea, but the "company owns what's in your dropbox" thinking forced everybody to have two separate accounts, and the "only one account can sync to your computer at once" restriction makes one of your two Dropbox accounts fairly useless.
We can only go back to Teams is when multiple accounts can sync to one computer seamlessly. All the problems around DB for Teams (privacy and TOS issues, personal and company data getting mixed up, weird consequences when people leave the plan) are caused by the lack of multiple account sync.
That said, we'd be happy to just pay monthly for a shared folder that doesn't count against users' storage limits. In the meantime, we have the company Visa on a bunch of personal Dropbox accounts which sucks.
For personal use though Dropbox is great. Merry Chrismas!
Our company would also love the option of paying for shared folder, instead of teams. We currently have teams, but it's not quite what we want, and if we temporarily want to share a folder with a client it is an big hassle.
I'm considering Google Drive for company filesharing. I've always found shared folders on Dropbox are pretty awkward. Hopefully GD will be a bit easier, but only time will tell. The one thing I can be sure of is it won't need to interfere with peoples personal Dropbox accounts.
I've found that shared folders mostly just don't work - others can't see the files I put in there - or at least most of them. I don't know what else I could do. (If I share one file with a direct email link generation, that works, so the files certainly are there.)
Maybe I ran into some undocumented antipiracy feature as they were self made AVCHD videos of a few minutes length.
It's probably against their TOS, but it is possible to run your personal dropbox account and a team account on the same machine. It's a pain to set up, but works well once it's running. And since you're paying for both anyway, it seems like it's at least in the spirit of the TOS.
Well it works but it's like running on 3 pistons. It's a serious pain to set up, when the Dropbox client is updated you need to update the 3rd party standalone client which breaks half of the time, and you need to manually switch to either one or the other (if you have one 'official' and one 'pendrive' version). Oh and it messes with your icon overlays on Windows. Massive pain, I regret going Dropbox for Teams but I'm sort of married to our setup now. The way it works is not clear at all - I had issues similar to the OP's which required support to resolve.
First of all, in Dropbox's defense, I probably did not give you guys enough time to respond before going public with this (also, living in Israel I sometimes forget that Sunday is a weekend for most people in the world). I apologize for that. I am not a journalist/blogger, and I was driven more by my emotions than by "journalistic ethics" (?) when submitting to HN.
Second, according to your support email (updated the original post with it) it looks like my particular case is going to be resolved. However, it does not seem to resolve the malicious use case I hinted at (and which people on HN did not seem to want to discuss that much): Give someone a terabyte as a gift, and then delete their account. In fact, from the support email it seems like it's even worse: The support staff will need the team admin to approve the account re-enabling. In the malicious case, the admin would not approve.
[EDIT: Recalled a third point I wanted to make.]
Third, regarding what you said "it's not possible for us to differentiate between Team data and personal stuff in the same account". I simply don't understand why this is true. Maybe the general case is not like mine, but my Dropbox folder just has a bunch of subfolders, exactly one of which belongs to the "team". Is it possible that certain folders have mixed personal-and-team content? How does that even work?
If you saw my account (I don't know if you can... but your code can) it would be blatantly obvious which folders (all but one) are personal.
When migrating an account into a team, the account and all the data in the account becomes managed by the Teams admin (shared or otherwise). Letting an existing account to join a team lets us smoothly support the situation where a user has created a Dropbox account separately and needs to move that account into the team.
For most cases though, users should create a new account for the team. The Dropbox for Teams sign up process guides users towards creating a new account when joining a team for this reason.
From your reply, it sounds much worse than I thought. I had no idea my team admin could see all my data until now!
If joining a team account changed after your October changes, then maybe it's okay now. But I included in my post the email I got for joining the team, and it makes no mention of the fact that the team admin now owns my data. One would think this email should be less bland and more cautioning.
> ... individual Dropbox users are encouraged to create separate accounts ...
If only your software actually worked in that scenario. Every single piece of your software is hard wired to believe you only have exactly one account. For example the Windows, Linux, Mac, Android and iOS clients, as well as the website. To see your other account involves logouts and logging back in again having to enter the other account email and password (not saved).
> Users can quickly switch between using personal and Teams accounts on the web today
Huh, the only thing that works for me is to logout and log back in again. I have a personal paid account and a separate team account. What I would expect is how google do it - the account dropdown having your other logged in accounts available for selection.
I had checked before posting and there was no such option. I did manage to make it appear in the end but it required logging out as my personal user and then logging as the team account and then logging in as personal again.
Why not just make 'Switch Account' always be present rather than having one blessed starting point account? The wording also implies this won't work if you have two team accounts.
I meant the wording on the website. If I log out, then login to the team account and then choose "Switch Accounts" the resulting page says "Sign in to your personal account". That is what implies to me that it would have issues if I had two team accounts.
You should really consider redesigning this feature. Having "Teams" control all the personal accounts of their employees means none of my companies can use it. So, we're stuck with folder sharing for the foreseeable future.
I'd expect Teams to work like folder sharing, but with the storage not counting towards the individual account space limits.
Our company has a corporate account but none of the employees want to use it for this reason. If we could make "corporate" folders and share them using the same account, but not lose the storage credit, that would rock.
The second link is basically a modification of the method from the first link, and has been working well for me for a few weeks. One caveat is that any apps which make us of dropbox for syncing (e.g. 1Password) seem to automatically use the first/normal dropbox instance, which can't be changed.
Out of pure curiosity, why is it that the OP's original requests for help were ignored? Is it recommended that to get better support you should make a public case of it? A lot of larger companies seem to operate this way.
Honestly, OP didn't give them enough time to respond. OP said that it was only yesterday that the team revoked his account. I could imagine that given it was both a sunday and a day or two before Christmas (depending on where you are) that the Dropbox support probably wasn't fully staffed.
* Teams admins want better control of data within a Team, and users want easy access to their personal stuff, but it’s not possible for us to differentiate between Team data and personal stuff in the same account.*
Of course it's possible. You just don't want to put the work in it make it happen.
> * Better support for multiple accounts: Users can quickly switch between using personal and Teams accounts on the web today and we intend to make this better across our platforms.
On Linux, I've gone out of my way to install the (pseudo-unsupported?) daemon headless under multiple users to get concurrent access to work and person accounts; would like to see this improve (while dealing with the "I have ten free accounts" and other pathological cases).
How hard can it be? At least you could restore all the files that were present on the user's account before it was made into a Team account... If courts can do it in case of a divorce, I guess a technology startup could as well.
tl;dr: The scenario described by the OP is now less likely to occur. Since October, individual Dropbox users are encouraged to create separate accounts when invited to a Team and warned that Teams admins will have control over the account.
I am going to be blunt here and just say it: the 'Team admins will have control over the account' part is completely stupid and not well thought through.
Becoming part of a team should be about sharing data. Not about handing over accounts.
Sounds like you ought to figure out why people want to merge their personal accounts, and then implement that functionality in a way that doesn't require merging such that it's not possible to reverse the change afterwards.
This may be way harder than I make it sound, since I don't know about the hurdles this would face. But that does seem like the place to aim.
> Disabled accounts are not immediately deleted: We can work with Teams admins and users to sort out account issues and recover users’ files.
Wait a sec, does this mean when I delete my files they are ... not being deleted?
> We’ve contacted the OP to help resolve his case and are sorry for any pain this caused!
Great, good for him. So basically next time when I or others have problems with Dropbox, post a rant about it and pray it will get to #1 spot on HN to get customer support response? Ok, dully noted, thanks.
Wait a sec, does this mean when I delete my files they are ... not being deleted?
This is true of almost every service on the internet. If it wasn't, whenever the only disk that had your account on it failed you would be absolutely livid about the fact that they never made any backups. (It is generally infeasible to purge backups of any deleted data, because they're stored in compressed blobs.)
I don't know where to start collecting anecdotes and appeals to authority, but I shouldn't need to. See my parenthetical above--how would you go about digging through terabytes of compressed backups to find every instance of a single file and its corresponding database record so that you could delete it? It would be possible to engineer such a system, but in most cases it would be an extremely poor allocation of resources.
If you upload something to the internet and you want to be the only person who is ever able to pull it back out, you should encrypt it.
Ability to recover deleted files for up to 30 days is a prominently advertised feature of Dropbox. It would be unthinkable for them not to allow that, given how easily files can be lost by users who aren't familiar with how syncing works.
Look at all of the people who consider Dropbox a "backup" service, and imagine being the guy on the phone who has to explain that yes, actually, deleting a file on one system is supposed to delete it from all other clients.
I will say that this is a really interesting story, in that it illustrates how Dropbox, like all of the similar services on the market, is handy for many use cases but ideal for almost none. A service that's basically a "scriptable" Dropbox would be pretty compelling.
I appreciate that they don't delete files because this lets them offer some convenient features related to backup, account history and uploading. The biggest reason not to delete concerns sharing. If I share a folder, share a file, then a user deletes the file from the folder, I want to be able to retrieve it as I've sometimes had to do. I don't want someone else to be able to irretrievably destroy it on the Dropbox server.
However, I agree that users should be able to delete a file from Dropbox' servers if the user, since the beginning of the file's life on Dropbox, had the only account possessing the file on Dropbox. The procedure should involve a lot of red warning buttons and re-entering of the account password. I think this should be possible because some files really are too sensitive to be shared, and I can see how they might accidentally be pasted or dropped into the Dropbox folder and instantly duplicated. If the file is dropped into a shared folder, then there is probably nothing to be done because obtaining multiple permissions to permanently delete a file would be so complex socially.
> it’s not possible for us to differentiate between Team data and personal stuff in the same account.
I am really curious about this statement. Maybe you can explain a little further. I am thinking about something like this:
git add .
git commit -am "Account state prior to joining teams"
git branch personal
git branch team_x
git branch team_y
git checkout team_x
git commit -am "Initial commit for team_x"
git commit -am "Did some more work for team_x"
git checkout team_y
git commit -am "Initial commit for team_y"
git commit -am "Did some more work for team_y"
git checkout personal
git commit -am "Added Dad folder for xmas pictures"
When someone leaves a team:
git branch -D team_x
git branch -D team_y
Obviously merging branches would not be allowed.
In other words, I am thinking that most of the heavy lifting required to manage the on-boarding and off-boarding of someone from teams, even multiple teams, is already done for your in the form of Git. Wrap it with some business-specific code and it could be really slick. Of course, that alone could be a six month project.
Anyhow, just seat-of-the-pants without much thought applied to it. Curious to hear your opinion.