Hacker Newsnew | comments | show | ask | jobs | submit | donmcronald's comments login

How do you handle a scenario like this...

I have a Redmine install that I want to backup. It uses both a database and the file system. If I attach a file to an issue, the attachment is stored on the FS with a reference in the DB.

I don't know how Redmine deals with keeping the FS vs the DB consistent, but assume it uses some type of transaction across both when an attachment is added.

How do you back that up without possibly getting the FS and the DB out of sync. Ex: What if you snapshot the FS right after an attachment is written, but before the reference is added to the DB? The transaction in the app will succeed, but your backup is inconsistent compared to what the app expects.

How can you get a truly consistent backup without either a) stopping the app or b) integrating with the app to make sure you're not breaking assumptions needed for consistency?

Basically, almost every backup solution I've ever seen is crash consistent at best. How is yours different?


I've been struggling with this myself in various cases. The conclusion I've come down to, is that the app has to take some responsibility to provide for consistent backups, either via checkpoints / write barriers, or via a quiesce command that can finalize in-flight transactions, and pause / buffer operations long enough for a volume snapshot to take place. The other thing I've seen apps do is provide a data export functionality, so you end up backing up the export file not the live data. But that requires extra time and disk space. Another option is to run the app in a VM, and do a live VM snapshot which includes the running state of the VM.

Bottom line is if you can't live backup your app, file a bug report with the app vendor. Although it would be really nice if there were standards in this area, so that a backup tool would just need to call one operation to put all supported apps into a hot-backup mode.


This is why I prefer to have objects in the DB if they are not too large or numerous for that to be practical, that way the transactional integrity of the DB and its backup handles this. Unfortunately it is not always practical or under your control.

From a developers PoV you can increase the consistency of your users backups by being careful how you order operations. If you make sure the DB is updated after the file is in place and tell the app administrators to backup the database first, and your blobs are insert-and-soft-delete only (you can do updates in an insert & soft delete only way with versioning instead of in-place updates). That way you are never in a position where you have a missing blob, though you might end up with orphans where inserts happen during the time the backup took to run.

Other than that, as you say, you have to stop the app or the app has to have built in support for consistent backups (or integration into your chosen solution).

To minimise the downtime associated with this you can use tricks like LVM snapshots or your OS's equivalent. That way the downtime is only as long as it takes to stop the app, take the snapshot, and restart the app, instead of the full length of time it takes to make the backup (which could be a fairly long time if it involves a full backup of a large DB), and it means you can coordinate the backups of apps that are integrated (or just used in sync) but not in a transactionally safe way, without having to stop them all for the full backup run.

That is the best you can do without explicit support in the app - there is only so much a backup solution can be in control of.


Our core value is around the simplicity of use, so instead of having to read a novel of a manual written by a crusty UNIX sysadmin, along with combining it with storage, you can create an account and copy-and-paste a single command that does all the work for you. We've even tested it with people who've never used Linux before and they were able to install it and restore a file without much direction.

With that said, we do provide the ability to hook your own scripts and commands into the backup process itself, for instance backing up MySQL you'd just put in the mysqldump command (with relevant DB and user/pass info) into the hook script uncleverly titlted 'run-before-backup.sh' in the config directory.

I personally don't have any experience with Redmine. Application-specific backups are outside the scope of our initial launch, but as we get more user feedback we'll be able to build plugins that integrate with specific applications. In the mean time, I'd both: 1) pester the developers to provide a simple backup solution for Redmine, and 2) look into either putting it on a VM that you can snapshot, or use something like ZFS snapshots and send/receive it to a remote location.


Um. From your docs:


This feature is currently in development.

So... My server goes up in a puff of smoke and I can't access the backup?


This is a beta product, and we put that up on the docs so people can understand in its current state what it's capable of and what it isn't. One thing we didn't put in the docs is that we can change the target of a new system to an old server's backup. So in the case of your server going up in smoke, you would simply open a uservoice ticket (or email me using my profile info), fire up a new server installed with JARVYS (disabling the cronjob until you're restored), and we'd change the target UUID to that of the new server, allowing you to restore from your old backups.

Soon we will have it where you can simply select any other server from your account to restore from it to any other server on your account, and the docs will be updated with how to do it.


A mirror and a front facing camera :-)


Wouldn't you need at least 2 mirrors to correct the image?


A front facing camera already flips the image so that the user sees what they usually see when they look in a mirror. So one mirror would end up being right. Except when you open the camera app, the mobile site is no longer displaying the QR code.


It seems like the basic idea isn't unique. I was thinking of something similar a while back and found https://tiqr.org while looking for similar ideas.


My experience getting a personal code signing certificate with Comodo was just as bad, but their support had the English language skills of a 5 year old and they offered fewer options for verification.

For me, they couldn't even find my phone number on my bill and asked me to highlight it and re-send it. Plus, they were absolutely fixated on getting a phone bill that showed the logo of my service provider. I've had the same phone number for 10+ years and even offered to arrange for them to call my phone company and verify the info.

Nope! After days I got fed up and used Photoshop to merge an old paper bill that had logos with a current PDF bill that didn't have logos. They don't care about verifying your identity. All they want to do is follow a pre-determined checklist of criteria so they can cover their ass and take your money.

As an added bonus, the Windows 8 SmartScreen Filter treats personal code signing certificates that aren't popular the same as unsigned code.


I'm late to the party, but I also dislike OnSwipe. The scrolling is terrible, I can't zoom at all and sometimes it crashes Safari. Even worse, the 'standard site' link on the WordPress theme is at the bottom of the page. It should be at the top.

Hopefully someone will create an 'unswipe' bookmarklet that we can use to disable it on sites that don't have an opt-out. Add me to the list of people that would like a global opt-out.

If your site uses OnSwipe and I can't disable it, I'll find content somewhere else.


Applications are open for YC Winter 2016

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact