

Moving data centres in 9 minutes - cornet
http://engineering.freeagent.com/2013/02/07/going-underground/

======
jewel
I was hoping for details about how it was accomplished in such a short time,
but the article mentions nothing but "special magic sauce".

There are obviously complex ways of accomplishing this, but here's how I've
managed to pull off fast server switches in the past using nothing but rsync:

1) Lower your DNS TTL.

2) Do an initial rsync -a --del of the data, with the master still in
production. It's OK if this takes a long time.

3) Do more rsyncs of the data, which should be faster each time. Time it so
you can find your minimum time. Since the filesystem layout gets cached, this
can be very fast (just a few seconds for hundreds of thousands of files).

4) Shut down services on the old server, such as cron and apache.

5) Run the final rsync with services down on both hosts.

6) Switch DNS entries to point to the new server while the final rsync is
running.

7) Bring services up on new server.

8) Raise TTL back to normal values.

This even works well for medium sized mysql databases, as long as you're OK
with the downtime being I/O bound relative to the size of the database.

~~~
cornet
That is one way of doing it and I've done similar migrations using a similar
method in the past. A lot depends on your existing infrastructure and how your
application works.

Hopefully we will reveal more about the "special magic sauce" in future blog
posts.

------
deleted_account
Why after months of research, stringent requirements for data centers, and
general hand-wringing, would you consciously pull the plug for 9 minutes?

Edit: That sounds snarky; didn't mean to be. It just struck me that a lot of
energy was put into choosing The Best[1] data center and still there was down
time.

[1] <http://dcurt.is/the-best>

