

Deployinator: Etsy's deploy tool open sourced - cloudman
http://www.slideshare.net/OReillyOSCON/put-a-button-on-it-removing-barriers-to-going-fast

======
stephth
The slides without the talk are mostly useless, you're better off reading
their Deployinator blog post:

<http://codeascraft.etsy.com/2010/05/20/quantum-of-deployment>

------
Cushman
Does anyone know if this talk was recorded? It's the sort of deck that leaves
you wondering what the presenter was saying.

Edit— Found this from NYLUG in April:
[http://www.archive.org/details/NYLUG-2011-04-20-Etsy-
Deployi...](http://www.archive.org/details/NYLUG-2011-04-20-Etsy-Deployinator)
(Talk starts at around 4:20, runs to 1:21:06.)

Haven't watched it through yet, but it looks like mostly the same slides.

------
cdavid
Continuous deployment sounds right to me in principle, but I have yet to see a
system to handle rollback in the face of database handling (one slide
mentioned this briefly). When you have a bug (say you delete the wrong data),
how do you rollback this ?

The only solution I can see is one where databases are handled differently
(they generally are anyway at the deployment stage), and IMO, this is the
challenging issue. Continuously deploying app servers is not trivial, but I
would consider it a mostly solved problem.

~~~
Pewpewarrows
The problem is pretty universally solved:

1\. Use some sort of schema migration tool to keep track of and version all
changes you've made to the database since your very first commit.

2\. Never do a backwards-incompatible schema change. Never. Time and time
again these are the ones that will completely bite you in the ass when the day
comes that you broke production with a deployment and desperately need the old
version back up and running. You're going to end up losing data and shit will
hit the fan. It doesn't take much code-wise to support the old scheme, and it
will ensure that you have zero downtime while rolling out your updates.

3\. Never delete data. The only exception being if some user-submitted content
violates a law, like child pornography. In which case, it shouldn't be part of
your migration/update process to begin with. Just use deleted flags / boolean
fields. Note in your code when certain fields are no longer used anymore
because they're only there to support legacy versions.

~~~
cdavid
I am pretty much a beginner in database handling, what would you suggest for 1
?

Concerning 2: how do handle schema fixes (example: at my previous job, the
tables were very badly designed, and the application+schema combination prone
to frequent race conditions). What to do in this case ?

As for 3, I already came to this conclusion on my own, so I was not completely
on the wrong track :)

~~~
quicksilver03
I've used DbDeploy and Mybatis Migrations with good success, they're written
in Java but they've also been handy in PHP projects.

------
AlecSchueler
Direct link to the project: <http://github.com/etsy/deployinator>

------
mscarborough
I'm curious about how HN feels about doing feature management all in trunk
with if/then blocks (Flickr, and I assume Etsy), versus a more branch-based
workflow. By having config flags you can be very particular about what
features or bugfixes are enabled/disabled at any time, but it seems like what
you are saving in merge time, you are paying by having a bunch of dead code
hanging around until someone gets rid of it.

Even in svn, and much better in git, if you can keep a sane release and
merging strategy then it's not going to introduce much overhead to getting new
features or bugfixes out. Our 20+ person team deploys a few times a day using
a branch system of (sprint + trunk/QA + release), but always looking for
different ways to do things.

~~~
mcfunley
Config flags and how we commit code are two completely separate issues. Very
simply, you need config flags to operate a site that degrades gracefully when
something is going wrong. They cannot be avoided. Is there a query in the
forums suddenly hitting the database too hard? Turn off the forums while we
fix it so the rest of the site isn't affected. Etc.

When a feature is in development, its config flag is off or enabled for
admins. We deploy it as we develop it, in pieces that are generally not more
than a few dozen lines of code long. This is another critical idea: we don't
push out all of the code for an entire sizable feature all at once, ever.
Doing that is a recipe for having a problem and having to review thousands of
lines of code to try to figure out what it is.

For new features, we generally do not remove the config flag once a feature is
live because we want to be able to disable it if anything goes wrong.

If we are replacing a feature with another one, we generally want to keep the
old one around for a little while (we generally do A/B testing or ramp up new
versions whenever we do this). After that, we do delete the old code and
reduce the config flag to an on/off switch which was probably there in the
first place.

~~~
mscarborough
Thanks for the replies. I didn't mean to frame it in a mutually exclusive way;
I'm up for ini or yaml configs any day. Am looking at this from a rel-eng
perspective.

------
tszming
For those who are interested in continuous deployment, there is a recent talk
from Flickr: [http://sna-projects.com/blog/2011/06/continuous-
deployment-f...](http://sna-projects.com/blog/2011/06/continuous-deployment-
flickr/)

------
czzarr
i'm very interested in learning more about deployment, does anyone know where
to look ? say for a small webapp that has 1000 daily users

~~~
NoahSussman
If you are specifically interested in this kind of continuous deploy process
then "Continuous Delivery[1]" is a decent book on the subject. The IMVU post
"Doing The Impossible 50 Times A Day[2]" is another good place to start
reading.

[1] <http://www.amazon.com/gp/product/0321601912>

[2] [http://timothyfitz.wordpress.com/2009/02/10/continuous-
deplo...](http://timothyfitz.wordpress.com/2009/02/10/continuous-deployment-
at-imvu-doing-the-impossible-fifty-times-a-day/)

~~~
czzarr
thanks a lot

------
drivebyacct2
Anyone have these slides in a non-Flash format? I try to advance the slide and
it just corrupts the display. Using flash to swap out static images.
Dumbfounding.

~~~
jgoulah
PDF format: <http://bit.ly/qRhsj4>

