

Continuous Deployment: just ship your code to customers as often as possible - TimothyFitz
http://timothyfitz.wordpress.com/2009/02/08/continuous-deployment/

======
akronim
_> Continuously deploy. Every commit should be instantly deployed to
production._

ah... no. Maybe this is just viable for a single developer, as a substitution
for continuous integration. But with multiple developers checking in, on a
complex system your site will be down. A lot.

If you're frequently having problems where new deployments are bringing down
the site, then doing that more often isn't the answer. As un-fun as it may be,
you should be looking at how you could have caught the problem _before_ it
went live, whether that entails more testing, more robust design or something
else.

~~~
moe
Agree'd. It seems like the author either has no customers or very
understanding customers. Pushing every commit to production is just non-sense
by any metric. What about commits that muck with the data model where bugs
might potentially destroying production data? What about complex developments
that consist of more than one commit?

This almost reads like a link-bait. I somehow doubt the author really believes
what he's writing there.

~~~
chadaustin
Well, it's certainly not "no customers"... Timothy is referring to IMVU.
Whether the customers are understanding or not is another issue. However, when
we implemented this "cluster immune system", we had outages all of the time.
But every single time, we took steps to prevent that class of failure from
happening again, and now we deploy code to the cluster twenty times a day.

~~~
moe
Well, I'm not against frequent deploys. I just found the article a bit light-
hearted in tone - as if no testing at all was going on.

No matter how many classes of failures you have ironed out, in any reasonable
complex system you will still regularly have regressions that are not caught
by a quick "two eyes" check.

------
ivey
I wonder if you could do some automated A/B system here, and instead of every
commit, just do it once a day: Every night, update one set of servers with the
latest stable branch code. Route N% of your user to those servers. If faults
start showing up, push N to 0 and notify developers. Otherwise, slowly move N
towards 100 throughout the day. By end of the day you're either fully
deployed, or rolled back.

(I already see faults with this...it's presented here as a thought experiment,
not advice)

~~~
inerte
What faults?

I ask because I swear A/B and multivariatate tests have been around my head a
lot lately, and when I finished reading the article, the first thing I thought
was: Why not just deploy to 1% of the users and see if it works?

Then I thought about how hard would be to manage multiple versions of the same
software, specially data, amongst different user. Certain features presented
to the 1% might be incompatible with the other 99%. But that's a technical
problem. Very hard to solve, but manageable. Then I imagined somekind of
framework that would make communication between different versions of the data
floating around easier, with "how to transform" data from-and-to version 1.1
and 1.2 easily.

Anyway, I am really curious, because it sounds like a good solution :)

~~~
DenisM
As far as data transofmration goes you have two options:

1\. Something akin to ActiveMigration is RubyOnRails world. This allows going
back and forth different versions of your data's schema.

2\. Use a more open data scheme such as Google App Engine uses where
adding/removing properties to an object is not as disruptive compared to SQL-
based solutions.

~~~
TimothyFitz
It turns out the system works fine with SQL based alters. We do have to do
real work to deploy expensive alters (apply them to standbys, fail over,
repeat, or worse) but in general it's cheap to change schemas.

Unfortunately, it's very manually intensive to roll back schema changes, so
it's one of the few places where we put old school process in place (a DBA who
reviews all schema changes prior to deployment)

~~~
DenisM
Did you look at ActiveMigration? Even though I never used ruby/ror I found it
a very sensible approach to thinking about schema evolution.

~~~
TimothyFitz
ActiveMigration really solves a different problem. Our problem is that adding
indexes or altering popular tables is impossible to do on a live and in
production database. To get those changes out we have to go through quite a
bit of extra work. It's really a MySQL limitation, not a process problem.

