

Why you should use Continuous Integration and Continuous Deployment - narfz
http://blog.teamtreehouse.com/use-continuous-integration-continuous-deployment

======
akbar501
Great post. I could not agree more. To further the article, CI/CD are often
neglected by small teams / startups. However, startups would benefit more than
large teams (as a percent of time) by automating the build and deploy
processes.

The only point I've heard a counter to is "Automate Rollback". Noah Sussman
(previously of Etsy) mentioned that they never rolled back (at a Groupinar).
When a bug was pushed to production, they would fix in dev, test, and deploy.
If I remember correctly, he mentioned their time to fix was < 7 mins (or
better).

Also, when I first started in my career, one of our most senior admins stated
something very similar. In a meeting, I suggested we have a rollback
mechanism. The response was that code is like a river, keep it moving in one
direction.

In practice, I've found that rollbacks should be reserved for catastrophic
events where you are literally running your deployment software to reset
infrastructure, dependencies and application code.

~~~
pjungwir
Came here to say the same thing. You can't roll back without rolling back your
database, but I've yet to see a good way to accomplish that. What happens to
the data you collected since your last backup? How much downtime do you need
for the db rollback?

Also, when you do your rollback, what happens to?:

\- data in caches like memcached or redis

\- images on S3/etc whose URLs are stored in the database

\- session cookies on users' browsers

\- state you've created on systems you integrate with, e.g. fulfillment,
warehouses, email lists, . . .

The database is only a portion of your app's "state", and if you suddenly
wrench it out of sync with the other systems, you're going to cause problems.

If you absolutely need to stop the world while you develop & deploy a fix,
then have an "unavailable" message you can quickly drop in for a few minutes.

Maybe you'll find a bug that really does require restoring to a backup, but
that should be rare and will cause a lot of pain, so making this automated
seems very scary. You want to do your restore in a controlled, considered way,
so you don't cause even more data mangling than you've already got.

------
aytekin
"Code rots. It should always run somewhere."

Words of wisdom.

