

What is Continuous Deployment? - SanjeevSharma
http://sdarchitect.wordpress.com/2013/10/16/understanding-devops-part-6-continuous-deployment/

======
mattjaynes
First, Continuous Deployment is really awesome.

Second, you should completely ignore Continuous Deployment until you have sane
systems.

I do contract systems work for a living and have seen hundreds of live
production systems. Nearly every system I see is in some kind of serious
peril:

    
    
      - no backups
      - if backups, no docs on how to restore
      - no monitoring (or very little)
      - production passwords in the wild (former employees, etc)
      - no configuration management
      - no path to scale
      - no path to replace defunkt servers
      - etc... (I could go on for hours)
    

Often when I'm talking with a potential client, they are SUPER interested in
Continuous Deployment (or whatever the hotness is at the time), but they start
to yawn when I start talking about getting their core systems in order.

Your systems are the _foundation_ of your application and your business. You
wouldn't build your office building on top of an active volcano or below sea
level on the coast, yet many businesses happily do this for their systems.

You know all those constant news stories about massive outages, security
breaches, crippled sites, etc? I'd bet 99% of them are due to them failing at
the fundamentals of sane systems. Having seen behind the curtains of so many
companies, I'm honestly surprised there isn't more massive systems failure in
the news.

This is a critical problem in the tech world. Does it affect you? A few of you
will have amazing solid secure systems, but the vast majority of folks reading
this won't.

If you're using a PaaS like Heroku or Parse, then you have most of the systems
worries taken care of for you. You can breath a bit easier.

But if you are managing physical or virtual servers and aren't even using
configuration management (puppet/chef/salt/ansible/etc), then your systems are
probably in a very precarious situation. You might not think so, since
everything just happens to be working at the moment, but you are essentially
coding without using version control.

You would probably think that a developer that used email for his code version
control is an idiot. You would be right. But consider that emailing code
around for version control is actually _better_ than what most companies do
for their systems. Most companies set up their servers manually. They often
don't even have any docs or even shell scripts.

At one client, it took them 2 weeks (!!!) to bring up a new server. The
engineers thought it would take only 4 hours. Those engineers nearly killed
that company.

Do you want to be a 10X or 100X systems engineer? Then use configuration
management! After we put that client's systems in Puppet, we could bring up a
new server in under 5 minutes. Yep, that's 4000X faster.

If you aren't using configuration management and don't know where to start,
just use Ansible. It's by far the simplest and easiest to get going. I've
written a quick intro to it here: [http://devopsu.com/blog/ansible-vs-shell-
scripts/](http://devopsu.com/blog/ansible-vs-shell-scripts/)

If you're curious about how the configuration management tools compare, check
out my book: [http://devopsu.com/books/taste-test-puppet-chef-salt-
stack-a...](http://devopsu.com/books/taste-test-puppet-chef-salt-stack-
ansible.html)

~~~
tieTYT
IMO, the example you're using in your blog should be simplified even more.
I've never heard of Passenger before and it's not something I've needed.

But I have needed to ensure that a file exists in a particular folder on the
server. An example of ensuring that would be more universal. Then you can
explain that this vs a shell script is better because it's error
prone/difficult to run the shell script twice, etc.

But I did get a lot of value out of this article because now I know that
Ansible exists. Thanks

~~~
axisK
Ansible is pretty decent but new release seem to break backwards compatibility
quite a lot so be prepared for playbooks not working on newer versions should
you need to reuse them at a later stage. Of course virtualenv gets around this
quite nicely.

------
wdewind
> You may only continuously deploy to an environment before ‘Prod’ – UAT/Pre-
> prod…, but the environments you do deploy to should be ‘production-like’, so
> you know with very high confidence, that the final deploy to production will
> work without issues, when you do really need to deploy to it.

In my experience this is a pretty dangerous way to think about it. One of the
best reasons for continuous deployment is to allow you to push small changes
to production. This is desirable because small changes are easier to debug
when things go wrong, and are less likely to make things go wrong overall. But
the core assumption behind this is "the system is really complicated and has a
ton of moving parts that are virtually impossible to think about at all
times." Continuous delivery makes the assumption that a preprod environment is
highly likely to be similar to a prod environment, and making that true is
usually extremely hard at scale. Couple that with the fact that prod will
(especially for large, consumer facing sites) often expose you to many more
edge cases, and the fact that a developer who has pushed to preprod may not be
there to help debug when his or her code finally gets to prod and breaks
things, I see continuous delivery as a pretty suboptimal alternative to
continuous deployment.

~~~
nimblegorilla
Coming from a banking and healthcare background it seems pretty suboptimal to
build a process around waiting for stuff to break in production.

It also seems rare to find bugs in production that cannot be replicated on
developer machines so I don't think getting an exact match on the preprod
environment is a huge obstacle.

~~~
wdewind
The important part is being able to push very small changes without it slowing
you down (you need a quick and easy deploy process).

If you have a small change, it is easily testable, and the chance of it
breaking something is small. The goal isn't to discover bugs on production,
it's to make it extremely easy to fix bugs on production. Continuous delivery
batches up many small changes and thus makes it more difficult to figure out
what exactly the issue is when you discover one. I have yet to see a process
that is able to release bug free code, so knowing that bugs do get to
production as a matter of life, it is highly optimal to make your process make
those bugs easier to fix. Ultimately you end up with fewer bugs, when you do
have bugs they have smaller impact, and when you need to fix them they are
easy to track down and fast to change.

Getting an exact match on preprod is extremely difficult when you have a
relatively complex data environment (sharding etc.), a relatively complex
build environment (assets built differently in production than dev) etc.

~~~
nimblegorilla
I still don't believe it's that hard to create a preprod that matches
production. What kind of asset generation are you talking about that can only
run on the production server?

Is continuous deployment just something for teams that don't know how to make
a copy of their production server?

------
kevinpet
I don't really consider continuously pushing to staging or QA true continuous
deployment. This still leaves you with the big push when you deploy features.

At Wealthfront, we prefer to push out everything and releasing features
amounts to pushing out a single small change of whether a feature is hidden
behind a user flag or experiment.

~~~
icebraining
Yeah, I think the term for this workflow is Continuous Delivery, as in Jez
Humble and David Farley's book:
[http://martinfowler.com/books/continuousDelivery.html](http://martinfowler.com/books/continuousDelivery.html)

------
matlock
Continuous Deployment and Delivery both are there to remove the fear of
change. Coupled with testing this is definitely a way to 10x your engineering
productivity.

We've stopped counting how many times per day we push changes at Codeship as
the number is meaningless in itself. If you feel you can't push regularly you
giving up before starting the fight already.

It's just a very different and way more relaxed method of software development
where you can actually focus on pushing the product further without getting
stopped by infrastructure all the time.

Immutable Infrastructure will be the next big iteration in getting this 10x
productivity I think.

------
sharkweek
I really like the resources that ThoughtWorks lays out on their CI page --
down at the bottom they have some whitepapers on the topic that are certainly
worth a peek --

[http://showcase.thoughtworks.com/continuous-
integration](http://showcase.thoughtworks.com/continuous-integration)

------
ambiate
We are language and environment restricted to waterfall at my job. The
language does not have any form of unit testing and an extremely archaic
version control. The language does not have a viable way to encapsulate small
changes without coding the whole shebang. The environment itself follows some
of the same pitfalls. CD sounds amazing in theory, but some languages simply
do not fit into the paradigm.

~~~
Joeri
If your language has a way to put an if condition around a block of code and
change the value in the if condition at runtime, then you have enough to hide
experimental changes behind a flag.

If your language has a way of calling an existing block of code from a new
location and checking the results, you have enough to write unit tests.

I don't know any languages which are unable of those things.

~~~
Jtsummers
I can't think of languages, but I can think of environments where it can be
difficult. Maintaining software for embedded systems developed ages ago, for
example. Unit testing of your new algorithm is, on the platform, not feasible.
You may be able to put the code segment and mock up the rest of the
software/hardware environment on your PC, but then you'll find out about
undocumented features when you push it to the platform, or performance
characteristics that make one algorithm less suitable than another (memory
usage, time consumed). And you have to persuade those above you to give you
the time to do this, when they don't see the value and want to get the code
changed, reviewed and pushed to their normal testing procedures (which _might_
be white-box testing, but is likely black-box testing at its finest).

And when your office has been built around this model for years, decades
likely, introducing unit testing for new desktop apps (support apps like I
make), is hard because they don't see the value of it. They have a process, I
should just stick to it (that's their attitude, not mine).

------
wastedbrains
I wrote up a bunch of pros/cons on continuous deployment about 3 months ago

[http://mayerdan.com/programming/2013/08/04/thoughts-on-
conti...](http://mayerdan.com/programming/2013/08/04/thoughts-on-continuous-
deployment/)

I am pro continuous integration, continuous delivery to staging. I think it
takes extra care to do continuous deployment to production. There needs to be
a number of systems in place some of which @mattjaynes mentioned others really
depend on having a very solid and accessible deployment pipeline.

------
yeukhon
For smaller product, continuous deployment can move at a really fast pace. For
larger project, continuous deployment has to have a release and deployment
cycle.

Google I heard does an amazing job at this. If I remember everything I heard
from a Google talk, before peer review there is a pre-build/testing process.
And while waiting to be reviewed, there should be a build triggered (or the
build should sits in a CI queue waiting for testing).

