

Ask HN: Good multi-server deployment practices - maverhick

Hello, what are some of the better ways to deploy code/modifying databases across multiple machines? How do you guys do it and keep your processes swift, servers secure and have the ability to roll back if something breaks.
======
kls
We script everything so that it is an automated script further all of our
process is managed in JIRA. So when we do a release, the production manager
decides whats in and what out via JIRA, based on that, our script takes those
ticket numbers, goes to our source control system and moves the associated
code into a release branch, tags it and automatically deploys it to a testing
server, the release is verifies and tested. If good, the production manager
then goes back to Jira and marks it as tested. The system then deploys the
release to production and once verified merges that branch back to the head.

OK so what happens in the actual release? We do a rolling release in which the
script goes to the router and re-routes traffic to the other servers
(basically takes a server off line). The script then deploys to the server,
brings it up verifies it and then turns traffic back onto the server.

A couple of principals that we live by are that are back end applications are
REST services, we don't do Java, or RoR, or .NET apps, we do REST middleware
with a language, ours is Java but it could be any. This is a huge distinction
because it means that we don't do JSP, JSF or any of the other server side
frameworks. This makes our server side very easy to mange.

Also we don't use session. All of our services are stateless. Which means we
can drop in a new service or section of an overburdened service to dedicated
hardware with out all the implications of session. This allows us to handle
traffic without having to deal with the context of that traffic. If we
experience heavy load on the profile service we can drop in 4 new boxes with
the profile service on it and route request to those boxes. We don't have
constraints that require them to go back to the same box they have a session
on or complex session management infrastructure to ensure context is there for
a user when a request comes into that box. Basically any request, at any time,
can be services by any box, with no need for external information from outside
the request. It is completely stateless.

On the database side all developers write delta scripts that take the database
from A to B. We also take the cumulative patch set apply them together and
generate a clean version x.x DB script that is more efficient. when a release
goes out the scripts are run against a DB that is taken off line and that DB
is brought up in unison with the app server that is at release x.x.

As well with the DB, we have a script that exports fairly static information
that is in the DB out to static json files on the web server.

Finally, we don't template html files on the server, we write JavaScript UI
apps that pull data from the json files and the REST services, we use a CMS to
do templating and the CMS exports static HTML to the web server. Dynamic info
is overlaid on the client side via JavaScript. This makes the web assets
deployments drop dead simple.

~~~
maverhick
Thanks a lot. Really helpful!

