
Python Application Deployment with Native Packages - craigkerstiens
http://hynek.me/articles/python-app-deployment-with-native-packages/
======
cageface
I've said this before here but I think the ability of Go to create a server as
a single statically linked binary is a HUGE deployment advantage. It's so much
easier to just compile exactly what you want and push out a single file.

~~~
hynek
Does it also help with deployment of non-code related files (eg. static files
in web apps)?

~~~
nakkiel
Definitely a problem in Go. It's still possible to embed resources in the
binary but if you're talking about a lot of such resources, the resulting
binary size can be an issue as well. Not to mention resources won't be
available outside of the binary.

~~~
stock_toaster
I don't think it is any more of a problem with Go than any statically compiled
C/C++ binary.

You would still want to use chef, puppet, or some home rolled package
deployment toolchain, if for no other reason than being able to ensure a
consistent state and bring up new nodes quickly.

------
onedognight
> Sometimes git pull fails halfway through because of network problems

Separate the pull into "git fetch" into a shared bare git repository (if you
care for efficiency) which is atomic and then a "git checkout" into a _new_
directory using a single symlink(2) and rename(2) which is also atomic because
rename is.

> Imagine you’re pulling from git and at the same time, the app decides to
> import a module that changed profoundly since its last deployment. A plain
> crash is the best case scenario here.

> On the other hand, deploying using self-contained native packages makes the
> update of an app a near-atomic,

You can't have your cake and eat it too. Native packages have the exact same
race conditions as an (inplace) git update.

~~~
hynek
You make fair points here however I was arguing against the widespread “git
pull” culture.

It's clear that some of the points can be mitigated using more sophisticated
git+fab magic. Nevertheless, the other points remain true and the combination
of them all makes me believe that native packages are the better way.

------
j-kidd
I think native packages make things too rigid. A private PyPI server hits the
sweet spot for us. The steps to deploy an update is as simple as:

1\. activate virtualenv

2\. run `easy_install -U app`

and no root permission is required. It also has the added benefit to allow
having multiple copies of the same package in the same machine [1], which
can't be done with native package.

[1] In one deployment, we have staging instance and production instance in the
same machine...

~~~
sciurus
Sure you can have multiple version installed with native packages; for
examples, see the discussion at [https://groups.google.com/group/devops-
toolchain/browse_thre...](https://groups.google.com/group/devops-
toolchain/browse_thread/thread/e99e93b34433e14d/1fc5b50feae310a7)

~~~
j-kidd
Of course, it has always been possible to have multiple versions installed
with native packages. Gentoo supports that since day 1. What I was talking
about though (and I didn't make it clear), is having the same version
installed multiple times in different locations (i.e. separate virtualenv).

------
nakkiel
I personally use a good old Makefile to create debs. I'm not sure using Fabric
for that gives much of an edge. I also handle base system configuration
through a hierarchy of debs that perform tasks ranging from user configuration
and database setup to application setup.

I've used that sort of setup in various configurations and it's solid. After
all, that's what our favorite distos use (whether it's deb-based or not is
irrelevant, concepts are similar).

If and when configuration needs to be "shared" or "propagated" between
machines, I store in a store of some kind (RDBS, KV).

------
othello
Deploying web apps as self-contained binaries reminds me of Go [1]. Except in
Go you get this for free as a simple (and swift) compile step.

Of course, this is true of any compiled language that can produce self-
contained binaries, but Go has the added benefit of being well adapted to web
app development.

1: <http://news.ycombinator.com/item?id=3722514>

~~~
mbreese
This isn't the same as a self-contained binary. This is packaging up your
application, all associated files (javascript, css, etc) and virtualenv setup
into a .deb (or .rpm). Then the package can be added to a local repository and
installed like any other program.

Even with Go, you'd have to package up your single binary in order to get the
same experience. (Although, that package would be pretty small!)

------
akheron
In postinst:

> virtualenv /vrmd/$APP_NAME/venv

This step could be skipped if you built different debs for different distro
versions (in chroots for example, like sbuild does it).

Might be too much of a hassle, though, for a little gain. It just looks
slightly wrong to me that you first install a deb and then go and overwrite
many of the files the deb installed.

~~~
hynek
I believe the reason is that the position of the virtualenv changes (from
/home/buildbot/package-version/vrmd/venv to /vrmd/package/venv) and some links
didn't match. I'm not 100% sure though, it's some time ago. :)

------
obtu
I'm a bit wary of calling an fpm-generated deb a native package, those are
about as well integrated as a checkinstall (make install -> deb) or an alien
(rpm -> deb) package. fpm may still be useful for this use case, but it
doesn't take me much more effort to get a proper source package.

~~~
mattyb
How are FPM-generated debs any less integrated than native packages? You have
the same control over the package metadata using its flags as you do with a
Debian control file, and using `-s dir`, you have complete control over the
package contents. You can ship rc/Upstart scripts, use pre/post-inst/rm
scripts to add users and backup data, etc. These are all the same tasks that
lots of Debian-maintained packages perform.

Please look at the FPM wiki, you'll see that it offers many more options for
integration than checkinstall (which I used to use, and hated):

<https://github.com/jordansissel/fpm/wiki>

And why would you want to create source packages? One of the benefits to
generating binary packages is that you compile once and roll it out to
hundreds of servers. I'd rather not have to compile, say, Erlang on every
target server.

------
lifeisstillgood
from vrmd.fabric.ubuntu.deploy import Deployment

How do you differentiate builds on say redhat / BSD / windows as opposed to
Ubuntu ? I would _guess_ vrmd.fabric.fedora.deploy contains a lot of the same
code except for pathname changes?

~~~
hynek
TBH, we currently just convert the debs to rpms when necessary. But RPM-based
servers are only legacy here so it doesn’t happen too often.

We have no windows and no BSD _yet_. There might come some FreeBSD for ZFS's
sake soonish, then I’ll have to refactor. :)

~~~
lifeisstillgood
Thank you.

