

Git-based fabric deploys are awesome - drtse4
http://dan.bravender.us/2012/5/11/git-based_fabric_deploys_are_awesome.html

======
maybird

      One other valid problem I've heard raised about git-based
      deploys is that you can end up with cruft in your working
      copy that sticks around like .pyc files where the
      original .py file is deleted and there is the chance that
      this file could still be imported even though the
      original .py was deleted.
    

I'm shocked. If a py file is modified, the pyc is rebuilt. So we know it's
poking at the original source file. Why not fail to import if the original
source is missing?

Maybe this behavior is meant to support binary-only distribution of Python
applications, but there really should be an option to override this behavior.

Edit: Did a bit more research and found this:

(1) <http://docs.python.org/release/1.5.1p1/tut/node43.html>

    
    
      It is possible to have a file called "spam.pyc" without
      a module "spam.py" in the same module. This can be used
      to distribute a library of Python code in a form that is
      moderately hard to reverse engineer.
    

Makes sense.

(2) <http://docs.python.org/using/cmdline.html#cmdoption-B>

    
    
      If given, Python won’t try to write .pyc or .pyo files
      on the import of source modules.
    

So you can mitigate that behavior by removing existing pyc files and using
"-B".

~~~
julien_p
When I ran into this problem I simply added a post-checkout hook that deletes
*.pyc files within the git root.

~~~
eli
Same here. Is there a downside to this approach that I'm missing?

------
calpaterson
" Packages are, of course, a legitimate way to push out changes but the
downside of deploying with packages is that it takes time to build them and
upload them"

Hopefully your CI server is building them. Packages (whether native or
otherwise) come in quite handy for loads of reasons. Native package managers
mature? and have fantastic tools available

~~~
calpaterson
I find it interesting that this comment has been given some points, and it's
actually the draft of the comment I was trying to write on my phone, but gave
up on (Hacker News doesn't work very well on my phone). Here's what I meant to
say:

Packages do take time to build, but your CI server should be doing that. For
most kinds of deployments, you can rely on unix's copy-on-write filesystem by
installing the package and then restarting you process manager (ie: supervisor
or apache). This means your program is only down between restarts (assuming
you have no migrations to apply). The outage due to a deployment is then
typically a couple of seconds, assuming your program is quick to start up.
This is a sufficiently short downtime period for many situations.

There are lots of additional benefits with using packages, especially native
ones, like dpkg/deb. You can add specific dependencies onto the package (for
example python 2.7), which will be checked at install time. Multiple language
programs are often handled better. The program is easily un-installable. If
your CI server is building the package, you can download it for manual testing
(for example, for UAT).

------
veeti
> Why anyone would want to write a billion line init script now that upstart
> exists is beyond me. Perhaps they don't know about upstart. It could also be
> that they are stuck on CentOS or RedHat. My heart goes out to you if that's
> the case. I know how that feels.

RHEL/CentOS 6 uses Upstart (and the next major version will use systemd).

~~~
bmelton
1) Because a billion-line init script is a gross exaggeration.

2) What do you do if you have 1,000 servers and your upstart script has to
change? Fabric is nice to have in situations like that.

~~~
286c8cb04bda
If you have 1000 servers and you're using Fabric to modify your init scripts,
then I weep for you.

Well before you get to that level you should be using configuration management
software (cfengine, puppet, et al.) to handle your servers.

------
viraptor
> we got sick of waiting for the packages to build and upload

I don't get it. What did they do, that took longer than git commit / git push?
It's not like you're compiling anything in python and when you do have deps to
compile, they would take that time on deployment anyway.

------
5h
Been doing this for a while, came with another bonus: I no longer have a
single ftp server running anywhere on my servers now, using codebasehq.com as
my repo / deployment source ... works extremely well.

------
kgrin
The pattern that I've seen fairly often (and use myself) is to git pull into a
"working copy" on the deployment target, but do a git archive/symlink to
actually build the copy that's served (before doing the service restart).

That gives you a more atomic process and several points to back out gracefully
if something goes wrong, without having to worry about *.pyc files or any
similar cruft.

