On: Starting a Django Project the Right Way

[yummyfajitas]

Don't use Fabric. Seriously, just don't.

It's easier to set up than puppet/chef, but you lose a huge amount of flexibility/robustness. All fabric does is runs commands on host machines. Dependency management is your responsibility.

[hcarvalhoalves → yummyfajitas]

I wouldn't be so harsh. Both Puppet and Chef come with their own (enterprisey) overhead and it's Yet Another Tool to understand and maintain. Chef in particular feels like an over-engineered solution for anyone managing less than hundreds of servers, it adds a lot of cruft (centralized server, authorization, protocols, etc.) that most people wouldn't need. I believe a lot of people like it for the sole reason they are not experienced managing servers, so they can just use pre-made recipes and call it a day.

Anyway, you can go a long way with just a bunch of scripts leveraging Fabric's API. I have setup ~10 servers for a news portal I run from the ground up in just a few lines of code. Managing dependencies is not ridiculously difficult as you make it sound, package managers (apt-get, pip) already handle that for you without any overhead.

[unohoo → hcarvalhoalves]

completely agree. Chef and puppet could be overkill for several small to mid sized environments. The learning curve for both is relatively steep as compared to fabric. With the parallel exec feature, fabric more than meets our requirements for a small setup (<10 instances).

[arocks → yummyfajitas]

Interestingly most posts which recommend using Puppet/Chef depend on Fabric for deployment. Is a mutually exclusive or pure Puppet/Chef approach better in any way?

[yummyfajitas → arocks]

If you want to use fabric (or a shell script) to run puppet, go ahead. I'm just suggesting that you really want to use a deploy system with proper dependency management.

The issue I ran into with fabric is that I often got stuck in dependency hell. The following is fabric's simplest method of dependency management:

    def install_foo():
        install_foo_dependency()
        ...

Unfortunately, you don't want to do this every time you deploy because install_foo_dependency() might take a while to run. You can work around it by checking inside install_foo_dependency whether it's already there. In practice, you probably won't always do this. Puppet usually has recipes which already do this for you.

So you typically have functions like:

    def full_deploy():
        install_foo_dependency()
        install_foo_without_dependency()

In theory, you can do things right with fabric. In practice, you have to do a lot of work to replicate what puppet (together with assorted easy to find recipes) gives you out of the box.

[irahul → yummyfajitas]

> I'm just suggesting that you really want to use a deploy system with proper dependency management.

Why not use pip(pip install -r reqs.txt)?

    def install_deps():
        local('pip install -r reqs.txt')

Or if you are talking system dependencies, use apt-get or yum or whatever comes with your system from within fabric.

[yummyfajitas → irahul]

Often installing a dependency involves more code:

    with in_tmp_dir():
        run('wget http://someserver/project_bleeding_edge.tar.gz')
        run('tar -xvzf project_bleeding_edge.tar.gz')
        run('./configure; make;')
        sudo('make install')
        put('conf/server.conf', 'server.conf')
        sudo('mv server.conf /etc/server.conf')

(I forget if in_tmp_dir is builtin, but if not it does exactly what it sounds like.)

[mattdeboard → yummyfajitas]

As someone who has worked with a fairly involved fabric deployment & provisioning process, I'm forced to agree. Fabric is great for what it is, but you lose so much by not using chef.

[herrwolfe → arocks]

Eric Holscher also has an excellent blog post, http://ericholscher.com/blog/2010/nov/8/building-django-app-... , describing how to deploy using both fabric and chef. There are certain instances where one tool works better than the other, and in that situation that tool is used.

[slig → yummyfajitas]

He's using fabric to automate the deployment of his app, not to create and manage a new VM.

[eli → yummyfajitas]

I agree that puppet/chef is better... But easier? That has not been my experience.

edit: Doh, I can't read

[mdehaan → eli]

This is why I created http://ansible.github.com

[yummyfajitas → eli]

Reread what I said. Fabric is easier to set up, puppet/chef works better after it's no longer a small project.

[ehutch79 → yummyfajitas]

i donno, i mean, how often do you think my server gets its dependencies out of sync? (note the singular there)