

Django Advice - stevelosh
http://stevelosh.com/blog/2011/06/django-advice/

======
fakeempire
I'm a big fabric fan and use if for server bootstrapping and deployment. I've
tried to get into puppet a few times but it always seems overly complex to me.
Is puppet useful if I'm only running a handful of servers or is it designed
for 10s (or 100s) of servers?

I noticed he used puppet and fabric in this article so it made me interested
what puppet it doing better for him than fabric.

Is there a repository of examples? Like just bootstrapping a debian box with
nginx/apache/postgres (or mysql)?

Thanks!

~~~
stevelosh
I'd say Puppet is useful even if you're running one server, especially when
you use it with Vagrant. If you use it for EVERY bit of setup on your server
(no manually installing packages!) you can be sure that your development VM
and production server are both the same environment.

Puppet is declarative, Fabric is imperative. With Puppet I can say:

    
    
        Make sure the directory /var/www/foo exists with permission 755 and owner foo.
        Make sure the nginx vhost "foo.conf" exists in sites-enabled with content X, and this depends on /var/www/foo existing.
    

With Fabric you have to handle all the edge cases. What if /var/www/foo
already exists? What if it's owned by someone else? You have to manually code
all the 'mkdir' and 'chmod' commands instead of just saying "this directory
should exist like this".

Right now there's no public example repo, but I'm planning on cleaning up and
open-sourcing our Puppet/Django/Vagrant skeleton this week so we can use it
during the Django Dash.

~~~
bryanallen22
I'd be interested in seeing this once you're done. Is there some github
profile I can watch or somewhere that you plan to put it?

------
itodd
A general tip that has served me well: use tmux (
<http://tmux.sourceforge.net/> ). My typical environment consists of a a few
vertically stacked panes. Up top is my vim session, below that is a shell.
Below that is a horizontal split with runserver on the right and cake watch on
the left.

~~~
stevelosh
I used tmux for a while and it's great. I've since switched to DVTM + dtach
because they're more lightweight and simple than tmux.

I never used 80% of tmux's features, so the simpler tools are better for me.
If you're a tmux power user DVTM+dtach might not work for you.

dvtm: <http://www.brain-dump.org/projects/dvtm/> dtach:
<http://dtach.sourceforge.net/>

------
Tomek_Kopczuk
Great post.

But I still don't think Vagrant is that big of a help. I agree that compiling
stuff like PIL on OSX can be a bit of a bother, but on production machines
it's no bother at all.

And developers usually have to do it once, as, at least in our company, they
do not change the projects they work on that often, but rather do them from
start to the finish (we find it more productive this way). PIP requirements
file with proper package versions is a must of course (Fabric deployment). As
is staging server, but speaking from the experience - it's just a precaution,
it works well.

Cheers for a part about Django Admin. I think it's the single most annoying,
hack-needing piece of Django (as Django is great itself, we should expect
perfection).

We are starting our own series of Django tips, mainly related to deployment,
performance and scaling - at least one tip weekly. But thanks to you I decided
that first we'll write about things we use to make Django admin work with you
not against you - <http://www.askthepony.com/blog/>.

------
jboutros
Great article, but I'm still not sold on using Vagrant. Is there a case where
it can help if I'm not deploying one app per vm? I often host multiple apps on
the same VM slice.

------
endlessvoid94
Wow, this is pretty comprehensive. Definitely gotta try django-annoying and
automate things more with fabric/puppet/vagrant.

------
andrewcooke
vagrant looks interesting. here are some answers to questions i was going to
post here but ended up finding myself via the docs:

\- what this adds to just using VB itself is that they provide pre-configured,
minimal environments, kept as small as possible (less than 500MB).

\- pre-configured means that it includes ssh, VB extensions, sudo, assumes NAT
addressing.

\- it has some support for multiple vms (DB and web separate, for example).

BUT i still can't work out what environments are available for download. there
must be an obvious list somewhere... :o(

~~~
stevelosh
_BUT i still can't work out what environments are available for download.
there must be an obvious list somewhere... :o(_

<http://www.vagrantbox.es/> has quite an extensive list of user-created
Vagrant boxes.

------
mikhuang
What have you found to be the best way to implement a REST interface? I
noticed the @ajax_request shortcut but nothing response code specific. I've
been using Django-Piston which has been... challenging at best.

~~~
true_religion
If your rest interface is deeply tied to the Django DB, I'd suggest TastyPie.

Otherwise Django-Piston sadly is the way to go.

~~~
niels
I find it to be exactly the opposite. Django Tastypie is in no way tied to the
Django Orm.

~~~
true_religion
It's not intrinsically tied to the ORM. Sorry if I gave that impression.
However it has many helpful shims and subclasses that interface with the ORM.

I'm looking at ModelResource as my example.

If you aren't working with a NoSQL database or a search engine, or grabbing
data from an tertiary resource then the sheer complexity found in TastyPie
might not be for you.

Django-Piston doesn't offer many features, but its dead simple in its usage.

~~~
niels
Yep, I agree. Tastypie has a steeper learning curve, due to beeing more
flexible. When I used Piston, jespern was not actively maintaining it, but
using a private branch for bitbucket.

------
joecasper
_Very_ informative article. I always find it interesting to see what
experienced Django developers are doing. I will definitely be trying out some
these tool, especially Vagrant and Puppet.

~~~
2mur
Yeah I knew a lot of these tricks. But I've been considering learning chef and
I didn't know about Vagrant. Now I know what I'm doing this weekend!

------
waterside81
Geez, I've been using Django since 0.92 and there's quite a few things here I
never knew about. Good stuff.

------
cabacon
I would love to see more details of the fabric+vagrant integration. For
instance, you don't show how the prod/dev prefixes in your fabric commands are
used to set the vagrant destination host, or the site urls. The vagrant way of
things seems to be using "vagrant ssh". I know it sets up port forwarding to
accomplish that, but how are you establishing the map of port# -> environment
type? Just by convention, within the Vagrantfile?

~~~
stevelosh
Good point. Here's the 'dev' task:

    
    
        def dev():
            '''Run on the development (i.e. Vagrant) server.'''
            env.env_name = 'development'
            env.site_name = 'MYSITE'
            env.process_name = 'gunicorn-MYSITE'
            env.process_owner = 'gunicorn'
            env.puppet_manifest = 'dev.pp'
    
            ssh_info = local('vagrant ssh_config', capture=True).splitlines()[1:]
            ssh_info = dict([l.strip().split(' ', 1) for l in ssh_info if l.strip()])
    
            env.key_filename = ssh_info['IdentityFile']
            env.hosts = ['%(User)s@%(HostName)s:%(Port)s' % ssh_info]
    
            env.site_path = '/var/www/MYSITE/MYSITE'
            env.venv_path = '/var/www/.virtualenvs/MYSITE'
    
            env.site_url = 'http://localhost:4565'
    

In a nutshell: it uses `vagrant ssh_config` to get the SSH info at runtime.

------
skept
Vagrant looks very interesting. For writing code I prefer Vim but I also use
PyCharm on OSX mainly for its excellent debugger and code browser. Does anyone
know if there's a way to get PyCharm or perhaps other IDEs (running on OSX) to
run the Django dev server through Vagrant?

~~~
rytis
you can set PyCharm to upload files on save. it'll scp all changed files for
you and can run post upload scripts, for example to restart httpd.

Settings -> Deployment

Point to your Vagrant instance IP

Settings -> Deployment -> Options

and select 'upload changed files automatically'

EDIT: sorry I lied re opst upload script, must have confused with something
else

~~~
skept
Thanks for that tip. What I'm really curious about though is if there's a way
to set PyCharm's Python interpreter to something not on the local host (e.g.
the Python interpreter on the Vagrant VM). I'd be surprised if this turned out
to be possible but thought it wouldn't hurt to ask.

Edit: I just found out about the remote debugging feature in PyCharm which may
actually do what I want:

[http://blogs.jetbrains.com/pycharm/2010/12/python-remote-
deb...](http://blogs.jetbrains.com/pycharm/2010/12/python-remote-debug-with-
pycharm/)

------
AndrewRadev
I'm not really a Django user, but I was interested in the Vim section of the
post. I work with Rails and my personal approach to setting framework-specific
commands, mappings, etc. is by using the proj plugin:
<http://www.vim.org/scripts/script.php?script_id=2719>. It's fairly simple, it
just sources a project-specific vimfile when you tell it to. I keep a ton of
settings in "~/.vim/projects/rails.vim" and I place a "runtime
projects/rails.vim" in the project file. This kind of a workflow might give
you more freedom in customizing your vim for Django projects, but it really
depends on whether you can get used to that. Just a suggestion :).

------
peregrine
Great Great article. Just started using Django and this basically covers every
single pain point that I've had thus far.

Thank you so much.

------
niels
It's a little bit odd, that the first tip is using Vagrant, and almost all the
other tips are hacks to make up for that decision.

~~~
stevelosh
Uh, what?

    
    
        Topic                                       Vagrant-Related?
        ------------------------------------------------------------
        Vagrant                                     Y
            Why Vagrant?                            Y
            Using Fabric to Stay Fast and...        Y
        Wrangling Databases with South              N
            Useful Fabric Tasks                     Y
        Watching for Changes                        N
            Using the Werkzeug Debugger with...     N
            Pulling Uploads                         N
            Preventing Accidents                    N
        Working with Third-Party Apps               N
            Installing Apps from Repositories       N
            Mirroring Repositories                  N
            Using BCVI to Edit Files                Y
        Improving the Admin Interface               N
            Enter Grappelli                         N
            An Ugly Hack to Show Usable Foreign...  N
        Using Django-Annoying                       N
            The render_to Decorator                 N
            The ajax_request Decorator              N
        Templating Tricks                           N
            Null Checks and Fallbacks               N
            Manipulating Query Strings              N
            Satisfying Your Designer with Typogrify N
        The Flat Page Trainwreck                    N
        Editing with Vim                            N
            Vim for Django                          N
            Filetype Mappings                       N
            Python Sanity Checking                  N
            Javascript Sanity Checking and Folding  N
            Django Autocommands                     N
        Conclusion                                  N
    

5/32 sections are Vagrant-related. 5/32 != almost all.

EDIT: My bad, there are only 31 sections total. Still, 5/31 is not almost all.

~~~
niels
You are right. My bad. My point though, was that I think the cost in terms of
complexity of using Vagrant for development is too high. The goal of having
your dev environment match your prod. env. is not the right approach in my
opinion. This the role of the staging environment.

~~~
dolinsky
The goal should be to keep environments as similar as possible, which vagrant
helps to accomplish. Just because you can develop locally on your Mac before
pushing to a QA/Dev/staging environment doesn't mean you should.

Obviously your Dev environment isn't going to be able to match every aspect of
your production environment, but keeping the same platform builds across
environments is something that all teams should be aiming for.

~~~
niels
That might be your goals. My dev environment should be as painless and fast as
possible. This is what I spend all day using.

~~~
dolinsky
It's what I spend all day using as well. The dev environments I've either
implemented or been a part of (web centric envs) have all used local folders
mounted to VMs that provide that painless and fast requirement you speak of
while removing any pain points associated with varying platforms. You dont
need to sacrifice the former to achieve the latter.

------
Runcowboy
Excellent article!

Would you detail more on puppet please? "Run Puppet to initialize the VPS."

------
zmanji
I find that the warnings and hints from <http://jshint.com/> are better than
what jslint offers.

~~~
stevelosh
I'm sure jshint could be integrated into Syntastic pretty easily.

------
dbinit
Steve, when you say that you mirror 3rd party apps, does that mean you don't
install anything from PyPI (like South)?

------
john2x
I love your articles, but this one hits the sweet spot. Thanks for another
great article.

------
thedjpetersen
Unrelated: His blog looks very nice, what is powering it?

~~~
stevelosh
It's a static site rendered with Hyde.

[http://stevelosh.com/blog/2010/01/moving-from-django-to-
hyde...](http://stevelosh.com/blog/2010/01/moving-from-django-to-hyde/)

------
chopsueyar
Great gems in this article. Thanks for the great info.

