

Eighteen months of Django - dangoldin
http://dangoldin.com/2013/05/07/eighteen-months-of-django/

======
tomkinstinch
I'll add that if you need to background anything, django-celery makes it easy.

celeryd is also good for scheduling tasks with either single or periodic
execution.

[http://docs.celeryproject.org/en/latest/getting-
started/firs...](http://docs.celeryproject.org/en/latest/getting-
started/first-steps-with-celery.html#first-steps)

You can also use it in a typical task queue role. It will pickle or serialize
the Python objects that accompany tasks for transmission over the wire.

~~~
obviouslygreen
Note that periodic execution, one of the more useful aspects of celery, is not
supported on all backends. If you're using rabbitMQ, you're golden. If you're
using most or all other transports (including SQS), you're likely SOL and will
have to resort to either bare cron jobs or something like django-cronograph
that accomplishes the same thing.

celery is definitely nice, but it does have its limitations (like everything
else).

~~~
crdoconnor
This is good to know. I've just started using it with redis.

I never really saw the point of that part of celery anyway. Cron's perfectly
adequate on its own (and installed just about everywhere).

~~~
obviouslygreen
The nice thing about it from our point of view is the ability to keep
everything in source control. Granted, you can keep your cron jobs as scripts,
but celery+rabbitMQ allows you to keep it all in Python without the extra
deployment step. It's not a huge deal, but it does help a bit with
maintainability and getting new devs up to speed.

~~~
crdoconnor
We keep all the deployment stuff in source control anyway. For me, it would
just mean shifting it from being declared by salt in the "infrastructure"
folder to being written in python in the django folder.

I agree that you should keep the code itself in python, but that can be
handled just by making it a management command that gets called by cron.

Sysadmins understand and love cron, too. I think celery reinventing the wheel
might make some of them a bit mad.

------
jordn
For those who would like to get started with Django, here's a skeleton project
<https://github.com/jordn/heroku-django-s3> and guide helps get you set up
quickly with:

    
    
      - hosting on heroku
      - static files on Amazon S3
      - virtualenv(wrapper) isolated packages
      - environment and security critical variables removed from settings.py and stored as environment variables (following the 12-factor design ethic)

~~~
iends
I have a couple of thoughts:

1\. Why not use django-admin.py startproject --template=<url>? I typically do
something like django-admin.py startproject
--template=<https://github.com/rdegges/django-skel/zipball/master>
mynewproject so I don't actually have to clone

2\. No runtime.txt for heroku?

3\. Not specifying version numbers in your requirements.txt is very dangerous.
I prefer something like "Django>=1.5.0,<=1.5.9"

~~~
jordn
Thanks! Basically the answer is that I wasn't aware of the possibility of
those things and have tried to keep it as simple as possible.

I made a skeleton project for me as I couldn't find a decent guide for
deploying django static's on s3. I fumbled my way through it and corrected and
formalised it later so I could do it again. The
<https://github.com/rdegges/django-skel> project looks really useful as it
contains everything and more you might want to use for a django project (and
more), though not yet knowing what magic's it's doing I wouldn't yet jump to
use.

1\. That is very neat. I hope to try and copy that.

2\. Noted. I didn't realise the need, as python 2.7.4 is the default. Can see
it causing headaches if this were to change however.

3\. Why is it dangerous? I preferred to keep the version numbers so it would
always just install latest versions without any extra effort.

~~~
iends
The reason it's dangerous is that packages can introduce backwards
incompatible changes. If you have a project that you don't work on for a few
weeks/months and then need to fix a bug, you're going to have to spend
additional cycles tracking down the issue and fixing either the backwards
incompatible change, or adding a specific version to your requirements file
anyway.

I like to try and scope the package versions at the level the maintainer
promises not to introduce breakages at.

------
scott_w
For dealing with packages such as python-psycopg2 and python-mysql, your best
bet is to use --system-site-packages.

This allows the virtualenv to depend on packages provided by your distro
instead of trying to compile its own. Wherever possible, I try to do this with
Python packages that require a C compiler to make deployment a little easier
and reduce dependencies.

Personally, I'm not a fan of Fabric and I much prefer configuration management
(Salt, Chef, Puppet) because of the added flexibility you get. About the most
I use Fabric for is logging into the Salt Master and kicking off a deployment
process.

~~~
antihero
Absolutely not. An added bit of compile time is the least of your worries and
if you _really_ don't want to compile stuff on production, look at packaging
your virtualenv as a compiled deb/rpm.

Never ever ever mix/depend upon system packages in your virtualenv. Ever.

~~~
scott_w
You're always depending on system packages somewhere. Compiling isn't the only
reason - sometimes it's necessary just to make the damn thing work.

Some examples:

* PIL - Using Pip's version on Ubuntu, you have to hack your directory structure and re-compile the whole thing or PIL doesn't compile in support for your image libraries.

* psycopg2 - This is hit and miss (OS X comes to mind), but depending on the system version guarantees that it has been tested to work with the specific version of Postgres on your system.

* Crypto libraries - Broken packages have ended up on Pip and just cause a bloody headache. The system package is, again, tested and works.

Compiling a DEB/RPM would be a good idea, but it's not a well supported system
at all. If there were a simple pip compile --type=deb or something, we would
be all over it.

~~~
antihero
PIL - Don't use PIL, use Pillow.

Psycopg2 - Well, it's just a client library - surely you will be making sure
your client matches your server's version (whether that's local or a clust or
whatever)? Sure your system will likely need Postgres headers installed, but
why is that an issue? Actually, let me just check that. e: Yeah, you need
libpq-dev installed, but I fail to see the point here? You need to make sure
you have python-dev and a correct version of Python installed, too.

PyCrypto - Hmm, not had a problem with this hugely.

What I was suggesting doesn't involve a deb of individual packages, but a deb
of the entire app/virtualenv.

------
ludwigvan
Here are a few other Django packages that I haven't seen mentioned in this
thread:

\- Gargoyle: Feature switches in Django <https://github.com/disqus/gargoyle>

\- webassets and django-assets: For minifying static assets

\- raven + sentry for logging. Sentry is the server, so actually can be used
with any project, not only Django.

\- django_extensions: Provides some helper management commands, for example
manage.py shell_plus which automatically imports your models

\- dbbackup for backing up your db to S3 or Dropbox

------
bluepanda_
Interesting post. I decided to use Django for our startup because Python was a
requirement, we're planning to have a lot of customers, and Django seems like
the most appropriate choice for large scale apps. While I've mostly used non-
rel databases, I'll be using postgresql as a database, because it seems to be
highly compatible with Django, because Heroku makes scaling it easy, and
because it's a good occasion to learn some RDBMS.

------
adamrt
Using S3 for static media makes perfect sense, but does anyone have
information on S3 when your contributors (users in the admin) are uploading
lots of images for via something like django-filebrowser or a simple zipfile
upload (that extracts the images into a folder)? I haven't seen anything like
this in any examples.

~~~
stevenwei
I think the standard method of dealing with this is to have your users upload
the items to the local web server and then have some sort of async task or
cron job responsible for moving the files to S3 later on.

Of course this means you'll have to keep track of where the uploaded file is
currently located, but if you ever want to scale to multiple web servers
you'll need to do that anyway.

~~~
dangoldin
Yea - this is how I've been doing it. The server that gets the upload does
some processing and then moves the content over to S3 and adds an entry to the
database that references it. It also forces you to write stateless code which
is nice bonus.

------
boffo9
I'll add that you should read Two Scoops of Django at
<http://django.2scoops.org>

------
tesmar2
As someone who is thinking about doing a python project, should I learn Django
or just go with Flask? Any thoughts on those two?

~~~
aidos
Neither! Actually, as the others have said it completely depends on what
you're making.

Django is great if you're making a standard cms type website. News feed, blog,
etc. There's a bit of a learning curve but it gives you a lot of the
scaffolding out of the box (and there are loads of examples to learn from). It
has the biggest community (I guess?) so you'll always be able to find answers
/ libraries / sample code.

Use Flask if you really want to get a website up and running with half a dozen
lines of code. You'll have to build everything else yourself; admin,
authentication, etc etc - though there are good extensions for a lot of things
too.

Thinking about it - if you don't know either of them, spend a day with Flask,
by the end of it you should have a fairly good idea of what you get with it.
Then run through the Django tutorials. It'll take longer but you'll be
impressed with how quickly you can get something fairly complete up and
running.

For me, I lean towards Flask (heavily). When you think about what Django does
- it's basically a web container with a db/model layer and sensible
scaffolding for the views (front and admin). I prefer to use SQLAlchemy (if
I'm working with a relational db). In terms of the admin I think that the days
of writing admin grid and edit screens that post back and forth to the server
are numbered (I use REST and Angular now) - so that's not a bit of Django I
use either. By the time you've ripped all that out you may as well save
yourself all the config and just use Flask. Having said that - LEARN THEM
BOTH. At least just a little bit so you can see for yourself where each one
shines.

These days I approach it completely differently. My last two projects have
involved a fair amount of complex business logic (and processing work). I've
written the libraries I needed first as standalone packages, only introducing
Flask as a final step. The web wrapper is just there to provide a web
interface (mostly an api) access the db layer and glue it all to my libraries.
I'd recommend this approach as it stops you from thinking along the lines of
"ok, I'll need to create 3 Django apps, here are my models for each, should I
put this code in the models.py or the views.py?" Instead you concentrate on
making the real python code to do the heavy lifting, and you architect that
sensibly without having to worry about how the web framework would normally
prescribe it to be done. You'll end up with far more portable code in the long
run.

~~~
mailshanx
What is your opinion about web2py? I started exploring it a week back, and it
seems to have a lot of built in functionality. I thought it would be the
easiest web framework to get started with (since it was originally built to
help teach students about web programming). However, it seems to have a vast
number of components, and it looks like learning all of it would entail almost
the same effort as learning Django...

~~~
ceol
Not the person you replied to, but my two cents.

The creator and lead developer, Massimo Di Pierro, is incredibly active and
always willing to lend a hand with whatever problem you have. I've seen him
around reddit (not sure if he's on HN) and even when he's met with the
inevitable haters, he's always been friendly and cordial.

Just like you've said, web2py was originally made to teach people how to build
websites, which really shows in its architecture. It forgoes traditional
Python conventions ("explicit is better than implicit" being a major one) in
order to be more beginner-friendly. Some of the more experienced developers
find this annoying, but if you're coming from PHP or a fresh background, it
can be much easier to get started. Just take a look at the overview on
Wikipedia[0]. It gives a good outline.

This is coming from someone who doesn't even use web2py. I prefer Flask for
personal projects and Django for work/group things, but I would whole-
heartedly recommend web2py. Run through the book[1] and, if you don't like it,
give the Django tutorial[2] a shot. If that doesn't mesh with you, there's
Flask[3], Pyramid[4], TurboGears[5], and CherryPy[6]. (The last two I would
not recommend for beginners, but they're still good!)

[0]: <http://en.wikipedia.org/wiki/Web2py#Overview>

[1]: <http://web2py.com/book>

[2]: <https://docs.djangoproject.com/en/1.5/intro/tutorial01/>

[3]: <http://flask.pocoo.org/docs/>

[4]:
[http://docs.pylonsproject.org/projects/pyramid/en/1.4-branch...](http://docs.pylonsproject.org/projects/pyramid/en/1.4-branch/)

[5]: <http://turbogears.readthedocs.org/en/latest/>

[6]: <http://docs.cherrypy.org/stable/index.html>

------
porker
How well does Django handle complex forms? The kind where "if x = y show
fields p-r (and make them required), if x = z jump the next screen)?

I've just implemented one of these in PHP using the 'old' Symfony 1.4 and it
was _painful_. I ended up hardcoding most of the frontend, and writing
ridiculous modifications for "add another row" and validation.

Is Django any better at that? Complex forms have been my pain point for over
11 years, and if a framework handles them... I'm in!

~~~
vially
I've recently (2-3 months) switched to Python/Django and my experience so far
has been very positive.

Multi-step forms can be implemented in Django using Form wizards [1]. In order
to optionally show fields or make them required I think you could build it
easily using the WizardView.get_form method [2]. The functionality to skip
steps is built-in and it's very easy implement using conditional views [3].

In conclusion, it's fairly easy to implement the functionality you described
using only the core framework without the need to rely on external packages.

[1] -
[https://docs.djangoproject.com/en/dev/ref/contrib/formtools/...](https://docs.djangoproject.com/en/dev/ref/contrib/formtools/form-
wizard/) [2] -
[https://docs.djangoproject.com/en/dev/ref/contrib/formtools/...](https://docs.djangoproject.com/en/dev/ref/contrib/formtools/form-
wizard/#django.contrib.formtools.wizard.views.WizardView.get_form) [3] -
[https://docs.djangoproject.com/en/dev/ref/contrib/formtools/...](https://docs.djangoproject.com/en/dev/ref/contrib/formtools/form-
wizard/#conditionally-view-skip-specific-steps)

~~~
porker
Interesting, thanks for sharing your experience!

Have you seen anything saying it helps to build adaptive forms like [1]?

[1] - <http://screencast.com/t/SutggDU5>

~~~
vially
I'm not aware of any library which could help you build these kind of adaptive
forms automatically.

However, the way I would go about building something like that is by using the
form wizard (mentioned in my previous comment) with a combination of
forms/modelforms [1] and formsets [2]. Formsets are used in situations when
you need to display the same form multiple times on the same page (like the
multiple addresses/benefits income in your example). However, you would have
to implement all the client side interaction yourself since Django does not
provide this by default (there are some libraries which are supposed to help
with this but I haven't tried them myself [3]).

One thing worth mentioning is that by default, the form wizard only allows you
to display either a Form, or a Formset in a step/page. However, there is a
ticket open for this [4] which links to a helper class: form_container.py [5]
which helps overcome this problem. Hopefully, this will be included in future
versions of Django.

Hope this helps.

[1] -
[https://docs.djangoproject.com/en/dev/topics/forms/modelform...](https://docs.djangoproject.com/en/dev/topics/forms/modelforms/)
[2] - <https://docs.djangoproject.com/en/dev/topics/forms/formsets/> [3] -
<https://code.google.com/p/django-dynamic-formset/> [4] -
<https://code.djangoproject.com/ticket/18830> [5] -
[https://code.djangoproject.com/attachment/ticket/18830/form_...](https://code.djangoproject.com/attachment/ticket/18830/form_container.py)

------
hiddentaco
Great short and concise article.

I'm in the process of writing a pet project using Django and would really
appreciate your insights on what packages you find useful (as mentioned at the
bottom of your post).

~~~
obviouslygreen
From someone who wasn't asked but has input... ;)

It's not a Python package in the sense that you might mean, but using
virtualenv is absolutely fundamental to good Python development (at least the
kind that lets you keep your sanity).

South is also exceptional; however, the developer recently ran a successful
Kickstarter campaign dedicated to creating an improved version intended to be
merged with Django itself, so that's something to watch for.

Django-CMS has proven invaluable for many of my projects and clients, though
it's very heavy and more of a project-tacked-on-to-your-project than just a
package. It's great to know, though, and very powerful.

django-registration can be useful if you're into full-on user profile usage,
though many sites don't actually need this. There's a separate project that
provides a set of default templates for this package, and it's almost more
useful than django-registration itself.

livesettings is an outgrowth of django-cms that's pretty useful when you need
what would otherwise be settings.py constants that can be modified at runtime
without a deployment and server restart. Be careful, though, as overuse of
this means you might have design issues.

feedparser is a very, very easy way to consume XML feeds.

If you want to parse something in a very user-friendly way, BeautifulSoup is
hard to beat. If you're into the nitty gritty and/or need serious performance,
lxml serves the same purpose but kicks far more ass.

If you need to interface with Amazon's AWS environments (EC2, S3, etc.), boto
is a brilliant library.

These are more Python-in-general than Django-specific, but that's often a good
thing.

~~~
tacticus
So a stupid question about virtualenv for you. Why does it default to full
paths for everything?

~~~
chrisdl
I actually don't "know" either. But I suspect it has to do with knowing which
python to use (system generally) and if the user wants to use global packages
as well.

checkout virtualenvwrapper
([http://virtualenvwrapper.readthedocs.org/en/latest/install.h...](http://virtualenvwrapper.readthedocs.org/en/latest/install.html))
as well for handling virtualenvs.

~~~
obviouslygreen
+1, I definitely should have mentioned this. virtualenvwrapper makes it vastly
more convenient.

------
groundCode
Nice post thanks - I'm about to tackle a fairly large Django project so it's
always nice to see some best practice guidelines

------
jspiral
nice summary, sent to a colleague who is starting with django.

~~~
dangoldin
Glad to have helped. If he/she has any questions you can have them reach out
to me. I'm definitely not an expert but can offer some suggestions. dangoldin
gmail is the email.

------
porter
One more addition: Use heroku.

~~~
porter
Not sure why this got downvoted. I was being completely serious. Beat having
to deal with fabric.

