Hacker News new | comments | show | ask | jobs | submit login
Why Django Sucks, and How we Can Fix it [slides] (scribd.com)
208 points by twampss on Sept 8, 2010 | hide | past | web | favorite | 105 comments

For those that don't know, this was a keynote at DjangoCon, and was pretty well received. There's a common theme emerging from everyone that there are some warts, and that they can and should be fixed (both code-wise and socially).

Eric's slides should not be taken on their face as a damnation of Django; rather Eric likes Django, and simply wants to make it better.

To add a bit of context, I suggested Eric as a keynoter and encouraged him to give this talk, so the idea that the Django core team -- of which I'm a member -- isn't responsive is bullshit.

In fact, we've had a "what's wrong with Django" talk at every DjangoCon -- the first had two, actually: Cal Henderson's hilarious talk, and Mark Ramm's incredibly insightful analysis of how Django can avoid some of the mistakes Zope made.

It's a bit of pet project of mine: I'm firmly convinced that we need a steady stream of constructive criticism. I unfortunately wasn't able to attend Eric's talk, but the slides have some great and valid points, and I suspect the video will be even better. Many of his conclusions (and the points made here) make me uncomfortable and unhappy about the status quo, and that's exactly the point.

If you look over the history of the "what's wrong" keynotes, in most cases the issues raised get fixed in the next release or two. For example, past "what's wrong" keynotes have complained about a lack of multiple database support, the weak "if" template tag, and puny CSRF protection -- all things fixed in Django 1.2.

So look, y'all can look on this as a set of complaints, or reasons not to use Django, or whatever the heck you want. To me, it's a todo list.

[edit: typos]

I also need to point out: Even core committers here admit there are issues which need to be fixed.

Of course. Those of us who're closely involved in a project can see areas of wrongness very clearly. I'll happily explain to my classes what I hate about the software I'm teaching them - doesn't mean I don't think it's still less hateful than the alternatives :)

One thing: I would consider Mercurial first, Git second. Python has a very close relationship with Mercurial already (obviously) and I find that Bitbucket is as good as or better than Github in the most important ways.

Do you know how long it will be before the actual video is up?

Maybe a week or two at the most (I hope) - looking at the slides alone strips ALL of the context off of the talk, and results in serious confusion.

I wish he hadn't posted the slides; instead waiting for the video.

The overwhelming dominance of Zope back in the day really held back the Python web development world and I worry the same thing is happening again with Django. Django could really benefit from a Merb breathing down its neck.

The Rails core team and the dev process is totally open and the DVCS-centric model makes it really easy for anybody to jump in and contribute. I can't see why this wouldn't work for Django too.

I think there is plenty of other opinionated* people/groups in the Python web developer community to keep the Django community on their toes. For example, Flask which was mentioned in the slides we are currently discussing. Flask is from the same person/group that did Jinja, Werkzeug, and a couple other cool things. Pylons has a good user base and plenty of steam. There is also mavericks like Ian Bicking who wrote Webob and Paste (which Pylons is build on) who prefer the sans-framework approach because it is simple in Python to glue together the various libraries needed for a "framework".

Plus the addition of AppEngine with doesn't play nicely with Django due to BigTable means people look at other solutions.

Also, the Django community, especially some of the core developers do seem to respect the opinions of these other people and groups. I have seen in mailing lists and IRC where core devs agree that Django to a wrong turn here or there, but correcting that turn is a slow process. Overall, I think the philosophy of "Do it right, not right now" is a great one, and it needs to be considered with correcting mistakes.

Opinionated may not be the best term, but it is close. I really mean people that are vocal, that are respected in the community, and groups or projects that great support and communities.

*There are ways to use Django with AppEngine but all have issues. Also, AppEngine now has some sort of SQL support, but I haven't looked into it.

Zope seemed to have great people behind it, and was very solidly built, but it seemed a lot more monolithic, which ended up polarizing people.

Django really benefits from the ability to swap out everything. People who hate the default libraries just find something better to use.

No migration? Use South. Some people run Django on Twisted or Tornado. There are different template libraries that people use. I'm not sure if the model layer can be swapped out so easily (without messing around with custom user authorization), but there's nothing to stop people using django models for authorization, and a separate database interface (even a no-sql one) for other data.

If you're going to swap out everything, are you really using Django to begin with? Why not just use TurboGears[1], which seems based on the idea of swapping out standard components?

[1] http://www.turbogears.org/

Yeah, you lose the full-stack benefits of Django pretty quickly once you start swapping out components. If I wanted a more customizable Python stack I'd pick one that was designed to work that way from the beginning.

Do you really have used Zope and Django if you say that the former is monolithic and the latter allows easy swap out its components?

I've avoided Django on AppEngine due to the vagaries of the AE datastore, and the extra time and planning it takes to get things working efficiently; I'm wary about using any framework not specifically designed for AE. Fortunately the webapp 'framework' that comes with AE is quite usable, though not exactly feature-packed.

Perhaps I'll look at Django on AE again in the future, once I have a better understanding of how to build things correctly.

EDIT: I don't think the advent of App Engine should have a major effect on Django development; my general thought when developing AE projects is to wish I was using a real server, one under my full control.

I have tried Django on AppEngine using AppEngine Patch, but at this point, I don't think it is worth it till Django has non-relational database support. If I were to build something on AE right now, I will try Flask or Kay.

Have you tried Django Appengine?[1] It works pretty well and supports most things in django (including the admin interface).

[1] http://www.allbuttonspressed.com/projects/djangoappengine

I had wondered whether that works, I'll have a look at it for the next job. Anything with a working admin system will make my day.

I haven't, but I am not currently doing anything major on GAE right now.

I've built a couple of small Flask apps and I like it too. Django doesn't really seem to have a direct competitor in the same sense that Merb was for Rails though. Merb and Rails provided a similar developer experience but had some significantly different architectures. Things like Pylons and Turbogears are much more roll-your-own than Django and microframeworks like Flask are great for what they cover but usually app requirements will dictate Django vs Web.py, for example, and not developer preference.

Django has plenty of not-so-alternative projects breathing down its neck and they're well aware they have to keep up. Personally I think it helps the rest of us to some degree for Django to be out at the front accommodating all the crushing masses of new users while smaller projects, not as quickly discovered (but discovered enough) enjoy moderate but well-tempered userbases.

I agree that the Rails model seems to be working well for them (especially since seeing Rails 3), but it always comes down to the accepted decorum of the community. Its really up to them.

Anyone can contribute to Django right now; just like anyone can contribute to Rails.

Only a group of core maintainers can merge those contributions into a core, shipping release. So how does the DVCS make that core think, work or release any differently?

I think the DVCS issue is not part of "what is wrong with Django."

Like you said, anyone can contribute, and core developers will be responsible for reviewing and integrating those contributions, regardless of the VCS. There are even unofficial mirrors on Github and Bitbucket if someone wants to go that route.

All that said, if Git or Hg (or other DVCS) would make some of the developers tasks easier, like reviewing patches (in the form of pull requests on Github), managing branches, tagging releases, etc, I think it is a worthwhile switch.

Edit: Changed core developers to developers in the last paragraph/run-on sentence because it might benefit all contributors.

Except that if the Django core developers thought that using Git, mercurial, or github would grossly improve their workflow, don't you think they'd use it?

They may be perfectly comfortable with their current tools - we agree though, this is not a source control issue. It's is a personnel, and social issue. DVCSes are not magical, and github isn't a cure-all.

If the problem is with the people; we have to fix the people, get new ones - not change the tools.

I'm surprised at how nascent the Django developer community is portrayed in the slides. I've spent a fair amount of time in the drupal community, which seems to have addressed some of the issues that are presented. (Please -- this is not a "Drupal" vs "Django" question.) I more interested in how we share development processes across projects. Given all that we've learned from the various open source communities (Apache, Plone, MySQL, etc), it would be interesting to develop a "project framework." Not a framework for code,or the management of it, but rather for the community around it. How to set up an event, how to encourage collaboration, etc. Is anyone familiar with anything like this?

I think Django was started with that kind of "project framework" in mind. It's evolving and, like any project that breaks new ground (it's a metaproject in that sense) it's bound to hit a couple snags.

I pretty much agree with all of the points raised here about Django, and I've been using it professionally for almost 3 years. The core development team is incredibly hostile to improvements that don't fit some fluffy definition of where they see Django going, to the point where some of the best coders I know are incredibly discouraged from contributing patches or improvements, and just release their stuff on GitHub instead. That's the wrong way for an open source project to be, in my opinion.

I'm really sorry that you found us to be hostile, and I apologize for any part that I played in that. I hope you believe that hostility isn't at all our intent -- in fact, I try very hard to be open to criticism -- so if you experienced hostility that's a failure.

If you can, could you point to any specific moments -- mailing list threads, blog posts, responses on HN, IRC conversions, whatever -- that you felt to be hostile? I want to learn from my/our mistakes. Please feel free to post 'em here, or if you don't feel comfortable doing private email (jacob@jacobian.org) is fine, too.

I'm so glad this presentation is getting attention. The presentation really nails it, and I can only imagine the well-deserved awkward moments it caused a few people.

All the issues are big, but those that touch on the nature of the closed nature of the Django community are, I feel the most important criticisms. A project like Django cannot scale effectively (in terms of pace of innovation, willingness of people to contribute, mindshare, etc) unless they address the community issues they have.

If the Django community is so closed why are over a third of the attendees new to Django? If the community was closed and insular we wouldn't be seeing so many people who are at their first DjangoCon and who are eager to contribute to Django.

The current setup is not perfect. The core team, especially Russel, have been very forthcoming about this. They are openly talking about things that need to change and how to change them.

The apps stuff is dead on, even Drupal does this better (but they have their own problems), the "Design decision needed" and "wontfix" attitude is another one of those 'signals to the community', for the most part the community is forced to patch the django core after each release in order to keep going.

It's a pity, django could go a lot faster than it does. I would not be surprised at all to see a mambo / joomla style fork happening to django if the core team does not heed these warnings.

There already is a fork (django-experimental), though interest seems to be limited at the moment.

There has been limited work on it. Essentially I just fixed the demented logging system... though I will probably fix the demented templating system, and ORM in due time.

Isn't the logging system being fixed in 1.3?

Not all ideas are good ideas. Just like python-ideas threads that crop up, not everything belongs in the framework, and some things are better off elsewhere.

Where though? Django lacks and clearly defined interface for framework plug-ins, python eggs don't exactly ooze with win here...

Just because python packaging sucks doesn't mean Django should drop it's "design integrity" just to throw things in people "really feel the need for". Otherwise Django bloats, gets complex and to big to deal with, and it goes the way of Zope.

I'm glad a talk like this is encouraged. How else are you going to move forward without a little humility?

Yes, and I hope more people realize this, the Django community is good at this, and in large part, so is the wider Python community. It's one of the traits that makes me proud to be a member of it.

Great slides. As a relatively new and inexperience Django user, the points I found most troubling are the points on Apps, and the points on the Auth model.

I also think that getting started with Django is harder than it could be. There are a lot of things you're forced to do, and decisions you're forced to make about how to structure your code, that should be made for you. For example, reading thedjangobook, several times the authors tell you "you can put that code in this file, or in this other file, or maybe do it like this". As a new user, you want there to be a consensus on code structure, and for that consensus to be forced on you: one less thing to worry about.

One point I would like to mention: I haven't taken part in the Django community at all, but if jacobian's comments to this thread are any indication, it looks like it's a great community. Seems to be a really helpful and open person.

It really is a great community. The django-user mailing list and #django on freenode are very newbie-friendly. I would recommend lurking on both and picking up a lot of best practices by osmosis, then asking questions when you get in a bind.

Anyone have any idea if the roadmap is going in the direction of smaller more focused apps? Maybe easier configuration of them as the way they are configured now is a bit awkward.

If you want this, you have to get involved either on the dev or the users list, and advocating this. Eric was setting forth a set of suggestions to make things better; but it still takes manpower, which is more than likely missing right now.

Based on the presentation - and the evidence is pretty compelling - it seems the manpower is there; it's just that the core developers don't want it.

I don't think it's that cut and dry - especially talking to all of the core people (and near-core) here at DjangoCon, and even Django core is well aware and willing to change, it's just not as easy as adding commit bits.

Django people should really look how it's all (configuration, reusability) done in Zope/repoze.bfg. The problems, mentioned in keynote was solved 4-5 years ago there.

And what's happened to Zope since then ?

It's all ok with it — there are two successors — bluebream and repoze.bfg. I prefer the latter. ;-)

I like repoze.bfg as well. My point was that Zope has dropped in usage and mindshare in the last 5 years, and the community was unable to prevent that (for technical reasons, the Zope2/Zope3 confusion, easier frameworks like Django, etc).

The problem with Zope is that it's not scale down to simple tasks. Repoze.BFG can do that, but it's Django-time now, sadly...

I think this is a major problem with Python (and Ruby, for that matter). Why does it have to be an all or nothing ? Why "Django-time" to detriment of all else ?

The PHP community seems to be happy with a rich ecosystem of frameworks offering different approaches to problems - from Drupal and Wordpress, through Symphony, Zend, CodeIgniter etc. Maybe it's because Python is such a smaller community, but it's a shame there has to be one-framework-to-rule-them-all (yesterday Zope, today Django).

Different frameworks offer different approaches and suit different projects. As Python developers we should be much more open minded.

Actually, we're far from "one framework" - just because one has a lot of mindshare doesn't mean we have like 30 of them floating around. Web.py, Django, Repoze, Flask, on and on and on.

I think writing a python web framework is a rite of passage.

I was referring to mindshare, and related jobs market, rather than the number of frameworks.

What is the merb for the django world ?

Is it web2py, flask, cherrypy ... maybe even pylons ?

IMHO it's Flask.

Seconded. Flask is really awesome. The guy behind it (Armin Ronacher) is a beast of a programmer.

Thirded(? Sorry, that's probably a bit too reddit-ish for HN.)

I've been developing with Django for the last two years (a screencast series by ericflo is actually what got me started), and I just recently started work on a major Flask project. Flask is terrific.

What Armin is doing with his curated extensions gallery is one of Flask's biggest strengths. You get all the awesome flexibility of a no-batteries-included architecture, but with a great set of fallback tools if you just want to get up and running fast.

Not to mention SQLAlchemy is just a ridiculously awesome piece of software, and it integrates with Flask very well.

Since you mentioned SQLAlchemy. Not that it's not understandable, but it's a shame how much attention the Django ORM has sucked away from the solid and powerful SQLAlchemy project. It's easy to get hooked on the Django ORM because of the admin contrib component which is tightly coupled with it.

I'm kinda confused - from the homepage, it looks like Flask is Werkzeug plus some extensions. Is Armin, the great thing about Flask ?

I and a lot of people trust his judgement and understanding of the problem domain. Flask uses Jinja2 and Werkzeug (both awesome libraries, and again by the same author and display an amazing understanding of what and what not to do for the respective solution).

More likely WebOb or Werkzeug. Maybe Paste.

I really feel what they said about the community and core developers:

Reading the correspondence on the following bug really turned me off to the project. http://code.djangoproject.com/ticket/5172.

Specifically: "Please do not reopen a ticket that's been marked "wontfix" by a core developer;"

I'm not sure how the issue jives with the whole Django philosophy, but with that much interest in changing the templating language, I found that a bit harsh...

Someone has to make a decision and since here that person is a django-core developer, I'm not sure there's much to argue with. If they don't want to do that, bug will get marked "won't fix" every time it's created.

What else do you propose developers should do? I may disagree with this decision, but someone has to lead the project.

How else will a core developer let it be known that this feature/issue won't be fixed? The "wontfix" tag/state is there for this very reason. And the core developers are there for this very reason -- to make decisions.

I'm curious though. Why not just fork the project and be more open in terms of getting patches in?

As someone who just very recently began to use Django extensively, it seems like the issues mention do carry some weight. While Pinax is a great help in avoiding to reinvent the wheel, I have had to copy the app out of the package into my own to customize it. I'm not sure if this is prevalent in other frameworks but it didn't seem to be too much of a hassle. But of course, maintenance-wise it might create more problems.

Major reson Django sucks much less than Zope3: There's "only" one way to do things.

Less is more. Choices lead to confusion. Let's continuing making the less even better.

I guess a lot of the initial slides are in terms of reusable django apps out there. While what Eric is saying is not wrong, I just think that its hard to enforce certain programming style on these app developers. Some of the apps are coded to be generic enough and can be plugged in easily, others not so much.

None the less, I dont see how / why this is django's fault ?

I think his point is that, together with settings issues, the structure of apps makes it hard to make them generic.

Are they finally fixing their nomenclature?

C'mon really? That's one of the most broken parts of Django ... a view is a controller, a template is a view?

I suspect some people think it's more problematic to force web apps into a MVC pattern that was meant to describe conventional desktop apps.

By calling it something different, even though it's the exact same thing?

Is there a non scribd version of this pdf? Scribd says "all download limits exceeded from your ip" and won't let me download or read anything beyond slide 26.

I have downloaded it from Scribd and put it on Google Docs publicly:



I like how the author keeps mentioning Flask - I've used flask and liked it a lot.

Aside from the title being a bit on the link-baitish side, this isn't too bad a set of suggestions.

Well, that was the talk name, so I don't think it's linkbaitish :)


Wasn't there a keynote at Django-con a year or two ago with the same title even?:)

Close. It was "Why I Hate Django" by Cal Henderson in 2008. He is way funnier than I am: http://www.youtube.com/watch?v=i6Fr65PFqfk

You can't link to a talk?

I dumped django for rails after a brief foray, and even participated in the Pinax group at Pycon in Atlanta. This presentation hits all the major issues, and is particularly strong regarding community. Incetuous communities support personal interest over quality, and that's reflected in the mess of django core.

That said, the framework is not a terrible choice, just not mine.

From some Rails/Ruby guys (who are also Django people) Rails, and the Rails community gets the same criticism, or at very least the same themes. Python core gets it too, etc, etc. Sooner or later Node.js will get it, and so on.

I think accusing the Django core as being incestuous attributes a lot more malice to what they say and do then is warranted. All communities have scaling issues, and all open source communities have growing pains.

The sad part about this is that people who didn't get "their thing" into core is going to use this as self-validation, even if the idea really was bad to begin with.

It's a lot easier to get the 'top stuff' from the community to the top if it the core team is inclusive rather than exclusive.

The examples you give are from communities that have for the most part learned how to deal with those issues (python itself especially, in the talk GvR is actually quoted as having reversed his position on specifically that point).

Communities tend to be one or more orders of magnitude larger than the core team of most projects, and to disregard such an enormous pool of talent is a loss. So what if 90% is junk, let the community sort it out and come to a consensus on which things deserve to be part of contrib or core, and then respect that decision, unless there are really earth shattering reasons not to.

I know, I'm a member of python-core, and I was happy with the change. And Django core is here, and listening, and was in the audience of the talk.

It's growing pains - and even python-core, despite guido's recent change of mind, is still learning, and finding ways to reduce friction. Django core is listening, and will change. There's nothing much to worry about.

And if they don't, they die, and we move on (see, past python web frameworks, and other things which wander into the woods to die).

"The sad part about this is that people who didn't get "their thing" into core is going to use this as self-validation, even if the idea really was bad to begin with."

Most of what is mentioned in this talk are not that much of an improvement, non-issues, or all around horrible ideas.

The fact is though there are major issues with the framework (not mentioned in this talk), and much of it is due to how things are being managed at the moment rather than a lack of ready, willing, and able contributors.

What do you see as the major issues?

The templating engine is around 10 times slower than similar competitors.

The ORM has quite a few failings. Granted I like the API, though the backend code is nothing short of a cluster-fuck.

The template engine isn't that slow anymore, and it's easy to swap in Jinja2.

The ORM also works quite well for what it does. It doesn't allow you to construct real SQL queries and non-existant objects, but that's it.

Jinja2 is indeed much faster and offers a more flexible expression syntan than Django's template engine. However, Django's native template engine has one huge advantage right now - when there is a syntax error in the templates, it almost always gives informative errors the correct location. Jinja2, as it is right now, often gives no errors and completely wrong locations for the problem. This has been a serious disadvantage for me when dealing with more complex templates. This is surely temporary, but is the state of affairs right now.

The speed of the template engine isn't really the problem (it's unlikely to be a major bottleneck anyway, compared to database operations).

It's the restrictiveness compared to e.g. Jinja2 or Mako. With those engines, I can just drop in a Python function and be done with it; with Django template engine you end up writing tons of template tags for the simplest things. It's designed for a particular work environment (designers work on templates, so should not be allowed to do anything dangerous) that I've found to be quite rare in most projects.

I think if your writing more than a couple template tags and for simple things you're doing it wrong. I would really limit templates to being primarily HTML. Put any advanced python functionality in your view function, Django templates are minimal and not for making queries like other templating systems (a good thing IMO). If you need to do anything crazy advanced like recursively display stuff for items, don't write a template tag, use a simple include tag to insert HTML for subelements. Using the include tag when possible is also great for AJAX, as you have the HTML for a subelement sitting in a file if you want to render just that to return to a XML request.

OK, come back when you have done something moderately more "crazy advanced" than recursion.

> not for making queries like other templating systems


I was not implying that the concept of recursion is at all advanced. I was implying that the problems to be solved in the template layer are very simple.

I won't be able to come back to you because it would be too inefficient doing something more advanced than that in the template layer. Because rather than write the python to write template syntax to be interpreted back into python, I simply wrote the logic in one-pass python in the view function. It's more direct, runs faster, and your templates are clearer to designers if you're working in a group project.


I've also never seen a project where the templating engine was a noticeable part of performance. I did see one where the decision was made to switch to Jinja2 which brought a huge amount of work duplicating a bunch of existing Django template tags – which didn't make much of a difference in the face of the 8,000 database queries needed to generate a page.

I have a friend that runs a popular music-related startup and template rendering has been their #1 bottleneck.

If template rendering is your major pain point you have either an amazingly well tuned system or are doing something very very wrong.

This is definitely an extremely well-tuned system.

No doubt, the Rails community has issues, although on the whole, it seems to me larger, more dynamic, and open. I saw the django issues as coming from the fact that it was tightly held for a specific purpose for a specific company for a long time.

People were generally really nice in the community. Some of it may also be the more introverted characters attracted to python.

Also, I have lots of friends who are django fans and I hope that django improves on all these fronts. The character of a community is both hard to quantify and hard to change.

The rails community has grown so fast and been so open that it doesn't even seem to have a specific character, which is something I quite like about it.

Some of it may also be the more introverted characters attracted to python.

Interesting observation. Ruby and Python do seem to attract somewhat different personality types. Functionally they're actually very similar.

Python limits what can be changed at runtime: Ruby doesn't. That in itself attracts more outgoing, adventuresome people, whereas Python is changeable enough to be flexible, but not unstable -- which attracts people more interested in stability and security.

Some big Django shop like Discus could create a team and sponsor the development. Something called Django2 Django-ng.

Without hype and stimulation individual developers are passive.

Why not just have Disqus sponsor the DSF/Django core. Forking it is exceedingly short sighted.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact