Hacker News new | past | comments | ask | show | jobs | submit login
Python and Django on Heroku (heroku.com)
414 points by craigkerstiens on Sept 28, 2011 | hide | past | web | favorite | 74 comments

Python 2.7? They’re years ahead of Google App Engine!

Addendum: The previous statement is slightly tongue-in-cheek, but: (1) GAE is running a version of Python (2.5) so old, that it’s hard to get fully patched binaries of it anymore (depending on your platform); and (2) By “years ahead,” I don’t mean it will take Google years to catch up, but merely that their Python version is years old. Puzzling, given that Guido works for Google and has for years.

Second Addendum: Linked: an issue opened in 2008 pleading Google to add 2.6 support. Three whole years ago. (Ironically, perhaps, the issue was closed as a duplicate of a 2010 issue asking for 2.7 support.) http://code.google.com/p/googleappengine/issues/detail?id=75...

App Engine announced they will release a Python 2.7 runtime in several months. It's currently in their trusted tester program.

I can confirm this. It's also very easy to switch old apps to 2.7.

I know this pain; I am forced to work with GAE every day now. And let me tell you, the only thing worse than not having 2.7's features is having un-patched bugs from as early as 2008--like one that prevents keyword arguments from being unicode strings, for instance! Seeing the other list of changes/"mandatory improvements" in child's link makes me weep all the more. I will have to start another crusade to get 2.7 support and between stuff like documentation, testing (gasp!), Django-nonrel, and HRD, I am fresh out of crusade slots.

As much as I've come to enjoy and appreciate the various start-ups whose mantra was "We're Heroku with Python/Django capabilities", it will be interesting to see which of those survive the next 8-12 months now that Heroku officially supports that stack.

It'd be a shame to see ep.io and Gondor go the way of the dodo bird, but what sets them apart now?

Epio cofounder here - we're not worried by this, and we've known it was coming for ages now.

The hosting market is plenty big enough for more than one player - we're expanding to multiple languages, much like Heroku expanded from Ruby, which provides an ample feeding ground of old-style hosts to slowly steal business from - and there's also still a lot of room for innovation.

Heroku, as much as I like their product - and I do, which is why we started this a year ago - still has its flaws and idiosyncracies, some of which are those low-level-design type of choices where if you go the other route, a different set of people complain. You can't be all things to all people, and I don't think anyone will end up being that.

"some of which are those low-level-design type of choices where if you go the other route, a different set of people complain"

This is actually a wonderful scenario. If either party successfully grows the market, both can benefit.

Well said and epio is still my favorite implementation so far so don't go anywhere.

These comments make me want to try ep.io to see if it would work for some of our projects. Would you be able to send an invite to my username at gee mail? Thanks.

Heroku is owned by Salesforce now and subject to their agenda. Could be a good thing for some reasons, but it leaves room for a nimble startup to use the start-up advantage (fast moving, close to customers, unencumbered by a corporate leash) to compete. They're going to have to keep moving though.

A year ago I would have agreed with you, but Heroku has been on a roll lately introducing features at a very fast pace. The party may end eventually, but I've been really pleasantly surprised by their post-acquisition actions so far.

Choice. For many, simply that there is an alternative is enough to not only keep other providers around but also thrive. The hosting world is huge.

> ...but what sets them apart now?

As a very happy user of some of Python PaaS services [1], there are a few reasons why I don't see myself switching to Heroku:

* These services are written by our community, for our community. The principles at most of these companies are people I've known for years. I've reviewed their code; they've reviewed mine. We'd argued, commiserated, and bought each other whiskey. They've helped shape the Python/WSGI/Django ecosystems. Their tools embody the best practices from our communities because they were there when those practices were debated and determined.

This is by no means a slight on Heroku's staff: I've met people from their team, as well, and they're wicked smart, very motivated, and highly focused on delivering an awesome product. Which it is! But as a company, Heroku is going to have to do some work to get to the same level of trust as the companies that have grown organically out of our community.

* Deploying a Python stack on Heroku is something of a small pain: there are few defaults, so you're left to do most of it by hand.

Take, for example, the bits from the Heroku/Django tutorial about configuring Celery. Not a lot of work, to be sure, but it is some work, and the result won't perform well at all (it uses a database for message transport instead of something that'll perform better). On mot Python-specific PaaS offerings, a single line in a config file (or checkbox in a web UI) gets you a fully-configured, optimized, ready-to-roll Celery setup.

I'm pretty damn good at deploying Python setups. I'm not going to use a PaaS offering unless it provides similar features to a hand-rolled setup for less work/money.

* On Heroku, I'm a small fish in a large pond. I'm the weird customer doing Python, but the bulk of their cash (presumably) comes in from their Ruby customers. Deploying Python on Heroku feels rough around the edges. A trivial example is the broken links in the Python documentation: it's no big deal, and easily fixed, but the many similar rough edges send the message that Python support is a hobby project, a sideline to the main business.

This will probably be fixed with time, I think, but again: it's going to be quite a while until I feel I'll get the same level of support on Heroku as on a tool that comes out of the Python community.

* Finally, Heroku really expensive compared to most of the Python-specific offerings. Maybe a typical Django site features more "pieces" than a typical Ruby one (???) -- a standard "smallish" site of mine will consist of Django, a task queue (Celery), a search server (Solr), a non-relational DB (Redis), and a relational database (PostgreSQL).

On Heroku, this will easily run me $100/mo. By comparison, I'm paying between $15/mo and $50/mo for sites of similar size on some of Heroku's competitors.

So all this to say: I welcome Heroku to this space; more competition is awesome. It'll really make life better for every Python web dev. But it's going to take some serious work to convince me to switch.

[1] I'm really sorry to be obtuse here, but I'm trying to be careful not to write anything that might be construed as an endorsement. As a Django core dev I don't feel comfortable playing favorites (just look at the drama that ensued when GvR had the temerity to say that he liked Django...)

> These services are written by our community, for our community. The principles at most of these companies are people I've known for years.

Of all the points you raise, this is probably the least relevant in business terms. PaaS is about opening up platforms to all comers; one's standing in relatively insular language communities doesn't say much about one's marketability outside those communities.

> Of all the points you raise, this is probably the least relevant in business terms.

He wasn't listing business reasons per se - He was listing reasons why he isn't considering Heroku. And that's a very valid reason - work with people you know and trust, rather than be a second class citizen at a primary ruby PaaS.

heroku used to be a ruby paas. calling it a ruby paas or primary ruby paas is not fair as now it supports node.js, clojure, java and python along with ruby.

There is a significant difference between "supports" and "optimized and designed for". I think Heroku is awesome, but I'm still going to give them at least another 9-12 month or so to sort out all the kinks before I'd trust them as a python PaaS.

For some value of "supports" that apparently doesn't leave Jacob comfortable that it gets equal developer attention at Heroku. Of course the other language support could be perfect ... I'll prefer to trust the opinions of people I know with no apparent axe to grind.

How many domain registrars and hosting providers are there? As long as a lot of people are using Django, there is plenty of room for competition.

I loved Heroku, many moons ago when I was working in Rails. As I've emigrated away from Rails to Django, I've found dotcloud to be the premiere platform -- I want to qualify this, it is the premiere polyglot platform, but for each individual environment I've deployed on dotcloud, their experience has been the best.

I haven't used dotcloud's Ruby/Rails stack, so I can't compare that, but Heroku is definitely fighting a hard battle if they're going to swing me from dotcloud, but it's always good to have competition, and if anybody is going to bring their A-game, it will be Heroku, who were sort of pioneers in the space.

The Heroku Python Free platform might be better in some instances than dotcloud's, and I'll definitely investigate that, but for anything I can think of using, dotcloud has been amazing for me.

I still find Dotcloud's pricing violently expensive (and I have a free VIP plan). YMMV I guess.


   Q. Are there discounts for education and non profit users? 

   A. Yes. DotCloud provides special free and low cost plans for open source 
   developers, educational users and other qualifying non-profits. To inquire   
   about adding free or low cost capacity to your DotCloud account, please
   contact support.

Any reason why you left Rails for Django?

Broadening my horizons? I'm not a Ruby genius, and I really like the Python/Django mindset of "not being clever". The code is MUCH more readable, especially to laypersons, and I value very highly the capability of revisiting my code a year later and knowing exactly what I was doing with it.

I never really felt that way with Ruby, though I still like and occasionally use it. I'm actually working on a Ruby project right now, mostly due to circumstance (hosting was donated for the project, and it supports RoR but not Django or Flask).

The other main thing, at the time, was that there were a LOT of very valid options with Python. I could crank out small projects very quickly in Flask or Tornado, work on larger, more custom solutions with Django, or if it was pretty cookie-cutter, fall back to Zope. At the time, AppEngine and web2py were catching on as well. For the most part, a module written for one of those ports really easily to any of the others.

Bottom line though, learning one has really made me a better programmer and taught me to appreciate the WHY behind some of the decisions a given framework has made. If you only know one, I encourage you to look at the other side of things, even if just for perspective.

I've been using Flask on Heroku since the Facebook apps announcement, and set up a boilerplate template here: https://github.com/zachwill/flask_heroku

Heroku's cedar stack can now detect apps using:

  * Ruby
  * Node.js
  * Clojure
  * Python
  * Go ??
  * Scala
  * PHP
  * Java
  * Perl ??
Anything missing here? That covers a lot of whats out there.

R -- http://groups.google.com/group/rrook/browse_thread/thread/33... (although detect isn't really the case, but run certainly is)

Everything else? Because you can just deploy anything actually?

This is great! Looking at the Django tutorial in the docs [1], I think it's too bad, though, that they don't provide a build-in, well tuned WSGI Server. This way, you have to choose your own WSGI server, configure and update it yourself.

[1] http://devcenter.heroku.com/articles/django#using_a_differen...

We do not automatically use a wsgi server because we want to offer flexibility to our users. Additionally we felt that having less magic in what we automatically do to your application (other than run it) was more of the Python way.

> "was more of the Python way."

That you tailor each environment to the language/framework around it is a great touch.

$ ls


$ git init

$ git add .

$ git commit -m "Init"

$ heroku create --stack cedar

$ git push heroku master

With that Heroku becomes the Heroku for Django.

$ epio upload -a appname. epio is epio for Django.

Anyone know if there's a self-contained way to spool up a dyno sporadically to complete scheduled tasks?

I have a webapp now that's hosted on a VPS and uses Celery to schedule tasks. The tasks themselves only take about a second, the dyno needs to stay active for ~5 minutes to service a bunch of HTTP requests from another webservice that will result from the task, and then shut down until the next task. There are O(100) tasks spaced throughout the day, and they each must be completed at a very specific time. My aggregate dyno-hour requirements are very low, but I haven't figured out a way for a dyno to turn itself on and off for scheduled tasks this way. Admittedly, my use case is a bit niche, but a solution sure would be useful. Whiteboxing my webapp in such a way that it could be deployed on Heroku would be great, but keeping a dyno running all month to run the Celery polling process, when 'actual' computing is happening << 1 percent of the time, is a bit steep :)

Use a single dyno, and set the caller's HTTP timeout threshold to a high enough point that it can deal with the delay associated with reallocating your app.

Some time ago Ask The Pony blog made a detailed tutorial on deploying Django on Heroku and how to make it perform 8 times faster than usual using some nice tricks: http://www.askthepony.com/blog/2011/07/getting-django-on-her...

Interesting... I wonder if this makes Google App Engine more appropriate for internal apps for Google Apps customers? For quick/easy personal hacking/development having a Python stack on Heroku seems pretty attractive now.

Personally I don't like Google App Engine, but I can understand its value -- it has reasonable free quotas (even now, after the pricing scheme changed), and your app ends up running on Google's infrastructure which is rock solid.

However, I never understood why people like Heroku ... it's like renting instances on EC2, only 10 times more expensive.

Of course, people then start enumerating a whole bunch of stuff that they don't have to worry about when using Heroku. However, when starting out, configuring a server is just like configuring your localhost development environment. You just have to start with something that works, then gradually keep learning.

Heroku doesn't provide anything to me other than an unreasonable free quota that's basically useless for serving anything other than static content (poorly) ... even GitHub does a better job with their static pages option, since GitHub's servers don't go sleeping when unused.

"it's like renting instances on EC2"

False. I'd suggest you take a look at their architecture overview. It's not like renting instances on EC2 at all because dynos != virtual machines. http://www.heroku.com/how

"Of course, people then start enumerating a whole bunch of stuff that they don't have to worry about when using Heroku. However, when starting out, configuring a server is just like configuring your localhost development environment. You just have to start with something that works, then gradually keep learning."

This may be fine for some, but for those that _don't_ want to worry about sysadmin work and want to focus on rapid iteration, Heroku makes more sense. Besides, it's not the initial configuration that is the most painful, it's the maintenance and cost associated with managing your own virtual instances, and infrastructure pieces such as reverse proxies, caches, etc.

"since GitHub's servers don't go sleeping when unused."

I'm not even sure what this means. Can you elaborate?

      dynos != virtual machines
Well, yeah, a small instance on EC2 is the equivalent of 20 dynos, maybe more.

What you do get with dynos is scaling out when you need it, however you can do the same thing by having a prepared AMI, a load-balancer and a bunch of scripts with which you can start new instances in seconds.

     focus on rapid iteration
I really do think that sysadmin work and rapid iteration are orthogonal. When you're starting out, administrating you servers is something that hardly takes up any time ... but the flexibility is priceless.

     ... infrastructure pieces such as reverse 
     proxies, caches, etc.
Well, Cedar doesn't have Varnish anymore -- and IMHO, setting up Varnish is just a day's work, which includes configuring it for your own needs.

Yes, Heroku takes that away by (1) giving you a useless Varnish configuration OR (2) eliminating Varnish altogether, as they couldn't figure out a common denominator.

Also, I know that "reverse proxies" and "caches" sound bad ass, but it's really a solved problem.

Of course, infrastructure can get very hairy further down the road, but that's what I've been saying -- when starting out, you just need Passenger or mod_wsgi, as in one "sudo aptitude install" or "gem install" away. It took me a day's work to configure an EC2 instance, including a deployment workflow with Capistrano (for the first time ever). That server is still running just fine, with no further maintenance.

And if small deployments is not Heroku's strength, than what is? If you've got a successful app that needs special infrastructure care and you can't afford a good developer/sysadmin to take care of it, then you're doing it wrong.

> you can do the same thing by having .... and a bunch of scripts with which you can start new instances in seconds.

Right, so if you already have everything you need, you don't need to buy it. This argument is similar to "I don't need water when I'm not thirsty". Hardly compelling.

> administrating you servers is something that hardly takes up any time

Sure if your servers never get any traffic, or you never scale.

> It took me a day's work to configure an EC2 instance, including a deployment workflow with Capistrano (for the first time ever). That server is still running just fine, with no further maintenance.

Right, it took me less than a minute to do that with heroku, and with less dependencies (on capistrano). When I decide my app needs an extension written in node, that is another minute on deployment for me, and another day for you. I'd gladly compete on those terms :)

> And if small deployments is not Heroku's strength, than what is? If you've got a successful app that needs special infrastructure care and you can't afford a good developer/sysadmin to take care of it, then you're doing it wrong.

This is just your highly opinionated view of things. Who ever said anything about 'special infrastructure'... Heroku deal in commodity infrastructure (that's the point). You also never successfully made the point that small deployments are not their strength.

At Heroku, free applications get idled out when no request comes in for a while. As a result, when the first request comes in after some time, it can take a few seconds to find a piece of iron to run the process on, run it, and hook up all the tubes.

The advantage is you get to run arbitrary code rather than, say, serve static pages, though.

I'm sure someone will correct me if I'm wrong, but I'm pretty sure that's only true if you're on the free level (i.e. only running 1 dyno). Otherwise, your process is always active.

If you read the message in which I said that, I was specifically referring to their free quota.

I don't think that any of Heroku's paid plans swap out Dynos. Sure, for the free plans they do - seems fair to me.

Also, to get something up quick and dirty with a vps or shared host, is fairly easy, I'll agree with you. However, as you need to get more power, it becomes more problematic. With Heroku, the "quick and dirty" way is actually quite robust and can scale well.

But I'm with you that I find it a bit too costly as I know (and actually like) configuring my stuff.

Awesome news topped with even more awesomeness that Heroku is continuing to truck along post-acquisition.

We're now looking into +/-'s of moving from RAX Cloud Servers to Heroku.

I'm currently developing django app on a ec2 instance. What's the advantage of heroku over just plain ec2?

Ease of use — especially for prototyping ideas. It's easy to push a git repo to heroku and have it fire up your site — but if you've already got a fabric script to do the same thing with EC2, then that might be the better approach for you.

This is pretty much what I've been waiting for to finally migrate my project to Heroku. I know you've been able to unofficially run Python/Django on the stack for a while now, but a lack official support (even in beta form) was all that had been keeping me from taking the leap.

Python minimizes magic and maintains backwards-compatibility? LOL.

(I'm a programmer who recent dived into python - it's awesome and it's easily my favourite language to use now - but those statements are fallacies).

Compared to Ruby on Rails, there's less magic going on.

Does anyone know if the stack will only support WSGI or could I also run a Tornado instance on the cedar stack? I know when I tried dotcloud several months ago it only supported WSGI.

You always have to work with the information available at the time, and personal tolerances for risk, but Heroku has become a case study in selling too early.

Can Haskell be next?

Great news! But why is the example for Flask?

Flask is very lightweight and simple to get up and running. We also have a guide within the devcenter to get up and running with Django: http://devcenter.heroku.com/articles/django

Maybe because it takes less code to write a "hello world" with Flask than with Django.

Definitely this. A database is a requirement for a Django app, but not for Flask. I always seem to go back and forth about whether to start a micro-framework like Flask and use plugins and Python libs to assembly a framework that has only what I need, or use Django and (possible) include a lot of code that will never get run.

It really depends on your vision for the app. If this is a single weekend project or a toy web site that you'll likely never touch again, or if you do will only be for minor updates, then I say go for Flask. It'll get in your way less and let you just launch it sooner.

For anything larger than that though, Flask has always turned into a giant headache for me. As soon as I ever start to try and organize the project better because it's become too large for a single file, all shit hits the fan. Supposedly this has gotten slightly better with 0.7's Blueprints, but I've yet to see it. Circular imports and Flask's global variables bite me in the ass on a regular basis. I also have come to really miss tons of features that Django offered, like URLconfs. The decorator technique works great when you can count the number of URLs that your app has on two hands. Beyond that, ack/grepping my project to find where anything lives isn't my idea of maintainable. With Django I can very cleanly trace through the whole request/response cycle in my head and by looking at the code paths.

I've been running multi file apps since before 0.7. If you read the documentation or searched mailing lists or looked at example applications you can see how. Blueprints made it even better.

2. In the documentation you will find that you can list out routes just like django. http://flask.pocoo.org/docs/api/#flask.Flask.add_url_rule

I've read through all the documentation, and researched extensively. I still ran into problems way too often to ever warrant using it over a full framework for larger projects.

I'm not trying to get you to use Flask or anything. But I definitely find it interesting that you have "read through the documentation or researched extensively" but were unable to find `add_url_rule`.

Either way, I'm glad we have selection and choice. At least we both like python! Peace

A database is not required fr Django.

Ah, my mistake

found that interesting as well, given that the release is around python+django, but it makes sense for the sake of quickly doing a demo with minimal requirements.

[edit] does adding Flask in requirements.txt automatically handle the easy_install portion?

anything in the requirements.txt is picked up by pip and installed into a virtualenv from what I have been able to gather.

Django has been able to run on Heroku for a while now. This topic keeps coming up over and over again and it is very old news. There are a lot of major flaws in running Django on Heroku right now too simply because of how their system works.

Hi, care to list some of the flaws?

Could you elaborate on why?

id like to know too, and what would you prefer instead?

Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact