Reinventing the wheel with microframeworks?

[stavros]

Recently I started working on a new project, and I decided to go with AppEngine to learn a bit about it (and because it looks great so far). Since I'm going in that direction, I decided to use a microframework to have the most control over the programming, so I went with Flask (this isn't specific to Flask, however).

After coding a bit, it feels like I'm always reinventing things that Django already has. I'm pretty sure that this is because I have years of development with Django under my belt, but the mere fact that I have to basically write my own framework is alarming.

I know that Django on GAE has a few problems (mainly that the ORM is not designed for it), but I hear that the django-nonrel people have done a good job porting it.

What are your experiences with microframeworks, on or off GAE? Has anyone used Django on GAE? Is it worth giving up on all the batteries that Django provides, and what do you gain?

[znt]

Two reasons come to mind about using micro frameworks:

1- You're working on a very customized architecture so having a large but powerful framework is a hindrance rather than an advantage.

2- You're starting to learn web programming on a particular language and you want to be closer to how things work, "inventing" or duct taping extensions on the go.

I've used django-nonrel on GAE, which has an ORM that works with GAE but now I'm reading Programming on Google App Engine and realized that learning the low level basics is a must as App Engine differs a lot from a regular database.

PS: Transactions does not work on django-nonrel, one more reason to go native python.

[stavros → znt]

Learning the low-level basics of GAE was important to me too, that's why I went with Flask, but having created the app structure and written 50 lines of code I think I know all the low-level stuff now (they aren't really that many).

With transactions not working, there might be one reason to go native, but there are a million others to use Django (all its batteries). I'm more worried about the lack of migrations et al, but you have to do those by hand anyway, so it's not that big a difference. One of the reasons I heard against Django is the slow startup, which can kill you if your app isn't very popular. I wonder how valid that is.

[wkornewald → stavros]

1. You can still use transactions with Django-nonrel. Just use App Engine's run_in_transaction() function in combination with Django models. No problem.

2. The slow startup problem doesn't really matter, anymore, because App Engine provides reserved instances and warmup requests. It only matters to those who don't want to pay a few $ for reserved instances (but even without reserved instances there are ways around that ;).

[stavros → wkornewald]

That's what I thought too. I wonder if this means that Django is (or will be, when it's mature enough) a very good alternative for GAE. A memcache caching module would be especially useful, as it would enable the entire Django caching infrastructure to work on GAE.

[wkornewald → stavros]

Caching is already pre-configured (in default settings) to use GAE's memcache in djangoappengine/Django-nonrel. The session app is also pre-configured to use cache+db. Django's mail API also is pre-configured to use GAE's mail API via a special backend. Even file uploads work with FileField if you use django-filetransfers. A large part of Django already works out-of-the-box and it's easy to write fully portable code.

With django-dbindexer you can even use QuerySet features which are not natively supported by App Engine. At some point django-dbindexer will even support JOINs via automatic denormalization (or via in-memory filtering where desired) and maybe also simple aggregates and scalable counters (via QuerySet.update()). This will simplify complex DB code a lot (compared to App Engine's model API) and by reusing Django's ORM as the abstraction for common NoSQL patterns you can not only move between NoSQL DBs, but also between SQL and NoSQL (at least to a certain extent; the goal is to provide a better high-level NoSQL API and only implement scalable features, not build a SQL emulation layer that wouldn't scale).

[wkornewald → znt]

Transactions via run_in_transaction() do work (it's even mentioned in the docs).

[wwortiz]

I think this blog post by the flask author is pretty relevant http://lucumr.pocoo.org/2010/6/14/opening-the-flask/

I can't really respond to the GAE thing as I don't have experience with it.

[stavros → wwortiz]

I'm not sure how relevant Armin is being there, in all my time doing Django development I've never had to do that much work to extend it. Extensions were always clean, repeatable and reliable.

Even if they weren't, I saved countless more hours by using what Django provides than I spent working around what it doesn't.

[upinsmoke]

Try http://tipfy.org. It's based on WSGI and Werkzeug, and it has many extensions like Jinja etc. It's only a thin layer on top of GAE.

[stavros → upinsmoke]

I did, but, as it was my first contact with GAE, I found it too big of an abstraction to understand how GAE worked. Flask allowed me to develop while still grokking the entire system, so I went with that.

Plus I didn't like Tipfy's view classes, they feel too Java-y and unpythonic.