If you want to use Python then use Django. As with any framework, it can be hard to work around but Django gives you the most options to work around the framework than any other framework I've found.
Normally, if you are having a hard time with Django its because you're trying to work around it the wrong way. This takes time to learn but Django can be really elegant to work around and with.
The second option is Flask or Tornado which are pretty great too. Both would give you more freedom but also less structure. I find it better to have the structure because unless your an experienced engineer your Flask or Tornado project will turn messy quick. Django projects can turn messy too, but at least there is a more training and protocols for people to follow.
I use Flask/Tornado if I want to use Python and I know my project is a small service and will probably not get big.
I use Django if I want to use Python and I know the project will grow over time or I need features from the start like a database, migrations, cache, etc.
My test for using Django is similar to yours: If I think the project will best be represented with more than one "app" then I will use Django. If not, I'll use Flask.
App can be an API + another part of the site, for example.
Agreed. I've been using Django for only a little while, and sometimes find it quite frustrating. Then something finally makes sense and I realise I've been doing it wrong and the proper way makes everything soooo easy. The trick is to know when you're doing it the wrong way which isn't always apparent.
No, I think you're misinterpreting the comment. Django guides you the right way and how it guides you works most of the time. However, when you do hit the edge, Django's very well written docs and thoughtfully designed architecture guide you in building around its limitations. For example, if you want to swap in Jinja over Django templates or SQLAlchemy over Django ORM. Really needing to do so is still the exception to the norm, but just know that Django does more here than most other frameworks.
If you are interested in understanding this a little deeper, I highly recommend picking up the Two Scoops book [1].
I second the Two Scoops book. Django documentation is excellent (try the tutorial if you want a quick sense of django). Two Scoops help you understand why to do things in particular ways and make good design choices within all the things django can do.
We've built web applications based on Django for small CMS sites, large multi-language CMS sites, up to large custom-built applications (mostly API-based) for the likes of Mercedes-Benz and Deutsche Bahn. I can honestly say that I never regretted the decision.
It scales beautifully (see the likes of Instagram, Disqus, Sentry, etc.), has a great security track record, sane deprecation schedules, and the core team keeps up with the times and new technologies but does so at a sane pace and without rushing things.
Not to mention the documentation - ohmygod, the documentation!
In a nutshell: pick Django, keep learning Python, and try to build something that your users love.
I've stopped citing Instagram as an example of Django scaling after learning from one of their engineering managers at PyCon 2015 that they're running a heavily modified internal fork of Django 1.5 or 1.6. It sounds like they really have built their own framework (before today's Django + DRF existed).
Maybe this has changed post Facebook acquisition. I know Facebook is mostly on Python 3 and modern stuff now.
Things I really like about django that are objectively good:
* A lot of _very common_ situations and patterns are already solved for you, and they're solved in the cleanest way possible.
* The documentation is excellent, both with "guides", examples, and API reference. Django (along with OpenBSD), has probably the best documentation I've seen around.
* Clean design (of django itself) means that you can extend it, or avoid using bits that for some reason don't suit you.
* Very transparent development, and the developers are really into open source - everything is done out in the open. This last point might not be really important as a user, but it helps if you ever want to contribute (or understand) anything to django itself.
---
Even after years of django, I very occasionally still come across a functionality in django that I'd overlooked and saves be a bunch of my own code.
If you have a usecase that matches Django's niche - i.e. a CMS style CRUD application - you would have a hard time doing better than Django. If you're not in that usecase, think long and hard about it. Django is fantastic to use when writing with the framework, and a nightmare to work around when you need to do something it doesn't easily support.
Your app does sound like it is in Django's wheelhouse (CRUD against a DB with users), so it is probably a good fit.
Nightmarish (without building additional systems): Asynchronous operations (i.e. any task that takes place after you return a response), anything involving websockets or long poll responses, anything involving cross-client communication through the server (like chat). Intelligent master/slave DB routing for read-only queries.
Hard: Using any DB but PostgreSQL. You can use other DBs, but Django silently disables features. Doing anything with the DB that isn't directly managed by Django itself. Doing anything but basic pre-defined URL routing as supported by their routing system.
Use something else: Not using a relational DB as a backend. Not using their user management and authentication system. If you want to use something other than their built-in ORM, templating, and routing.
I personally recommend Flask in the cases where Django is not a good fit. A threading/forking fastcgi model with basic nginx caching is incredibly performant. That said, Flask is pretty terrible at most of the "nightmare" level tasks as well.
They supposed to make this sort of thing easy and straightforward. I'm still learning them, but this system seems to be well designed and makes a lot of sense.
Django + Django REST Framework + React should give you pretty much everything you need to build whatever you want, including single page apps. At least as far as I understand it.
Channels don't guarantee at-least-once delivery, so you can only use them at best as a refinement - you have to assume your clients will miss messages. Which makes it unusable for many use cases (though I'm sure many negligently ignore it too).
In my experience, celery handles the asynchronous tasks pretty well (and with minimal configuration)
For web-sockets, yes django is a (or was) a nightmare to work with. I had to use nodejs and django with a common redis backend to implement chat functionality (not fun at all).
I have high hopes from django-channels and the asgi server. Hopefully, we will get plug and play asynchronous chat apps for django in the future.
Like any omni-tool Django can usually solve most problems a web application framework can be applied to. Depending on your problem space however you might need to pull out a crow bar or a blow torch.
Leaving the metaphor, python in general fails at some async and cpu-intensive tasks, while other languages (or even frameworks/libraries of Python) excel at it. Whatsapp for example excelled through its use of Erlang, and would not have worked well had it been build with Django.
Django always gets a lot of praise here (deservedly) but I feel a counter-weight should also be applied to prevent less experienced, and easily influenced, developers from getting the wrong impression (though that may be a lost cause with tech hype).
Anything involving asynchronous operations has historically been painful with Django. I know that new versions are improving on this, but I think for now it's limited to web sockets.
How about using Celery and sending tasks to it? A customer of mine has a Django application which is little more that a web interface over a bunch of long running Celery tasks and it works pretty well.
No status updates over websockets, but it could be added pretty easily.
Earlier this year I worked on a project which did exactly this. I find it much more difficult to get insight into the system with Celery & RabbitMQ than with other tools. I also found Celery's documentation to not be super awesome for my uses, and it took a lot of trial and error to get it running. YMMV of course.
Celery is indeed great, but the doc falls far behind django's. The "celery+django" doc is actually outdated in a couple of places, and getting it setup isn't such a breeze.
I regret not having documented what I did at the time to actually push an update to this. :(
For example if you have a very special user model or you need object level permissions. Django supports to customize the user model to some extend and you can integrate an object level permission system, but depending on your needs Djangos ORM could hinder you and/or you would have to abolish some of the out of the box functionality of Django.
I was recently surprised that Django can handle object level permissions very nicely. The support is built in, but nit implemented in Django itself, but you can simply use django-guardian and everything just works.
when you need to deal with extreme amount of data with a small latency that comes from the database than django is not that pleasent, at least not it's orm.
It very much depends. The ORM in Django is one of the better ones, and very powerful. If it doesn't suffice it still offers query wrapping. Of course, if you need to process all 1 bazillion records in application code, then you're going to take a huge speed hit with any framework/ORM and language. Processing data at the storage site (the DB server) is simply very, very efficient.
When lots of objects are loaded then annotations usually improve performance greatly, since they avoid the "for x in y: query_database" execution, instead executing a single query with allr equired data.
well reordering 1000 objects in scala/java is fast or doing map/flatmap/filter or put them into a map, however with django orm it's not. not a problem in python tough since when you directly use pg from python it retuns a iterator which is faster when you directly put everything correctly in your list. problem with django orm is that it copies it into a list structure etc. pp. (at least pre 1.4)
Well, you're probably going to be looking for something other than Python in the first place if latency is such a major issue. Python can be fast enough to handle most use cases, but really low latency is not one of its strengths.
Often these requirements are exaggregated. "Real" low latency applications are far harder to write than just changing a language or framework. It would mean, for example, to forego any allocations in critical req-resp paths, or exactly bounded runtime of processing. Note how most software that is perceived to be "low latency" doesn't do any of that (eg. audio/video streaming)
Could you elaborate a little about DRF? I spent a day or so evaluating it among some other options for an API in a Django project, and it seemed to be a relatively poor player in the Django ecosystem and be quite complex for the same outcome compared to other libraries. It seemed like it would be ok for a DRF specific project, but in as a single part of a larger project it didn't seem to fit well. That said, I have very limited experience with it.
DRF is at its best when you develop an application from the ground up using it. It also "does the right thing," and helps you stay RESTful. In particular, if you use its HyperlinkModelSerializers and its baked ViewSets, it just plain works, no fuss. But it's important to know what these shortcuts are doing so that when you need to write custom APIs that go beyond CRUD, you know what to do.
For very simple APIs, DRF is overkill, and if I only need a couple JSON endpoints, I don't bother with it. But if I need full access to my models, you just can't beat it.
> ...it seemed to be a relatively poor player in the Django ecosystem...
DRF fits in very nicely with Django, especially if you are used to class-based views. We use it at edX for nearly all of our APIs. We don't have many complaints.
As someone mentioned regarding Django, it is important to not try to work around the framework. If you're not making a RESTful API, you may have a hard time taking full advantage of DRF.
Bringing it into a Django project of ~250 apps, I found it didn't play particularly well with app nesting and fitting into a complex URL hierarchy, although part of this could be that we might have been trying to fight it slightly.
Before I was sophisticated enough to investigate such decisions on my own (given the mass of discussion freely available on the internet), I really don't think it mattered which framework I chose.
So I'd suggest you just some popular framework and don't worry whether it's the right one. Pick one and run with it!
I picked up Django early 2015 and it has been my choice of backend ever since. Though I haven't used Django to build a fullstack application, I have used Django REST framework extensively for my backend. Impressions over 1.92 years:
1) Great documentation. Well organized, well written. Good balance between theory and examples.
2) Superb ecosystem. Coming from PHP land, I got the sense of a "anything-goes" mentality. On SO, people who help are very much interested in doing things the idiomatic way.
Agreed. I moved to Python form Perl 5 years ago because of Python. I feel I never know Python as well as I knew Perl, but I do know Django really well. But then the built in functionality means I rarely need to go too low level with my code.
Consider Pyramid web framework.
Deploy it on gunicorn server with eventlet workers.
Use pushpin if you need realtime.
Its easy, it scales, use any DB and Pyramid framework has good documentation too.
Or go with web2py if you need to start simple and easy and fast prototyping. It has an ORM which supports many Relational DB and MongoDB too if you need it. It has good documentation. Its authentication and authorization system is too easy.
Django is a good framework, because it's opinionated about the right things (ie those that fit most very well).
Also, "there's an app for that". The ecosystem is large and healthy.
If you don't want Django cause it's Django, then Flask and Pyramid are other excellent alternatives. There is also a swath of more specialized frameworks (eg. Tornado).
I'm copying here what I wrote back in 2013 to some other forum, much of it is still true...
Django is mature, full-stack, flexible, has sound architectural decisions behind it, relatively good security track record. There is a built-in admin and permission systems to help with what you require. Batteries included such as an ORM, templating, routing, forms. The Django third-party apps/module ecosystem is rather large and accessible: from authentication, social media integration, payment gateway integrations, APIs, to whole apps like invoicing, helpdesk, etc.
I don't have experience with Django, but you could also just build everything yourself on plain WSGI (the python version of (Fast)CGI).
Note I am not saying "build a new, generic framework because all other frameworks stink". Building libraries with the goal to support other projects is a neverending task.
But only building the support paste as you actually need it is not a lot of work. WSGI is a quite simple interface. I'm doing this myself and the support I currently need is only ~500 lines. The only external library I use is jinja2 templates. As database I use plain sqlite3 from the python standard lib and in some places text files.
If you don't know and don't want to learn HTTP and Web application architecture you will of course end up with a broken design. On the other hand it's very rewarding to learn how to decompose an application into independent components (templates, services, routing, database access, business logic, authorization, session management, config parsing, project directory structure, what have you).
The other big advantage is of course flexibility. If you do a good job the decomposition is better than with a framework. This means flexibility. It's easier to fix problems than with an opinionated framework.
I think frameworks like Django, Rails or Grails are excellent tools for building many business applications. Especially when there are plenty of CRUD operations and complex workflows with different administrative roles. These frameworks excels in developer productivity, stability and maintenance over years.
I've been using Grails for over 10 years, my clients are super happy with what I have delivered. I've built complex issue tracking systems, e-commerce web sites, administration panels, data management tools and so on. Code I wrote 10 years ago is still working fine today! I highly recommend learning one of these frameworks, you will make new friends and good money!
If you're using Python and don't use Django (I've tried it as an exercise for one of my pet projects), you have 2 options.
Create your own framework - writing http headers etc from scratch which will take you forever or Use a lightweight framework like Flask or Web.py.
Either way, you'll end up creating your own ways to query the database and creating patterns to communicate with frontend.
As you write more code, you'll abstract your code to reusable methods.
About a month into the project, you'll realize that you're slowly rolling out your own framework which has grown organically.
Soon you'll realize you're better off using a polished framework like Django which has abstracted many of the details so you can focus on delivering value instead of clobbering together your own framework which will become a burden to maintain alongside your app.
I've always preferred microframeworks for exactly this reason. As you build, you tailor things to what you actually need instead of building features that match the framework. I've got one project that's built on flask + sqlalchemy, but I've been slowly hacking sqlalchemy out of the fast paths because it is just too slow and I'd rather improve my grasp of SQL than spend my time learning SQLAlchemy.
Django's opinionated nature means it fits together better. Random django packages will do admin, ORM handling, etc. in a single consistent way whereas flask packages typically don't.
E.g. what seems to be flask's most popular CMS uses mongo as a backend.
Any Django developers who have made the switch to Play Franework (in Scala), would you share your experience? About to go down that route, and would like to know if it's as great as everyone says, challenges, etc. The usecase is fairly mundane CRUD app, 10-15 interconnected tables.
Django does a lot automatically: authentication, schema creation, ORM, default forms, admin interface. These are things useful to 95% of CRUD apps.
At a certain complexity, nearly every project would have to replace or mostly replace all of those. A full table scan is prohibitive, you want to customize indexes, you want to customize the design of everything, etc.
The Play Framework provides just that: an web MVC framework with basic routing, templating, and HTTP utilities. It's more general and will scale further in project complexity, but you should expect to spend more time figuring stuff out initially.
---
Other than that, there are the obvious differences between Python and Scala. Personally, I love, love, love the confidence of static verification, but there are good things about both sides.
Django is great for fast prototyping if you need very dynamic pages (because views written in Python are so powerful), a complex site structure (because it has excellent url routing), and lots of database interaction (because of the ORM). If you only need the first two, I recommend Flask instead.
But be careful if you intend to create a stable, heavily-used site. I have so many problems with Django in this use case that I've gotten tired of listing them each time it comes up and collected them into a blogpost: https://alexcbecker.net/blog.html#django-is-terrible
Django gives you a ton out of the box, with lots of sane defaults, but it still allows you to override those defaults when you're ready for more complicated functionality or business rules. I like the templating language, it lets you drop a lot of dynamic functionality into existing html templates, while staying out of the way for the most part.
Lots of people will say not to use Django because ORMs don't scale. In my experience, if you don't use an ORM you end up writing one (and doing a poor job) in the early stages of a project. Usually this happens because you're changing things quickly and favoring iteration time. You don't know what you're building yet, so you can't choose the super optimized solution. Once you need to scale things can become specialized and it's easier to go that route because you have a better idea what you're building.
I like Django's ORM specifically because it automatically creates the database migrations for you. Just change the model, then "makemigrations" and "migrate", then you continue with the next task. Great for prototyping.
If you want to use python I would give web2py + intercooler.js a try. It is "good enough" for most CRUD pet projects. Reading here the comments about Django's advantages an inner voice always says "fine, web2py has it too" except the excessive documentation and the weird non-pythonic syntax.
If you want to make something really professional just ignore the words of a n00b.
Figuratively speaking, Django gets you from zero to 60mph in 2.4 seconds. Getting from 60mph to 100mph takes 20 seconds. Going above that takes a lot more time.
Lacking details, it seems like your app could be a good candidate for Django.
For me, the main appeal is the Django admin. It gives you a great way of interacting with your data with minimal effort.
As far as the backend frameworks go, you have the 3 most popular, very good options:
- Ruby on Rails
- Node + Express
- Django
The major reason I have picked Django is that it is written in Python, which allows me to use an elegant and beautiful language that I can apply in any field, and utilize a huge amount of python apps, scripts, and stackoverflow examples.
Using Django has also enabled me to learn the solid foundations of web development, which I can expand upon and apply anywhere.
For example now I am experimenting with React and Express, and knowing Django helps me enormously, it makes understanding these things way easier.
I have built a bunch of projects with Django, and I am extremely happy about my choice, so I highly recommend it to anyone who is beginning to learn web development.
Django has excellent documentation. Whether you're a beginner or an expert, it is appropriate. That is to say: there are good tutorials for starting out and there is good reference material for going deep.
As others have mentioned, Django is opinionated and I think they get it mostly right.
I use Flask myself, in part because I wish for my data modelling language to be a separate part of my stack, but my use of Flask is heavily influenced by Django's philosophy.
I am the primary developer of Flask-Diamond, which attempts to adapt Django's opinionated choices to Flask. However, I would still recommend that most people start with Django.
The reason I keep sticking with it is the "optional batteries included" approach. User auth, for example, is something that just works out of the box with the admin/ORM, and has a lot of basic security necessities addressed.
In a more a la carte framework, such as Flask, there are packages to help with user auth, but you have to do a lot more work to get them working with whatever ORM you choose.
This means I can spend my time making products instead of wrangling packages together.
After spending several years developing in Django, I am currently building something in a Scala web framework (not gonna name names) and the level to which Django just works is utterly amazing.
If your application fits the use case - a mostly CRUD application based on SQL, HTML-based or REST, no heavy real-time component - don't think twice about it. Of course, none of those factors are a hard limitation, but if you stick to them, Django fits like a glove.
Couldn't anyone comment on the pros/cons of using Flask? Currently I've been using DRF and it's been serving my needs pretty well. I'm curious if there's anything to be gained from Flask.
If you are learning Django is best. First, it will show you the basic way of structuring your app. If you use something like Flask while learning, you'll just mash everything together in some disorganized fashion. Flask is great for the experienced folk who know what they want, and want nothing else.
Second, Django is batteries included, meaning it does a lot for you, allowing you to progress fast. You won't get bogged down in the weeds. You can dig deeper on specific parts after you have more experience.
Third, there is so much learning material with Django. "Tango with Django" is my favorite. If you are learning, you need learning material.
If you're just learning Python, I would not recommend to start with Django, since Django does so much for you. It does sound like Django would be a great choice for your side project.
It seems it's one of those things where you have to decide: Do I want to learn or do I want to build something and optimize for that.
I actually taught myself python through Django. Basically started the tutorial without previous python experience. When you start with the ORM and models, views, etc, there is enough Django-specific stuff you have to learn anyway.
Not that I'd recommend this way... but 6 years later it seemed to have worked.
DJango or RoR seem like the two "best" frameworks to learn on though. Both are fairly opinionated, and both have tons of documentation online. I am biased towards DJango, but I could not fault someone use RoR.
I prefer Flask myself, but its all kind of silly if you aren't reasonably proficient in Python or Ruby to start with. You are going to end up throwing out the first few projects you make anyway.
Isn't that the book that only teaches python2 and just wrote a blog post yesterday confused on what a turing complete language is? Or did I get my wires crossed?
I would 100% start in Python3 and DJango personally, but YMMV.
Didn't hear about any of that. The Python 2 to 3 gap isn't wide in my opinion, I move from one to the other without issue. I guess it depends on what kind of programmer you want to be. I believe you will have a deeper understanding if you work from the standard library outward.
Because you're a gentleman of exceptional demeanor, character and surpassing technical judgement, who unfailingly chooses the best tool for the job at hand from the Maelstrom of pretenders.
> Because you're a gentleman of exceptional demeanor, character and surpassing technical judgement, who unfailingly chooses the best tool for the job at hand from the Maelstrom of pretenders.
if you want to use sqlalchemy use Flask.
If you want to use DjangoORM, use Django.
Pros and cons to both. I guess you can find weird solutions that look better one way or the other except the ORM.. but I don't really see it.
I currently like DjangoORM like 10% better, so I would use Django. It is my favorite ORM I have ever worked with. Right balance of saving you SQL writing time, while still having a good SQL escape hatch when needed.
If you hate DjangoORM, seems silly to suffer through it though.. use Flask or something else.
I read in an answer of yours that you're learning Python. I learned Ruby myself with Rails in 2005 because I had no other reason to use Ruby back then. Rails was a much lighter framework than Django is now. I would recommend to start with something very lightweight, get the gist of it and move on to Web2py or Django.
What you're looking for could be Flask or even something simpler like
I inherited a Django project and after a few months working on it I wouldn't start a project of mine with it. Obviously customers can ask me to do anything :-)
Disclaimer: I might be biased against it because I don't like Python much (it looks like a badly engineered Ruby, and probably the other way around if you like Python, no bad feelings).
First problem: coming from 10 years of Rails, Django is very weakly opinionated, so you can arrange the project as you want, name the db fields as you want, etc. This is not as bad as having no framework or using Flask, but a developer joining a project might have to spend time to get familiar with the general structure of it (and the bill for the customer keeps growing, so the chances he won't be happy).
Second problem: it doesn't import automatically every model and module. It gets tiresome to do that in every file and in the shell (and the bill...) This is not a language thing because Ruby has to import modules too but somewhat Rails works around that. Surprisingly it raised problems for me only a couple of times. The sum of the time lost with Python in the last few months is greater than the one I lost in Rails because of that.
Third: the templating language is not Python but some crippled down language. Somewhat people still manage to write nice code with Web2py which uses full Python in the templates. Rails too. Why not Django? More time lost to learn one more thing.
Web2py is an alternative but it has its share of problems: the first two of Django plus a very risky approach at migrations which are automatically enforced as soon as you run the code. If there is a way to turn that off and you really like Python or have to work with it, look at Web2py.
About the nice points, it's got everything that it has to be taken for granted in a past 2005 web framework: tests, migrations, ORM. It doesn't miss any particular feature, it's got a big community so it's a safe choice and I won't steer customers away from it.
Those are all intentional choices and benefits of Python
1. This gives you flexibility to structure your app in a way that makes sense for what you're doing. In addition, I've found Django's approach to modular apps to be far more manageable and approachable than Ruby's "all the controllers go in the controllers folder" approach.
2. Explicit imports is a Python thing and a huge benefit. I've looked at plenty of Rails code bases where I had no idea where different functions and variables were coming from. In Python/Django, you can always see the explicit definition at the top of the file.
3. The "crippled down" templating language is very intentional as well. Django is preventing you from writing your application in the templating logic (something I've seen too much of in Rails) All your template should be doing is specifying how the data is to be rendered in HTML/whatever.
Really Python/Django could be said to be Ruby/Rails flipped on it's head.
Ruby itself isn't very opinionated (there are usually several ways to do something), while Rails is.
Python itself is pretty opinionated (there should be one right way to do things), while Django isn't.
In general it's a matter of opinion and perspective but I personal find Python and Django to be much easier to work with than Ruby and Rails.
In theory, one should take a look at web.py and flask first.. It will teach essentialism and minimalism (web.py) and doing the right thing in a right way (all of them and Python in general).
Normally, if you are having a hard time with Django its because you're trying to work around it the wrong way. This takes time to learn but Django can be really elegant to work around and with.
The second option is Flask or Tornado which are pretty great too. Both would give you more freedom but also less structure. I find it better to have the structure because unless your an experienced engineer your Flask or Tornado project will turn messy quick. Django projects can turn messy too, but at least there is a more training and protocols for people to follow.
I use Flask/Tornado if I want to use Python and I know my project is a small service and will probably not get big.
I use Django if I want to use Python and I know the project will grow over time or I need features from the start like a database, migrations, cache, etc.