One of the things that has been really fantastic about Django is that we're still happily using it for the vast majority of code in the project, and every time Django comes out with a new release, I read the changelog and get excited about several improvements that actually make my life better.
Overall I think we've gotten a ton of value out of Python and Django and would recommend it to anyone starting a new full-featured web application project today.
My only frustration with this release announcement is dropping Python 3.5 support now. Because Python 3.5 is not EOL and used in LTS OS releases that have vendor support through 2021, this forces us to choose between our policy of supporting vendor OS releases until they reach EOL, not upgrading to Django 3 for the next year or more, or shipping our own Python on Ubuntu Xenial.
Sometimes I've knowingly relied on framework internals in ways I expected to eventually break, but those situations don't represent the bulk of my experience with painful upgrades.
One of the things I have learned early on is to not depend on the system-bundled Python. I either just use pyenv to install whatever Python version I like, or I use a Python Docker image (https://hub.docker.com/_/python/). That way, I'm decoupled from the system-bundled Python and it's easier to test out new Python versions.
Supporting only Ubuntu releases is one thing, but the code base actively fights installing it on anything else but the supported distributions (ubuntu). At which point, why even have a installer, just ship an image.
Since Zulip has so many moving components, I would love to see it be installed with something like Nix. Doing so would allow it to be installed in a consistent way across many different OSes, and decouple your reliance on software provided by the OS. It would also make upgrades seamless.
You can install nix on any single linux distribution, not just nixos, and since your installer already hard requires running as root, you can install nix and then install zulip in nix.
I wouldn't say the installer is actually a complete machine takeover; we're pretty cautious about using the system versions of dependencies shipped by the OS vendor. But we do __document__ that you should treat it as one, mainly for support reasons. The support experience that we currently provide is that where possible we respond to, investigate, and debug all reports of issues installing Zulip, with the goal of making it Just Work (this includes working to give nice error messages for common user errors).
Before we added those warnings in our documentation, we got tons of reports from users who were trying to do things like install Zulip in a shared hosting environment with other software that doesn't expect to share its database (etc.) with something else, and where they don't have root access. Also, investigation often determined the system in question had low RAM/disk resource limits from the shared hosting environment that meant running Zulip on that system wasn't going to work in any case.
At one time, that sort of shared hosting setup accounted for the majority of reports from folks having issues installing Zulip. Investigating those reports was a huge waste of time for our development community. We solved that problem by making our documentation recommend installing Zulip on a dedicated system. This also provides a lot of benefit in terms of our being able to do things like tune the configuration for postgres and memcached to allocate resources in an appropriate way for a Zulip installation with the system's RAM allocation.
There is occasionally discussion in the community of adjusting our approach here, but given how easy it is to get a VM or container these days, we're currently happy with guiding users to the dedicated VM/container path that we can guarantee works ~100% of the time and provide a great installation experience.
Django has an LTS release (2.2) that supports Python 3.5 just fine. Doesn't it make sense to use the LTS Django on an LTS operating system?
There are plenty of ways to use newer python, too. Docker, PPAs, etc.
Don't ever rely on python that is included with OS. It makes upgrades really difficult. This is especially true with RedHat and CentOS where they are way behind. There are other issues, like dependencies, you are bound then to only using the dependencies version they provide. If there is a bug in one, tough luck, you can try to upgrade but then you're risking that you will break system tools (yum etc). Just assume that the system python is for the system.
Use the right version you need (if you use redhat/centos I highly recommend ius.us repo, they make sure that it can be installed side by side and doesn't conflict with anything in the system) then create virtualenv and install your application there.
That way you have full control of the python version you use and all of your application dependencies. Also when ops team wants you to move from version 7 to 8 of the OS, your application should work the same without no or very small changes.
So you can chose whether you want python 3.5.x, 3.6.x, 3.7.x, 3.8.x or maybe still you want 2.7.x.
How much does the framework help you, and where? What are the places where you have to go against the grain, if any?
While there are some land mines with any ORM, the Django ORM does allow us to write the vast majority of our database operations with clean Pythonic code, rather than going to raw SQL. We need SQL in a few performance-intensive code paths, but it's the exception, rather than the rule, and the ORM is constantly getting better.
And in contrast with some other ecosystems I've worked with, the libraries generally are well-written, well-tested, and stable, so even if they're not well-maintained because the old maintainer changed jobs or got busy or whatever, it doesn't feel like a burden to fork or adopt the library.
I'm not particularly happy with Django forms or Django REST framework with how they organization validation code. For the former, I think the problem is just kinda messy; we mostly solve it by only using Django forms for the authentication code paths where we can reuse its libraries. For the latter, we have a framework we built that I'm very happy with and we plan to spin out for other projects to use when we have a bit of time. There's some documentation here: https://zulip.readthedocs.io/en/latest/tutorials/writing-vie... for those curious; though that doesn't cover cool details like the mypy integration we did more recently.
I'd love to have you on as a guest for a podcast I recently started called Running in Production.
Dan from https://realpython.com and I just did one recently. That site runs Django too. You can check that out here: https://runninginproduction.com/podcast/4-real-python-is-one...
Click the "Become a guest" button to get started if you want.
Note: If you're reading this and you're not from Zulip and you are running Django in production (or any stack) hit me up too.
To be fair, I think the next LTS release of Django will be the last minor release before 4.0, 3.0 is not LTS.
If Django can now also support annotations and async code I dream of an scenario where apis can be built using this two elements.
Does anybody know a good resource to learn/catch up with this release?
Am I correct to understand this as meaning async views can’t even read from the database yet? I guess the only use cases for ASGI views currently would be interacting with outside-Django backends that implement async support and such?
Also, thank you and all the contributors for such amazing resource.
If I am willing to hack a bit, is it possible?
What is the main obstacle? Is it all middleware?
middleware is a part of it, then there's the ORM, caching and anything else that does IO.
Another option is using something like Tom Christie's orm project (https://github.com/encode/orm), which is a wrapper on top of sqlachemy with a django like interface.
In a way it feels kinda lispy (in a good way) to me, data is code, small DSL for working more efficiently with specific domain.
There’s a couple reasons:
1. you get a breakdown of all the new features, which only takes a few minutes to kind of quickly go through each version and decide which bits you care about
2. you get a list of backwards-incompatible changes, which resolves any upgrade regressions, as long as you’re not using internal APIs that have changed
(Partly inspired by FastAPI)
Coupled with Pydantic for validation it really does make things easy :D
1. Performance, it's a wrapper on top of starlette/uvicorn, which brings the performance closer to nodejs (https://www.techempower.com/benchmarks/#section=data-r17&hw=...). (I did run into some issues with it though due to the default response validation when serializing large response bodies)
2. Lightweight background tasks (from starlette)
3. Documentation generation from type annotations.
It's a nice tool for microservices but coming from django you'll have to roll your own database management, authentication, sessions, caching, admin and etc. I'm also not a fan of the magic request unpacking using type annotations and prefer getting a request object as is done in django and starlette. IMHO most people would probably be better off with plain starlette and a 3 line decorator to handle request validation and response serialization.
Why do you think plain starlette is better? Are the type annotations annoying to debug?
I was trying it out for a machine learning service that had to send back large dense vectors and I hit some other performance issues (like https://github.com/tiangolo/fastapi/issues/360#issuecomment-...), due to the coupling of validation with serialization.
But the most commendable value of the DSF – which I believe is a model to be replicated – is how effectively it has managed to both attract new developers and also mentor them so well that they have gone on to become core developers who maintain the framework later on.
I'm still subscribed to the django-developers mailing list, even if I have had no free time to contribute with more than a couple patches over many years, simply because I enjoy watching how incredibly smart and efficient developers collaborate.
Kudos to all of you guys, and congrats on achieving this big milestone!
You also gain access to new features as they come out, this way.
Do you have some examples? Are you sure you are not confused with dependencies?
The release notes mention less than ten deprecations related to django. Other stuff which is not supported anymore is mostly non supported database versions.
There's a few times I had to dive into Django's internals, and every time it was logical and straight forward to understand and modify.
Using django in a notebook is definitely not a typical use case so I wouldn't worry about it too much.
Ideally the ORM would be a standalone package that could be used outside of the web server context. (I know sqlalchemy is an option but then you lose all of the benefits of django)
It's still kind of a mess though because you have two universes that are not really compatible, forcing you to rewrite everything that touches IO. For django that probably means a rewrite of the ORM, caching and middlewares.
Alternatively, you can also use it with a plain Promise, which makes for a nice mutex. I've also used this for imitating async constructors, which is not currently possible.
Using it for more than 10 years professionally, full time with no stop.
It helped me to pay my bills, bring food to the table and build my ideas and get one step closer to my dreams.
Thank you Django and everyone around or behind or even remotely related to it.
Almost all of my work with Django is through full time and once a while contracting gig here and there (very short ones.)
Not just my day time job, I also implement all my side/personal/commercial projects with Django (gonevis.com is my latest project).
Django never gave me a tough time to understand anything, for almost everything there's a well-maintained library and properly tested with enough community around it, it's ORM is crazy easy to work with, optimize, tweak and change when needed.
Every new release is easy to upgrade, you may have some problem with third party libraries to catch up if something is not compatible with them but in my experience with medium-size code base (~300k LOC) the whole upgrade would take 2-3 hours and that was just changing code and running the tests, all in all, in 2-3 week most of the other third parties would have upgraded as well.
It's mature framework, doesn't get crazy with new shiny things (NoSQL, MongoDB, WebSocket, etc), it might be late to some technology but it's because it stays in the corner until all these shiny things have worked out their problem and issues, then those things will become part of the official code base, otherwise if you're in hurry, there's always another third party that bring those for you, either NoSQL, WebSocket, push notifications, OTP, etc.
For me, the most fascinating thing in Django is the ORM, Authentication, Admin, Views and template engine and the crazy support of PostgreSQL (almost the same applies to other database backends in Django), sessions, caching and every other single thing in that carefully have been layered upon each other and work in harmony. The database migration (formerly South migration then part of official) is something you can't easily find in other frameworks (Rails migration is great as well).
Now, DRF is something else, since starting to work with DRF, I've become an API monster :D
The framework that lets you implement your idea without getting in your way and allows you to bring ideas from zero to production very quickly, is the one I look forward to using (I use Java Spring once a while as well).
For me, Django and it's surrounding makes sense, it's logical and it's not magical, maybe it's just me and it may not be something tasteful for others, if you find a framework you're comfortable with it, then use it, if you get into Django, I hope you feel the same.
I find the productivity somewhat higher from a business point of view, to turn around projects for customers when using Rails instead - largely due to the gem ecosystem and the way community libraries tend to play well with one another.
REST APIs are also incredibly easy to throw together using DRF.
django-stubs is black magic and helps a lot.
Here's an example setup that I use for my library:
the "sample" site is 100% type checked. Look at scripts/test.sh for running mypy in ci (and .github for the CI config)
Ran into a couple issues with the typing of ‘client.force_login()’ where the ‘user’ parameter was typed as the ‘get_django_user()’ which wasn’t the same as the custom user model I defined in the ‘models.py’. I also had issues with ‘from django.conf import settings’ not being the correct type so I’d suggest changing that to ‘Any’.
You might be able to override the few types that don’t work for your given setup or you can just fork the repo.
Overall, I think the pain is worth the benefits.
> I also had issues with ‘from django.conf import settings’ not being the correct type so I’d suggest changing that to ‘Any’.
This one also has plugin support, and should be working mostly correct. Fill the issue if you can reproduce, I'd love to help with that.
What's your experience with that whole "plugin has to actually run django.setup() before typecheck" thing?
Try 'django-stubs' mentioned above too https://github.com/typeddjango/django-stubs, it's mostly complete. Mypy plugin inside takes care of a lot of stuff.
Disclaimer: I'm a creator of the 'django-stubs'
The idea of Django throwing errors when you try and run async unsafe code within an event loop gives me cause for concern. I can just imagine folks who don't understand adding 'yield', 'await', 'yield from' until the errors go away - while being confused with the resulting execution. I think there is always a place for synchronous code in Python - I think Django should lean in to the simplicity afforded by it.
Last note - I also am very weary to mix asynchronous code with other primitives like threading, multiprocessing. If I am going the route of async programming - I usually just intend to optimize a single thread - and then I leave it up to the OS to optimize across multiple processes.
Agree. On the other hand, Trio (https://trio.readthedocs.io/) gives me plenty of reasons to switch.
A lot of the time you simply can't not have another thread or process, for example, when you have a blocking library call or CPU intensive task.
Django is incredible. It was useful to start with the minimalist frameworks to understand the true value of Django when I got there.
Django is amongst my software top 10. Maybe top 5.
The thought going through my mind right now is "I wonder if there is some way of creating a Django project and make it resilient to future Django updates and releases with minimum fuss?"
I've had to deal with ongoing and inherited legacy projects which run on Python 2.7, use the long-deprecated Pylons, for which there is no easy upgrade path other than a total re-write in less deprecated technologies. No practical way to convert that Pylons project to Pyramid, for example.
I guess the answer to my thought above is to "Not do/use anything /too exciting/ when creating your Django project" - i.e. only use the most basic and generic Django features, but even then what's to say those won't get deprecated in favour of some new exciting replacement?
I guess that's one of the dilemmas developers have day to day?
You probably already know, but the long term release timeline is shown at https://www.djangoproject.com/download/
Sticking with LTS releases really helps. You'll still have nearly a year to migrate from one LTS release to the next.
In my experience it also helps to avoid small (e.g. a single developer) and young Django apps and stick to the big ones. The amount and quality of the documentation can also be an indicator of longterm viability of the project.
I've done a few more significant migrations (including Python 2 to 3), and about the only bad things are apps whose development stopped and those with developers implementing hacks using Djangos internal API. Everything else is usually done in less than a day for the whole Django project. I only did small projects with 5-10k loc, though.
> "I wonder if there is some way of creating a ???? project and make it resilient to future??? updates and releases with minimum fuss?"
- Use control version (mercurial, git, ...) in everything, including utils and side projects.
- Record all dependencies somewhere in the project.
- Automate the builds, so you also have how reconstruct the project
- Use VM or containers or virtual envs or similar
- Rely in a small "core" of solid dependencies (in my case: PostgreSQL, sqlite3, nginx, ubuntu LTS) that could cross across projects and versions.
- Reduce dependencies (but each day is HARDER and HARDER) or better, NOT USE VERY FANCY STUFF (rely on PAAS or kool-aid kind of tech.) For example, If I rely in a http client library it can be dead years after I need to upgrade. That sucks, but, a "http client library" is so "normal" to have that I can rely in found a alternative easily. However, if I base everything on top of Firebase or other cloud stuff, if it die... good luck with that!
They said "with minimum fuss". Write every non-trivial functionality yourself from scratch is the opposite of minimum fuss.
When you use something fancy, and the devs stop supporting it, worst case is you have to write it from scratch yourself at that point, or hope that an alternative exists.
Your suggestion is acting like they don't exist in the first place and write stuff yourself like fancy stuff disappeared - it is a huge hassle and investment all the same.
I know everything have "but..." somewhere (for example: We rely in operating system and the browser, 2 things too hard to recreate) but the idea is always about minimization.
Sometimes you NEED to rely in VERY fancy stuff... but then you are walking on luck...
I don't think you need to do this with Django. Features are generally very mature, stable, and are not deprecated frequently. When they are, they are deprecated years in advance with long term support still provided.
In all honesty, I've seen very few projects run feature lifecycles as well as Django. I'd use everything you need to in it without concern about this.
What you want to do is follow this recipe:
You put a 404 handler in pyramid to delegate not found views to pylons, and move views one by one.
In a pyramid tween you can push pyramid session, settings, request,response to pylons app globals.
This approach works really well - you don't have to do total rewrite.
You have same request/response objects, same sqlalchemy ORM, so you can certainly take advantage of that during migration. I don't think anyone at our company regreted the path we took and it was a massive codebase that was making money - no one can afford rewriting that from scratch in $WHATEVERISCOOL today solution.
It certainly is possible - if you need any hints contact me.
The company in question has plans for new areas of business for which this site won't be able to handle. It's simply not worth spending the time, bother (and money) trying to hammer that round peg into new square holes, and hence not worth rewriting to Pyramid. The revamp mentioned above will tide them over until something more suitable (and bespoke) is created, which is where Django comes in.
But in all honesty: django is the project I would worry about the least here. Even when stuff is deprecated, an improved alternative tends to exist and is pretty well documented. The level of care that goes into each release is very high; the number of brown bag bugs that get out very low. If you want a quiet life, django is a very good fit.
Have lots of automated tests. The more surface area you cover, the easier it'll be to upgrade.
> [...] use the most basic and generic Django features, but even then what's to say those won't get deprecated in favour of some new exciting replacement?
In my experience, Django maintainers are slow to rush to new exciting replacements, and even slower to deprecate and remove things. From a compatibility perspective and ongoing maintenance, it's a great approach!
It's rarely a good idea to use the latest version of Django simply because most projects will be reliant upon 3rd party libraries which will often be slow to catch up to supporting the latest version.
The beauty of this type of architecture is that you don't need to worry about upgrading your entire stack just because your underlying framework bumped a version number. You can create a service w/ Pylons here, a service w/ Whatever there, and lock them down without needing to refactor them all just because you want to add another feature.
Edit to add: I would class an advantage of microservices if it made it easier for you to have automated tests or deployment/rollbacks, since that’s much more effective for this problem in my experience. I’ve seen people do that with monoliths and fail to do that with microservices so again, no magic wands.
You push all your complexity to your infra, where you end up having an order of magnitude less visibility, way more problems debugging trivial issues, and making your life more difficult for dumb, simple problems that have been solved two decades ago.
Monolithic frameworks are amazing. They have well-designed APIs by people who don't want to waste time upgrading to the latest framework of whatever, and splitting them into actual services is equally complicated provided you've architected them well. And you haven't, it's as much of a PITA as any "serverless" thingy of the day.
(The release note erroneously still show the "Under Development" banner.)
I’ve seen python used extensively for data science work, but I’m not sure how well dynamic typing works (in terms of support ability/maintenance) in a larger code base, especially as some trivial knowledge gets lost and newer folks are introduced to an older code base. Do you just set break points to trace? I digress, but open to hearing folks thoughts.
That said, jumping a bunch of releases (e.g. 1.11 -> 3.0) actually often works out just fine, especially if I've got good test coverage. It's just that when it doesn't work, it can be tricky to find out what's going wrong. That's because the release notes are typically written with just a single step in mind, so with a bunch of releases I find myself scrambling to figure out which set of notes I need to read.
The best thing to go is run the dev server with `python -Wd` to get the depreciation warnings, and fix them before each upgrade. Worked perfectly!
Of course it depends on your codebase. In the past I worked on a project which interacted a lot with Django internals, with a lot of code reflection involved. Upgrading that was a pain even between minor releases.
It's not much work, and you might just take the opportunity to stop and read the release notes of each release and find something useful there.
So DB calls aren't async?
The asyncpg library could be used for direct, asynchronous database access, from within a Django 3.x app.
The asyncpg library is a low-level async Postgres adaptor, similar to psycopg2, except that it provides much better abstractions and automatic encoding / decoding (vs just piping bytes back and forth like psycopg2), not to mention it is much faster than anything else out there. IOW, asyncpg would serve as a much better foundation for a future ORM. My best guess is that, in the future, Django's ORM will be refactored to optionally support async and will supply an adaptor for asyncpg.
You're not going to get maximum performance out of a python program, generally. But unless your python program is using 100% cpu that's not really an issue, and there are very few cases where I'm using python and it's CPU that's slowing down the program. Mostly it ends up being some kind of IO.
If you're looking to do complicated mathematical operations, you can call out to an external library like `numpy` of tensorflow to take better advantage of cpu/gpu compute.
You can also use a faster python interpreter, like `pypy`, although then you start getting slower startup times and more memory usage. The performance is closer to something like java but drawbacks are generally enough that they don't get used, since it's very rare that you need that extra performance.
There's also micropython, which lets you run python on a microcontroller and makes it pretty easy to optimize things for your hardware, although you lose some useful debugging and introspection features.
But for the most part, I haven't run into anything I'd call "performance issues", just the knowledge that python is a scripting language and it's never going to be the fastest thing around. Python's performance just isn't an issue very often.
You have to work pretty hard to hit a CPU bottleneck, and I can't imagine how you'd do that building a simple CRUD website.
Can you explain a bit more about how the people you know are hitting that bottleneck? I mean I've literally built CRUD apps on an esp32 microcontroller using python, and they work performantly. If real python with a real web-framework can't do the same then something is going horrible wrong.
But then maybe your problems are deeper that choice of language.
So many people immediately dismiss relational databases, then re-implement all that functionality in their apps with various bugs and performance bottlenecks.
ORM is another bad technique. It supposedly promises you that you don't need to know SQL to use, and that is true for the simplest examples, but you absolutely have to know it for anything less trivial, then you have to figure out how to write query in ORM to get a desired SQL statement (it makes it very difficult to use advanced SQL functionality), and that's not the end. ORM constantly will make unnecessary SQL queries and by default request all fields, even if you don't use them, adding additional performance bottleneck.
In my current workplace thanks to ORM we make on average 10 queries per request and at peak generate 1Gbps throughput from/to database because of those inefficiencies.
I think the way to go is SQL support in IDE, I recently saw PyCharm with DataGrip where if you configure it to connect to a database it will start recognizing SQL in string statements and start treating it like code (so you now have autocomplete, refactoring etc). I think this is probably the proper way to do it and I wish that other IDEs would have similar features.
please support this assertion with an ORM whose documentation promises this.
> ORM constantly will make unnecessary SQL queries and by default request all fields,
as does "SELECT * FROM table" if you don't write out the fields and use a buffering database adapter (which is the case for nearly all Python database adapters), so, when using an ORM, you need to give it instructions over what columns you need to fetch. This is not unusual nor even anything a library could possibly guess for you if you do not give it this intent.
I don't think any ORM would officially claim that, but many people decide to go that way to avoid SQL. For many simple examples it looks simpler than SQL.
I also experimented myself and wrote my code using SQLAlchemy's ORM. Then edited my code to use psycopg2 natively. I realized that SQLAlchemy didn't save any code for me, in fact it was more verbose. It also heavily encouraged me to get the raw data from database and do processing of it in the application. It also did unnecessary extra queries.
So with it now I need to:
1. still understand SQL
2. trying to figure how to write the SQL statement using ORM
3. figure out how to make ORM not to do extra queries
> as does "SELECT * FROM table" if you don't write out the fields and use a buffering database adapter (which is the case for nearly all Python database adapters), so, when using an ORM, you need to give it instructions over what columns you need to fetch. This is not unusual nor even anything a library could possibly guess for you if you do not give it this intent.
My problem is that in addition to knowing SQL, I also need to know how to do in that specific ORM that's for the specific language I'm using.
ORMs that are more honest about this, such as SQLAlchemy, are generally better than those that try to pretend that you really are dealing with entities.
But anyway even query builder is not that great. Recently used PyCharm's integration with DataGrip. Basically the way it works is that if you configured a database in your project and let PyCharm fetch its schema, suddenly the IDE started recognizing the SQL statements providing autocomplete not only the statements but also table names etc, it also offered refactoring which created migration scripts.
After using that I think that's the proper way of solving the impedance mismatch and at that point you no longer need ORM or query builders. I hope other IDEs will start doing the same thing.
- to make my application scalable I want two endpoints one that is read only, that goes to replicas and one that goes to master for modification. That way in my application I have full control which queries go to replica and which need master
- use datatypes that are recommended by the database
- do upsert
- do select and only return fields that I need
- perform insert and return only fields that I want (my ORM was performing another select)
- ORM also does bunch of queries that I don't need, I don't want it to add extra load to database if I never use the results of it
Those are very simple things, but I'm also curious how you can do some advanced operations like group by having, window functions, CTEs, aggregate functions that return JSONB structure (so you don't have to send duplicate data and avoid N+1 queries). How do you use PostGIS?
You don't! That is not what an ORM is for. Use SQL. Any decent ORM with allow you to integrate SQL.
Check out the Fluent (query builder) and Eloquent (ORM) from Laravel. They allow all the features you want.
It’s possible, of course, but I’ve usually seen it as a symptom of not having a good culture around monitoring and troubleshooting — e.g. I remember someone porting an entire site to Jinja2 alleging performance wins, and it’s true that template rendering got (IIRC) 10-15% faster but they’d missed that 99.999% of the total runtime was being taken up by unoptimized database access generating many thousands of queries.
What I mean is: when you start optimising, no matter how well you do, you will eventually hit a CPU performance wall. Then you will realise that only a rewrite will get you out of that hole, and by then it will be late.
That said the Python VM is a well optimized beast, it's very dynamic, you can monkey patch anything and everything at runtime on-the-fly. This is of course what makes it hard to speed up, too many things to "check for overloadedness" when doing anything. A JIT could help, but that's a lot of work. (Hence why PyPy took a decade, and the whole Python C API compatibility problem is still there - PyPy can only JIT the Py parts.)
Anyway. With that much dynamic stuff and with the VM and the language already well tuned for concurrent memory safety its 'async' story is great and performant. (Like NodeJS'.)
So Python is great for orchestrating whatever you need to handle requests. And if you want to scale up, just spin up more instances. (Just like with Java, but you don't have to think much about the overhead garbage collection, because CPython uses simple reference counting.)
Finally, my concern with Python was the lack of static typing, which made any large project hard to manage. (And resulted in a lot of boilerplate tests and excessive defensive coding.) But mypy is great, and with that it's a joy to work in Python. (Just like TypeScript made the JS world exponentially much more saner.)
Nowadays it works correctly with almost all C modules. Mostly you just need to use `pypy3 -m pip install foo` instead of `python3 -m pip install foo`.
It seems that any new/greenfield Python project should target PyPy though, just in case, and the performance never hurts.
You can use Python (or Ruby, etc) to build fast websites that scale well if you are smart about how you use it and how you design your application, but it still isn't a particularly fast language compared to others.
That's how asyncpg can outperform some libraries that you normally would think was not possible.
Instagram can afford it so it works fine for them. Reddit can't, so their site is always slow or down.
> Django 3.0 begins our journey to making Django fully async-capable by providing support for running as an ASGI application.
Bleh. ASGI is probably the worst implementation I can think of for a common async interface. Stringly-typed callbacks? async generators exist for a good reason. It's also very asyncio-centric, and asyncio is Not Good.
Having worked and developed with it extensively for over a year now, ASGI is one of the most elegant and well-designed interfaces I've seen.
Just wanted to address this:
> async generators exist for a good reason.
While the idea of using async generators might sound attractive at first, there's so much more to ASGI than exchanging messages in both directions.
Even on that aspect, I'd be interested in knowing how you'd achieve bidirectional communication async generators. How would the ASGI server call into the application and handle messages coming both ways? Surely, it can't just `async for message in agen: ...`, as that would only replicate the `send()` ASGI callback — what about `receive()`?
Besides, how would middleware look? Would it be as simple as wrapping `receive()` and `send()` into whichever other callable adds the functionality your middleware needs?
And you probably wouldn't be able to support all the possible syntactical implementations allowed by the notion of a Python callable: functions, classes, callable class instances.
So, yeah, I think ASGI is a beast.
Also, I wholeheartedly congratulate the Django team for pushing the standard out there and driving the Python async web ecosystem forward. You rock!
Yield is two-way. ``result = await agen.asend(next_message)` ` et al.
> Having worked and developed with it extensively for over a year now, ASGI is one of the most elegant and well-designed interfaces I've seen.
Having worked with Python async for 3 years, anything that asyncio touches is completely poisoned. ASGI is another in the line of terribleness of the modern async ecosystem.
> Besides, how would middleware look?
A second async generator.
> And you probably wouldn't be able to support all the possible syntactical implementations allowed by the notion of a Python callable
Require an async generator. Done!
> Also, I wholeheartedly congratulate the Django team for pushing the standard out there and driving the Python async web ecosystem forward. You rock!
Much like asyncio, something isn't good just because it's being pushed.
It's still limited. To get a result you need to send something. Using it for something that's not strictly request-response can get complex.
> Having worked with Python async for 3 years, anything that asyncio touches is completely poisoned. ASGI is another in the line of terribleness of the modern async ecosystem.
That's your own bias. I also used asyncio from the beginning and got a different opinion. The asyncio package that comes with python is low level and is meant for building frameworks. If you used aiohttp, asyncpg and other packages packages build on top of asyncio it's actually quite enjoyable.
There's the problem. WSGI is fundamentally flawed too - it could also be using generators for a two-way communication channel instead of stringly typed callbacks.
In a world where Python has optional static type hints, it would be nice to have concrete objects passed too.
* You can absolutely stay within structured concurrency constraints using ASGI.
* The ASGI spec doesn’t have anything to do with asyncio. It’s purely an async/await interface - You can build against it just fine using other concurrency backends.