Looking through some of the code for Sourcehut, there’s an insane amount of boilerplate or otherwise redundant code. The shared code library is a mini-framework, with custom email and validation components. In the ‘main’ project we can see the views that power mailing lists and projects.
I’m totally biased, but I can’t help but think “why Flask, and why not Django” after seeing all of this. Most of the repeated view boilerplate would have gone ( could be like 20 lines), the author could have used Django rest framework to get a quality API with not much work (rather than building it yourself) and the pluggable apps at the core of Django seem a perfect fit.
I see this all the time with flasks projects. They start off small and light, and as long as they stay that way then Flask is a great choice. But they often don’t, and as the grow in complexity you end up re-inventing a framework like Django but worse whilst getting fatigued by “Python” being bad.
My current favorite way of building APIs is this Frankenstein's monster of Django/FastAPI, which actually works quite well so far:
FastAPI is a much better way of writing APIs than DRF, I wish it were a Django library, but hopefully compatibility will improve as Django adds async support.
I still want to rip out SQLAlchemy ORM and replace it with pure SQL via `asyncpg`, as SQLAlchemy ORM is not async and that causes a bunch of extra switching in the backend that certainly doesn't help eek out more perf, but at the moment it's a bit too much effort and users are happy.
Scaling is handled by just throwing more instances of the application at the problem, behind a load-balancer.
So yeah, still seeing good speedups in our own benchmarks even though most of our endpoints are sync.
What was arguably more important though was how much switching to ASGI helped with handling WebSockets. We're using SocketIO, and trying to get a fundamentally async protocol working within sync (Flask) land was a massive pain. We had repeated reliability and deployment issues that were very hard to debug. Switching to FastAPI made that much easier.
For WebSockets, all of the code is async, so I'm already using `asyncpg` for any database stuff that is happening there.
With regards to why are the sync endpoints faster, I think it is a number of things, some of which are userland changes that could've been made under Flask, but all of which are somewhat related to the switch. With regards to things that FastAPI itself has changed, I think using a (de)serialization lib like Pydantic and serializing to JSON by default (which is what we were doing under Flask anyway, though with Marshmallow) makes a lot of the code paths in the underlying lib a bit faster, because with Flask there was more "magic" going on behind the scenes. For userland stuff, I think partly because there is less magic going in the background (I really like FastAPIs dependency injection system), it's made it easier to identify the bottlenecks and optimize hot code paths.
Can you show me a comparable codebase in django and how it looks? I'm genuinely curious how people deal with edge cases.
The first was when they removed tracebacks. Singularly useless thing to do IMO. But there's a --show-tracebacks option (or something like that, it was a long time ago) to show tracebacks, but it didn't work. I dug into the code for this one. IIRC, the guy who added the code to suppress tracebacks didn't take into account the CLI option. I patched it to not suppress tracebacks but there turned out to be another place where tracebacks were suppressed, and I eventually gave up.
The second incident (although, thinking about it they happened in chonologically reversed order) was when a junior dev came to me with a totally wacky traceback that he couldn't understand.
All he was trying to do was subclass the HTML Form widget, like a good OOP programmer, but it turned out that Django cowboys had used metaclasses to implement HTML Forms, and utterly defeated this poor kid.
I was so mad: Who uses metaclasses to make HTML forms? Overkill much?
(In the event the solution was simple: make a factory function to create the widget then patch it to have the desired behaviour and return it. But you shouldn't have to do that: OOP works as advertised, why fuck with a good thing?)
So, yeah, Django seems to me to be run by cowboys. I can't take it seriously.
FWIW, I'm learning Erlang/OTP and I feel foolish for thinking Python was good for web apps, etc. Don't get me wrong, I love Python (2) but it's not the right solution for every problem.
Here’s the ~20 lines of cowboy code you’re referring to - collecting the declared fields and setting an attribute containing them.
Not exactly the kind of thing that should make you mad, and rather than overkill it’s exactly the use case for metaclasses.
And to top it off, that metaclass is completely optional, if you want to create a list of fields and pass it into BaseForm then go for it. Most don’t.
> Django never “removed tracebacks”
I don't want to get into a juvenile back and forth, but I must insist that Django did so suppress tracebacks. I don't know what it does today but I remember clearly patching the server code to re-enable them and that the '--traceback' CLI switch didn't do it.
> I would love to see a declarative form that didn’t use metaclasses in some way.
Here you go: https://pypi.org/project/html/
Clever design, elegant code, under 20k (including docs and tests), no metaclasses.
> Not exactly the kind of thing that should make you mad, and rather than overkill it’s exactly the use case for metaclasses.
There is no use case for metaclasses. GvR called that paper "The Killing Joke" for a reason. ( https://www.python.org/doc/essays/metaclasses/ https://www.youtube.com/watch?v=FBWr1KtnRcI ) I read it for kicks, because I'm that kind of freak, but it's not the kind of thing that you should ever use in production code.
What made me mad is that Django's gratuitous use of metaclasses broke my junior dev. The kid was doing the right thing and it was exploding in his face with an inscrutable error message: that's on Django.
That’s not even the same thing - it’s a (horribly old) library for generating HTML.
We’re talking about forms: sets of typed fields including validations that can be optionally rendered to HTML. Think wtforms
> There is no use case for metaclasses.
There are, that essay (about Python 1.5 no less) does little to dissuade people from using them, going so far as to offer concrete code samples.
It’s also hopelessly outdated: nobody uses metaclasses like that at all, especially not for tracing! It’s hard to blame it though, this document was written before even decorators where introduced.
And let’s not ignore the call to authority by pointing out that GvR uses metaclasses extensively while working on type hints, GaE libraries, and even in his early asyncio code.
In actual fact metaclasses a few use cases, including the most common: syntactic sugar. Like anything this can be heavily abused and is most useful when creating libraries rather than used within traditional application code. In any case, shunning it wholesale is stupid.
Half remembered issues with junior developers are not great arguments against a useful part of a language. Who’s to say that it was even related to metaclasses, and your apparent allergy to them isn’t colouring your memory?
You're not going to convince me that Django isn't an overblown toy. I'm not going to convince you that it is.
Same with metaclasses, you're not going to convince me that they're a good idea (despite what GvR does with them) and I'm not going to convince you that using them is irresponsible.
So what are we left with?
> That’s not even the same thing
But html.py (would have) solved the problem we had. without the nasty surprise.
> In any case, shunning it wholesale is stupid.
No, it's conservative. That's a different thing.
I want to be able to hire someone who can modify a form. The more complex and obscure the the code is (even if it's only twenty lines long) the smaller the pool of folks who can use it with mastery.
Think about it.
Anyway, I'm off learning Erlang/OTP now and it really makes Python's runtime look like a joke in comparison. Web-app backends are Erlang-shaped, not Python-shaped. Not using it sooner makes me feel stupid.
Then they build their own abstractions. And then, congratulations, they’ve spent longer than they should have to end up with a worse version of Django, that nobody but them finds “clear” or “unambiguous”.
If only we could capture these common, repetitive and important patterns and put them in some kind of library. A “framework”, if you will. That way you don’t need to copy-paste this stuff over and over again, and anyone who knows the library will find it clear and unambiguous!
In fact this is such a good idea that I’m going to do it myself. I’ll call the library Franz, after a famous pianist.