
Molten: A modern Python (3.6+) web framework - dternyak
https://moltenframework.com/v0.5.2/index.html
======
dternyak
Bogdan has put together an awesome framework with molten that I think can use
a lot more visibility. With his permission, I've posted here to get more eyes
on it. I'm excited to see how much attention it's getting.

For those who want to get started with molten, I've put together a boilerplate
for a more full fledged production application here:
[https://github.com/dternyak/molten-
boilerplate](https://github.com/dternyak/molten-boilerplate)

It demonstrates:

\- OpenAPI (swagger)

\- SQLAlchemy

\- alembic

\- py.test

\- invoke (task management / CLI)

Feedback is super welcome on the boilerplate, which is partially inspired by
[https://github.com/Bogdanp/molten_cookiecutter](https://github.com/Bogdanp/molten_cookiecutter)

Bogdan has been incredibly friendly and helpful while I've been learning the
ins and outs (and pestering him over email). I'm sure he'll be happy to answer
any questions here as they come up.

Happy hacking!

~~~
tankerdude
How does this compare with Connexion?

Uses Swagger/OpenAPI that then uses convention (or fully defined) packages,
and methods for REST. Runs on top of Flask, and does a lot of all of the
validation all from the yaml file. Plumbing taken care for you, etc. so you
can focus on business logic?

~~~
Bogdanp
I'm not familiar with Connexion (this was my first time hearing about it), but
it seems, from reading its README, that the two approaches are polar
opposites: in Connexion you write the schema first and that is used to hook up
your API to your business logic, whereas in molten you write your API using
normal Python code and idioms and an OpenAPI schema is generated from that
code. As a user of Molten, you don't need to know anything about OpenAPI to be
able to use it.

~~~
tankerdude
Molten does what I've seen in RoR land do. You decorate and document your
schema. It then can spit out an OpenAPI spec, use its schema and tooling
around OpenAPI.

So it becomes a choice. A decision of writing mostly OpenAPI, or writing
Python via an API that is defined by someone (or a small set of people).

I'd personally rather learn OpenAPI specs as you're going to live in that
world anyways, as it gives you so much tooling.

It's not hard, and you could take that yaml file and generate code for other
languages in case Python's performance specifically for that API (especially
it if is a microservice), runs too slowly in the real world and need a faster
runtime, etc.

------
zzzeek
So refreshing to see a new Python web framework that isn't trying to hoist
more event-based non-blocking IO on us.

~~~
dfee
I get that SA ORM is unfit to run on the eventloop thread, but I’m
increasingly unclear if you hold your position that asyncio is bad solely
because it makes SA ORM less competitive - and less of a ecosystem standard.

The alternative being just using SA Core and something like aiopg, using
asyncpg, or using SA in a threadpool (unfortunately, a technique that’s
involved enough that the recipe for using SA isn’t a 5 min “gotcha”, re:
flask).

I like SA. I’ve used it for years, and you’ve helped me on the forums, so I’m
appreciative for the work you’ve done. I just don’t understand why you’ve
written off an entire style of programming.

~~~
zzzeek
> but I’m increasingly unclear if you hold your position that asyncio is bad
> solely because it makes SA ORM less competitive - and less of a ecosystem
> standard.

It's not that the ORM is less "competitive" because I am certainly able to
make an async version of SQLAlchemy, it's just I don't have the time /
interest / resources to do so, and as I've gone through the effort to write
about in detail, it is a _bad idea_. If you want to pay me a salary to make an
async SQLAlchemy I can totally do that. It would be a worthwhile effort in
that it would make people happy. But also it would be a wasteful effort in
terms of advancing the field of Python database programming. That's the
decision I'm facing and it would be a lot easier if folks would just realize
asyncio isn't getting them much overall.

It's like, suppose I make bikes. But everyone wants to get to work on pogo
sticks instead. Everyone who uses pogo sticks is terminally late to work,
they're getting injured all the time, they are miserable, but someone told
them "hey pogo sticks are FASTER than bikes!", people were just bored with
bikes so much that they didn't even bother testing this assertion, and now
everyone just has discussions like "oh well I have this shiny new pogo stick,
so I only broke three ribs this week! hooray!" and people have just forgotten
that there is nothing wrong with bikes at all and they are in fact a lot
better than pogo sticks for the task of getting to work every day (you might
find pogo sticks to be more _interesting_ and _fun_ in the moment, but as a
real solution to the problem, they are not any better than bikes and probably
worse). Someone who makes bikes tries to point all of this out. That puts you
in the position of saying, "all your arguments about bikes being better than
pogo sticks for getting to work, who cares. you are just concerned pogo sticks
are a threat to your bike sales". Well, yes, they _are_. But the bigger issue
is that it is ridiculous everyone is killing themselves trying to get to work
on pogo sticks. Somehow I feel that should not be lost.

> I just don’t understand why you’ve written off an entire style of
> programming.

I've written and tweeted about this a lot so feel free to point out the
reasons that might not be clear. To recap
[http://techspot.zzzeek.org/2015/02/15/asynchronous-python-
an...](http://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-
databases/):

asyncio is essentailly intended to provide an interface around the concept of
non-blocking IO. The relational database use case, for the vast majority of
uses (e.g. CRUD), gains nothing from using non-blocking IO (most databases
don't even offer a non-blocking API anyway) and only complicates the
application, introduces subtle bugs, and does not improve performance. The
other argument made about asyncio is that replacing the OS'es task scheduler
with a cooperative one defined by when IO happens to occur is "safer", because
context switches are no longer implicit. The second section of the post above
makes many arguments why this is not the case and especially not for modern
server-side application design which is inherently multi-process.

~~~
dfee
I don’t know, an asyncio ORM version seems very challenging, but if anyone
could do it, it’d be someone with deep domain knowledge (like you).

Relationships as properties for example, I just don’t know how you’d solve
attribute access in an idiomatic way (sure it could return a future, so maybe
the idea is that you’d get a future subclass that still supports operations)?

Anyway, the problem is that it’s really difficult to mix sync/async code. And
as soon as you’ve got a need for websockets, you’re pretty much thrown into
the world of asyncio. To use the ORM is then a threadpool issue.

Anyway, my approach has instead been to use SA Core (with aiopg). I’ve
probably always gotten to involved with my Mappers anyway.

~~~
zzzeek
> Relationships as properties for example, I just don’t know how you’d solve
> attribute access in an idiomatic way (sure it could return a future, so
> maybe the idea is that you’d get a future subclass that still supports
> operations)?

I think the answer would be that you just ditch "lazy loading" altogether, in
the spirit of asyncio's theme that "implicit IO is bad", you'd have to have
explicit conversations with the DB. Loading attributes from relationships is
one thing, but there's also the notion that other kinds of attributes can
refresh themselves if they happen to have been expired, which is something the
ORM does when a server-side rule or a transaction boundary that might mean the
attribute changes on the database side. It's a very elegant and simple
solution to the problem but if you have decided that implicit IO is bad, it
means you don't really want that anymore. You'd just start treating an ORM
mapped object more like an active row and you'd have yields on most
operations, and you'd also want to add more coarse-grained boundaries to when
state is expired and re-loaded (e.g., instead of expiring particular
attributes as needed to be reloaded when accessed, there would need to be a
more formal "refresh the object" kind of step).

That's one way to do it. Another way would be if an implicit non-blocking
approach such as gevent could be cleanly integrated with asyncio - then object
heavy database interactions could simply be inside of a block written in
"blocking IO" style, and that entire block is entered as a yield. There is no
technical reason this can't be done. The reason is cultural; people who like
asyncio really hate implicit IO. So trying to make that work seems like it
wouldn't be very worth it either.

> Anyway, the problem is that it’s really difficult to mix sync/async code.

I think this is because people just aren't working on the problem. You read
asyncio tutorials and they say things like, "if you really MUST use evil
threads...here's this crappy executor thing we think you should never use",
I'm exaggerating a bit but that's kind of the vibe they send off. Making the
asyncio / thread transition could be done _very_ nicely, and there can also be
integrations between gevent style and asyncio style, and finally there are
many other ways to use non-blocking IO without using a full event style
programming interface. The asyncio crowd isn't there yet. They are still
having the endorphin high of asyncio "clicking" for them and they are rigidly
dogmatic and pretty short on real benchmarks that show actual gains. I have
nothing against using non-blocking IO but the next time I have to use it, I'm
going to purposely build a non-blocking IO library that seamelessly makes use
of threads and queues and shows how easy it is if you really need to reach out
to a dozen slow web services at once to get the non-blocking IO advantages
without turning your entire codebase and dependencies inside out.

> Anyway, my approach has instead been to use SA Core (with aiopg)

right, so another thing to keep in mind, Postgresql is the _only_ database
that has a native non-blocking protocol. So building an async ORM is
immediately limited by that - for every other database there would still be a
threadpool running the socket conversation, and having every database message
be a separate threaded unit is less efficient than just having the entire
transactional conversation in one threaded unit (hundreds or thousands of
shifts in and out of the thread pool vs. just one). There's really many
reasons to not write an asyncio ORM right now.

------
JustSomeNobody

        app = App(
        components=[
            DBComponent(),
            TodoManagerComponent(),
        ],
        routes=[
            Include("/todos", [
                Route("/", list_todos),
                Route("/", create_todo, method="POST"),
            ]),
        ],

)

Wonder what this would look like for a _real_ app and not a toy.

~~~
dternyak
Had the same thought, so I built [https://github.com/dternyak/molten-
boilerplate](https://github.com/dternyak/molten-boilerplate)

Gets you pretty close to what a production app looks like.

Feedback welcome!

------
no_wizard
If the author or someone cares to share I’m curious about the decision not to
build this on asyncio

~~~
Bogdanp
Here are my reasons:

* I am not a fan of asyncio's API (`ensure_future`), docs (though I understand these two are improving in 3.8) and of the fact that it is incompatible[1] with much of the existing ecosystem of libraries (eg. the SQLAlchemy ORM or any existing libraries that talk to... anything over the network (postgres, redis, memcached, etc.) which need to be written from the ground up in asyncio style or rewritten so their protocol parsing logic is completely separate from the IO that they do (not a bad thing, mind you, but taking battle-tested libraries and rewriting them from the ground up is not the greatest idea)), tools and even builtins (queue.Queue vs asyncio.Queue).

* I don't buy into the claimed benefits of explicitly marking yield points: any sufficiently complex code will have enough yield points (`async with`, `async for`, `await`) that reasoning about concurrency in the program isn't going to be any easier than if it had implicit yield points.

* I think ergonomics matter for developer happiness and programming with asyncio is not ergonomic in the slightest.

* File IO is blocking.

Put all of those together and you end up with a system where you can't use any
of the popular, battle-tested, libraries out there, where your code is more
verbose for minimal gain and where seemingly benign things (reading a file,
using a `for` instead of an `async for`) may greatly impact the performance of
your program all for a small increase in throughput (assuming it does
increase!).

[1]: [http://journal.stuffwithstuff.com/2015/02/01/what-color-
is-y...](http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-
function/)

~~~
thomasahle
1) This is fair.

2) Explicit yield points certainly save you from most of the classic threading
errors, and having to use special maps and datastructures everywhere.

3) Other than tying into (1) I guess this is a matter of taste.

4) Supposedly this refers to the `open` function? This can be solved using the
`aiofiles` module, but again I suppose this ties into (1).

~~~
zzzeek
> 2) Explicit yield points certainly save you from most of the classic
> threading errors, and having to use special maps and datastructures
> everywhere.

"classic threading errors" don't apply to the users of a web framework where
you are programming logic that shares nothing with other threads.
Additionally, the most free-of-race-conditions monolithic application loses
all of that advantage the moment it is placed in a multi-process environment,
where other copies of it on the same host or other hosts are competing for the
same resources. There is no non-trivial web application today that is deployed
to production in a single process on a single host only. Web applications are
not scaled vertically.

~~~
mixmastamyk
> Web applications are not scaled vertically.

Should be rare, but believe stackoverflow was doing that for a while.

------
gravypod
This looks very cool. Is there a reason the HTTP response codes and many other
things are exposed as magic string constants?

------
aportnoy
How does this compare with Flask?

~~~
Kpourdeilami
It seems to come with ORM and request validation, metrics, etc. built-in.
Flask is fast for getting started but you end up rebuilding most of the things
this framework does on your own as your application grows.

It seems like a good middle ground between Flask (minimal and no batteries
included) and Django (bloated but very powerful).

~~~
scrollaway
I don't see the ORM or metrics in the docs.

After a cursory look at it, this seems like an interesting, modern replacement
for Flask. Types-first frameworks are interesting, as they're still very rare
right now.

~~~
tyingq
_" The molten.contrib package contains various functionality commonly required
by APIs in the real world such as configuration files, prometheus metrics,
request ids, sessions, SQLAlchemy, templating, websockets and more."_

~~~
scrollaway
Ah! Missed that, thank you :)

------
perturbation
Kind of reminds me of
[https://github.com/timothycrosley/hug](https://github.com/timothycrosley/hug)
, especially using type hints for validation of request parameters.

~~~
jrs95
Also similar to Tom Christie's APIStar
[https://github.com/encode/apistar](https://github.com/encode/apistar)

~~~
Bogdanp
APIStar was a major inspiration[1]! One of the reasons I built Molten was
because Tom Christie took APIStar in a different[2] direction.

[1]:
[https://moltenframework.com/motivation.html](https://moltenframework.com/motivation.html)

[2]: [https://docs.apistar.com/#where-did-the-server-
go](https://docs.apistar.com/#where-did-the-server-go)

------
drcongo
I recently started my first Falcon project, but this looks closer to what I
was actually hoping Falcon would be. Definitely going to give this a spin,
thanks.

~~~
drcongo
OK, I've been playing with this for an hour or so now and it's magnificent.
Exactly what I needed, thank you.

------
voycey
Bitdefender sees this site as dangerous? Admittedly it is quite overzealous
but something you should look into!

------
Waterluvian
In Python 2, omitting (object) inheritance in a class definition had an actual
effect. Is something different in Python 3 that makes this okay, or are we
doing the less okay thing because it's cleaner?

~~~
joshuamorton
Yes, it has no effect in python3. There are no old style classes.

~~~
Waterluvian
Oh that's just wonderful.

Thank you.

~~~
meowface
Python 3 cleaned up a lot of other language warts like that one (though some
may disagree as to whether or not they were warts). No more
`super(SuperLongClassNameHere, self).method()`, no more having to teach
beginners to always use raw_input() instead of input(), `0 > None` now raises
an exception rather than evaluating to `True`.

I personally wish they would just make `self` implicit, but I understand that
that adds a lot of magic in a language that currently doesn't have a lot.

~~~
another-cuppa
> I personally wish they would just make `self` implicit

Hell no. Go use Java if you want that. If you find typing self is making you
_more_ work then you're not using classes correctly. Write functions and use
classes if you need them.

~~~
meowface
edit: To be clear, I mean making `self` implicit in the method signature -
_not_ implicit in the method body like the way Java does it.

It's a very minor and aesthetic/cosmetic thing. No, I don't write classes for
everything. I write functions for things that should be functions and classes
for things that should be classes. But when I do write classes, it's annoying
to remember to write `self` in the method signature, annoying to get
exceptions because I forgot to add it, annoying to see it since it's just that
tiny bit of additional redundant line noise when I just want to quickly glance
at a method signature. And literally no other OO language does this that I'm
aware of; even ones that are generally way less dynamic than Python.

But as I said, I do understand how it came about and why it is the way it is.
But on the other hand, it was the exact same story for `super()`, and they did
resolve that with "magic". The "magic" floodgates have also opened up a bit
more with string interpolation added [1]. So now I think a more reasonable
case can be made for making `self` optional. If they could do it for
`super()`, they can do it for `self`. It doesn't annoy me that much, though,
and I still plan to use Python for a long time even if it's never changed.

[1]
[https://www.python.org/dev/peps/pep-0498/](https://www.python.org/dev/peps/pep-0498/)

~~~
Waterluvian
Do you mean remove self from the method signature or altogether?

I think the latter is fundamentally impossible. There would be name collisions
and it would be horribly ambiguous. I assume you're suggesting the former.

I can see it being plausible. I'm not sure I have a really clear case against
it. I do really like that things are explicit. There's no magical place "self"
emerges from. I hate this about some other languages, like JavaScript's
'attributes' collection that magically exists in a function context.

Being able to juggle binded and unbinded functions is nice. The explicit self
or cls kind of forces you to declare what the intent of your function is.

Not super clear. But the ship has sailed on that issue. Not sure it could be
changed in a non breaking way.

~~~
meowface
Ah sorry, I meant from the method signature. I wouldn't like the totally
implicit `self` / `this` found in Java.

>Being able to juggle binded and unbinded functions is nice. The explicit self
or cls kind of forces you to declare what the intent of your function is.

Sure, but we can already do that with the @classmethod decorator. They could
just make it so `cls` is a variable that can be accessed from any method. If
it's a class method, `cls = self`, otherwise, `cls = self.__class__`.

------
jihadjihad
This looks pretty clean. I would like to see benchmarks that compare between
Flask and especially Falcon, which IMO is the gold standard for fast, minimal
API frameworks.

~~~
Bogdanp
There are some benchmarks[1] in the molten repo as well as instructions for
running them yourself. On my machine it ends up being faster than Flask and
nearly as fast as Falcon.

[1]:
[https://github.com/Bogdanp/molten/tree/master/benchmarks](https://github.com/Bogdanp/molten/tree/master/benchmarks)

------
hguhghuff
What’s the benefit / key differences compared to falcon?

[https://falconframework.org](https://falconframework.org)

~~~
oblio
Well, the front page illustrates them quite well, I think.

~~~
hguhghuff
Errr, I’m looking for some words to answer the question, not to try to infer
at an expert level by reading a large amount of detailed technical
information, and I didn’t see a comparison with falcon, so no, it does not.

~~~
Bogdanp
I've used Falcon extensively for about 3 years while at LeadPages.

I would say the biggest difference between the two is Molten gives you more
out of the box: input validation, doc/schema generation, ORM support, support
for multipart/form-data requests (Falcon doesn't actually handle these out of
the box!), etc.

~~~
hguhghuff
So molten is batteries included essentially, whereas falcon is not.

Like flask versus Django.

Straight question, wouldn’t it have been easier to fork falcon and add the
batteries?

~~~
Bogdanp
Probably not, because many of the differences are fundamental:

* Falcon revolves around the concept of Resources, i.e. classes as the primary handlers of requests; molten revolves around functions,

* Falcon supports Python 2.x,

* automatic dependency injection via type annotations is core to molten.

------
krob
Does python 3.6 support type hinting? This reminds me of php 7+ with scalor
types, and return types now. Didn't think python supported this.

~~~
jakereps
Yes, Python introduced static type hinting in version 3.5

[https://www.python.org/dev/peps/pep-0484/](https://www.python.org/dev/peps/pep-0484/)

~~~
ubernostrum
Python introduced the 'typing' module in the standard library in 3.5, and
declared that henceforth the annotation syntax was for type hints.

Python introduced the annotation syntax (for everything except
variables/attributes) in 3.0. The annotation syntax for variables/attributes
was introduced in 3.6.

------
wodenokoto
Is this competing with flask or with Django?

~~~
Chirag
I would say Flask.

------
andybak
Post title needs to have "API" in it.

