
Monads in Python (with nice syntax) - llambda
http://www.valuedlessons.com/2008/01/monads-in-python-with-nice-syntax.html#6799472260664409462
======
andymatuschak
This approach to the "do" syntax requires that the function implementer know
the monad type a priori.

That defeats some of the purpose of monad as general abstraction. One of the
big ideas here, as far as I'm concerned, is that monads represent an
abstraction over a common _structure_ of programs. Lists, Maybes, threading
models, etc, all share this same language of structuring computations. This is
significant because it means that one can build functions which manipulate
those structures, and then those structures can be deployed generically.
Depending on how you're counting, that means third-order abstraction.

For instance, consider liftM from Control.Monad:

    
    
      -- | Promote a function to a monad.
      liftM   :: (Monad m) => (a1 -> r) -> m a1 -> m r
      liftM f m1              = do { x1 <- m1; return (f x1) }
    

This represents a common idea across e.g. both List and Maybe. I've got a
function of values. I want to turn it into a function of Lists / Maybes. Well,
liftM recognizes the similarity of List's and Maybe's structure and provides a
single implementation suitable for both places.

The decorator described in this article cannot function at liftM's level of
abstraction: at least as far as I can tell (as a non-Python programmer), it
requires a monad type parameter. If that's true, I think it's missing a key
idea behind the power of monads.

~~~
ufo
Not really. Python doesnt have typeclasses so liftM would be a fucntion
receiving the monad dictionary as an explicit parameter. At that point, you
can pass that dictionary to the decorator:

    
    
        def liftM(m, f, x):
            @do(m)
            def doNotationBody():
               ...
            ...

------
almost
Can anyone explain to me how the continuation Monad is working there? As far
as I know there's no way to return from the same yield twice, which is what a
callcc would need to do. I'm going to carry on trying to puzzle it out but a
pointer would be helpful!

EDIT: I just tried out the code and it doesn't support multiple returns. Isn't
that pretty much the thing that defines the Continuation Monad or am I just
not getting it?

~~~
qznc
Technically, it is still a continuation, even if it does not match the
behavior of Haskell's continuation monad.

<http://c2.com/cgi/wiki?SingleUseContinuation>

~~~
almost
I didn't know that, thanks. Seems less useful though.

What's the correct term for a continuation as I understand them? (eg: as they
are in Haskell, Ruby, Scheme etc)

~~~
ufo
Continuations that are not single use are usually called _multi-shot_
continuations.

Single shot continuations are very useful though. They are enough for a very
large percentage of the situations you would use continuations for while being
much more efficient to implement (no need to save the stack)while also being
more reasonable theretically in other situations(call/cc can be used to break
some important invariants when it interacts with mutablestate and other things
like that. Its kind of subtle)

------
freework
This is missing the one thing that all other "let me explain monads to you"
articles is missing: a real example of what problems monads solve. As far as I
know, a monad is some kind of mathematical construct that is very useful when
solving mathematical proofs, but has no use for the every day programmer. The
examples in these kinds of articles are always completely abstract. You can
very easily describe a database by using the "customer/phone number" example,
yet I've never seen such an example with monads.

~~~
dons
<http://www.haskell.org/haskellwiki/Monad#Interesting_monads>

Nothing to do with proofs.

Monads are a generic interface -- an API -- for changing up the computational
rules of your programming language.

Just think about that for a second -- its a big deal.

Maybe you want to work with non-deterministic threads, or with continuations,
or quantum computations, or backtracking search, or exceptions or partial
results, or some other notion of computation. But you want to still write in
your usual language.

So you embed that computational style as a monad, and now you have a really
nice API for programming in that different environment -- inside your everyday
language.

Its simple, but profound. With (good support for) monads you aren't
constrained to the default computational rules of your programming language
anymore.

You are free.

~~~
lhnz
The problem with this description is that I don't know what kind of benefits I
can get from using different computational rules... Can you show me?

~~~
frio
dons' explanation will be better/more rigorous than mine, but in a more
familiar syntax (Python), let's say you've got some functions:

    
    
        def get_user(id):
            # returns None if no such user exists
            return Database.Users.get(id)
    
        def send_message(user):
            user.send(message)
    
        def main():
            user = get_user(1)
            # uhoh -- this could be None and we didn't handle it!
            send_message(user)
    

So, we add a None check:

    
    
        def main():
            user = get_user(1)
            if user != None:
                send_message(user)
    

But... what if we want to log the status of that send_message call?

    
    
        def send_message(user):
            try:
                user.send(message)
                return None
            except Exception, e:
                return "Failed to send message; {0}".format(e)
    

(Note that this is a contrived example, please don't critique the general
dumbness ;)).

Now...

    
    
        def main()
            user = get_user(1)
            if user != None:
                response = send_message(user)
                if response != None:
                    log(response)
    

As you can see, it's getting a bit hairy. But, what if we had a class that
dealt with the Nones for us? This is what a monad is: an interface, that many
different instances implement, which provides some behavior. The following
only deals with single-argument functions, but:

    
    
        def NoneMonad(Monad):
            def __init__(self, arg):
                self.arg = arg
    
            def bind(self, func):
                if arg != None:
                    next_arg = func(arg)
                    return NoneMonad(next_arg)
                else:
                    return NoneMonad(None)
    

Now, we...

    
    
        def main(self):
            NoneMonad(1).bind(get_user)
                        .bind(send_message)
                        .bind(log)
    

And, boom, the Nones are handled. Real, non-contrived Monads do more (and
implement other behaviours -- like in dons' example, the Either response, or
the Maybe monad, etc.), but hopefully this demonstrates the strength.
Haskell's typically used as the example language, because there's syntax sugar
in Haskell for making dealing with Monads prettier :).

Having finally grokked them, it surprised me how simple the concept is. I
think for people (like me!) coming from Java, Python, etc. who've not had to
deal with the concept, there's this sort of imbued myth that they're a much
more difficult concept than they really are. It's as simple as: a Monad is an
interface, that can be implemented, that provides a construct for executing
code in.

Please forgive any blinding mistakes I've made in my code.

~~~
StavrosK
Ugh, this made more sense than anything else I've ever read about monads.
This, together with the Wikipedia sentence that monads are "programmable
semicolons", helped me finally understand what they actually are.

Like the other commenters said, I don't understand why someone would use
Haskell syntax to explain monads. If you know Haskell, you already know what
monads are!

Thanks for this great example.

~~~
tel
I think it's always worth studying Free monads to understand them a bit more
completely. After you can see how to construct and map free monads then
there's not much left to the concept to debate.

------
dustingetz
i've worked through this article a couple times and his implementation never
really sat well with me, the "with nice syntax for monad comprehensions" bit
really obfuscates the implementation.

here's python source to a monadic lisp interpreter, which i wrote to follow
along the paper "monad transformers and modular interpreters" [1]. i think
this is a much simpler implementation of python monads than provided in the
ValuedLessons article. <https://github.com/dustingetz/monadic-interpreter>

Study of this implementation will teach you why nobody actually uses monads in
python for non-toy projects. A literal port of this code to clojure would feel
so much more idiomatic and not hacky at all.

here are some smaller code dumps demonstrating the fundamental concepts to
that monadic lisp interpreter:

[http://www.dustingetz.com/2012/10/02/reader-writer-state-
mon...](http://www.dustingetz.com/2012/10/02/reader-writer-state-monad-in-
python.html) [http://www.dustingetz.com/2012/10/07/monads-in-python-
identi...](http://www.dustingetz.com/2012/10/07/monads-in-python-identity-
maybe-error.html)

[1] [http://haskell.cs.yale.edu/wp-
content/uploads/2011/02/POPL96...](http://haskell.cs.yale.edu/wp-
content/uploads/2011/02/POPL96-Modular-interpreters.pdf)

~~~
dbaupp
I think the point of this article is to make a "nice" syntax for monadic
operations in python (i.e. avoiding all the lambdas and extra parens in the
"bind(mx, lambda x: my)"/"mx >> (lambda x: my)" pattern), no matter how
unclear and non-pedagogical the implementation ends up.

------
hxseven
BTW: This was already submitted to HN.

[http://www.hnsearch.com/search#request/all&q=Monads+in+P...](http://www.hnsearch.com/search#request/all&q=Monads+in+Python&start=0)

But that doesn't make it less interesting ;)

------
klibertp
I went through this article some time ago and, while it's certainly neat
implementation and worth understanding (that was the first recursive generator
that made sense to me - thanks to this I had much easier time understanding
Scheme's call/cc and it's uses), it needs to be noted that it is not a generic
monad implementation. You can not, for example, implement list monad (I tried
very hard and couldn't, I saw the same said in comments below the article, but
if it's possible please let me know!) using this implementation and syntax,
because you can not "rewind" Python's generators.

So, while it's nice implementation for some kinds of monads it's not nearly
general enough to be (IMHO) called "monads in Python" - it is possible to
implement fully general monads in Python, but you need nested lambdas and
there is no getting away from it.

Still, very nice hack and worth spending an evening (or two) to understand how
it works.

------
csense
The first line of his first example of using his monad implementation is:

@do(Maybe)

Ctrl+F Maybe. It's not defined anywhere!

I was excited to see that someone could maybe finally explain to me, _in
Python_ , why people are so excited about Monads.

I looked at Haskell once. It looks impossible to understand without
understanding Monads. (And near-impossible to understand even if you do
understand Monads -- the syntax is a nightmare.) I thought maybe this article
would help me, by using the syntax of language I do understand -- Python --
with a concept I don't understand -- Monads.

But this excitement was misplaced, since this isn't a usable demo. Please, if
you're the author of this article or someone who understands this code, post a
Github repo, or a tarball on your HTTP server, or _something_ with a
_runnable, simple example_ of code that uses the article's approach to monads.

------
temac
This is very interesting, very intelligent. Indeed this is too much
intelligent. There is a huge amount of highly valuable programmers that won't
be able to understand all that mess, and the gain is questionable at least
given that there are alternate constructs that are way better compromise
between expressiveness and maintainability.

Use the right tool in its intended way: don't try to retrofit constructs in
languages were they don't fit, just for an accessory characteristic. (Every
line of an imperative language is conceptually kind of a monad, so the article
here is just trying to convince us that yet another form of hidden goto is
good, by an ad-hoc extension in a language were more than half of monad
necessity is missing)

~~~
beering
It might require an intelligent programmer to create, but as a library, can be
used by regular, unintelligent programmers. If he documents his monads well
and shows how to use it, and it's effective, then it's not such a big deal.

Or, you know, this is more of an attempt to show programmers stuck in simpler
languages what other PL features have to offer.

------
crucialfelix
The problem with Monads is that the people who understand them are the least
qualified to explain them to other people.

~~~
StavrosK
The corollary would mean that the people who can explain well can't understand
monads, which means that some people just can't understand them. We're doomed!

~~~
ibotty
not that i agree with the top comment, but your reasoning only works for a
static view of people (i.e. they cannot learn something and at the same time
forget something).

