Can someone please explain in a simple way what Monads are and why I want to use them in normal daily programming (assume, incorrectly, that I only know Python)?
I know I can google this, and have, and unfortunately I came away even more confused as a non-Haskell practitioner.
My advice is not to worry about it (unless the itch bugs you so much that you're willing to invest a lot of time researching the answer).
I don't think you'll ever get a single, straight answer that satisfies you. Just reading this discussion, I'm sure you can see that people describe monads in wildly different ways. Some descriptions are minimalist (perhaps a bit reductionist): it's just an algebraic data structure, nothing to get excited about. Many rely heavily on metaphors: containers, "programmable syntax", and yes, even burritos have been used to explain them. Some conflate monads with properties that do show up often in monadic code, but aren't themselves inherently monadic (immutable data, pure functions, sequencing of evaluation). Some are very language-limited, in the sense that the description invariably turns into a debate over the semantic fine-points of the language being used (this will almost always happen when Haskell gets discussed, because expression evaluation is a very subtle thing in Haskell). Frankly it's a mess. The only safe ground you have is the algebraic definition, but that's not terribly useful for building an intuition about how one might use them.
I think that you (and me, and most everyday programmers) would use them in your code only if you were using some library that was designed in a monadic style. In Haskell, so many libraries use monads these days that it's a given: using Haskell for any practical purpose means using monads, whether you want to or not. In Ocaml (about as close to Haskell as you can get without being purely functional), they show up mainly in third-party libraries, mostly when you want to do something tricky like asynchronous I/O -- again, you use monads if you want to use the library. But in run-of-the-mill imperative languages, though, monads don't tend to show up very often, and when they do, they usually are very specific to one library or another.
I like this answer, but it leaves one important thing out. Mathematicians have been studying abstraction for thousands of years. When they decide something is useful, that probably means it has broad applicability and enough functionality to get interesting results out of.
Mathematicians decided monads are useful.
Take advantage of that. Even if you're working in a language that can't express the abstraction, you can allow it to inform your library design. If something could be a monad, that's usually a better choice than something ad-hoc instead. (Sometimes it's overkill and a monoid would suffice, but both are better choices than an ad-hoc interface without any broadly accepted theoretical basis.)
I'm glad you brought up monoids here as a (sometimes) alternative -- in addition to their inherent properties, people have a much easier time wrapping their heads around monoids.
And yes, while abstraction is generally good, principled abstraction is generally better. At the same time, a lousy monadic library is still lousy. The sets of "great libraries" and "algebraically informed libraries" intersect, but neither subsets the other!
The definition of the thing we call “monads” is a poor place to start understanding what they’re for. The only way to understand monads is to use a number of monads, and you’ll start to see why they’re useful.
One great monad is the Option monad. It lets you represent, at the type level, the two casss of having something or not having. Think about when you use a python query operation that may return nothing. You don’t know if you’ll get back an empty list, or a null, or what. If the API returned an option, this would indicate what happens when you get nothing back. That’s cool. You can then do computations against maybe having a value. Option.map(lambda) will apply that lambda function to the value only if it exists, saving your program from blowing up on a null. Even better, tho, you can chain a bunch of operations that may fail together. In the map example, ifyour lambda also returns an option, you would end up with the type Option[Option[A]]. Monads implement a flatten method that will turn an Option[Option[A]] into an Option[A]. They abbreviate this with “flatMap”, which can be implemented as a call to map, followed by a call to flatten.
Futures are another great monad that help model asynchronous operations. Val f = Future(mydatabasequery) will return a Future[A], read as “future of type A”. You can perform computations on the result using the map function: f.map(lambda). You don’t know when this will happen, you just know it will. If your lambda also returns a Future, you will end up with Future[Future[A]]. You convert this into a Future[A] with the Future.flatten function. The shorthand for this is.flatMap(lambda).
Another great monad is the Either[A, B] monad. It’s like the Option monad, but rather than having None or Something, you have Left or Right. If you have an operation that may throw an exception, you can wrap it in an Either. A left value will be some error, and the right value will be success. You can conditionally perform operations on the success using the Map function: Either.map(lambda). If the Either doesn’t have a Right value, it will not execute your lambda, saving you from performing an incoherent operation. In python sometimes you might return either a string or an int, representing failure or success. Either[String, Int] is just formalizing that pattern, and letting your compiler prove that you’ve handled both cases. You can model sequential computations that may fail using either: if your lambda returns an either, you’ll get an either[string, either[string, int]]. You can flatten these using the Either.flatten method. The shorthand for this would be flatMap.
In computer science, a monad is a notation representing an intermediate value and its computational context. E.g., data necessary to continue the next steps of a calculation, or some kind of marker of the value's source (IO monad.)
It's an abstraction which rarely carries its own weight, but it's necessary to understand Haskell for what seem to amount to cultural reasons. Other strongly typed functional languages don't place nearly so much emphasis on it.
While monoids are explicitly identified in some functional languages, they don't have to be. Monoids just exist whenever the axioms are satisfied.
A monoid is a set, and an operation for combining elements from the set, that satisfy a few properties.
Closed: For every combination of elements, the result of combination is in the set.
Identity: There is an element of the set that when combined with any other element, results in that same other element.
Associative: For any elements a,b,c a+(b+c) = (a+b)+c
Examples are helpful here. I'll list a few as (set,operation,identity)
(Integers,+,0)
(Integers,*,1)
(Strings,&,"")
(Lists,&,[])
(functions t->t, function composition, identity function)
(Bool,and,True)
(Bool,or,False)
(Bags of candy and the empty bag, dumping bag a into bag b, the empty bag)
For me, I really like the “programmable semi-colon” metaphor, where essentially a monad glues together computations and has additional context/actions carried along between the glued computations.
If you’re looking for external resources, it really helped my understanding by reading all the definitions you’ll find online, looking at every example (including the source code on hackage!), and building some play examples myself. It also helped me to first understand functors, applicative functors, then monoids as I was going through other resources.
The problem with the “programmable semi-colon” metaphor is that you don't need a monad just to "glue together computations" (sequencing). That only requires an applicative functor. What sets a monad apart is that you can use the result of the first action to determine the second action.
Functor - If you have a value in a context (f a), you can apply a function (a -> b) to that value to obtain a new value in the same context (f b).
"Pointed" Functor - A Functor with a special unit value (a "point"). In Haskell this is part of the Applicative typeclass and the unit value is written as "pure ()" or (deprecated) "return ()". If starting from the unit value instead, "pure x = fmap (const x) unit".
Applicative Functor - If you have a function in a context (f (a -> b)), and a value in a context (f a), you can combine them to produce a new value in a new context (f b). Combining the contexts is where sequencing comes in, though there are other interpretations besides sequencing effects.
Monad - This can be expressed two ways:
1. The "join" version: If you have a value in a nested context (m (m b)), you can "flatten" the contexts to produce the same value in a new context (m b).
2. The "bind" version: If you have a value in a context (m a) and a function from that type of value to another value in a context (a -> m b), you can combine them to produce a value in a new context (m b).
These perspectives may appear rather different, but "bind" can be recreated from "join" by first mapping the function (a -> m b) over the value (m a) to obtain (m (m b)) and then collapsing the nested contexts to (m b) with "join". Similarly, "join" can be seen as a special case of "bind" with the value (m (m b)) and the identity function (m b -> m b).
Anyway, I think the article is looking at this backward. The power of (pure) functional programming lies not in providing the most powerful abstractions all the time, but rather in deliberately using the least powerful abstractions that will get the job done. Knowing when you need—or more to the point, don't need—a Monad is a key part of making your program easier to reason about. A traditional imperative language offers the power of IO and Monad everywhere, transparently, whether you need it or not, which makes reasoning about the program very difficult—any part of the program could potentially depend on and/or affect anything in the program's runtime environment. Perhaps a certain function only needs the ability to sequence effects which don't depend on their results (Applicative). Or perhaps it only needs the ability to transform a value (Functor). Or perhaps it doesn't need a context at all (pure code). The less power you demand from the abstraction, the more you can statically prove about the code's runtime behavior, which is good not only for correctness but also for reusability.
Tom Stuart did a seriously great talk about this, which really helped my understanding of monads. You can read it as a blog post, but I recommend watching the video. It's worth finding the time for. https://codon.com/refactoring-ruby-with-monads
Monad is an interface which a lot of useful "context" types conform to - I find that if a function returns any kind of "secondary value" or "effect" (think the sort of things aspect-oriented programming is supposed to be for - permissions, logging, database transaction boundaries) this can usually be represented by a type which will form a monad. http://blog.sigfpe.com/2007/04/homeland-security-threat-leve... is a simple example if you don't mind the Haskell syntax.
The concept is useful because it means that we can implement generic functions that work for any monad - http://m50d.github.io/2013/01/16/generic-contexts was one practical example I met at my job (if you can read Scala - I feel it's quite Pythonlike but YMMV). Better still, these functions already exist for you - things like the "traverse" function I described there, or cataM (write a function that operates on one level of a tree and returns a type in a monadic context, get the corresponding function that operates on the whole tree), are just a library import away.
Why do you want to use them? They make it practical to replace "magic" with plain old values - somewhat cumbersome values, but values that are usable enough that the advantages of working with plain old code start to show. If you're using Python, anything you're using decorators for is a very good candidate; I already mentioned AOP. A really simple example would be error handling: in a lot of languages you have the choice between https://blog.golang.org/errors-are-values (explicit, clear, but too cumbersome to use in practice; means your functions are hard to read because all the error handling gets in the way of reading the business logic) and exceptions (concise, but magic and invisible; code with exceptions can end up with really surprising control flow). Monads give you a middle way: https://fsharpforfunandprofit.com/rop/ , where your error handling is visible enough to not be a surprise when it happens, but hidden away enough that you can read the "happy path" code without getting too distracted.
// A Box that holds stuff
public class BoxOfStuff<TStuff>
{
public TStuff Value { get; private set; }
public BoxOfStuff(TStuff value)
{
Value = value;
}
}
// Class that holds methods for working with Boxes of stuff
public static class Ext
{
// Takes something and puts it in a box
public static BoxOfStuff<TStuff> PutInABox<TStuff>(this TStuff thingToPutInTheBox)
{
return new BoxOfStuff<TStuff>(thingToPutInTheBox);
}
// Converts a Box containing one thing into a Box containing something else
public static BoxOfStuff<TOtherStuff> ConvertBox<TStuff, TOtherStuff>(this BoxOfStuff<TStuff> theBoxItself,
Func<TStuff, BoxOfStuff<TOtherStuff>> functionThatConvertsTheBox)
{
return functionThatConvertsTheBox(theBoxItself.Value);
}
}
// Class that holds a string value
public class StringHolder
{
public string Value;
public StringHolder(string value)
{
Value = value;
}
public override string ToString()
{
return Value;
}
}
var result = new StringHolder("A random string").PutInABox()
.ConvertBox(randomString => new Random((int)DateTime.Now.Ticks).Next().PutInABox()
.ConvertBox(randomNumber => (Guid.NewGuid()).PutInABox()
.ConvertBox(guid => ($"{randomString}, {randomNumber}, {guid}").PutInABox())));
Console.WriteLine(result);
// Compare the above result to this:
public static class LinqExt
{
public static IEnumerable<T> ToEnumerable<T>(this T t)
{
return new List<T> { t };
}
}
var linqResult = new StringHolder("A random string").ToEnumerable()
.SelectMany(randomString => new Random((int)DateTime.Now.Ticks).Next().ToEnumerable()
.SelectMany(randomNumber => Guid.NewGuid().ToEnumerable()
.SelectMany(guid => ($"{randomString}, {randomNumber}, {guid}").ToEnumerable())));
Console.WriteLine(linqResult);
BoxOfStuff + PutInABox + ConvertBox = Monad.
This is analogous to:
IEnumerable + ToEnumerable (the Identity function) + SelectMany (the Bind function). To be honest, this is a fairly loose, non-rigorous definition even by programming standards, but if you've used LINQ before it should give you an idea of how useful Monads can be and how they can be used.
LINQ works the way it does because of Monads. In fact, if I were to add a Map function (Select) to BoxOfStuff and rename ConvertBox to SelectMany, I could use query syntax eg.
var result = from thing in myBoxOfStuff
select thing.ToString();
Basically the traditional way to do I/O is to just call I/O functions (like printing to the terminal and reading keyboard input) in the middle of the program.
There is another way, which consists of having the program only consist of a pure function that returns the kind of an I/O function to call, its parameters, plus a closure that takes the result of the I/O function, and returns again a pair of I/O function kind, parameters and closure.
Then, an external "executor" can just perform the I/O, call the closure with the result, perform the I/O again, etc.
The result of this is the program can be executed to do I/O, but all functions are side effect free (they just compute a result based on the arguments).
The tuple of I/O function kind to call plus parameters plus closure is called an "I/O monad" (or equivalently it can be just the closure).
Now in this model if you want to have subroutines you need them to return an I/O monad, and need to be able to "chain" the rest of the computation to it, which is done by adding a method to the I/O monad called "bind".
These subroutines might not do any I/O, but it's still convenient to return an I/O monad, so an operation called "return" is added, which returns an I/O monad whose I/O function simply returns the parameter to return.
This interface with those "bind" and "return" functions is called a "monad", of which the I/O monad is an instance (along with some obvious properties, like that calling bind twice is the same as calling it once with the composition of the two functions, and that return and bind also commute in the same obvious way).
Another example is the "state monad", which is like the I/O monad except the operations are "read variable #i" and "write variable #i" (Haskell uses STRef objects instead of variable indices, but it's essentially the same).
While the I/O monad can only be executed "externally" to the program, the state monad can be executed internally: a pure function can be written that takes an array of initial values of the variables, a state monad instance and executes the state monad instance and gives back the final variable values.
So this gives a more complicated way of expressing imperative programming that has the advantage of only involving pure functions and thus being usable in pure languages like Haskell.
Now, why isn't this the best thing since sliced bread?
The obvious problem is that a naive implementation is massively inefficient, since every time you want to perform a memory write you need to stuff the stack of the program into the heap and return it as a closure.
Furthermore, monads for normal I/O and state are just not very useful: isolating the computation can be done with Rust's model of lifetimes and linear types, the ability to execute it in different ways can be achieved by providing a dynamically-dispatched set of I/O functions, and stopping the computation can be performed with unwinding.
The only practical advantage of monads is that you can pause the computation and have the state on the heap instead of the stack. This is why efficiently implemented languages like Rust (and in this case Java/C#/JavaScript) only use monads to implement futures/promises, where this ability is exactly what is wanted, and don't use state monads or I/O monads for synchronous I/O.
As an aside, the reason futures/promises are the best way to do I/O is that when the computation is paused, the state is stored on a heap in a compact layout (as a future/promise monad instance), while with either threads or coroutines it consists of an inefficiently laid out machine stack in userspace (that also takes a lot of virtual address space since it must be able to grow arbitrarily) and in case of native threads a machine stack and thread structure in kernel space as well.
The async/await and future/promise support in Java, C# and JavaScript is essentially a direct implementation of the Haskell monad model, with the downside of requiring an allocation for each await.
However, there's also way to have a monad-like style while preserving near-optimal performance, which is used in Rust's async/await generators.
The idea is that instead of having monads be closures that return closures, they instead become generic "objects" with a method that, upon being called, updates the state of the object and returns a description of the I/O operation to perform (or even just performs it with parameters provided by the caller), and is called again and again until it signals that the computation is done (this of course involves mutable state, but the mutation can be restricted with ownership).
"bind" can then be performed by simply defining a larger object that embeds the inner "monad" and the whole thing can be monomorphized, thus removing any dynamic dispatch and having a program that behaves as normal, except that instead of storing variables in the stack, it stores them in a single variable that needs to be immovable, either by heap allocating or ensure it's not moved on the stack. Note however that you need further allocations for recursion and in general for any dynamic calls whose state does not have a bounded size.
TLDR: monads primarily exists because Haskell must be pure and doesn't care about performance, but also because they are the only way to do memory-efficient async I/O
I know I can google this, and have, and unfortunately I came away even more confused as a non-Haskell practitioner.