
Monadic IO: Laziness Makes You Free - noelwelsh
http://underscore.io/blog/posts/2015/04/28/monadic-io-laziness-makes-you-free.html
======
eranation
So, I'm trying to explain this to myself in laymen terms, can you let me know
how far is my naive understanding?

So is this whole IO Monad idea is to simply delay the side effect to the
edges? e.g. no pure program is useful, as it will have no input and no output,
but if you separate the input and output from computation and delay side
effects to the very end using monadic transformations you end up with more
"pure" functions and less "side effects" functions? (you can never have zero
side effects if you want your code to actually do anything)

This is how I ELI5 this to myself, but not sure if this is the right way to
put it. Am I close?

I see it as this - it's simply one of the SOLID principles. (the first one)-
Separation of Concerns

The best example (that doesn't involve Monads) that I can think of is this:

    
    
        def logMessage(foo:String, bar:String) = println(s"your $foo has gone $bar")
    
        def logMessage(foo:String, bar:String) = s"your $foo has gone $bar"
    
    

This is obviously not a monad or has anything to do with functional
programming, but the second one has no side effects, yet is still useful.

In the end the CLIENT of the code can decide what side effects they can do
with it (println, log.debug, whatever)

tl;dr - I don't really get all this Monadic stuff, but I see this as simple
SOC principle, decouple the data from it's presentation, or in other words,
the library should work with pure data, the client should work with the
"unpure" input and "unpure output", the library should just transform the
input DATA into pure output DATA. the side effects are delayed to the CLIENT
instead of the LIBRARY code.

How far am I? Hope it doesn't sound too simplistic (or just plain dumb)

~~~
chousuke
Not quite ELI5, but Haskell tries to make IO pure by essentially representing
all side-effecting programs as data and separating evaluation from execution.

If you say, for example:

    
    
      x = print "Foo"
    

then the value of x is a program that prints "Foo" when executed and in

    
    
      y = x >> x
    

the value of y is a program that prints "Foo" twice.

The language runtime essentially executes the value of "main" (which may be
lazily evaluated).

Where monads come into play is the composition of such programs, especially
when a program may produce output that needs to be fed to another program. The
"bind" function does that, and the Monad typeclass abstracts the pattern so
that you're not limited to just sequential programs; for example, promises
(asynchronously executed programs) are monadic as well.

~~~
drostie
I have to comment to specifically highlight this comment above. You can easily
understand monadic I/O when you understand chousuke's comment; until you
understand that comment you will have trouble understanding how Haskell does
I/O.

So to say it again in slightly different words: when you are writing Haskell,
your job is to generate `main :: IO ()` (if you have never read Haskell type
annotations, that's short for "a thing named main which is a program which
produces nothing important." To do so, you have a lot of building blocks, like
`getLine :: IO String` (a thing named getLine which is a program which
produces a String), `putStrLn :: String -> IO ()` (a function named putStrLn
which takes a String as input and produces a program which produces nothing
important).

You also have functions which combine these together, like `(>>=) :: IO x ->
(x -> IO y) -> IO y` (an operator named >>= which takes, on the left, a
program which produces an x, and, on the right, a function which turns an x
into a program which produces a y -- the operator then giving you a program
which produces a y.)

So one example is:

    
    
        main :: IO ()
        main = getLine >>= hello where
            hello :: String -> IO ()
            hello name = putStrLn ("Hello, " ++ name ++ "!")
    

This is a program which prompts the user for a line (like, say, `Chris`),
wraps it in some text, and prints that text out.

The Haskell compiler just writes `main` to disk as the executable. The
compiler does not "run" any of these programs; that happens when you run the
executable which Haskell makes for you.

It is called by this scary word "monad" because that describes two things you
can do with programs, one of which is this fundamental `>>=` operator.

------
serve_yay
I guess I'm just slow, but I don't understand how this isn't just moving the
toothpaste around the tube, so to speak. You start out with an impure
`println` and replace it with an impure `Pure.println`.

edit: I was somewhat inaccurate here - as the name implies, `Pure.println` is
pure, but it returns a thunk which is not. That's what I'm talking about, I
have trouble understanding how this isn't just a little dance you do to
pretend your code isn't impure.

~~~
implicit
I use this technique to achieve bulletproof unit tests in Haskell HTTP
services by providing two run functions: One that runs on IO and does all the
real stuff, and another that twiddles some state and produces a pure result.

This nets us tests that are 100% reliable. The compiler rejects all code that
can introduce external inputs.

~~~
JackMorgan
Do you have an example demonstrating that pattern? I'd love to see it.

------
pron
To me this seems like an unnecessary abstraction in 99% of the cases that then
makes reasoning about execution so much harder. Once you go monadic IO, you
have to use it everywhere (at least when dealing with a specific API), and
then what? Suppose you wish to trace calls to println (or any other monadic
operation) with Byteman (one of the greatest JVM tracing tools) by capturing
all calls to the actual println (because, say, you want to know what operation
resulted in a line printed to the log). You get a useless stacktrace that you
can no longer pinpoint in the code. Monads erase all context.

Of course, I assume you can rebuild the concept of execution context,
replacing the stack with some other context abstraction. You've just recreated
yet another powerful and standard concept -- in addition to exceptions and
control flow -- provided by the platform. I guess that when using Haskell the
entire platform and all libraries already use this monadic context (as there's
no concept of a stack), but on the JVM you're now living in a "shadow" world,
completely separate from any built-in abstraction.

Laziness (on imperative platforms) may have its virtues, but its cost is that
it makes you abandon and then re-create in a foreign and incompatible way lots
of powerful -- and standard -- execution primitives.

~~~
chriswarbo
Consider an analogy to booleans. We can _get_ a boolean in many different ways
('true'/'false' constants, numeric comparisons, etc.). We can only _use_ a
boolean in one way: selecting the branch of an if/then/else construct. All
logical connectives, (de)serialisers, conversions, etc. and everything other
'users' of booleans that a language may have built-in can all be replaced
using combinations of if/then/else. More formally, we could say that booleans
have many _introduction rules_ but only one _elimination rule_
(if/then/else)[1].

We can easily remove booleans from a language by combining introduction and
elimination into one step. In other words, we can replace
introduction/elimination combos like this:

    
    
        bool b = lessThan(x, y);
        if (b) { foo; } else { bar; }
    

With a single construct, like this:

    
    
        ifLessThan(x, y) { foo; } else { bar; }
    

We can do this to every boolean introduction rule; in particular, if we do it
to the constants 'true' and 'false' we end up with a form of the "Church
booleans"[2]:

    
    
        ifTrue  { i_will_always_run; } else { i_will_never_run;  }
    
        ifFalse { i_will_never_run;  } else { i_will_always_run; }
    

If we think about first-order programs, what effect does this transformation
have? It brings together the _reason_ for a decision with the _consequences_
of that decision. In other words, to use 'ifLessThan' we must have the numbers
('x' 'y') which 'do the choosing' _and_ the branches ('foo' 'bar') which get
chosen, _at the same time_.

What if we inspect the stacks of these programs? They will always show us the
_reason_ for branches being taken! We're never just in the branch of an 'if'
construct: we're in the branch of an 'ifLessThan' construct, and we can see
what the numbers were; or an 'ifEqual' branch; or an 'ifPhaseOfTheMoon'
branch; or whatever. In other words, to paraphrase your complaint about
monadic IO, _booleans erase all context_!

In a language with booleans, we can write a complex, deep calculation which
returns a boolean; then we can use that boolean elsewhere. Reasoning about
such programs is incredibly difficult, since all of the stack information
built up during our complex calculation is thrown away once it returns; by the
time we hit a problem _using_ the boolean, we know nothing about it other than
'true'/'false' (this is also related to the idea of "boolean blindness"[3]).

This same argument can actually be applied to _every_ data type (numbers,
lists, etc.).

So, why do languages use booleans? Because _despite_ these problems, this
ability to split up reasons from consequences can be very useful too! It lets
reify decisions into concrete values. We can manipulate these values with an
intuitive, high-level algebra (AND/OR/NOT/etc.). It lets us separate _how_ we
make decisions from _what_ those decisions are. As long as these decisions are
given descriptive variable names like "queue_is_empty", and we avoid the
temptation to collapse everything together with logical connectives, then we
can still reason effectively: "we got here because the queue was empty".

In a language without booleans, we can regain this power using higher-order
functions (passing branches around as variables); or we can write first-order
boilerplate to model the same thing. But why bother when we can just use
booleans?

 _Exactly_ the same argument can be applied to reified ('monadic') IO. The
core idea is to represent IO actions as data, just like booleans represent
decisions. Just like with booleans, this lets us separate _how_ we decide on
actions from _what_ those actions are. Likewise, we can use (more or less)
intuitive, high-level algebras to manipulate these values. Monads are one
example of an algebra for doing this; applicatives, effects and arrows are
other algebras we could use instead. Note that we can also manipulate boolean
_values_ using algebras other than "boolean algebra"; eg. as a ring.

Yes, we lose the context of what caused an action to be decided on; but it's
exactly the same as losing the context of what caused a boolean to be
true/false. The "solution" is the same in both cases too: look at how it was
_produced_ , not how it's used.

Booleans are either "true" or "false", not matter which operations went into
their construction. We can't 'inspect' a boolean to recover whether it
'contains' 'AND's/'OR's/'other booleans', etc. Likewise, trying to 'inspect'
an IO action to recover whether it 'contains' 'PrintLn's/'open's/etc. is just
as futile and defeats the point of IO actions.

In other words, if you're recreating/rebuilding things to work with IO actions
then you're DoingItWrong(TM). Trying to, for example, look for a "println"
call 'inside' an IO action is like trying to look for an "OR(x < 5, ...)" call
'inside' a boolean.

[1]
[http://en.wikipedia.org/wiki/Natural_deduction#Introduction_...](http://en.wikipedia.org/wiki/Natural_deduction#Introduction_and_elimination)
[2]
[http://en.wikipedia.org/wiki/Church_encoding#Church_Booleans](http://en.wikipedia.org/wiki/Church_encoding#Church_Booleans)
[3] [https://existentialtype.wordpress.com/2011/03/15/boolean-
bli...](https://existentialtype.wordpress.com/2011/03/15/boolean-blindness/)

~~~
pron
I understand your sentiment, and while it may apply to languages such as
Haskell, it doesn't apply to languages running on such extremely flexible
platforms, like the JVM. On the JVM (and not just, of course), you can
represent any sequence of IO operations to be taken as an unstarted (or even
blocked) thread. Then, you can use the excellent bytecode manipulation tools
to transform that code. Doing so isn't easy, because it's not meant to be --
it's cleverness that should be limited to experts, and hidden from the
language as an extra-linguistic tool.

> Trying to, for example, look for a "println" call 'inside' an IO action is
> like trying to look for an "OR(x < 5, ...)" call 'inside' a boolean.

Perhaps, but they're both extremely useful, and luckily, if you keep to the
very powerful imperative abstractions provided by the JVM, you can do both:

> Reasoning about such programs is incredibly difficult, since all of the
> stack information built up during our complex calculation is thrown away
> once it returns

On the JVM, it's quite easy to transform any computation to record all state-
changes and decisions (AKA "omniscient debugging")[1]. Of course, using monads
will make things that much harder, because now you have to record not only a
very elaborate object graph, but one that is detached from a thread.
Production-time omniscient debugging tools can't do that.

> this lets us separate how we decide on actions from what those actions are.

Oh, absolutely; it's a terrific abstraction. Now all that's left to do is
weigh the cost of the benefits of this abstraction against its cost.

The costs include erasing context and making post-hoc reasoning hard (both in
debugging and profiling); also, this abstraction is infectious -- so it's hard
to limit it to just the places where you need it -- and incompatible with code
that doesn't use it. The advantage is manipulation IO operations much more
easily that with bytecode transformers (as the JVM already treats all code as
data, and makes it available for inspection and manipulation).

In short, this beautiful abstraction lets you do something that can already be
done on _all_ code on the JVM, but more easily, except it's limited to code
that actually uses it, which makes it incompatible with code that doesn't
(i.e. almost all code).

Why would I pay so dearly to do something I can already do much more
generally?

[1]: Open source:
[http://www.lambdacs.com/debugger/](http://www.lambdacs.com/debugger/),
[http://pleiad.dcc.uchile.cl/tod/download.html](http://pleiad.dcc.uchile.cl/tod/download.html)
commercial: [http://chrononsystems.com/](http://chrononsystems.com/),
_production time_ : [https://www.takipi.com/](https://www.takipi.com/)

~~~
chriswarbo
> Why would I pay so dearly to do something I can already do much more
> generally?

For the same reason you might pay to use a typed language, when an untyped
language is more general. Or why you might pay for encapsulation, when globals
are more general. Or why you might pay for structured programming, when GOTOs
are more general. Or why you might pay for a VM, when machine code is more
general. And so on.

I also don't see why it's difficult to mix and match monadic/non-monadic IO.
We can convert back and forth easily: "return" turns a non-IO value into an IO
action and "run" (AKA "unsafePerformIO") turns an IO action into a regular
value. We can think of it as just delaying and forcing function calls; in the
same way that we can think of booleans as their Church encodings.

What's more, in an imperative context like the JVM, everything is already in
IO by default, so we never need to do any conversion! Nothing becomes
'incompatible', since the JVM lets us perform side-effects whenever we like.

I think the more important distinction between the JVM and IO a-la Haskell is
the laziness in Haskell. Without laziness, it would be awkward for a JVM
language to abstract this stuff; in the same way that it's awkward to write
ifThenElse as a function, or to write short-circuiting boolean operators.

~~~
pron
> For the same reason you might pay to use a typed language, when an untyped
> language is more general. Or why you might pay for encapsulation, when
> globals are more general. Or why you might pay for structured programming,
> when GOTOs are more general. Or why you might pay for a VM, when machine
> code is more general. And so on.

Not all abstractions are created equal, and while all the other ones you
mention do have _some_ costs, they are dwarfed by the benefits; they're
bargains. This one? I'd say it's overpriced junk (though it's pretty junk).
The expressiveness it buys on top of what's already there (threads) is
negligible, and it's very pricey.

Not only do abstractions differ in benefits and costs in general, those costs
differ considerably depending on the platform.

> We can think of it as just delaying and forcing function calls

Delaying a "plain IO" operation is no different (in fact, it's identical) to
just using a thread. The abstraction is already built in. The laziness of
Haskell is a dual to (blocking) threads, and threads are already well
supported. There's no need to replace them, especially if doing so severely
harms your ability for posthoc reasoning (I'd say that even Haskell sacrifices
posthoc reasoning for apriori reasoning, which is a wrong choice for most
uses; but that's doubly worse on the JVM, which already has a lot of powerful
tooling in place for working imperatively).

------
tel
In Haskell the IO monad is a weird beast. In particular, other than it "being
a monad" and a couple of other instances it's not entirely clear what its
semantics are. The "Free monad semantics" described in this post are one
choice that is often used but it's a little bit tricky. Free monads are, in
particular, "fixed" structures while `IO` can be extended using FFI. There are
also questions about how to properly represent parallel threads in this way.

Edward Kmett explores a different method based on a kind of optimization of
free monads and Richard Kieburtz's `OI` monad [0]. In particular, we consider
values of the type

    
    
        data OI a where
          OI :: i -> (o -> a) -> FFI i o -> OI a
    

To decode this a bit, consider the abstract type `FFI i o` to be some unknown
"foreign, side-effecting" computation which intakes values at type `i` and
outputs them at type `o`. When we wrap `FFI i o` in `OI` we must pack it with
a value for its input along with a "transformer" function which ensures that
we're able to `fmap` over `FFI` [1]. The end result is that `OI` is a functor
which packs up a potentially open type of external, side-effecting values.

As a functor, we could `Free` it to get

    
    
        data IO1 a where
           Unit   :: a -> IO1 a
           Effect :: FFI i o -> i -> (o -> IO1 a) -> IO1 a
    

but Ed uses his interestingly optimized `Free` construct to instead give

    
    
        newtype IO2 a = 
          IO2 { runIO2 :: forall r .
                 (a -> r) ->
                 (forall i o . FFI i o -> i -> (o -> r) -> r) ->
                 r
              }
    

This is a funky type, but you can think about what it's like to "interpret"
IO2. Let's say you have a value of type `IO2 ()` and want to `interp` it

    
    
        io :: IO2 ()
      
        interp :: IO2 () -> ()     -- clearly a side-effecting signature!
        interp io = runIO2 io unit effect where
          unit :: () -> ()
          unit () = ()
    
          effect :: FFI i o -> i -> (o -> ()) -> ()
          effect ffi input continue =
    

What's neat here is that `effect` is essentially just asking you to give a
semantics for the `FFI i o` abstract type. You have a value of it, you have
its "input" and you have a continuation for its output. As the "side effecting
interpreter" of `IO2`, you have free levity here to do whatever you need to
"call" the `ffi` value. You also have free levity to "jump to another thread"
before calling the continuation and passing control back to the author of the
`IO2` value.

Ed goes on to note that as far as giving semantics to `IO` inside of Haskell
goes, there's a _very_ simple choice for the abstract `FFI i o` type

    
    
        type FFI i o = i -> o
    
        getCharFFI :: FFI () Char
        getCharFFI () = unsafePerformIO getChar
    
        putCharFFI :: FFI Char ()
        putCharFFI c = unsafePerformIO (putChar c)
    
        -- etc.
    

[0] [http://comonad.com/reader/2011/free-monads-for-
less-3/](http://comonad.com/reader/2011/free-monads-for-less-3/) [1] This is
Yoneda if you're into that kind of thing

~~~
tome
It sounds like you're claiming a free monad approach couldn't handle an FFI. I
don't think that's right.

~~~
tel
Eh, I'm playing fast and loose to validate a number of interesting things that
come out of that blog post. The `FFI a` functor is just Yoneda of (F o =
exists i . (FFI i o, i)) which you want because we can't guarantee that `FFI
i` is a functor. You can form the standard free monad over (Yoneda F) just
fine, but if you do it in the weird CPS style indicated here you get something
that's faster in Haskell ("well, ok, so what") but also has a really nice
story where the only remaining bit is "the interpreter is exactly giving
semantics to FFI".

It's also nice to do parallel threads in this way, but it's not infeasible to
do them with the standard free formation. Totally just aesthetics. I'll see if
I can dig up an example of two concurrent threads interacting over a shared
flipflop.

Also worth thinking about a definition of FFI which goes like

    
    
        data FFI i o where
          Unsafe :: (i -> o) -> FFI i o
          C :: (Storable i, Storable o) => String -> FFI i o

------
weego
_I’ve already hinted we need a monad_

Unless you are framework/lib programming, that is a phrase that should pretty
much never be uttered.

~~~
whateveracct
Or just architecting something in general. This talk [1] proposes a way to
approach application architecture that will result in the programmers writing
their own monads (or rather, their own FREE monads)

[1] video: [https://www.parleys.com/tutorial/composable-application-
arch...](https://www.parleys.com/tutorial/composable-application-architecture-
reasonably-priced-monads) | slides:
[https://dl.dropboxusercontent.com/u/4588997/ReasonablyPriced...](https://dl.dropboxusercontent.com/u/4588997/ReasonablyPriced.pdf)

