
Railway-Oriented Programming (2015) - tosh
https://fsharpforfunandprofit.com/rop/
======
mpweiher
What I found interesting is that if you have a pipe/filter architectural
style, your happy path stays completely free of error handling, because when a
filter doesn't have a good result, it just doesn't pass any data to the next
filter in line. Done!

You can then centralize the error handling by having a "stderr"-like output on
your filters.

When you have a call/return architectural style, you need to return something,
and thread the result along, so you need to do something more complex like
this pattern.

~~~
klodolph
This pattern isn't as complex as it looks, it's just the same pattern as
you've always used with some mathematical terms added to it. Having a
"stderr"-like output on a filter can also be expressed as a Monad. Instead of
Either or Maybe, or in general error monads, you could use a writer monad like
WriterT. (Haskell, not sure about F#.)

You can then keep your call/return architectural style or use pipes/filters.
Haskell and F# are pretty good at providing ways of describing these things
mathematically and making them easy to compose, it's just that the underlying
math is obscure enough that we get blog posts about it. Fortunately, there are
enough mature libraries that you don't need to understand the math to leverage
this stuff.

~~~
mpweiher
> just the same pattern as you've always used

What's the "pattern I've always used"?

Anyway, the problem is one of call/return in general, whether expressed in an
FP language or not. It really wants to _return_ something. With a real
dataflow approach, that just goes away. And nothing is simpler than something.

~~~
klodolph
> What's the "pattern I've always used"?

The pattern is to return a value and have some non-local return for errors.
For example, having a return value in C++ but throw an exception for errors.
By "always used", I'm just asserting that most people are familiar with this
pattern and use it frequently.

> Anyway, the problem is one of call/return in general, whether expressed in
> an FP language or not. It really wants to return something. With a real
> dataflow approach, that just goes away. And nothing is simpler than
> something.

This is not true for languages like Haskell. When you are writing monadic
functions, it doesn't "want" to return something, instead, you are free to
create your own data flow primitives as the monad's interface. This lets you
change how control flow works in these monads while providing enough common
structure that the novel control flow doesn't doesn't surprise people using
the monad. Common ways to do this are by composing monad transformers,
returning special value types, or using continuations.

That said, you get more flexibility to define data flow in Haskell if you
combine the ordinary monadic parts with data flow libraries like
pipes/conduit/machines. But you're still free to not return anything from a
function (or technically a monadic action) in Haskell, as long as you're in a
monad where returning nothing is a possibility.

~~~
gowld
Since Haskell is non-strict, it has "dataflow", and deprecates "call/return"
to some extent. (Parent poster might not have known that.)

Function "calls" can be passed around but not executed unless needed. Monads
like Maybe take advantage of this. Haskell functions always return something
_if_ they are called. Every Haskell program is "main :: IO ()", returning an
IO action executed by the runtime, but in the Maybe monad that something can
be _Nothing_.

~~~
klodolph
With monadic structure the non-strictness is not actually necessary. Haskell
is a research language with a lot of features dumped into it, the fact that
it’s non-strict has forced people to be more aware of whether their code is
pure, and purity has motivated the research into things like monads. But you
can do the same data flow programming even in an eager version of Haskell,
it’s just that in eager languages programmers are more tempted to write impure
functions in the first place.

------
meddlepal
This feels like a lot of words and fp "stuff" to describe a system where a
function takes two inputs (anyval0, maybe<error>) and ouputs (anyval1,
maybe<error>) and you chain them together.

~~~
lmm
The essence of FP is that if you apply a bit of cleverness and engineering,
you can replace things that look like they would need custom language
features, like multiple return or async, with libraries that let you work do
the same thing with plain old functions and values.

(All that said, if you're thinking in terms of multiple inputs and outputs
you're missing the point. The output isn't a pair, it's _one or the other_ ,
and that's really important. You need to be able to make invalid states
unrepresentable, so a disjoint union is not just a pair)

~~~
mpweiher
Same for OOP, example from 1988:

"Building a backtracking facility in smalltalk without kernel support"[1]

 _Languages like Snobol, Prolog, and Icon were designed with backtracking
facilities from the outset and these facilities are deeply intertwined with
the implementation. Retrofitting a backtracking facility in a language that
wasn 't designed for it has never been achieved. We report on an experiment to
retrofit Smalltalk with a backtracking facility. The facility is provided
through a small number of primitives written in the language (no modifications
to the kernel were made). The ability to do this is a direct result of the
power provided by the objectification of contexts._

There's tons of other examples, for example various distributed objects
mechanisms, my own Higher Order Messaging [2] the way Gemstone creates queries
from procedural selection code etc.

[1]
[https://dl.acm.org/citation.cfm?doid=62084.62094](https://dl.acm.org/citation.cfm?doid=62084.62094)

[2]
[https://en.wikipedia.org/wiki/Higher_order_message](https://en.wikipedia.org/wiki/Higher_order_message)

~~~
KirinDave
It's extremely frustrating that whenever anyone talks about even the slightest
positive mention of FP, this site tends to immediately have 2-3 folks pop out
of nowhere demanding that everyone recognize OO exists.

Yes. Yes. We know. You can write programs with OO. It's the dominant paradigm
of the software industry. No one here saidthat OO is somehow ineffective,
cannot abstract, or lacks cleverness.

Please for even comment section just put down the sword and let other folks
with equally interesting approaches talk about their work.

~~~
seanmcdirmid
I feel like the HN bias tends to go the other way, that FP is some kind of
awesome magic sauce and OOP is the bane of good software. In reality, most of
the techniques we discuss don’t really depend on a specific language
orientation.

~~~
KirinDave
You can probably prove this one way or another.

~~~
reitanqild
A good number of the posters here are fans of Paul Graham who is a loud FP
advocate.

Also I've been here for a number of years and I can never recall seeing anyone
bashing FP. (Which is totally fine with me.)

Bashing Java and and general OO however is to be expected in any programming
related thread here (a little tiring but understandable, I too have seen my
share of bad OO code.)

Neither of this counts as _proof_ but yes, this site comes off as extremely
pro FP for me as well.

~~~
KirinDave
Paul Graham is not a fan of FP, he's a Lisp advocate. Every Lisper will be
quick to tell you their approach is a superset of FP with its own unique meta-
syntactic tools.

This is a great illustration of my point. If you place every differing
viewpoint in one bucket and then take the sum, yeah it's gonna see
overwhelming. In actuality, there are lots of factions with interesting
viewpoints.

------
erk__
Old discussion:
[https://news.ycombinator.com/item?id=11955917](https://news.ycombinator.com/item?id=11955917)

------
cup-of-tea
Nice to see somebody else use a railway analogy. I like to use railways to
describe why interfaces and encapsulation are important engineering
principles. The track gauge is the interface between the people that build the
railway and the people that build the trains. As long as they both implement
the interface then together they get something much greater than the sum of
its parts.

~~~
xenomachina
Another place a railway analogy is used in CS is railroad diagrams for
representing context-free grammars:
[https://en.wikipedia.org/wiki/Syntax_diagram](https://en.wikipedia.org/wiki/Syntax_diagram)

------
Animats
Over 100 PowerPoint slides, which don't even play if you have tracking
blocked.

------
erikb
I either haven't gotten the point or have to assume there isn't one. Isn't
that normal error handling? The incoming error path is usually even done with
exception handling in normal languages, so you don't have to worry about it.
the pattern `if not <assertion>: raise Exception` should be everything one
needs. And for that this whole stuff is really fancy and long, especially if
one considers that most people (sadly) don't even check assertions between
function calls, if they are not forced to by bug reports.

------
protomyth
On the same train of thought, there was a book called "Plug and Play
Programming: An Object-Oriented Construction Kit by William Wong" that
detailed a similar concept in C++. Its an old, hard to get book, but did a
nice job of it.

------
k__
I like the way Cycle.js follows this principle.

It's basicall what Angular 2 should have been.

On the other hand, callbags seems to go a step further and could be simplify
things even more.

------
PLenz
And here I was hoping this was about CMRI/JMRI

~~~
ant6n
And here I was thinking this was about ERTMS/ETCS

------
develop7
A healthy person's lungs^W Rails.

------
DenisM
Way I could do this in C#. Tried doing something like this myself, but it
didn’t work out.

~~~
noblethrasher
Here is a moral equivalent in C#:

    
    
         public abstract class Maybe<T>
         {
         	public abstract T Value { get; }
         
         	public virtual string Failure => null;
         	
         	public SelectMany<U, V>(Func<T, Maybe<U>> f, Func<T, U, V> g)
         	{
         		var that = f(this.Value);
         		
         		if(that.Failure != null)
         			return new Failed(that.Failure);
         			
         		return new Success(g(this.Value, that.Value));		
         	}
         	
         	public sealed class Success : Maybe<T>
         	{
         		public override Value { get; }
         		
         		public Success(T value) => this.Value = value;
         	}
         	
         	public sealed class Failed : Maybe<T>
         	{
         		public override Failure { get; }
         		
         		public override Value => throw new InvalidOperationException();
         		
         		public Failed(string s) => this.Failure = s;
         	}
         }
         
         Maybe<string> UpdateCustomerWithErrorHandling()
         {
         	return 
         		from req in receiveRequest();
         		from v in validateRequest(req)
         		from c in canicalizeEmail(req)
         		from d in db.updateDbFromRequest(req)
         		from e in smptServer.sendEmail(req.Email)
         		
         		select "OK";
         
         }

~~~
noblethrasher
Actually, the above code doesn't compile (I just dashed it off while waiting
for a real world project to compile).

Valid C#:
[https://gist.github.com/noblethrasher/2d0e99f86b9853c646f3fa...](https://gist.github.com/noblethrasher/2d0e99f86b9853c646f3fa91c59b2223)

------
fortythirteen
So... try/catch?

~~~
steego
There's a difference. First, you're not incurring a large runtime penalty, so
this works better in higher performance scenarios.

Second, you're trapping and propagating successful values in a monad, which
means your obligating the caller to handle both success and error conditions.

~~~
dnomad
There is no performance penalty associated with try/catch. At least in JVM
languages, try is free (its a compile time phenomenon) and throw is also free
(memjmps), its only the _creation_ of the exception that is expensive and this
can be avoided by pre-allocating an exception (at the cost of no stacktrace
info).

Developers are better off in the end with proper exceptions because these
sorts of Result objects don't scale. Haskell ultimately embraces proper
exceptions and I suspect Rust will get there too eventually. F# actually has
proper exceptions which makes you wonder why the author is reinventing the
wheel badly.

~~~
thardus
I've had a guy come in on an F# project that used exceptions in a relatively
data intensive workflow and rewrite it to use Results like this.

The amount of boxing and unboxing as you go through the various switches
actually can ultimately make this much less efficient than the exception based
flow. The code was ultimately something like 8 times as slow, and topped out
CPU on beastly boxes which was surprising to me.

~~~
jstimpfle
Performance myths like this are really interesting. Usage of features is
usually not inherently responsible for bad performance, but _the way_ they are
used. Usage of these features can amplify the effects of bad software
architecture. Some features might just be elected as the culprit while there
is really a different problem.

If a little boxing and unboxing in a functional language takes 7 times as long
as the rest of the code, something must be very very wrong. It's hard to
believe that.

In this concrete case: a _data intensive_ workflow should really have no or
only few error situations. The performance of exceptions should not matter at
all. I would guess that reworking the code to avoid exceptions also included
reworking the program structure.

~~~
davidgrenier
I don't believe the boxing/unboxing performance claim either. I built a large
project in this style (in F# specificially) and the efficiency is not a
problem. It's possible said person built his custom implementation that had
issues but if you use ILSpy to decompile how union types end up compiled
you'll see it comes down to a boolean check against an integer followed by a
pointer dereference.

OCaml has very fast exception throwing/handling mechanism which apparently is
why some of the applications built in OCaml make heavy use of exceptions for
control flow (which as been mentioned by others, if you push too far is a
nightmare to to reason about). The appropriate strategy to port such code to
F# for example (which is a highly similar language to OCaml) is to switch to
unions in the return type.

The approach advocated by Scott is a general pattern with which you build an
entire application around. It does marvel when building a system surrounded by
others that you cannot trust and database with brittle consistency (as most
15+ years line of business database eventually become).

As for some of the other suggestion advocated in the comments, you have to
note that composability becomes a factor. If you are dealing with a collection
of entries each of which might fail or succeed, the railway oriented
programming approach scales to handling collection of errors/successes
gracefully which so many error handling strategies fail to meet.

~~~
jstimpfle
I have tried designing an application around Haskell's error monads in the
past, and it was an absolute disaster. There was so much typing. The types
became really hard to understand and the program structure became really
rigid. I do not know that it can't be done better, but I've heard prominent
Haskell guys say that combining errors (or any monads at at all) sucks in
practice.

And I've got zero problems with just handling errors procedurally. I realize
that errors simply combine very badly: If there are many possible error kinds
the best you can do in the end is usually to just quit. That's not what they
promised on the tin :-)

So in the end I just write procedural code and I'm careful not to get into
situations where many different kinds of errors can happen. I select a few
error cases that I care enough about to handle. Because handling even a single
kind of error means a _lot_ of complexity on top of a software project.

~~~
davidgrenier
Yes I agree it can become unwieldy if it is used throughout an application. I
think the sweet spot is too keep the pattern at the application boundary. You
can use business objects the way Scott describes them for the internals
without threading Either/Results monads throughout your application. Moving
calls to services and databases to the application boundary also helps keeps
the internals (where most of the business logic will reside) clean of
Either/Results. For the few things left that could fail in the internals,
stick to exceptions mechanism and avoid catching. I'm a big believer bugs
should make the application crash.

------
jgtrosh
I think that this uses a very specific model which can be represented with
railroads, but also any other 1-way system. For example, you could say it's
like hanging from a glider where you travel in a happy path, or can fall down
at any point and keep travelling on foot on a failure path. This analogy is
not much better but I feel like there's a better analogy out there than
railroads.

~~~
m_mueller
IMO the railway analogy works well, because it describes the right
dimensionality. It's sort of a 1.5D problem with a directed graph, with a
chain of events plus a fallback chain. A railway captures that perfectly while
giving people a clear picture they can understand, assuming they imagine a
train that can only move in one direction and decides its path based on
switches on the track. With something like a glider you could ask questions
like "but what if it flies off to the East" or "what about vertical air
movements that carry it up or down" and so on...

~~~
jgtrosh
I agree that's a problem with the glider image, but that's the issue I had
with railroads as well—there's much more freedom than what is shown here;
railroads can go multiple ways, join in various orders etc. What I mean is one
line change in a given direction does not imply subsequent ones will be in the
same.

------
hliyan
It is time someone wrote an essay arguing for _non-oriented programming_
(NOP?) and _requirements driven development_ (RDD?). In my darker moments, I
tend to see some of these trends in the same light as certain diet fads...

~~~
KirinDave
This style of programming has existed for about as long as Python has, and has
seen substantial improvements both in classical environments (ML & Haskell)
and experimental environments (Idris and Agda).

It's also increasingly popular in languages with less exclusively academic
DNA. Swift's community is considering adopting a moral equivalent. Rust uses a
very Haskell-ish variant. Golang uses a related variety cribbed from Common
Lisp called "product-type errors" (as opposed to "sum-type errors" used here).

If this is a fad, then so is Python, Golang, and Java.

~~~
junke
Common Lisp multiple values are not used for error handling as far as
idiomatic code is concerned. The also work differently, being symetrical to
optional parameters.

~~~
KirinDave
It's not common, but it's doable and Golang copied the approach for its error
handling, which is what I was referring to more directly.

But, people absolutely use MVB in cases where an ADT would work. Especially in
FFIs where CL's restarts would make the underlying bindings way more complex
or underperform. I can go find some examples when I'm off hours, if you like.

------
chatmasta
I'm surprised there's not a single mention of JavaScript promises in the post.
This is basically what they are, and their syntax naturally imposes this
technique as a solution to most use-cases.

Here [0] is an example of how I've used this technique in functional
JavaScript code. Please don't judge too hard as this was my first JS library
2+ years ago... certainly lots of problems in there (although it has been
running untouched in production since then!)

[0]
[https://github.com/milesrichardson/kewt.js/blob/master/kewt....](https://github.com/milesrichardson/kewt.js/blob/master/kewt.js#L326)

~~~
zaarn
It's not quite a Promise.

A promise would be the case in an async functions but railway like presented
work in sync functions.

Promises also sorta recursively build up and tear down (IIRC) while railway is
a "pass along" situation; each function that generates and error calls the
next function with the error where you then simply pass it along (or handle
it).

~~~
vimslayer
Obviously, not all railways are like Promises (they don't need to be async)
but Promises are kind of railways, as the parent commenter said.

Not sure what you mean by "recursively build up and tear down" but this JS
snippet sure looks a lot like the F# one in the slides

    
    
      const updateCustomerWithErrorHandling = (...args) =>
        receiveRequest(...args)
          .then(validateRequest)
          .then(canonicalizeEmail)
          .then(updateDbFromRequest)
          .then(sendEmail)
          .then(handleSuccess, handleError)
    

Here, each of the functions can return a result (async or not) or reject with
Promise.reject or by throwing. If any of the functions rejects, the rest of
the functions in this "happy path" are not called.

~~~
joshribakoff
Each .then replaces the promise before, instead of wrapping the source promise
around it. At each step you have a resolved value, with a monad you'd have to
.flatMap because mappining a monad to a monad nests it, which you have to
later flatMap to extract the boxed values. Compare rxjs streams which are
monads to promises which are functors. monads are functors but not vice versa.

