
Erik Meijer: Duality and the End of Reactive [video] - malbertife
http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2014/Keynote-Duality
======
AlexanderDhoore
(Shameless self-promotion) I just finished by thesis on a new programming
language called SolScript, which is reactive and based on "math".

It's a hard real-time language for avionics software, BUT it has type
inference and duck typing (everything is "symbolic"). SolScript is supposed to
look familiar to anyone with a basic knowledge of mathematics. It's also a
literate language, because every SolScript file is a Markdown file (and
SolScript code is inside Markdown code blocks).

[link redacted]

~~~
jacquesm
\- real time

\- avionics

\- type inference

\- duck typing

You like challenges don't you? ;)

I like the literate component a lot, I can see why that would be a very big
plus when you're building avionics software.

Bookmarked your thesis.

What was your reason for going for 'duck typed' and 'scripted' rather than
strongly typed and compiled? (Which I would assume to be a whole lot more
suited for the intended branch of industry, so that has me curious.)

At the surface of it this looks like a 'textual spreadsheet' with the lines
replacing the cells of the sheet.

This 'spreadsheet' is then evaluated 'x' times per second to satisfy the hard
real time requirements.

How do you stabilize the results from one 'tick' to the next? Do you keep a
shadow copy that you work on with all the results posted to the next
generation?

There is something funny about a person working for a small software outfit
choosing a format for textfiles (for really good reasons) because Microsoft or
Apple might not be around when the files need to be read. I get your reasoning
and agree with it completely but you have to balance that against the chances
of your own company not being around anymore either.

How much of the _software_ of the original Boeing 747 is still in use today?
I'd imagine there are no fuselages that have not been upgraded multiple times
since the first 747's took flight. So the lifespan of the documentation is
roughly the same as the lifespan of the software.

Are there any public sources out there?

Here are some of the oldest 747 airframes known to be still on active duty:

[http://www.airliners.net/aviation-
forums/general_aviation/re...](http://www.airliners.net/aviation-
forums/general_aviation/read.main/2863915/)

But that does not say much about the avionics in them.

Do you intend to release the source to solscript in order to profit from the
'many eyes' out there to reduce the number of bugs in the runtime? (Which
can't be written in solscript, so effectively will have to be written in some
unsafe language.)

I like that markdown trick a lot, that's applicable to just about every
programming environment, see:

[http://michael-sokol.tumblr.com/post/13112309135/literate-
pr...](http://michael-sokol.tumblr.com/post/13112309135/literate-programming-
with-github-and-markdown)

~~~
AlexanderDhoore
Haha, that's a lot of questions. Let me try to go over them.

I say SolScript is "duck typed" and "scripted", because that's how it feels to
program SolScript. However, under the hood, SolScript is a statically typed,
compiled language.

About the "spreadsheet" idea, that is exactly what the SolScript IDE (Solide)
is going to be. There will be a spreadsheet mode, which displays the source
code as a spreadsheet. The last chapter talks about Solide, if you want to
know more. (The creation of Solide is going to be my job, and I've been
watching Bret Victor videos, all day yesterday, for inspiration.)

We have no "deep copy" of the data. The data is simply mutated in place, for
performance. The compiler knows in what order it has to update the datacells,
so SolScript users don't notice any of this. If I say `b = a`, then `a` will
be calculated before `b` uses it. (If you want the value of `a` before `a` is
updated, you say `b = previous a`.)

The whole "plain text" argument is about the fact that companies can die. So
can (UN)MANNED, of course. The difference is that plain text files will always
be readable, in contrast to Word files. You don't need our software to read
plain text documentation.

Releasing the source is not something we plan to do (for now). We are not
against the idea, but right now we have other priorities. There will probably
be a gratis version, even for commercial use. Only that version won't be
certified.

Yes, SolScript is written in an "unsafe" language: C. This makes it extremely
cross-platform. We can run on Linux, OS X, Windows, iOS, VxWorks... The coding
standards of our C code are extremely strict. Avionics style.

And about Markdown, yes that could be applied to other programming languages.
BUT SolScript integrates with Markdown on a whole new level. For example,
sections in Markdown are the scopes or namespaces of a SolScript program.
Markdown is built into SolScript, not tacked on.

Edit: Circular dependencies really make no sense, in a declarative programming
language. Defining `a = b` and `b = a` has no clear meaning, so the compiler
will throw an error. However, you can define things recursively using
`previous`. For example: `a = b + 1 initially 0` and `b = previous a`. In this
case, `b` is 0, 1, 2, 3... and `a` is 1, 2, 3, 4...

~~~
jacquesm
> Haha, that's a lot of questions.

Well, you post your thesis, I dig in :)

> Let me try to go over them.

Much appreciated, thanks for all your answers, I've read a good chunk of your
thesis by now (that's hard work :) ), good stuff.

> We have no "deep copy" of the data. The data is simply mutated in place, for
> performance. The compiler knows in what order it has to update the
> datacells, so SolScript users don't notice any of this. If I say `b = a`,
> then `a` will be calculated before `b` uses it. (If you want the value of
> `a` before `a` is updated, you say `b = previous a`.)

Ok. That may cause some trouble when there are long chains of circular
dependencies but I'm sure you're aware of that and that you have a solution in
place (assuming circular dependencies are even allowed).

Again, thank you for all the answers and much good luck with your project,
I'll be following your progress, it's super interesting what you are doing.

------
Pitarou
TL;DR

Reactive programming is ill-defined and over-hyped, so let's talk about types
and math instead (plus some not-very-subtle in-jokes about Microsoft).

An enumerator is basically a getter with the ability to fail and / or
terminate. It might also return a promise rather than a value. An enumerable
is a getter that returns an enumerator. We can express all this very cleanly
in generic types. And if we take the category-theoretic dual of these types we
get the observer and observable types.

So who needs reactive? ;-)

~~~
seanmcdirmid
What people really want is managed time.

~~~
auxon0
What kind of scenario are you thinking of? Rx has virtual time with the use of
schedulers that enables you to abstract over time, which is very cool.
[http://channel9.msdn.com/shows/Going+Deep/Wes-Dyer-and-
Jeffr...](http://channel9.msdn.com/shows/Going+Deep/Wes-Dyer-and-Jeffrey-Van-
Gogh-Inside-Rx-Virtual-Time/)

~~~
seanmcdirmid
This isn't really virtual time though, at least as realized by time warp. It's
a virtualization of time, for sure, but almost completely unrelated to DR
Jefferson's work [1]; no rollback of bad interactions or anything like that.

[1]
[http://dl.acm.org/citation.cfm?id=3988](http://dl.acm.org/citation.cfm?id=3988)

Rx is a one trick pony that focuses exclusively on the duality between event
streams and data streams. Management of time in any real sense is completely
missing from the framework. No continuous bindings, control flow must all be
buried as data flow, etc...

React is a much better solution for writing reactive programs than Rx.

~~~
auxon0
Brian Beckman worked on the Time Warp OS, with Dr. Jefferson [1] and was
involved with the development of Rx virtual time. You can read his comment
about how Rx virtual time could be used to create a Time Warp monad. [2].

Are you speaking about React.js? Erik's point about reactive being a buzzword
and distorted is proven by that because it's not the same thing at all.
React.js is cool and all that but it has nothing to do with guaranteed
composition and dealing with side-effects and monadic operators. That's like
saying Angular.js is better than list operations and functions that operate on
lists.

[1]
[http://dl.acm.org/citation.cfm?id=37508&CFID=356264587&CFTOK...](http://dl.acm.org/citation.cfm?id=37508&CFID=356264587&CFTOKEN=35819486)
[2] [http://channel9.msdn.com/Shows/Going+Deep/Expert-to-
Expert-R...](http://channel9.msdn.com/Shows/Going+Deep/Expert-to-Expert-Rich-
Hickey-and-Brian-Beckman-Inside-Clojure#c633949462670000000)

~~~
seanmcdirmid
That is a nice link, thanks! The previous video one was not talking about the
same virtual time, as far as I can tell.

My point was that if you were building user interfaces, react would get you
much further than Rx, which is a hammer looking for a problem where it is
actually better.

If you want the ability to deal with side effects, I'm working that in my own
work while avoiding burying everything in data flow:

[http://research.microsoft.com/apps/pubs/default.aspx?id=2112...](http://research.microsoft.com/apps/pubs/default.aspx?id=211297)

Ironically, I started this work while working for Martin with the goal of
making the scala compiler more "reactive" (the ability to responsively react
to change); I actually succeeded while making minimal changes to scalac. Like
actors or FRP, Rx can't be applied to scalac either, which desperately needs
to be reactive to fit into the IDE. Rx's idea of reactivity revolves around
event streams, virtual, managed time and frameworks like react attack the
problem of change directly. Erik is right about useless definitions, but one
based around change really works.

------
shaydoc
i thought it was a good presentation, however, I recently watched a Netflix
one, published by Matt Marenghi, which to me was infinitely better.

I think the description of events as just another collection, albeit a time
based collection flow was nicer ( Observable === Collection + Time ), and also
that this collection must have the ability to tell you that it has terminated.

[https://www.youtube.com/watch?v=XRYN2xt11Ek](https://www.youtube.com/watch?v=XRYN2xt11Ek)

~~~
brodo
Watched both talks and the Netfix one is really way better.

------
virtualwhys
and Scala is now his favorite language, interesting stance to take for a
previously prominent member of the Haskell community.

~~~
skrebbel
He's an independent consultant now. I suspect the market for companies willing
to consider Scala is a lot bigger.

If you want statically typed functional programming, but also functional
programming that stands a chance of adoption in many environments, then Scala
is about your only choice.

~~~
eranation
And Kotlin, and I would dare say Rust.

~~~
skrebbel
I may be living under a rock, but is Kotlin really used in production a lot?

And Rust? Does _anyone_ dare basing production stuff on Rust like now? With
breaking changes all the time?

~~~
TheHydroImpulse
There's only 3 production deployments of Rust that I know about. It's not
recommended to use Rust in production atm.

------
stigi
Variance explained by Martin Odersky in the Functional Programming Principles
Course

[https://class.coursera.org/progfun-004/lecture/83](https://class.coursera.org/progfun-004/lecture/83)

------
dominotw
"Reactive is the new agile. Everyone is talking about reactive. Reactive is
this, reactive is that"

This is the first time I've heard of reactive and read HN everyday.

> Java's enumerable( Iterable?) is broken

anyone know what how its broken?

~~~
auxon0
He's saying people are turning it into a buzz word that is losing it's true
meaning, so instead of the buzz word, think of what it's really all about:
composing and managing side-effects. Take a look at Google Trends for
"Reactive Programming", it spiked over the last year or so,:
[http://www.google.com/trends/explore#q=reactive%20programmin...](http://www.google.com/trends/explore#q=reactive%20programming)

Take a look on GitHub at all the projects with reactive in the name or
descriptions; it's become a buzzword that people use to describe various
things that don't have any relationship to the original.

Iterable is broken, as Erik said, because it has a hasNext() method, and a
next() method, which is a problem because hasNext() causes side-effects to be
evaluated.

Imagine if the implementation of the iterator has to execute a long running
function, or wait for the next value in an observable to be pushed, like an
event. So, if call hasNext() twice, then it may cause a side effect so that
next() actually has a different result than it would without the side effect.
Thus, it is not mathematically, a function, and can't really be used to build
monads.

So, unfortunately, due to backwards compatibility, Java classes that implement
Iterable/Iterator can't automatically get the benefits of list monads etc...,
by simply defining a library of operators that take Iterable/Iterators, like
C# could with IEnumerable/IEnumerator.

In C#, IE only has side effects in the MoveNext() method, which returns true
or false if there's another element, and the effect is explicit in the name,
which helps. However, more importantly, if you get the value from Current,
there is no side effect, and you can do that as many times as you like.

With Iterable, if you call next() it moves to the next element. So, it is
impossible to get the same value twice with a guarantee of no side effects,
and impossible to guarantee mathematically sound composition of Iterables. The
interface is broken, and changing it would break nearly every Java program.

------
ahvetm
tl;dr: Languages offer ways of composing side effects. Delay is just another
side effect and "reactive" isn't that special.

~~~
Pitarou
I think there's a little more to it than that...

------
hyperion2010
Now I want to know who was in the audience (esp. the two 'math' guys off to
the right).

~~~
pjmlp
At very least, these guys

[http://channel9.msdn.com/Events/Lang-NEXT/Lang-
NEXT-2014](http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2014)

------
noelwelsh
The first six minutes alone are worth a listen if you're in a rabble-rousing
mood.

~~~
haberman
Wow, I listened to the first six minutes on your recommendation and was pretty
put off.

The speaker mocks his audience of Microsoft engineers (like not in fun, he is
really saying their culture sucks).

He is dismissive of "reactive programming" as nonsense from architecture
astronauts, without giving a remotely fair description of what it actually is.
His straw-man of "var x = 10; println(x); x = 42; println(x)" is not reactive,
because "x" is not a function of other mutable state in the model. I've never
heard of reactive programming until five minutes ago, but I can tell this just
by reading the Wikipedia excerpts from _his own slides_ (and a little follow-
up reading confirms this).

I'm all for rabble-rousing, but if you're throwing punches you should know
what you're talking about.

At 17:30 he makes a joke about a Steve Jobs function that says "iPhone",
"iPad", "iCloud", and then "terminates naturally". Holy bad taste.

EDIT: others have pointed out that with extra context this is more obviously a
self-deprecating and good-natured shtick.

~~~
bad_user
Erik is a little trollish when expressing his opinions. From what I noticed,
he loves flames. People either love his style or hate it. I guess you're in
the later group - personally I don't mind when he's mocking something I care
about, because from what I observed, he's always joking or half-joking and
he'll say the opposite of what he just said in a couple of weeks.

> _He is dismissive of "reactive programming" as nonsense from architecture
> astronauts, without giving a remotely fair description of what it actually
> is._

That's because his presentation is from React 2014, so he can fairly assume
that the audience has some idea of what he's talking about, because "reactive"
is the freaking topic of this audience -
[http://reactconf.com/](http://reactconf.com/)

> if you're throwing punches you should know what you're talking about

Well, he kind of does. He's one of the architects of Reactive Extensions (Rx)
and now he's contributing to RxJava. I don't agree with many of his opinions,
for example I think they made design mistakes while architecting Rx, but he's
OK in the knowledge department ;-)

> _At 17:30 he makes a joke about a Steve Jobs function that says "iPhone",
> "iPad", "iCloud", and then "terminates naturally". Holy bad taste._

He also makes the same joke about himself - was trying to explain how streams
are terminated, either naturally or by error (the joke on himself was that he
was fired, i.e. onError).

~~~
headinthebox
> for example I think they made design mistakes

Curious to hear more.

~~~
louthy
Same here. For the record I think Rx is a lovely piece of work (as is LINQ
btw). I can't think of anything I'd change in the bits of Rx that I've used so
far.

~~~
platz
One criticism is a lack of ability to express things like "back-pressure"
where consumers indicate to producers to slow down, stop, etc..

~~~
louthy
That implies a two way dispatch which is not in the remit of reactive surely?
A system that's closer to Haskell's pipes is probably what you'd want.

I think ISubject<I,O> has the interface for a pipe, with IObservable being a
Producer equivalent, and IObserver being a Consumer equivalent. So there may
be some cunning that could be done with your own implementations [to create an
Effect]. (This is just wild brainstorming without looking at the code, so take
all of this with a large pinch of salt).

Working within the Rx system, there'd be two ways of doing this:

The Observable has a mechanism for slowing down, but obviously it can't get
instruction from observers - so it would have to make a judgment on what is
'too much'; a slow down would affect all observers. Definitely (well probably)
not what you want.

The other way is to use the various buffering functions in Rx on the
subscription, or roll your own function that has some intelligence. That
localises the 'backing up' on a per observer basis, but doesn't slow down the
observable itself.

To 'Stop' you could switch your subscription to Observable.Never<T> until
you're ready to receive messages. Obviously you'll miss messages generated on
the other stream whilst you were not listening.

~~~
platz
Reactive = resilient + eventbased + responsive + scalable

I think it is relevant, what allows you to ignore the behavior of the system
in favor of a component?

Dean wampler in a recent talk spoke of "Reactive Streams" which add this
capability, but I have not seen it yet.

I have not seen anyone do this in Rx yet. Also ideally the back pressure is
out of band to avoid starvation of that channel

~~~
louthy
I guess have an 'out of band' observable? Each observer that needs to
communicate instruction back to the original observable can drop messages into
an OoB observable, the original observable can subscribe to the OoB observable
and then react to those messages as it sees fit.

Or do you think it should be an inherent feature of the stream itself? Isn't
the point of IObservable to be the dual of IEnumerable, so calling Select on
IEnumerable is a projection from A -> B, calling Select on IObservable is also
a projection from A -> B, I think having an additional effect breaks that
duality. So an Observable that has a side-channel or out-of-band stream
wouldn't be an Observable, it'd be another category (which is why I suggested
Pipes).

I totally understand why you'd want something like that, I'm just not sure
it's fair to suggest Rx has design mistakes because of this though.

~~~
platz
Well, speaking to the point of duality, I think Rx solves the problems it sets
out to solve very nicely, and I would not say it has "design mistakes".

It certainly seems possible to use Rx primitives to build more functionality,
yes.

Though everyone will have to build this themselves (if, of course, they even
need it), and they'll probably all build it differently, where other lib's
have this concept baked in (i.e. Pipes).

IMHO, libraries shouldn't try to do everything anyway, and to that end, Rx
does it's thing and does it well (unix philosophy), as it's based on these
mathematical abstractions.

------
cbeach
I'm confused as to why Erik would want anything to do with MSDN? He slates MS
on a regular basis. Also, I suspect very few amongst the YC/HN developer
community would touch Microsoft products.

~~~
dtech
Other than him having been employed by MS and making major contributions to
.NET like LINQ and Reactive extensions?

Yeah I wouldn't know why he would want to have to do anything with MSDN

~~~
louthy
He's not employed by MS. He left a few years back.

