Hacker News new | past | comments | ask | show | jobs | submit login
Erik Meijer: Duality and the End of Reactive [video] (msdn.com)
98 points by malbertife on June 13, 2014 | hide | past | favorite | 65 comments

(Shameless self-promotion) I just finished by thesis on a new programming language called SolScript, which is reactive and based on "math".

It's a hard real-time language for avionics software, BUT it has type inference and duck typing (everything is "symbolic"). SolScript is supposed to look familiar to anyone with a basic knowledge of mathematics. It's also a literate language, because every SolScript file is a Markdown file (and SolScript code is inside Markdown code blocks).

[link redacted]

Just had a little glance (I'm on my way out the door), but if it's possible to construct arbitrarily large lists (as seems to be implied by the availability of e.g. "range N" for any N, how can you guarantee that e.g. "sum large_list" can actually terminate within the time bound set by the frequency?

Also there seems to be no actual proof of safety of the language's semantics (are the semantics formally specified?), but perhaps I missed it.

The length of all lists is known at compile-time. So, yes, you can do "sum range hugeNumber", but the compiler will tell you that your program can't be run at the correct frequency. (The possible range of the `hugeNumber` variable is known to the compiler, so it can know the maximum size of the list you are asking.)

The compiler can find out the range of numbers and the lengths of lists using static analysis, because SolScript is such a "restricted" language. Functional programmers would probably dump all that information into a type system. We call it "dataflow analysis", but I think the end result is the same.

Note that, all of the static analysis is done on the dataflow graph underlying a SolScript program. In the dataflow graph there are no "functions" or "classes"... The static analysis is much simpler because it is done on something more primitive than the language itself.

- real time

- avionics

- type inference

- duck typing

You like challenges don't you? ;)

I like the literate component a lot, I can see why that would be a very big plus when you're building avionics software.

Bookmarked your thesis.

What was your reason for going for 'duck typed' and 'scripted' rather than strongly typed and compiled? (Which I would assume to be a whole lot more suited for the intended branch of industry, so that has me curious.)

At the surface of it this looks like a 'textual spreadsheet' with the lines replacing the cells of the sheet.

This 'spreadsheet' is then evaluated 'x' times per second to satisfy the hard real time requirements.

How do you stabilize the results from one 'tick' to the next? Do you keep a shadow copy that you work on with all the results posted to the next generation?

There is something funny about a person working for a small software outfit choosing a format for textfiles (for really good reasons) because Microsoft or Apple might not be around when the files need to be read. I get your reasoning and agree with it completely but you have to balance that against the chances of your own company not being around anymore either.

How much of the software of the original Boeing 747 is still in use today? I'd imagine there are no fuselages that have not been upgraded multiple times since the first 747's took flight. So the lifespan of the documentation is roughly the same as the lifespan of the software.

Are there any public sources out there?

Here are some of the oldest 747 airframes known to be still on active duty:


But that does not say much about the avionics in them.

Do you intend to release the source to solscript in order to profit from the 'many eyes' out there to reduce the number of bugs in the runtime? (Which can't be written in solscript, so effectively will have to be written in some unsafe language.)

I like that markdown trick a lot, that's applicable to just about every programming environment, see:


Haha, that's a lot of questions. Let me try to go over them.

I say SolScript is "duck typed" and "scripted", because that's how it feels to program SolScript. However, under the hood, SolScript is a statically typed, compiled language.

About the "spreadsheet" idea, that is exactly what the SolScript IDE (Solide) is going to be. There will be a spreadsheet mode, which displays the source code as a spreadsheet. The last chapter talks about Solide, if you want to know more. (The creation of Solide is going to be my job, and I've been watching Bret Victor videos, all day yesterday, for inspiration.)

We have no "deep copy" of the data. The data is simply mutated in place, for performance. The compiler knows in what order it has to update the datacells, so SolScript users don't notice any of this. If I say `b = a`, then `a` will be calculated before `b` uses it. (If you want the value of `a` before `a` is updated, you say `b = previous a`.)

The whole "plain text" argument is about the fact that companies can die. So can (UN)MANNED, of course. The difference is that plain text files will always be readable, in contrast to Word files. You don't need our software to read plain text documentation.

Releasing the source is not something we plan to do (for now). We are not against the idea, but right now we have other priorities. There will probably be a gratis version, even for commercial use. Only that version won't be certified.

Yes, SolScript is written in an "unsafe" language: C. This makes it extremely cross-platform. We can run on Linux, OS X, Windows, iOS, VxWorks... The coding standards of our C code are extremely strict. Avionics style.

And about Markdown, yes that could be applied to other programming languages. BUT SolScript integrates with Markdown on a whole new level. For example, sections in Markdown are the scopes or namespaces of a SolScript program. Markdown is built into SolScript, not tacked on.

Edit: Circular dependencies really make no sense, in a declarative programming language. Defining `a = b` and `b = a` has no clear meaning, so the compiler will throw an error. However, you can define things recursively using `previous`. For example: `a = b + 1 initially 0` and `b = previous a`. In this case, `b` is 0, 1, 2, 3... and `a` is 1, 2, 3, 4...

> Haha, that's a lot of questions.

Well, you post your thesis, I dig in :)

> Let me try to go over them.

Much appreciated, thanks for all your answers, I've read a good chunk of your thesis by now (that's hard work :) ), good stuff.

> We have no "deep copy" of the data. The data is simply mutated in place, for performance. The compiler knows in what order it has to update the datacells, so SolScript users don't notice any of this. If I say `b = a`, then `a` will be calculated before `b` uses it. (If you want the value of `a` before `a` is updated, you say `b = previous a`.)

Ok. That may cause some trouble when there are long chains of circular dependencies but I'm sure you're aware of that and that you have a solution in place (assuming circular dependencies are even allowed).

Again, thank you for all the answers and much good luck with your project, I'll be following your progress, it's super interesting what you are doing.

SolScript looks fantastic.

Considering Avionics domain, using Coq or ATS to prove the correctness of SolScript compiler will help boost confidence.

> I say SolScript is "duck typed" and "scripted", because that's how it feels to program SolScript. However, under the hood, SolScript is a statically typed, compiled language.

Duck typing refers to the ability of making shit up at runtime, as in the ability for functions, classes or code to manifest at runtime and to respond to user queries based on runtime info. It doesn't refer to a lack of explicit types in your code, since in a static language types are still there and still have to be something concrete and solvable at compile-time. It doesn't even refer to structural typing, because again, the types themselves have to be solvable at compile time.

I'm a static languages fan and I've also worked a lot with dynamic languages. The ability to make shit up at runtime is much more than the ability to call a method on an object without being explicit about the interface used. This is not about succinctness as as it about programming style. With an expressive static language used to its strengths, as a developer you want to write code that is as close to provably correct as possible, so even if the static language used is very expressive, the style is oriented around modeling the problem domain with types. In a dynamic language on the other hand, the focus is on the raw data that needs to be manipulated and code is written to make shit up on the fly (i.e. method_missing, eval, classes manifested at runtime by means of meta-programming, etc...).

Hence my opinion that duck typing shouldn't be used outside the context of dynamic languages, because people might get the wrong idea about it.

SolScript really looks cool. I'm also writing something, it's a generic paper about FRP though. Gonna cite your paper :) Cool that you used duck-typing, although it's causing much more complexity, but it's great for Reactive Programming. Have a look at this paper, it might be of help for you http://lambda-the-ultimate.org/node/2438

My paper is in german, but I'll publish it on github and translate it (time given). I've also got a hypothesis on how FRP can be abstracted into a domain model using AI and non-axiomatic reasoning. But I hope I don't run in a dead-end there though, it's just a theory :)

Very interesting work. Is there a development list or some other way to follow progress?

Well, there is the main company website[1], but that's more for "marketing" purposes.

In the future, there will be a website dedicated to SolScript itself. This website will be the "technical" website. We already have a domain [2], but there's no website there.

Right now, we are working on getting SolScript into production. This thesis was really the first time anyone ever wrote anything about SolScript. Part of my job, in the future, will be to write blog posts and create tutorials. I'll definitely keep HN posted!

[1] http://www.unmanned.aero/

[2] http://solscri.pt/

This looks really cool. Congratulations on the launch.

BTW, there's a broken link on your home page: /71of71 instead of /71objectives

Any reason why the thesis itself isn't presented a complete and valid SolScript program? ;)


Reactive programming is ill-defined and over-hyped, so let's talk about types and math instead (plus some not-very-subtle in-jokes about Microsoft).

An enumerator is basically a getter with the ability to fail and / or terminate. It might also return a promise rather than a value. An enumerable is a getter that returns an enumerator. We can express all this very cleanly in generic types. And if we take the category-theoretic dual of these types we get the observer and observable types.

So who needs reactive? ;-)

What people really want is managed time.

What kind of scenario are you thinking of? Rx has virtual time with the use of schedulers that enables you to abstract over time, which is very cool. http://channel9.msdn.com/shows/Going+Deep/Wes-Dyer-and-Jeffr...

This isn't really virtual time though, at least as realized by time warp. It's a virtualization of time, for sure, but almost completely unrelated to DR Jefferson's work [1]; no rollback of bad interactions or anything like that.

[1] http://dl.acm.org/citation.cfm?id=3988

Rx is a one trick pony that focuses exclusively on the duality between event streams and data streams. Management of time in any real sense is completely missing from the framework. No continuous bindings, control flow must all be buried as data flow, etc...

React is a much better solution for writing reactive programs than Rx.

Brian Beckman worked on the Time Warp OS, with Dr. Jefferson [1] and was involved with the development of Rx virtual time. You can read his comment about how Rx virtual time could be used to create a Time Warp monad. [2].

Are you speaking about React.js? Erik's point about reactive being a buzzword and distorted is proven by that because it's not the same thing at all. React.js is cool and all that but it has nothing to do with guaranteed composition and dealing with side-effects and monadic operators. That's like saying Angular.js is better than list operations and functions that operate on lists.

[1] http://dl.acm.org/citation.cfm?id=37508&CFID=356264587&CFTOK... [2] http://channel9.msdn.com/Shows/Going+Deep/Expert-to-Expert-R...

That is a nice link, thanks! The previous video one was not talking about the same virtual time, as far as I can tell.

My point was that if you were building user interfaces, react would get you much further than Rx, which is a hammer looking for a problem where it is actually better.

If you want the ability to deal with side effects, I'm working that in my own work while avoiding burying everything in data flow:


Ironically, I started this work while working for Martin with the goal of making the scala compiler more "reactive" (the ability to responsively react to change); I actually succeeded while making minimal changes to scalac. Like actors or FRP, Rx can't be applied to scalac either, which desperately needs to be reactive to fit into the IDE. Rx's idea of reactivity revolves around event streams, virtual, managed time and frameworks like react attack the problem of change directly. Erik is right about useless definitions, but one based around change really works.

I believe that you're right, but with that attitude "against" an institution like Erik Meijer, you're not gathering many popularity points :-)

i thought it was a good presentation, however, I recently watched a Netflix one, published by Matt Marenghi, which to me was infinitely better.

I think the description of events as just another collection, albeit a time based collection flow was nicer ( Observable === Collection + Time ), and also that this collection must have the ability to tell you that it has terminated.


Watched both talks and the Netfix one is really way better.

and Scala is now his favorite language, interesting stance to take for a previously prominent member of the Haskell community.

He's an independent consultant now. I suspect the market for companies willing to consider Scala is a lot bigger.

If you want statically typed functional programming, but also functional programming that stands a chance of adoption in many environments, then Scala is about your only choice.

And Kotlin, and I would dare say Rust.

I may be living under a rock, but is Kotlin really used in production a lot?

And Rust? Does anyone dare basing production stuff on Rust like now? With breaking changes all the time?

There's only 3 production deployments of Rust that I know about. It's not recommended to use Rust in production atm.

Variance explained by Martin Odersky in the Functional Programming Principles Course


"Reactive is the new agile. Everyone is talking about reactive. Reactive is this, reactive is that"

This is the first time I've heard of reactive and read HN everyday.

> Java's enumerable( Iterable?) is broken

anyone know what how its broken?

He's saying people are turning it into a buzz word that is losing it's true meaning, so instead of the buzz word, think of what it's really all about: composing and managing side-effects. Take a look at Google Trends for "Reactive Programming", it spiked over the last year or so,: http://www.google.com/trends/explore#q=reactive%20programmin...

Take a look on GitHub at all the projects with reactive in the name or descriptions; it's become a buzzword that people use to describe various things that don't have any relationship to the original.

Iterable is broken, as Erik said, because it has a hasNext() method, and a next() method, which is a problem because hasNext() causes side-effects to be evaluated.

Imagine if the implementation of the iterator has to execute a long running function, or wait for the next value in an observable to be pushed, like an event. So, if call hasNext() twice, then it may cause a side effect so that next() actually has a different result than it would without the side effect. Thus, it is not mathematically, a function, and can't really be used to build monads.

So, unfortunately, due to backwards compatibility, Java classes that implement Iterable/Iterator can't automatically get the benefits of list monads etc..., by simply defining a library of operators that take Iterable/Iterators, like C# could with IEnumerable/IEnumerator.

In C#, IE only has side effects in the MoveNext() method, which returns true or false if there's another element, and the effect is explicit in the name, which helps. However, more importantly, if you get the value from Current, there is no side effect, and you can do that as many times as you like.

With Iterable, if you call next() it moves to the next element. So, it is impossible to get the same value twice with a guarantee of no side effects, and impossible to guarantee mathematically sound composition of Iterables. The interface is broken, and changing it would break nearly every Java program.

>This is the first time I've heard of reactive and read HN everyday

You're not paying enough attention. There were tons of articles on reactive on the front page, at least 1-2 per week, for the last year or so. See also: "functional reactive programming". See also: "React.js". See also: "Om".


He starts talking about that at ~23min. You will probably need to watch the whole video up to there to understand the context and why it is important to him.

tl;dr: Languages offer ways of composing side effects. Delay is just another side effect and "reactive" isn't that special.

I think there's a little more to it than that...

Now I want to know who was in the audience (esp. the two 'math' guys off to the right).

The first six minutes alone are worth a listen if you're in a rabble-rousing mood.

Wow, I listened to the first six minutes on your recommendation and was pretty put off.

The speaker mocks his audience of Microsoft engineers (like not in fun, he is really saying their culture sucks).

He is dismissive of "reactive programming" as nonsense from architecture astronauts, without giving a remotely fair description of what it actually is. His straw-man of "var x = 10; println(x); x = 42; println(x)" is not reactive, because "x" is not a function of other mutable state in the model. I've never heard of reactive programming until five minutes ago, but I can tell this just by reading the Wikipedia excerpts from his own slides (and a little follow-up reading confirms this).

I'm all for rabble-rousing, but if you're throwing punches you should know what you're talking about.

At 17:30 he makes a joke about a Steve Jobs function that says "iPhone", "iPad", "iCloud", and then "terminates naturally". Holy bad taste.

EDIT: others have pointed out that with extra context this is more obviously a self-deprecating and good-natured shtick.

Erik is a little trollish when expressing his opinions. From what I noticed, he loves flames. People either love his style or hate it. I guess you're in the later group - personally I don't mind when he's mocking something I care about, because from what I observed, he's always joking or half-joking and he'll say the opposite of what he just said in a couple of weeks.

> He is dismissive of "reactive programming" as nonsense from architecture astronauts, without giving a remotely fair description of what it actually is.

That's because his presentation is from React 2014, so he can fairly assume that the audience has some idea of what he's talking about, because "reactive" is the freaking topic of this audience - http://reactconf.com/

> if you're throwing punches you should know what you're talking about

Well, he kind of does. He's one of the architects of Reactive Extensions (Rx) and now he's contributing to RxJava. I don't agree with many of his opinions, for example I think they made design mistakes while architecting Rx, but he's OK in the knowledge department ;-)

> At 17:30 he makes a joke about a Steve Jobs function that says "iPhone", "iPad", "iCloud", and then "terminates naturally". Holy bad taste.

He also makes the same joke about himself - was trying to explain how streams are terminated, either naturally or by error (the joke on himself was that he was fired, i.e. onError).

> for example I think they made design mistakes

Curious to hear more.

Same here. For the record I think Rx is a lovely piece of work (as is LINQ btw). I can't think of anything I'd change in the bits of Rx that I've used so far.

One criticism is a lack of ability to express things like "back-pressure" where consumers indicate to producers to slow down, stop, etc..

That implies a two way dispatch which is not in the remit of reactive surely? A system that's closer to Haskell's pipes is probably what you'd want.

I think ISubject<I,O> has the interface for a pipe, with IObservable being a Producer equivalent, and IObserver being a Consumer equivalent. So there may be some cunning that could be done with your own implementations [to create an Effect]. (This is just wild brainstorming without looking at the code, so take all of this with a large pinch of salt).

Working within the Rx system, there'd be two ways of doing this:

The Observable has a mechanism for slowing down, but obviously it can't get instruction from observers - so it would have to make a judgment on what is 'too much'; a slow down would affect all observers. Definitely (well probably) not what you want.

The other way is to use the various buffering functions in Rx on the subscription, or roll your own function that has some intelligence. That localises the 'backing up' on a per observer basis, but doesn't slow down the observable itself.

To 'Stop' you could switch your subscription to Observable.Never<T> until you're ready to receive messages. Obviously you'll miss messages generated on the other stream whilst you were not listening.

Reactive = resilient + eventbased + responsive + scalable

I think it is relevant, what allows you to ignore the behavior of the system in favor of a component?

Dean wampler in a recent talk spoke of "Reactive Streams" which add this capability, but I have not seen it yet.

I have not seen anyone do this in Rx yet. Also ideally the back pressure is out of band to avoid starvation of that channel

I guess have an 'out of band' observable? Each observer that needs to communicate instruction back to the original observable can drop messages into an OoB observable, the original observable can subscribe to the OoB observable and then react to those messages as it sees fit.

Or do you think it should be an inherent feature of the stream itself? Isn't the point of IObservable to be the dual of IEnumerable, so calling Select on IEnumerable is a projection from A -> B, calling Select on IObservable is also a projection from A -> B, I think having an additional effect breaks that duality. So an Observable that has a side-channel or out-of-band stream wouldn't be an Observable, it'd be another category (which is why I suggested Pipes).

I totally understand why you'd want something like that, I'm just not sure it's fair to suggest Rx has design mistakes because of this though.

Well, speaking to the point of duality, I think Rx solves the problems it sets out to solve very nicely, and I would not say it has "design mistakes".

It certainly seems possible to use Rx primitives to build more functionality, yes.

Though everyone will have to build this themselves (if, of course, they even need it), and they'll probably all build it differently, where other lib's have this concept baked in (i.e. Pipes).

IMHO, libraries shouldn't try to do everything anyway, and to that end, Rx does it's thing and does it well (unix philosophy), as it's based on these mathematical abstractions.

> Well, he kind of does.

If he has that much background I would expect him to know better than to give such a poor example when he's trying to make fun of a definition as "dilbert-esque."

Tongue-in-cheek is a fun style and I totally get that, but if you're taking unfair hyperbolic pot-shots I think it should be at your own expense, not other people's. But that's just me.

When Erik mocks reactive programming I believe he is reacting to the Reactive Manifesto (http://www.reactivemanifesto.org/) currently being pushed by Typesafe and others.

My reading and feelings concerning the Manifesto mirror Erik's so I enjoyed listening to his mockery of it. I didn't really notice the MS bashing.

The issue with the Reactive Manifesto is that it is largely waffle. There are styles of programming that account for time, such as functional reactive programming. They tend to have formal definitions, or at least concrete implementations, so one can make objective statements about them. The Manifesto on the other hand is vague and full of platitudes. It's great for the Pointy-haired Boss, but there is little of substance than one can discuss in any objective manner.

[Update] I can understand how it's off-putting to listen to the presentation without this background.

Erik is one of the original signatories of that manifesto.

Erik Meijer is actually one of the main guys responsible for the popularization of 'reactive programming' with his Rx extensions for .NET [1]. It's probably a fair assumption that he knows more about this topic than most people.

[1] http://channel9.msdn.com/Shows/Going+Deep/Expert-to-Expert-B...

> if you're throwing punches you should know what you're talking about.

Ironically, I think you completely missed his point, probably because of an unfamiliarity with his background and history. He's not dismissive of reactive programming at all, quite the opposite! For example, he gave a course on Coursera about Functional Reactive Programming with Martin Odersky and Thomas Kuhn[0].

So my take on it is that he's just making fun of himself and his friends in a typical blunt Dutch fashion. Which of course doesn't work if you don't know the context.

[0] https://www.coursera.org/course/reactive

Cool, I'm totally open to the possibility that I missed the self-deprecating shtick from lack of context. If so, mea culpa.

When you say that he's mocking Microsoft engineers, is that because of the Visual Basic comments? Because Erik actually enjoys VB and was responsible for adding some features to it. He's a former employee, so there's context...

And I don't think he's being dismissive of reactive programming, he's dismissive of that definition, which is too broad. He was one of the main proponents of Reactive programming inside Microsoft.

> When you say that he's mocking Microsoft engineers, is that because of the Visual Basic comments?

"In Redmond, like a lot of people talk about code but don't write it, so I think Microsoft can only come back if, you know, they start writing more code."

> And I don't think he's being dismissive of reactive programming, he's dismissive of that definition, which is too broad.

The definition seems pretty reasonable to me. It immediately resonated with me; it described a principle I had been trying to achieve in some code I was writing recently. The idea of having changes automatically propagate through your model as a data-flow network is a pretty noticeable contrast to other ways of programming.

> He was one of the main proponents of Reactive programming inside Microsoft.

Yeah I guess I don't have this context. But without it, the talk just rubbed me wrong.

Erik was fired which is unbelievable; it's like firing Feynman. He obviously wasn't happy with Ballmer or the culture, and frankly neither were most of the MS employees in that audience. Ballmer's gone and suddenly everything MS does is becoming open sourced, for example. "A miracle". And just because quantum might sound cool, naming a razor blade after it doesn't make it quantum physics.

Correction - Erik wasn't fired.

>"In Redmond, like a lot of people talk about code but don't write it, so I think Microsoft can only come back if, you know, they start writing more code."

It's like you've never worked at a tech shop with a sales team. Also keep in mind that it wasn't meant to be taken seriously or literally. If his audience wasn't terribly offended by it, why are you?

> I'm all for rabble-rousing, but if you're throwing punches you should know what you're talking about.


pwnd ;-) Next time I'm back in Seattle to visit my ex-Microsoft SDE buddies we should grab a beer together.

I definitely wouldn't mind watching more of his presentations. He's pretty good.

There's a classic one where he spends 20 minutes mocking the Gang of Four for not realizing that Iterators and Iterables should have a dual. I'll try and find the link.

The GoF wrote, "favor composition over inheritance". That much was right. Unfortunately, they missed the mathematical composition possible that was hidden inside these patterns all along, much simpler and fundamental than the design patterns themselves.

I'm confused as to why Erik would want anything to do with MSDN? He slates MS on a regular basis. Also, I suspect very few amongst the YC/HN developer community would touch Microsoft products.

Other than him having been employed by MS and making major contributions to .NET like LINQ and Reactive extensions?

Yeah I wouldn't know why he would want to have to do anything with MSDN

He's not employed by MS. He left a few years back.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact