
Flow-based Programming - jarsin
http://www.jpaulmorrison.com/fbp/
======
jpaulmorrison
There's a good discussion in the Google group of the ideas that differentiate
what we are now calling "classical" FBP (or CFBP) from FBP-like systems, more
and more of which are appearing all the time - in particular the topic
"Underlying models" \- [https://groups.google.com/forum/#!topic/flow-based-
programmi...](https://groups.google.com/forum/#!topic/flow-based-
programming/L_E7dEU6sN8) .

If you are interested in the more large volume, data-processing, "classical"
FBP implementations, a good place to start is my book, "Flow-Based
Programming", 2nd ed., and the web site
[http://www.jpaulmorrison.com/fbp/](http://www.jpaulmorrison.com/fbp/) .

~~~
SOMMS
Hey Paul,

Glad to hear FBP is going strong. You may not remember me but I had a chance
to work with you many years ago on getting the Java based FBP framework to
input/output HTML (ie like a servlet node). Always loved the paradigm, the
parallelism and genericness of the whole concept. I wrote many, many networks
and eventually wrote an FBP parser that would create programatic FBP networks
from XML files. I was wondering if this sort of thing
(interpreter/parser/language) was ever incorporated into FBP itself? If not -
are their any plans? Nowadays JSON would be the way to go. Thanks again - hope
all is well. Chris S

------
tbirdz
On the topic of dataflow programming I found the book "Dataflow and Reactive
Programming Systems: A Practical Guide to Developing Dataflow and Reactive
Programming Systems" by Matt Carkci [0] to give a pretty good overview of the
different types and implementations of dataflow programming systems. I think
that flow-based programming sounds incredibly cool, but I haven't found any
project where I could use it well... yet!

[0] [https://leanpub.com/dataflowbook](https://leanpub.com/dataflowbook)

~~~
amelius
I think the biggest problems with flow based or reactive programming are the
occurrence of glitches, and, more importantly, the problem of how to compute
updates without performing unnecessary work.

I wonder if these problems have been successfully addressed somewhere.

~~~
seanmcdirmid
My work [1] addresses glitches head on (hence the system is called "Glitch")
by tolerating glitches and at least making the extra work functionally benign
(via logging and rollback, like a transaction). Glitch is not based on data
flow, however, but somewhere in between even if still very reactive.

I've built a few data flow systems in the past (my dissertation work [2] was
based on one), but I decided to tolerate rather than avoid glitches early on
because I found that way to be more resilient. Flapjax [3] avoids glitching
via a topological sort, which I rather see as an optimization (to reduce
unnecessary work) rather than as a correctness issue.

[1]
[http://research.microsoft.com/apps/pubs/default.aspx?id=2265...](http://research.microsoft.com/apps/pubs/default.aspx?id=226574)

[2]
[http://research.microsoft.com/pubs/179366/mcdirmid06superglu...](http://research.microsoft.com/pubs/179366/mcdirmid06superglue.pdf)

[3]
[http://cs.brown.edu/~sk/Publications/Papers/Published/mgbcgb...](http://cs.brown.edu/~sk/Publications/Papers/Published/mgbcgbk-
flapjax/paper.pdf)

~~~
amelius
Interesting. Do you know of any work that computes a data flow graph as the
program runs (i.e. data flow is implicit in the source code), which performs
then reactively?

~~~
seanmcdirmid
Ya, that is exactly how Glitch works, except I just call the data flow graph
as a dependency graph :).

Computing the dependency graph dynamically obviously leads to a lot more
flexibility. I'm not really sure what you mean by "reactively" here: if you
mean does it reactively update computations as state (and even code, if you
want to get Bret Victor about it) changes, then ya.

Data flow by itself isn't necessarily reactive; actually the way it was
originally defined to work very lazy push things through. Continuous
interpretations where changes were propagated non-lazily didn't come until
much later.

------
harlowja
A framework that is similar to this in python (that I am one of the authors
of) [1] (and [2] for more in-depth docs/examples). It's used in openstack
(which some people may have heard of) where appropriate (and where
desirable...) to help in consistent, scalable execution using a flow like
methodology to declaratively describe workflows (and execute them using
various strategies/engines).

[1]
[https://github.com/openstack/taskflow](https://github.com/openstack/taskflow)

[2]
[http://docs.openstack.org/developer/taskflow/](http://docs.openstack.org/developer/taskflow/)

~~~
samuell
The one I did know in python was PaPy:

[https://code.google.com/p/papy/](https://code.google.com/p/papy/)

Will have to have a closer look at this one as well, though.

~~~
i000
Author of PaPy here.

Thanks for giving PaPy a try. Although PaPy [[https://github.com/mcieslik-
mctp/papy](https://github.com/mcieslik-mctp/papy)] has never gained traction
:(, I have been using it daily for over 5 years (no bugs in the scheduler so
far). By now our PaPy based genomics pipelines have probably processed
petabytes of data.

------
ColinWright

        Bandwidth Limit Exceeded
    
        The server is temporarily unable to service
        your request due to the site owner reaching
        his/her bandwidth limit.
    
        Please try again later.

~~~
walshie4
[https://web.archive.org/web/20141026114453/http://www.jpaulm...](https://web.archive.org/web/20141026114453/http://www.jpaulmorrison.com/fbp/)

------
samuell
For anyone wanting to read-up / discuss FBP in depth, don't miss the mailing
list, it's a gold mine of information:

[https://groups.google.com/forum/#!forum/flow-based-
programmi...](https://groups.google.com/forum/#!forum/flow-based-programming)

------
peterlk
What specifically differentiates this from functional reactive programming?

~~~
samuell
Matt Carkci's book discusses the different closely related topics of data
flow, flow based programming and reactive programming, very clearly and i
quite some detail:

[http://dataflowbook.com/cms](http://dataflowbook.com/cms)

Can highly recommend.

~~~
tbirdz
Wow, we just recommended the same book within seconds of each other. But
seriously it's a good book, if you have interest in the topic it's worth
reading.

~~~
corysama
I believe the author started
[http://www.reddit.com/r/dataflowprogramming/](http://www.reddit.com/r/dataflowprogramming/)
It's a bit slow, but there's a lot of good stuff in there already.

------
samuell
On a related note, there's a new Apache Incubator project, built on these FBP
principles:
[http://nifi.incubator.apache.org](http://nifi.incubator.apache.org)

------
coding4all
This can be done in Clojure with core.async [1] and transducers [2]

[1] [http://clojure.com/blog/2013/06/28/clojure-core-async-
channe...](http://clojure.com/blog/2013/06/28/clojure-core-async-
channels.html)

[2]
[https://www.youtube.com/watch?v=6mTbuzafcII](https://www.youtube.com/watch?v=6mTbuzafcII)

------
setori88
We implemented a flow-based programming language in Mozart Oz at
github.com/fractalide/fractalide if you're interested.

~~~
whitten
I can testify to the usefulness of Mozart Oz. It is a multi-paradigm
programming system. I didn't know it had dataflow as one of its paradigms. Is
this something written in Mozart Oz rather than an enhancement to it ?

------
analog31
Ask HN: Is Excel a dataflow programming tool?

~~~
kazagistar
Sure, but it is not "flow-based programming" (which is what this article is
about) because there are no named output ports.

------
namelezz
Is this similar to Flux of ReactJS?

------
kitd
ITT: lots of people hawking other (semi-)related content.

