

Graph: Faster Abstractions for Structured Computation - olenhad
http://blog.getprismatic.com/blog/2013/5/1/graph-faster-abstractions-for-structured-computation

======
taliesinb
At one point, I did a similar thing for a computer vision system I was working
on.

That was quite naturally a filter graph: every component was time-dependent,
caching and logging was important (rewind to find this blob 5 minutes ago),
and parallelism was crucial because the system had to deal with synchronizing
multiple IP cameras.

Amazingly, it was possible to do this in an entirely safe way in C++, with
templates.

I wish I could put the code up on github, but unfortunately the company got
taken apart and sold, so the code is in limbo.

------
msandford
This is great work. Stuff I've been thinking about for a number of years but
never actually doing. I wish there were something like this for Python. I know
it's not pure, but it's my drug of choice. Anyhow, kudos.

~~~
aria
Graph would be possible, but not as clean in Python. Consider switching up
drugs. Clojure is way more potent man.

~~~
msandford
Yeah agreed. But I can ready Python (and many non-functional languages)
effortlessly and Clojure (and most other functional languages) take WAY more
effort for me to read. I mean huge amounts. Enough to where I've never been
able to make a serious effort since getting anything done takes forever.

~~~
babs474
It's interesting because I had trouble reading lisp until somebody told me
that you don't read lisp by worrying about parentheses, but by reading it as
an indentation level aware language. Just like Python.

Here is a more in depth link: [http://stackoverflow.com/questions/1894209/how-
to-read-menta...](http://stackoverflow.com/questions/1894209/how-to-read-
mentally-lisp-clojure-code)

but the idea is, delete all the parentheses from you mind, the code should
still make sense. The parentheses are to help the computer out, not you.

~~~
grayrest
To follow up on this, I configure my editor to de-emphasize parens:

<http://gr.ayre.st/s/images/parens.png>

I can still see them if I need to but they have the same visual weight as
tabs/end of line spaces. I like this because it makes it feel more like python
to me.

------
w01fe
Coauthor of Graph here, I'm happy to answer any questions.

~~~
sigil
How does the parallel "execution strategy" work? Do you specify a maximum
concurrency level? Can you distribute the computation onto other machines?

~~~
w01fe
Graph is currently just in-process (no cross-machine distribution), although
it's definitely a possibility down the line. The currently released parallel
strategy is also just a pedagogical example, not really meant to be used.

Here's some WIP on a more practical parallel compilation:
<https://gist.github.com/w01fe/4710008> Once the kinks get worked out this
will go into the OSS project. Presumably concurrency level will be controlled
by a parameter, and/or passing an appropriate ExecutorService.

~~~
jared314
It would be very interesting if you could easily convert a graph into a
Storm[0] topology. An example of the Storm Clojure DSL is available in the
storm-starter project[1].

[0] <http://storm-project.net/>

[1] [https://github.com/nathanmarz/storm-
starter/blob/master/src/...](https://github.com/nathanmarz/storm-
starter/blob/master/src/clj/storm/starter/clj/word_count.clj)

