
Prismatic's "Graph" at Strange Loop - janaboruta
http://blog.getprismatic.com/blog/2012/10/1/prismatics-graph-at-strange-loop.html
======
salimmadjd
I love reading prismatic's blog. You have some really smart people there.
However, it seems like you're trying to solve everything from a purist view
and I think your product is suffering as a result. I have made the same
suggestion consistently that would make me use it more without any reply.
Basically, I've stopped using the product as the UX does not match the reason
I would want to use the product.

~~~
janaboruta
I'm sorry about that. If you're still interested in giving us UX feedback send
us an email feedback@getprismatic.com. I'll personally respond to it. Please
understand that we're still a small team and we get a lot of emails on a daily
basis. :-)

~~~
dmix
I agree with the author, the UX is definitely under-developed and I've also
given many feedback msgs.

We only say this because we care :)

------
jpadvo
This sounds very useful. I'm having trouble understanding the nature of the
system, though, because I don't really understand lisp / clojure. Is this a
pattern that could be applied in any language with first class functions, or
does it require a lisp to be practical?

~~~
w01fe
Sure! This particular way of expressing declarative structure may be specific
to Clojure, but the basic ideas should be useful in any (especially dynamic)
language with first-class functions.

The first idea is to take a complex system and express the set of components
and their relationships declaratively, rather than procedurally. This is an
old idea, and makes it possible to reason about the system, mock out
components for testing, map over the components for monitoring, and more.

The second idea is simplicity. By making this declarative language as simple
as possible, it becomes trivially easy to do all the things I just mentioned
using the existing tools in the language, without needing to write complex
library functions to support each use case.

For another example of a declarative system for composition you could look at
'react' in JS, which can be used in similar ways to Graph (and also supports
things like async composition): <https://github.com/jeffbski/react>

~~~
jpadvo
Thanks! That makes a lot of sense, and react looks like a really interesting
project. I'll have to go back and reread the article with an eye for applying
its ideas in Javascript / Coffeescript.

------
rafaelferreira
Pretty cool project, like a functional reinterpretation of dependency
injection frameworks. The basic idea behind dependency injection is to make
all dependencies for each unit explicit and externally supplied.

DI frameworks then build upon the pattern offering a declarative DSL to
declare dependencies and wiring. They also offer a selection of alternatives
to build dependent objects (e.g. build a fresh one when needed or supply a
global instance for every dependee) and a smidgen of generic functionality via
impersonation (e.g. monitoring, logging, timing, transaction
acquisition/release).

Interesting that untyped functional programming and macros allow for a lighter
expression of the same basic idea.

Also interesting that maps of functions are a fundamental part of the
solution. There are definitions of object-orientation that amount to basically
records-of-lambdas. I've long thought that there is a lot to gain by using
this basic abstraction as it naturally provides seams for
testing/observability/auditing/etc.

------
blaines
Is it just me or is the prismatic blog text being cut off on the side?
[https://twitter.com/blaines/status/256545302926995456/photo/...](https://twitter.com/blaines/status/256545302926995456/photo/1/large)

~~~
blaines
Appears this may be a retina mbp thing...

~~~
xtrahotsauce
happens to me on an iPad as well

------
a-priori
Very interesting, needs more examples though.

It seems like this fits a similar purpose to Nathan Marz' Storm framework.
Could you do a bit of a compare/contrast thing between Graph and Storm?

~~~
w01fe
Thanks! There are some more complex examples in the slides, I just didn't want
to make the blog post any longer.

The main similarity between this and Nathan Marz's Storm framework is that
they both rely on a declarative expression of the structure of the computation
graph.

However, beyond that there are many key differences. Storm is a graph
computation framework, which compiles your specification into a distributed
real-time computation pipeline. In contrast, Graph is just a _library_ for
expressing composition structure, but says nothing about execution strategy.

In principle (with a lot more code and some more annotations), one could
compile Graphs into distributed real-time topologies like Storm. For now we've
been using Graph in-process for real-time processing. But because Graph is so
simple and close to the language, it's very easy to apply to new situations
and build new abstractions on top of. For example, we also find Graph useful
for expressing the composition structure of our production services, which are
built up from many components.

~~~
jwr
But, from what I understand, if I structure my software using Graph, it should
be much easier to run it using Storm, even without automated tooling. That way
Graph becomes a stepping stone: structure your software, and _if_ you need
Storm's parallelism/scalibility, define a topology using the same functions.

------
moomin
This is great, but I wish they'd take it one stage further. At the moment, the
first example they give (monolithic let) is pretty idiomatic and readable. The
transformation from one to the other is pretty automatable. So for most cases,
why not just put a macro on top? e.g. defgraph-asfn

Storm requires some manual intervention because you need direct control over
how things are distributed.

------
aleyan
Nice. Assuming that the computation of each node is memoized, can we 'dirty' a
single node, so that during the next computation only the needed parts of the
graph are recomputed? This would make multiple computation of models where
only a few parameters are changing much more efficient.

Also, has this at all been inspired by SecDB's powerful graph oriented
features?

~~~
w01fe
We haven't had the need for that application yet, but it's certainly possible
-- that's why declarative specifications are great :)

I haven't heard of SecDB, but I'll definitely check it out -- thanks!

~~~
gajomi
This is definitely a use case I am interested in. I have been playing around
with computer algebra in Clojure for awhile where it is critical to keep track
of these dependencies (I have a hacky implementation which relies on metadata
so that the resulting data structure is out of site). But of course, when you
actually want to evaluate something, this is when you need to get
sophisticated. There can be multiple paths in the graph to arrive at the same
node and you have to decide. With memoization at nodes being an important
consideration. Anyway, I would be very happy to see an open source version of
this!

P.S. As I understand it all the graphs are dags. In my line of work (which
involves doing stuff at the interface of probability and statistics) there are
quite a few recursive functions that are difficult to unroll explicitly, or
are otherwise undesirable to unroll for the purposes of readability. Can we
expect to see circular dependencies at some point. Well, I know, the halting
problem and all, but still...

------
tel
This feels like lazy evaluation bolted on to Clojure (cool!). There's further
benefit since you get some dataflow abstraction—which then reminds me of the
Par monad.

<http://community.haskell.org/~simonmar/par-tutorial.pdf>

~~~
mattdeboard
There's no need to "bolt on" lazy evaluation since (future) and (delay) are
parts of the core language (not to mention lazy seqs)

~~~
moomin
Well, future, delay and lazy sequences are bolt ons to an eagerly evaluated
language. I'm not convinced the laziness is Haskell is worth it, tbh, but its
true that you really wouldn't need this project.

~~~
tel
Usually these kinds of demos are the same as the ones touting the virtues of
Haskell's laziness. The visualization is really nice, though, along with the
possible paralleization. I think the Par monad is a more interesting
comparison, honestly.

[http://hackage.haskell.org/packages/archive/monad-
par/0.3/do...](http://hackage.haskell.org/packages/archive/monad-
par/0.3/doc/html/Control-Monad-Par.html)

------
DennisP
Happy to see you're opensourcing. Are you planning to do the same with Flop
and Optimize?

~~~
w01fe
Flop is definitely on the list (with about 10 other libraries), but I'm not
sure about our optimization code. We plan to start a discussion of what we
should prioritize soon, so stay tuned.

------
jessepollak
Just a heads up, it looks like something is up with your CSS:

<http://cl.ly/image/241X1C2D3O35>

I'm on 10.8 with the latest version of safari.

In other news, this looks pretty cool.

~~~
janaboruta
Are you using a Retina Macbook?

~~~
renderkid
I have the same issue and i'm using a Retina MBP.

~~~
janaboruta
Thanks guys. We will look into it.

------
pnathan
If I understand the nature of this system, it's essentially a higher-order
function composition tool. Correct?

Nit, defnk? what about def-kw-func or something? not wholly readable as defnk.

~~~
w01fe
Yep, that's correct.

Point taken about the names, thanks for the feedback. Since Clojure already
uses defn and fn, defnk and fnk seemed natural. But maybe defn-keyword and fn-
keyword would be better.

~~~
pnathan
I really like this idea. Do you have a Common Lisp port of this, out of
curiosity? If not I might have to see what I can come up with. :-)

I like to have fully expanded names, ala Common Lisp. My editor will
autocomplete those on demand after I use a name once, so it's not a hindrance
to reading or writing.

My goal for identifiers is that I can read without consulting my internal
lookup table.

~~~
w01fe
Thanks! Nope, no common lisp port -- and we haven't released the Clojure yet,
but plan to soon. It's only 150 lines of code or so should be easy to port or
rewrite from scratch.

------
emeraldd
Graph reminds me of working with Spring (from what I remember of that). The
concepts sound very similar to Spring's auto-wiring.

