

NoFlo: Help Everyone To Understand And Visualize Code - sarreph
http://techcrunch.com/2013/08/01/noflo-launches-kickstarter-campaign-to-provide-a-way-for-everyone-to-understand-and-visualize-code/

======
davesims
How is this different than, say, Rational Rose or any of the other UML/WYSIWYG
tools that have come down the pike over the years? Or, for instance, Rules
Engines that in general no one uses because inevitably stakeholders stop
trying to understand the "plain and simple" english-like syntax that have
morphed into full-on semi-Turing Complete disasters that developers manage
anyway?

Most code, when visualized in sequence or relational form, is generally
incomprehensible even to developers until they've spent time understanding the
intent and history of the domain. I can't convince most senior developers to
learn to read and use the simplest of UML sequence diagrams. How are non-
technical stakeholders supposed to be able to grok this?

Code is complex in the best of times, early in development. Legacy code that's
actually used for business in the real world, even when visualized and
distilled down to its simplest form, is generally a labyrinth of polymorphism,
duplication, object/relational tension, algorithms, background jobs and
asynchronous callback madness.

We've seen this kind of thing many, many times before. So often that by the
early 2000s when some Golden Visual Environment For Code, the Holy Grail of
"drag and drop", would emerge every couple of years or so, we used to roll our
eyes and say, "here's another one." I wish I could dig up some of those links,
but ALL of the language in this pitch is exactly the kind of thing we've been
hearing for decades. And it never pans out.

The closest thing we have is UML. Look, I love UML, and I'm probably one of
the last holdouts for it -- I read Martin Fowler's UML Distilled years ago and
still use those techniques to this day. But I've never used it to generate
code (a la Rational), or to fully diagram an entire system. Even with a human
editor reducing the noise and focusing the diagrams on the core workflow, real
world systems become extremely complex and very difficult to follow visually
unless you have a long history with the system, an understanding of the
patterns and architectural principles that went into designing it, and enough
technical ability to grok things like inheritance, object/relational concerns,
asynchronicity and the like. For these issues, the code isn't the challenge --
it's the concepts themselves, whether expressed visually or directly in code.
Any system that grows to have significant business value will a) NOT be able
to be maintained purely through visual environment and b) become so complex
that the visual representation will be a mass of lines and boxes that, unless
edited down and vastly simplified, will be too complex to read _even for
developers_.

The Holy Grail of a visual, drag and drop environment that could create
meaningful business value would have manifest years ago if it were possible or
even desirable to do so. This latest attempt may end up having some limited
utility for simple applications, but I'm beyond skeptical or pessimistic. This
seems hopelessly naive to me.

~~~
slacka
> Code is complex in the best of times...The Holy Grail of a visual, drag and
> drop environment that could create meaningful business value would have
> manifest years ago if it were possible or even desirable to do so. This
> latest attempt may end up having some limited utility for simple
> applications, but I'm beyond skeptical or pessimistic. This seems hopelessly
> naive to me.

Before Einstein showed up, most experts in the field of physics thought that
they solved nearly all the major problems. It saddens me that the nascent
field of CS is showing the same close-mindedness. I refuse to accept your
claim that we must live with the complexity. My mom struggled with my Win
Mobile 2003 smartphone. If Jobs had your attitude, she probably wouldn't be
using one today. There are too many unexplored areas of CS for this defeatist
attitude toward complexity.

That being said, I actually agree with most of your criticisms of visual
languages on todays HW. But I have an open mind. Like what if the problem with
visual languages is that they need a 3D display and a minority report/speech
input for UI to resolve the spaghetti code readability issue and all your
criticisms? Look at leap motion/Siri. These UIs are not that far off. See what
I’m getting at?

I’m sure that with new ideas in HW and SW, we will be able to reduce the
complexity of parallel programming. And even if Visual programming is a dead
end, I’m thrilled to see people continuing to flush our this area. Call back
hell is a real and present problem with JS, I welcome creative attempts to
solve it like NoFlo.

~~~
davesims
The Einstein example is a _non sequitur_. This isn't a technical, algorithmic
or scientific problem that needs special genius. Where's the 3D rendering of
Relativity that makes E=MC2 perfectly comprehensible for the masses? Or is
that defeatist?

Neither is the problem with representation. We've had 3D visualization for
years -- spaghetti flying at you in perfectly rendered HD 3D is still
spaghetti.

The problem is not with the technology, but in the subjective perception of
complex domains, and the real world fact that codebases tend to be more
unclear than not, acquire massive amounts of technical debt, and even the
cleanest codebases, in pure Uncle Bob and GOF-approved architectural clarity,
present a cognitive domain that requires a lot of specialization to
understand.

It's like saying -- what if the problem with understanding French is that we
need better visual representations of French? Cool -- that might help the
French and other Francophiles, but it doesn't remove the conceptual barrier
and basic training required to understand the underlying language.

Likewise, if I presented you with a perfect 3D model (whatever that means) of
my pub/sub solution built with a Reactor Pattern that messages N number of
clients and takes asynchronous events from N number of other clients and
persists the results of processing those messages in a sorting algorithm to a
key-sorted Hash inside a NoSQL store -- you'd still have to have the
conceptual vocabulary and experience to have any idea what was going on.

In other words, the issue is not an extrinsic problem that needs innovation,
but a subjective problem of conceptualization that floats independent of
language and implementation.

I love Steve Jobs' attitude and take inspiration from it -- he put user
experience and elegance over the needs of programmers. But let's stay with
Jobs for a minute. Because he was focused on user-centric experiences, the
tools he left developers were serviceable but mostly afterthoughts. It was up
to coders to service tools, not the other way around. XCode is not widely
admired as an IDE, Objective-C is a good language but has its warts, and in
general Jobs himself didn't see the biggest challenge as technical or code-
centric, his innovations were mainly a finely-tuned intuition for the
difference between elegant and simplistic, combined with a will, attention to
detail and leadership drive to make those intuitions reality. He simplified
touch interfaces, and left coders mostly to their own challenges, and I think
he was right on with those priorities. In that, he totally "shared my
attitude".

I _do_ have an open mind, or I wouldn't still be coding and pushing myself to
learn decades after scribbling out my first lines of BASIC on a TRS-80. But I
also have enough experience with many, many codebases and have spent years
learning techniques for simplifying and expressing complex domain problems.
And the class of visual representation of those domains, of which NoFlo is
just one more iteration, has very limited utility in that regard.

Is it possible that a better visual vocabulary for code will come along? Sure,
why not? But it will be of a completely different class than what we've been
presented with so far, and it certainly won't have been invented by IBM in the
70s.

~~~
slacka
"Everything should be made as simple as possible, but not simpler." -Einstein

We are failing to communicate here. Did you watch Bret Victor's from
yesterday? I’ll do my best to explain. I said I agreed with most of your
critiques on visual programming, it's your assertion that we've tried it all
and just have to deal with the complexity that I was referring to. Einstein’s
refusal to accept status quo, led him to that beautiful, simple equation. I
was not saying that visual programming is or is not that solution. I was
questioning your dismissal of it at this stage.

My 2nd point about the 3d Minory Report UI was a whimsical example of being
open minded towards technology. Which is why I said "Like what if" I was
trying to demonstrate a creative solution to the problem by combining it with
an outside tech. I doubt it is a viable solution. On second thought,
programming with a Leap Motion controller on an Oculus Rift display sounds
very cool, but of course google beat me to it in the 2013 I/O.

> It's like saying -- what if the problem with understanding French is that we
> need better visual representations of French?

Of course, every field takes training, but some representations do have
steeper learning curves. I started off learning simplified Chinese. After I
changed representations to pinyin, I was able to learn words about 5-10x
faster. So I don't see how your analogy holds with natural languages.

> Likewise, if I presented you with a perfect 3D model....

If you describe it in words or show me a flow chart, I will pick up the flow
chart much faster. Machine code, not so fast. NoFlo does not limit you to the
visual editor. You can also use FBP text language if you desire.

> But I also have enough experience with many, many codebases and have spent
> years learning techniques for simplifying and expressing complex domain
> problems. And the class of visual representation of those domains, of which
> NoFlo is just one more iteration, has very limited utility in that regard.

"The most dangerous thought you can have as a creative person is to think you
know what you're doing." -Bret Victor

I don't know how much experience you really have with FBP like NoFlo. I have
done some programming in LabVIEW, and I found it much easier to reason about
concurrency than thread based languages. You may think it's a waste of time
with its "limited utility" but I'm with MZT on this one, "Let a hundred
flowers bloom"

------
cliveowen
Maybe I'm a pessimist but I think that programming is and always will be a
process of writing text line-by-line on a text editor.

Also I'm calling BS on this:

"[...]is a sort of visual mechanism for helping people not necessarily
familiar with code, like executives, managers and others[...]be able to
understand at a glance what developers are doing"

Bringing executives and managers in the development process is a recipe for
disaster, there's a separation of roles for a reason.

There's always chat about programming being stuck in the '80s but I think that
with the recent developments and improvements in compilers, programming
languages and toolchain the process of programming is anything but stuck in
the '80s.

~~~
delinka
I've been writing code as text line-by-line for decades. I think code
visualization tools are key to letting those with a higher-level concern have
high-level access. Let's go with the car analogy: most drivers only ever need
high-level access to their cars- accelerator, decelerator, directional
control, etc. Some can "drop down a level" and change their own oil.

I feel that once we (the programmers and computer scientists) get some more
experience, we'll start designing our systems (including programming
languages) for such high-level access. Think Star Trek and the 'ambiguous'
verbal commands given by characters to the computer to complete a task.

Somewhat tangential to this, I have a similar problem with math education. We
don't show students the high-level uses of math and then work into the depths
of the concepts and mechanics of producing a result. Instead, we teach them to
memorize the mechanics and for many, the understanding never materializes. We
expect these students to build their car from scratch.

~~~
RogerL
That was not my experience in regards to math.

------
bergie
Here is the link to the Kickstarter, since TechCrunch seems to be having
issues with that:

[http://www.kickstarter.com/projects/noflo/noflo-
development-...](http://www.kickstarter.com/projects/noflo/noflo-development-
environment)

I'll be happy to answer any questions on NoFlo or the campaign here after I
sleep a bit :-)

~~~
JPKab
So I first encountered your project a while ago, and it looks like its come a
long way. Congrats!

Just curious, but can you point to any folks who are using NoFlo (not
necessarily the new visual editor tool) for ETL? I do a lot of ETL type work,
and would love an excuse to begin to use Node/Javascript instead of Python.
Python is awesome of course, but there are certainly advantages to Node's
event-driven architecture that would help me, since a lot of times I'm
processing huge CSV files that can't fit into memory.

I think NoFlo, combined with the new visualization tool, would have a very big
advantage in ETL. ETL workflows are one of those things that are so often
shared with executive types in a visual format anyway.....

------
jgrahamc
_Software applications are one of those things that are easy to draw, but
impossible to describe without years of practice in arcane specialized
languages._

I agree. They are easy to draw. They are easy to draw because in doing so you
eliminate real world situations and just draw some boxes that elide all the
real detail of programming. And in doing so you barely express what will be
needed to make the program actually work.

Visual programming has been attempted over and over again. It's the future and
I suspect it always will be.

~~~
thelukester
Most people seem to focus on the visual programming model which is just one
aspect of NoFlo. I think its real power comes from the Flow Based Programming
model. Functional programming is the trendy solution to the problem of
concurrency these days. But I think FBP could offer some solutions in this
domain too.

In my experience with LabVIEW, I found that the FBP paradigm greatly
simplified process of writing highly parallel code. Whether you use the text
or visual UI, I’m excited that NoFlo will bring this concept to JS
development.

------
sz4kerto
There was a team called Manchester Dataflow Group. They have spent 25 years on
researching dataflow-based computers, languages, whatever.

They made some brilliant contributions to CS in general, but - as they
realized as well - DF is not the holy grail.

[http://cnc.cs.man.ac.uk/projects/dataflow.html](http://cnc.cs.man.ac.uk/projects/dataflow.html)

------
Detrus
Looking at the prototype graph UIs it feels like the design misses the mark
significantly. Without organizational or auto-layout features that makes the
flows easy to follow it's still spaghetti.

Looking at some other flow chart designs that weren't inappropriately inspired
by subway maps

[http://www.businessprocessincubator.com/media/catalog/produc...](http://www.businessprocessincubator.com/media/catalog/product/cache/1/image/9df78eab33525d08d6e5fb8d27136e95/n/e/new_employee_as-
is-example.jpg)

[http://www.businessprocessincubator.com/media/catalog/produc...](http://www.businessprocessincubator.com/media/catalog/product/cache/1/image/9df78eab33525d08d6e5fb8d27136e95/i/n/incident_2.png)

[http://www.file-extensions.org/imgs/app-
picture/2731/labview...](http://www.file-extensions.org/imgs/app-
picture/2731/labview.png)

[http://www.computer-
solutions.co.uk/gendev/images/Example_LA...](http://www.computer-
solutions.co.uk/gendev/images/Example_LABview_diagram.png)

Seems there were several design ideas they just ditched? Particularly
organizing flows around containers that hold smaller pieces. LabView has this
functionality and even then you get some flows that are visually useless
[http://forums.ni.com/attachments/ni/130/3008/1/Run_away_diag...](http://forums.ni.com/attachments/ni/130/3008/1/Run_away_diagram.jpg)
like their text based counterparts. But simpler flows are clearer than NoFlo.

Meemoo also has a confusing interface. They're twisting flow lines to make
more of the UI fit on screen. Is there a clear overview where the flow lines
are straight?

Quarz Composer and building Facebook in it is a convoluted nightmare. They
could have built a Facebook prototype far easier with Flash Builder and a bit
of ActionScript. Keyframe animations are best done in an interface that
supports them instead of generic connect the dots.

All that said I believe flow based programming has a lot of potential. But to
make it viable for flows that are more complex than straight command lines or
20 elements you need visual tools that are more sophisticated than subway
maps. Popularizing flow programming would at the very least motivate
programmers to build components that work together.

~~~
RogerL
Layout is NP-hard. Actually, it is NP-impossible. How does an algorithm know
that in this case I want to emphasize how my code interacts with the IP stack
(top of page higher layers), in this next page I want to emphasize change over
time (later operations to the right and down), and so on?

I assert without proof that any programming model that requires me to do
layout is one that does not scale and will fail. Any model that does not allow
me to suck in and work with 1MM lines of enterprise code will fail. I don't
direct this at NoFLo, which I haven't used, it's a more general comment
directed to the multiple attempts over the decades for a visual interface. It
works great for domain specific, small problems, and falls flat for larger
problems and larger code bases. Anyone trying to make a new tool should be
aware of the history and difficulties.

~~~
Detrus
I'd guess that for your example you'd have labels on the flow components that
approximate the order of events. So in the visualizer it can display them in
almost linear order.

When you want to view low level vs high level code, you switch layouts. May
have to manually label your components low level vs high level. So you won't
see everything at once but noticeably more useful.

Hard to tell how far ahead these guys thought this through. Maybe with
community interest and a cool looking product they'd attract enough people to
work out these details. But judging by comments and relative lack of interest,
won't be holding my breath.

------
BMarkmann
I know this isn't the enterprise crowd on HN, but this idea is pretty deeply
rooted in a lot of large organizations in the form of "Business Process
Management". The BPM vendors all pitch the "let business users create
software" promise. In reality (as others in the comments have said about
similar tools) it's very hard to deliver anything but the simplest
applications and maintain the ability for non-developers to grok what's going
on.

Similar (standardized) notation: [http://www.bpmn.org/](http://www.bpmn.org/)

Also, blatent self-promotion (sorry) discussing the issues with graphical
programming: [http://c20g.com/site/blog/entry/the-promise-and-peril-of-
bpm](http://c20g.com/site/blog/entry/the-promise-and-peril-of-bpm)

------
lechevalierd3on
Well it already exists, it's called Labview and it is awesome for building
tools.

------
delinka
I think they've got a good idea. But after spending many minutes "Getting
Started," I haven't seen the first graphical visualization yet. If their
target is "...executives, managers and others who are increasingly asked to
have some background in programming" as TC suggests, they have a long way to
go. Someone who wants "visualizations" wants graphical, not text.

Let's assume for a moment that TC has that bit wrong. In that case, I'm
finding the NoFlo tool much less impressive.

~~~
gozala
I believe the whole point of the kick-starter campaign is to raise a money to
actually build a visualization tool / engineering environment. You can also
find demo in it's current state here: [http://noflo.github.io/dataflow-
noflo/demo/](http://noflo.github.io/dataflow-noflo/demo/)

------
knowaveragejoe
I think this is has a ton of potential use cases, but the video(at least the
beginning half) kind of makes it seem like they're pitching the ability to get
away from code entirely. I don't think that will be happening any time soon,
instead, this allows for a high level mapping of the codebase and for non-
developers to link components together at whim(if they're built generically
enough, that is).

~~~
gozala
I don't think the goal is to replace code entirely, goal is to have a high
level tool for high level problem that today is solved with a code but could
be lifted. People that love low level stuff will still do it by implementing
low level components.

------
subtenante
Large code bases now will REALLY look like spaghetti !

~~~
riskable
Well that's one of the benefits of visualizing code... You get to see just how
much of a spaghetti mess you've got. It can be a good motivator to start
organizing your code better.

~~~
recursive
Yes. I need more motivation to organize this 7-figure-sloc behemoth I
inherited.

------
SimHacker
Looks like there's somebody using "dataflow" programming on a large scale:
[http://www.washingtonpost.com/blogs/worldviews/files/2013/08...](http://www.washingtonpost.com/blogs/worldviews/files/2013/08/saic.png)

------
CmonDev
Is it possible to use just the graph visualisation package as a standalone
thing?

~~~
bockris
There are lots of JS graph viz packages if that's all you want.

~~~
CmonDev
Can you please give me an example that would support specifying graph elements
in form of:

{ name: "", inputs : {}, outputs {} }

?

Specialized lib is always better. The best thing about noFlo is visualisation.

~~~
bockris
I'm not sure I understand your syntax. Graphs are typically specified as
relationships (edges) between nodes.

    
    
      a->b
      b->c
      d->c
    
      { name: "a", inputs : {}, outputs {"b"} }
      { name: "b", inputs : {"a"}, outputs {"c"} }
      { name: "c", inputs : {"b","d"}, outputs {} }
      { name: "d", inputs : {}, outputs {"c"} }
    

???

~~~
CmonDev
OK, tell me which lib takes whatever JSON and allows laying it out like NoFlo
and allows custom styling of nodes AND connections (colors, icons, font other
styling). Something higher-level then D3.

~~~
bockris
entitled much?

[http://lmgtfy.com/?q=javascript+graph+visualization](http://lmgtfy.com/?q=javascript+graph+visualization)

------
CmonDev
Guys please just implement it in Mono/CLR rather then in Java/ObjC/Python -
you will get more platforms AND languages supported this way.

------
jamwaffles
I wish KickStarter had an option to subtract money from a project. This is the
biggest abomination in programming I've seen since PHP.

~~~
Robbo89
I signed up to this site just to comment here. Fucking agreed! I still hope
this is just a big joke.

------
ckdarby
I am the developer in the basement that comes to save everything and we do
exist.

------
joshdance
Backed the project. I really want Objective C support.

------
ttty
I've been looking for this for a long long time!!

------
xiiiiiiiiiii
awesome

