
NoFlo Kickstarter, the hacker's perspective - svetly
http://bergie.iki.fi/blog/noflo-kickstarter-launch/
======
Oculus
Watching the Kickstarter video, it felt very forced. I get that you are trying
to revolutionize programming, but try not to keep telling me that and instead
show me. I feel that if you had a screen cast of working on a simple demo
application (longer than 10-15 seconds), it would've conveyed the message of
how powerful NoFlo really is, much better.

I'm not trying to take away from all the work you've done (it looks like a lot
and keep at it). It's just a suggestion on how to really show me the ground
breaking-ness of NoFlo.

~~~
meemoo
Good suggestion. This will be coming in the project updates.

------
kybernetyk
From the kickstarter page:

    
    
          > If you’re a programmer, deep down, you know code is bad, it rots, 
          > and with each new line your software accrues complexity debt. 
          > New code means more bugs, more refactoring and more time needed
          > to bring someone else up to speed.
    

Yeah, and moving little interconnected boxes around a project workspace will
solve that? It's not like visual programming is something revolutionary new
and different - it's just another representation of the same old thing. So the
chances that it will solve the current problems with large code bases are the
same as a green car being faster than a yellow car only because of the color.

~~~
polymatter
Good point, but poor analogy. Everyone knows that red cars go faster
([http://tvtropes.org/pmwiki/pmwiki.php/Main/RedOnesGoFaster](http://tvtropes.org/pmwiki/pmwiki.php/Main/RedOnesGoFaster)).

I think more to the point I can't see anything about visual programming that
will make refactoring easier or less necessary.

~~~
dkersten
_I think more to the point I can 't see anything about visual programming that
will make refactoring easier or less necessary._

I don't fully agree. Well, I do agree that it is just as necessary and good
software engineering principles are necessary to prevent the code from (quite
literally) becoming spaghetti code, but from the work I've done in Max/MSP, I
feel that as long as you follow software engineering principles (most Max/MSP
people don't because most are eg musicians and such who have no software
engineering training) I found it to be a very pleasant experience with many
advantages over traditional textual languages. Granted, some things really are
best represented textually (certainly complex expressions or formulas) but a
lot of code is just routing data around and I found that the visual languages
I've used (mainly Max and Synthmaker) really shine at that. in fact, I find
that moving data around is the single most important programming task in
_most_ programs.

Other areas where visual programming really shines is in experimental
programming and I think a big part of that is that 1) you can easily "wire"
things up differently and change things just to see what happens and 2) you
don't have to name things until you're good and ready. You still should
properly name things, of course, but during prototyping and experimentation I
find this often gets in my way when using textual languages and I end up
giving things temporary names like foo or x. Another area where (some) visual
languages shine is debugging because they often let you visually see the flow
of data through the code and intercept it and such. I found Max/MSP's debugger
to be very nice to use, though I also feel there is a lot of room for it to
improve even more still and unit testing tools are needed too.

As for refactoring - you know how some IDE's have a feature where you can
select a bunch of code and hit refactor and it puts it into its own function
for you? Max does this really well too: click-drag to select all the
components you want to factor out and it places it all into a new component
for you. Really quick and easy. This isn't a visual vs textual thing but
rather a tooling thing, of course.

------
adamnemecek
I'm not quite sure what problem is this solving. What exactly does dataflow
programming allow me to do that is not possible with non-data flow
programming? It appears that the single components are still text, and data
flow is only used to hook up these components. So how is this different than
say an IDE with definition and reference lookup?

I've used Pure Data (which they mention) a little bit and here is an example
of a somewhat advanced program.
[http://i.imgur.com/NRtG8Ai.jpg](http://i.imgur.com/NRtG8Ai.jpg) [1]. For Pure
Data, dataflow somewhat makes sense (considering the problem domain) but if I
had to pick between debugging the schema above and debugging text code, I
think that I'd choose the latter.

[1] Image is from here [http://www.spazioclang.org/eventi/live-electronics-
con-pure-...](http://www.spazioclang.org/eventi/live-electronics-con-pure-
data-e-maxmsp/)

EDIT: Lol, here's an example of another PD program I found.
[http://i.imgur.com/Flwviol.jpg](http://i.imgur.com/Flwviol.jpg) Now imagine
that you inherit a legacy codebase that looks like that.

~~~
rits
Flow chart can be beautiful or ugly just as a textual code can be beautiful or
ugly. Also shape of nodes and connectors as important as indentation in text.
Look for example of some graph from Nuke - [http://www.spaille.be/dev/wp-
content/uploads/2010/10/works_n...](http://www.spaille.be/dev/wp-
content/uploads/2010/10/works_nukeNG.png)

------
31reasons
Code at Compile time is a different entity than the Code at Runtime.
Programmers learns to visualize expanding and collapsing data structures and
the web of pointers and call-backs. One simple tiny recursive function can
become a massively complex tree at runtime. These two complexities are similar
yet very different. Any system that doesn't respect the runtime will fail to
understand the mindset of a programmer. I hope they succeed in dealing with
both.

~~~
werg
Exactly this is the problem with grafting visual programming interfaces onto
textual programming semantics. One of the main important reasons to do VP is
to be able to inspect the code, to easier see what it does, ideally at a
glance. Yet as you point out, simple recursion can make this really difficult
to do. My thesis is that we need VPLs that have semantics specifically
tailored for the visual interface. It's possible, but takes a bit of thinking
to work out

------
bunkat
Looks like an interesting way to enforce the pattern of testable, reusable
components for developers that tend to end up with spaghetti code. However, I
think it's a little dubious to say that designers without coding experience
will suddenly become empowered to contribute to the code base (or write their
own applications) in any meaningful way.

Looking at one of the sample pictures on the Kick Starter page I see AnchorY
out connected to SpringY anchor and Touch start connected to DragX open. I
literally have no idea what this is doing. I probably would have connected
Touch start to DragX in, though apparently that would be wrong. So as a non-
coder I need to understand what every component does (which probably is not
explained in layman's terms), what inputs are, what outputs are, what the
specific component inputs and outputs are, and what those actually do (again
in layman's terms please!).

More importantly, I need to be able to think like a coder and know how to
break large problems down into small components and be able to figure out how
to then wire those together to get the desired outcome. And as soon as I hit a
bug or it doesn't do what I want, I'm guessing I'm just out of luck because
everything is a black box to me.

I remember some usability studies that we did at Microsoft that was testing a
simplified flow control UI to do basic programming. We brought in people that
claimed to have at least scripting experience if not coding experience. We
should them a constrained UI that had some operations (ping a server, start a
server, stop a server) and basic flow control (if, for, etc). We asked them to
reboot a server as a warm up question - since no 'reboot' operation was
provided they just need to combine a stop server with a start server. We lost
75% of the participants at that step. It was pretty eye opening to see first
hand that not everyone can think like a developer, it has nothing to do with
providing a nicer UI or pretty visuals. Problem decomposition and
recomposition are learned skills.

I really like the approach and think it can be very effective at improving the
overall experience for developers. But to say this is somehow suddenly going
to allow somebody with no coding or developer experience to "skip the tens of
thousands of hours becoming fluent in syntax and start visually hacking on
your application now" just reeks of marketing bs.

~~~
ericHosick
What would have happened if the question had been rephrased to "Ok, you have
stop car, start car and check car. What do you do to restart a car"? Perhaps
they would have done a lot better because they have a better understanding of
the car domain.

The purpose of a visual language is not to replace domain experience. If you
put me in front of a drawing program with a pallet of red, blue and green and
ask me to create the color cyan... Then we're gonna have a bad time.

Visual programming does not claim that critical thinking is not required: aka
programming in this context.

We've chatted with people who've said "I can not get the coding thing. That
has always been hard for me. Designing and taking bigger concepts and parts
and putting them together: assembly. I get that." These people can solve
problems, they understand critical thinking. They have expertise in their
domain.

What they don't get is a mapping process of automating a real world system
(what programming is) in a computing device by coding out the solution.

Don't even get me started on education where students are expected to learn
how to think critically (do that mapping thing) and fight with, to them,
totally nonsensical compiler errors as they try to learn coding and syntax.

In my opinion, a lot of brilliant future programmers are lost because they
don't enjoy fighting the compiler/interpreter.

~~~
bunkat
The participants in the study were IT professionals that managed servers for a
living. Servers and actions regarding servers were definitely within their
domain experience. So we did exactly what you were suggesting we did.

I still think that connecting up a hundred little boxes is just as complicated
as actually looking at code syntax, but even more so since everything is a
black box. How do you suggest people fix errors in the output when the only
action they can take is to connect boxes with lines?

~~~
meemoo
I'm imagining a debug view that slows down the flow and shows data traveling
through the graph, so you can see it at every step.

Or with the circuit board metaphor, a debug "probe" that would show the data
(or a graph of data) traveling through that point.

Different debugging tools will make sense in different domains, but we're
considering the challenge.

------
freework
As much as I want to love this thing, I can't help but feel like they are
trying to solve an unsolvable problem.

We don't consider Michaelangelo a genius because he thought up the David
statue. We consider him a genius because he went through the process to make
the statue exist.

Great software exists not because someone thought it up. Great software exists
because someone went through the process of making that software a reality.
There is no shortcut, in the same way that there is no shortcut between
thinking up a really nicely painted portrait, and making that portrait a
reality. The only way to make a painting a reality is to learn how to paint.
The only way to make a program a reality is to learn how to code. Any tool
that claims to make coding easier is nullified by the fact that you have to
still have to learn that tool too!

~~~
meemoo
More than making coding easier, I think we're trying to give strict structure
to code and pair it with a visual metaphor to help manage complexity. There is
a learning curve, but we're designing it to stick close to JavaScript.

If we succeed, I imagine that we'll just increase the complexity of the coding
that we take on. That's an interesting possibility. I'd like to imagine a new
generation of startups with more interesting tech than CRUD.

~~~
rubinelli
The strict structure kinda bugs me. I expect situations where a clean solution
using higher order functions outside this framework is possible, but if I
implement it, I no longer have this clear picture of the system. Or I submit
to it, perhaps duplicating code and spreading gnarly business rules on top of
what should be "pure" data flow logic.

EDIT: Still, I'm all for experimentation. If they can pull this off, great!

------
mks
This

    
    
      > NoFlo graphs instead are only the coordination layer 
      that manages the control flow of your software.
    

reminds me a lot the whole SOA/BPEL/BPMN mess that was hot few years ago in
enterpriseland. The surge in visual programming just resulted in developing
the same logic with less powerful tools in languages that were not designed
for programming (XML and in this case JSON).

These kind of tools focus on the wrong side of the problem. It is not about
how to connect the nicely designed services, but how to actually get services
to be nicely designed. Once you have service that can be easily connected in
the visual tool, it can be trivially used from the conventional programming
language as well.

Reasons why services can not be just trivially composed include:

    
    
      - every service that is comprehensively mapping real problem is inherently complex 
      (even for seemingly simple things like address geocoding)
      - any two services will have slightly different APIs 
      (consider various signatures of substring method in various languages)
       and oftentimes the backend system details 
      - limitations are leaking over the service API
      (datatypes, formats, paging, ...). 
    

For these reason there exists the large branch of programming that deals with
integrating services (sometimes called Enterprise System Integration) that
basically wraps, formats and preprocesses data before shoving them to the
target service.

For restricted domains - it could probably work. But for general service
integration within larger company, the best approach so far has been a
lightweight approach with mixture of Spring Integration and plain old Java.

------
johnw
This reminds me of Quartz Composer[1] in a lot of ways: you wire the inputs
and outputs of a bunch of different nodes together (called patches in QC) and
each patch gets you a step closer to the solution. This seems to work
especially well for graphics programming type problems where there are many
different variables to tweak and you're applying many transformations to the
image at each stage.

The problem is each component in the process still needs to be written in
code. In QC I would often be frustrated when a patch didn't quite do what I
needed and I had to keep trying to make my idea fit the limitations of the
available patches.

So while this might help non-programmers understand on a big picture level
what's happening, to go beyond that they're still going to have to learn how
the individual components work which comes back to learning to code textually.

[1]
[http://en.wikipedia.org/wiki/Quartz_composer](http://en.wikipedia.org/wiki/Quartz_composer)

~~~
meemoo
Diving in and editing the component-level JavaScript is a big part of the
design. There will be much less friction than opening XCode.

------
gridspy
Interesting.

Gridspy ( [http://gridspy.com](http://gridspy.com) ) is built on a lot of
discrete components. We've got a variety of servers all talking via several
APIs and pretty complex data flows in twisted and also in javascript in the
browser.

I'd love a tool that simplified this - even if it just helped ease the pain of
API integration or the flow of data.

I need to be led to more concrete information around what happens on deploy.
Do you compile applications from the flow diagram?

I want to see a nice migration to and from the noflo platform. The last thing
I want to see is lockin either way.

For me I need Python and Javascript support. We're built in Python.

Very nice video, very artistic. You certainly had fun pulling focus and
playing with exposure.

~~~
bergie
The network is assembled at run time by instantiating the various components
and connecting their ports. There is no compilation step.

As for Python, we'd love to support the language! It is one of our stretch
goals. In case we don't hit it, the Python community can of course do it
themselves :-)

------
cmhylton
We at Interface Vision (
[http://www.interfacevision.com](http://www.interfacevision.com) ) are working
on a Visual Programming Interface as well and we wanted to say congratulations
on a successful first day on Kickstarter. It is great to see someone else
share the space. Great job guys and again congratulations!! If you ever want
to chat please don't hesitate to reach out - great job guys and good luck to
us all!!

------
dangoor
Flow-based programming and the actor model have some overlap. There's an
interesting discussion of this on the c2 wiki:

[http://c2.com/cgi/wiki?ActorsAndFlowBasedProgrammingDiscussi...](http://c2.com/cgi/wiki?ActorsAndFlowBasedProgrammingDiscussion)

One big difference between actors and FBP is that actors can give feedback
whereas flow-based programs can only send data from one port to another (so,
providing feedback to the process feeding it information requires the creation
of an input port on the feeder and a feedback output port on the processor...)

As with anything else, FBP isn't a hammer that turns everything else into a
nail, but it's an interesting architectural pattern. I'll be curious to see
their noflo-based Jekyll replacement.

------
VikingCoder
I think of this as the "Rick Allen Solution." Rick Allen is the drummer for
Def Leppard, who lost his arm in a car accident. He says that overcoming his
disability has turned him into a better drummer.

Fundamentally, forcing yourself to do visual programming like this, is like
tying one arm behind your back. It's so impossibly difficult, that you'll
demand that the tools you use are the best quality, and intelligently solve as
many of the problems at your layer as possible.

The truth is, if developers insisted on good tools with good abstractions,
they'd achieve all of the supposed benefits of visual programming.

Most developers forget they're responsible for making their own living space
well-organized. They just code, leaving a mess everywhere.

------
MarcScott
From a teacher's perspective this looks excellent. I start teaching my
students programming using Scratch. Continuing on to a more advanced visual
programming interface would be great. It reminds me a little of the visual
scripting interface inside Cinema4dD- Xpresso.

------
Gonzih
Lets solve spaghetti code with more spaghetti code hidden behind some nice UI.
How about actually thinking about problems and finding fundamental flaws in
workflow? I know it's harder, I know you can't then film fancy promo video in
some expensive hotel. But it actually might work.

I'm not saying that ideas of visual programming are bad. What I'm saying it
that hiding complexity behind some more complexity and saying that we invented
simplicity is bad. NoFlo doesn't look like something new but more like just
wrapper around existing methodologies. Nothing new, just hiding details from
programmer. OOP way of thinking.

------
jjsz
I concluded that they are trying to open source what
[https://www.mashape.com](https://www.mashape.com) does behind the scenes,
visually. Connecting the different parts of the SOAd web, in the sense of
docker and containers. It's true- why can't newly released services just work
like legos? Say you want to support a new payment system, or a new analytics,
it's just drag and drop with this, hopefully; right?

------
Schuback
Throwing this out there, I have shelved a similar web-based project that did
this with Verilog designs, which lends itself more naturally to this struture
imo. I stopped because I don't believe there's a big enough market for it.
That, and I wasn't sure how to tackle the problem of thousands of I/Os in a
visual format (would make things real hard to see).

~~~
meemoo
One method for dealing with this is what Pure Data does. You have a message
component that can set a specific property of the next component.

    
    
        [innerHTML "hello"] -> [div]

------
KingdomSprite
I think this is a great idea, but experience is everything, as others have
stated, the diagrams presented still have quite the learning curve, knowing
what names connect where. I think if NoFlo allowed actual coding through a
visual interface like Scratch, with the ability to group components into
functions and classes, this could really go somewhere.

~~~
meemoo
At the Scratch level I'd rather just write code. NoFlo's design will allow you
to open and edit any component in a live graph.

------
ape4
Well, I liked the music (in the video).

------
gbrits
Interesting. This would seem a natural fit for stream / event based
processing. Digesting server-logs, social signals, etc. Something like Twitter
Storm but with components on a more fine-grained level. Wonder how concepts
like iterative map/reduce map to this.

------
ronaldx
I would like to know if NoFlo is being bootstrapped, i.e. are NoFlo developers
actively using NoFlo to further develop NoFlo?

Some video of this type of development process would be an amazing proof-of-
value.

~~~
meemoo
The UI is currently written in JS with Backbone. Once it reaches a usable-
enough state rewriting it in NoFlo is high on my list.

------
mugenx86
noflo looks like a novel yet familiar way of designing software, which really
glues well with TDD.

I don't sense a very strong emphasis on runtime profiling from the video or
the description, but if this were a core feature then it would certainly add
value to the product.

