

Graphical Programming: I Really Hope This Is the Future - pcmonk
http://pcmonk.wordpress.com/2013/10/14/graphical-programming-i-really-hope-this-is-the-future/

======
vidarh
People have tried this. Over and over and over. I love the idea, and I too
have tried (and scrapped my effort), but it is an immensely difficult problem
to tackle for a number of reasons:

\- The moment you force a user to pick up the mouse, you have lost.

\- It seems very compelling to augment the display with more information, but
a lot of it is simply distracting, or reduce the density of relevant
information that will fit on the page. There's a reason that even in most
IDE's very little additional information beyond syntax highlighting have
"stuck".

\- Some weirdos (like me) won't touch tools that won't let me work over a dumb
ssh connection and terminal when I need to, which is one of many reasons
why...

\- ... everything needs to have a sane textual format. Another reason is that
it is essential to communicating about the code.

If you want to think about this, at least thoroughly review the long history
of visual languages, as there are plenty of mistakes to repeat.

~~~
ArbitraryLimits
> People have tried this. Over and over and over.

Seriously, it's the future and it has been for fifty years.

~~~
CmonDev
So it will probably be a part of final HTML5 specification. Worst case
HTML5.1.

------
danmaz74
Shouldn't an article about graphical programming show something graphical,
instead of just ascii characters?

~~~
Doublon
That was exactly my thought. This article is just plain text. It feels like
something is missing.

~~~
CmonDev
It's a "wouldn't it be nice" kind of article. Concept art would be nice even
if there is no intent to make a prototype.

~~~
pcmonk
Actually, I do have a prototype; I'm just not ready to release it yet. So,
it's probably best to interpret this as a general discussion of the subject.

------
erock
I've spent the past year "working" within graphical programming, and I have to
say it is a slow form of torture.

I have been "writing" IVR applications for a call center. The interface is far
from intuitive, and it is jsut too easy to miss what is actually going on. The
loops and if statements are all there, but you have to click through on the
item to actually see what is going on.

Forget trying to find an easy way to locate where something is. at least in my
case (VXML) I was forced to use grep to at least give me an idea of where to
look, then spend 10x the amount of time it would take in plain ol code to find
where to make the change.

ugh.

give me ASCII code any day

------
brudgers
[Terms of Service: Let me start by saying that I admire the author's passion
and effort to create something wonderful, and that this is not a crap on their
project, but rather a reaction to their essay. And the fact that their essay
raised a reaction to the point where I wrote something about it is, in this
case, a testament to its quality.]

Lisp programs can be represented as trees because they are trees. They can
also be represented as lists, and this is because they are also lists.

The important difference between these representations is that trees the
primary representation by which computers unravel the intent of a Lisp program
and lists are the primary representation by which programmers express intent
in their programs.

Sure some people sometimes picture their lisp program as a tree, but usually
when a programmer pictures something as a tree, it is when viewing it as a
data structure. Picturing a Lisp program as a tree is easier than picturing
the tree represented by code in most other languages because of Lisps simple
parsing process, but the way in which a programming language is parsed is an
arbitrary feature of the language [though one worth design consideration].

Although it is easy to draw a tree. Often in Lisp and other languages, the
verbal diagram "a list of lists of ..." is adequate to represent a tree along
with a description of its interesting properties. Again, while the interesting
properties can be represented graphically, what makes them interesting are the
maths underpinning their structure not the aesthetics of their spatial
representation.

 _" The purpose of this project is to assist the coder in developing more
complex code faster and easier"_

This is orthogonal to Rich Hickey's goal of making it easier and faster to
produce simple code.

[http://www.infoq.com/presentations/Simple-Made-
Easy](http://www.infoq.com/presentations/Simple-Made-Easy)

We already have a great tool which allows graphic methods to be used easily to
create complex code. It is called Excel.

~~~
pcmonk
Hm, interesting. I've always pictured my Lisp programs as trees. I think it's
a more useful representation, but I see what you mean.

You're right that this is orthogonal to Rich Hickey's goal. That's because I
think his goal is great for writing simple programs. His goal is to make more
programs simple, which is great. My goal is to make it easier to write complex
code. As Fred Brooks talks about in No Silver Bullet, there are some problems
that are accidental and some that are essential. Hickey is trying to solve the
accidental problemms, and I'm trying to make it easier to solve the essential
problems. Some logic is just inherently complex -- I want to create tools to
make it easier to reason about.

And Excel is not nearly programmable enough to be better than a textual
interface to solve the same problems. It's benefit is that it can be used
(relatively) easily by non-programmers.

~~~
brudgers
My impression is that Rich Hickey is seeking ways to allow programmers to
solve complex _problems_ , without writing complex _programs_. He's been
fairly successful relative to other people who write new languages.

Excel is entirely programmable using the standard .NET stack and languages
[from PowerShell to F#].

If the idea is to target the tool as a new IDE for experienced programmers,
how does improve productivity over EMACS or Vim etc.? Or without it being an
extension of one of those existing ecosystems, what is gained in exchange for
all that is lost?

Which suggests to me that incorporating the functionality as an extension to
one or more of those ecosystems might be more simple than competing with them
for experienced coders.

------
taeric
Again I look at how our industry completely ignores every other industry out
there. Some things do lend themselves to graphical viewing. Some things do
not. Many things lend themselves to a series of graphical representations. The
vast majority of things benefit from both graphical representations _and_
textual elaborations.

One of the things I wish programming tools offered, was a way to "zoom in"
details wise on a solution. More than just offering an outline, I want there
to be some sort of representation of how things went from conception to
realization. Consider, the story boards for movies/games. Those story boards
are lost in the final product. Worse, when you are looking at a piece of code,
you don't really see the "why" of what it was made for.

Sure, you get some of this with descriptive names, but that really doesn't
help with what I'm wanting to say here. An example, that I really want to
explore some more is graphviz. It can go from text to graphics. This process
is fairly static, though. If it doesn't turn out how you want it, you have to
edit the text and fire it again. I want something that can possibly go from
text, to initial graphics where I manipulate it some more, to final
representation all in a repeatable way. (I think this is possible, but I am
not familiar with the toolchain.)

More, there is no reason I shouldn't be able to start from graphics, go to a
textual details portion, at varying levels of detail, make edits, and view
what effect they have in the viewer.

Imagine, something similar to developer tools in chrome/firefox, where in
addition to being able to look over at the boxmodel for details, you can
physically move elements around.

Of course, I'm a bit of a heretic. I would kind of like to be able to export
absolute positioned style sheets for forms designed this way. :) Still not
sure on all of the reasons that would be terrible.

~~~
MetaCosm
... wanting Visual Basic doesn't make you a heretic, it just makes you aching
for a foregone era a bit. Visual coding has been pushed for ... at least 30
years.

The thing you are looking for is all over the place, in various IDEs, in QT
MOC, in literally DOZENS of VPL (Visual Programming Languages) and even to a
lesser degree in UML (Unified Modeling Language). Since we have QT MOC in the
mix, we could even through in ALL the meta compilation tools, which adds in
hundreds of others.

What you want isn't so much unique, as already done and discarded. You end up
with tightly bound IDEs and code, you end up with hard to manage binary
nonsense, you end up with ridiculously hard peer review and change tracking.

As for tracking the "why" \-- use any method you want -- my team uses -- story
boards! They work perfect for software.

~~~
taeric
I was not trying to paint my view as a unique novel thing. Probably what I
left out is "well done." I want this product in a well done package.

So, is there a well done version of something like this out there? I should
stress that I do not spend most of my time looking for this. I'm too busy
getting things done in my current flow to reinvent a new one.

------
CmonDev
This is what you get when you lock yourself in the *nix Vim-niverse. Try
Visual Studio 2012 Ultimate:

[http://msdn.microsoft.com/en-
us/library/vstudio/jj739835.asp...](http://msdn.microsoft.com/en-
us/library/vstudio/jj739835.aspx)

[http://msdn.microsoft.com/en-
us/library/vstudio/dd409365.asp...](http://msdn.microsoft.com/en-
us/library/vstudio/dd409365.aspx)

PS: pricing is the sad bit :(, but there is no competition in this area. I
hope JetBrains will work harder.

~~~
dman
If youre dependent on auto generated diagrams to understand the structure of
your code you have bigger problems.

~~~
sparkie
Diagrams may not be necessary, but we all depend on auto-generated
visualizations, whether it's a "find all references" feature of some IDE that
produces a hyperlinked list, or in the most primitive case, the use of grep to
produce a list of locations in files for some identifier.

I'd argue that you need help if you're not using some auto-generated
visualization to view relationships in code, because the alternative is to
scan through files manually.

The choice of diagrams, lists, or some other visualization is irrelevant, it's
the preference of the programmer.

------
outworlder
I can postulate that there is absolutely no way a purely graphical
representation of code will be the future. The same way we do not talk using
pictures.

A graphical language would be best used to display structure. Bonus points if
it allowed changing. The rest, modules if you will, can be better described by
text.

~~~
CmonDev
Don't want to argue with the spirit of your post, but Chinese hieroglyphics is
derived from pictures and it's probably somewhat representative of their
language.

------
svantana
Personally, I would prefer an IDE/debugger that allowed me to visualize my
data really well. This ought to be a lot easier also - I've done some simple
scripting with GnuPlot which has given me a lot of mileage. But imagine being
able to put a breakpoint anywhere in your program and interactively browse
your trees, dictionaries and arrays using OpenGL-accelerated real-time
graphics. I don't understand why xCode and VS aren't putting more efforts into
things like this.

~~~
CmonDev
Well actually Visual Studio team put efforts to address exactly this need:

[http://visualstudiogallery.msdn.microsoft.com/eedc48e7-5169-...](http://visualstudiogallery.msdn.microsoft.com/eedc48e7-5169-40d3-b602-ad451990a420)

[http://blogs.msdn.com/b/vcblog/archive/2012/07/12/10329460.a...](http://blogs.msdn.com/b/vcblog/archive/2012/07/12/10329460.aspx)

[http://msdn.microsoft.com/en-
us/library/zayyhzts.aspx](http://msdn.microsoft.com/en-
us/library/zayyhzts.aspx)

[http://visualstudiogallery.msdn.microsoft.com/c7e02633-86d9-...](http://visualstudiogallery.msdn.microsoft.com/c7e02633-86d9-4262-b745-6cc647afb3a8)

~~~
CmonDev
PS: if there is a need for something IDE-related it is either already done in
VS, available as a third-party extension or there is an API to build it.

------
super-serial
I've always thought that a visual interface for programming would help with my
procrastination problems.

It's so painless for me to open up a webpage and solve daily chess puzzles...
on the other hand it seems more painful to open up an IDE and start solving
code "puzzles." I wish I could visualize all the pieces of my software similar
to chess.

~~~
Kurtz79
I absolutely doubt that graphical programming, even in its best
implementation, will be about solving "code" puzzles.

Regardless of the interface, programming is "work", linking up different
interfaces, checking edge cases, conform to protocols...

It would feel more like glorified data entry than solving chess puzzles.

------
rtb
What's the difference between this effort and the countless Graphical
Programming efforts that have come before (many with large teams as well as
many other single-author efforts)?

~~~
Jare
This is still at the "idea" or "vision" level and hasn't failed yet.

Sorry about the snark, but while I'd love someone to succeed in this type of
quest, I believe this one is based more on ideology than rigor. Stuff like
"WYSIWYG editors are basically evil" just doesn't give me confidence.

------
collyw
Users will use Excel. That is the closest thing we have to graphical
programming that works. They don't need to understand data structures, just
columns and rows. Anything more than that, and you need to understand new
concepts. By the time you understand these, drag and drop doesn't gain you
much over typing.

------
sesm
Did some LabView programming as a student, didn't enjoy the experience.

Graphical programs are similar to circuits without memory, and thus are less
powerful then Turing machines. As a result, graphical programs quickly grow in
size, and growth happens in 2 dimensions, which makes it very hard to read
them.

~~~
Houshalter
To be fair, text is merely a special case of graphical programming where the
output is limited to a handful of characters and is completely linear. Is it
impossible to improve upon that? Syntax highlighting and indentation were good
improvements for example.

~~~
lmm
It's possible to improve, but I think the key is progressive enhancement. Look
at how IDEs tend to offer typing and implicit information in scala: it's
there, it's very useful, but it doesn't stop you from seeing the plain text
source code.

------
chipsy
I don't see graphics as a silver bullet, but I do think they're underused or
misused. Graphics provide a wider variety of feedback; they let you see graph
structures and summaries of data, and allow connections to be spatially
represented.

All of which means that they excel at connecting the big chunks of the system
that build on top of some model of service provision and protocol definition,
but poor at engaging in the "direct dialogue" of imperative or applicative
computation.

The part that makes people say "it never works" is that we're almost always
thinking from an inverted model, where we know how things scale up and down
with text, and apply that learned experience to graphical systems.
Subsequently, our engineering instinct leads us to put a similar degree of
granular agency into graphical coding and allow the standard constructs
(looping, branching, subroutines) to appear in it, which massively increases
the complexity of the system and brings us away from problem domain thinking.

My recommendation now is - figure out a data structure that could be
interesting to visually edit. Make a good editor for that. Allow imperative
code a runtime model for accessing it. Allow imperative code to be called from
it. Then it stands at least some chance of not being a bad wheel reinvention.

~~~
pcmonk
I understand what you're saying. My hypothesis is that (some version of) Lisp
itself, being a tree, could be interesting to visually edit. All my code, by
the way, can be directly accessed from Racket (and vice-versa), so it's
definitely not a requirement that the whole program be written graphically. I
want to make it a smooth transition from textual to graphical programming.

------
smoyer
In the '80s I was working on a (never released) system I called FlowPro, but
EGA adapters and 8088 processors weren't really capable of the kind of DnD I
wanted. The idea was that flow-charts would be turned into code, with some
meta data added if you wanted specific variable names etc. (instead of the
words used in the diagram's box).

Then in the early-to-mid '90s, I came across a product called Prograph that
ran on both Windows (I did a lot of programming for WFW with it) and Mac (OS 8
I believe). I loved that system, and realized it solved some of the syntax
issues I had using flow-charts, but it still never really took off. The
concept lives on with the Marten IDE though I'm not sure how many people are
actively using it.

[https://en.wikipedia.org/wiki/Prograph](https://en.wikipedia.org/wiki/Prograph)

[http://andescotia.com/products/marten/](http://andescotia.com/products/marten/)

------
TeeWEE
I worked with numerous tools for graphical programming, mostly for academic
work.

Such as graph-transformation based tools for software verification, in which
your programs are graphs.
[http://wwwhome.ewi.utwente.nl/~rensink/papers/spin2006.pdf](http://wwwhome.ewi.utwente.nl/~rensink/papers/spin2006.pdf)

Which is useful for software verification, but writing your actual program in
it is cumbersome.

Software is a lot about abstractions, and there is no reason that graphical
programs are better at representing abstraction that text (or maybe im wrong
here?).

In theory, that what you can represent in graphical-form you can also
represent in textual form.

However the information density with an 2D graphical form is much higher than
textual form. And the more information you put into the same space, the more
difficult it is to reason about it. Thats why textual programming is still
very much popular.

------
vinceguidry
I find myself looking for graphical interfaces to things I already know fairly
well but don't really care to drop into shell every time I need to use it. I
have an SSH tunnel manager and an rsync manager. I use Sequel Pro instead of
the mysql command line.

In fact, I really only find myself using the command line for, you know,
running commands. If I find myself there a lot, I'll check myself and start to
think, "can I start putting all these things I'm typing into a Rake task or
shell script? After all, the more I'm there, the more chance for user error.

I think the only major task I still use a terminal for is looking at log
files. I intend to eliminate that soon enough when I can 12-factorify the app
I'm responsible for and find a decent graphical log viewer.

------
buro9
Show, don't tell.

------
pizo
The idea of graphical programming seems in a lot of way like graphical
mathematics.

Works in some cases, but not in all.

I used graphics to represent databases, and their relationships, because it's
more easily understood than several hundred lines of code. It fails to tell
the whole story though, and thus only serves as a thousand foot view of what
is going on.

~~~
pcmonk
You're right about the similarity with graphical mathematics (and Bret
Victor's Kill Math project was something of an inspiration for this (even
though I have some fundamental disagreements with his methods), even though I
didn't see it until after I'd started it).

You've identified a critical problem with graphical programming today: it
fails to tell the whole story. This is exactly what I referred to in the post,
and that's not inherent in the medium; rather, it's just that people have been
trying to solve this problem in the wrong way.

------
lnanek2
Everything's solvable and everyone else is wrong, but he has none of his own
work to show. Very funny post. Maybe he's just a student? These constant
failures at trying to implement visual programming often seem to come from
academia where the goal is to write papers rather than actually accomplish
anything.

------
RivieraKid
I have been thinking about this too recently and I'm thinking about writing a
simple IDE to demostrate these ideas. If anyone would like to discuss this
over mail (or here of course), let me know.

------
lcedp
The future is what? Nothing's been proposed so far.

------
stinos
I doubt this is _THE_ future, and I also hope it is not: still too limited. As
far as I can predict, the future is a proper direct brain-machine interface.
Controversial yes, avoidable: I seriously doubt it. (would I use it: maybe if
it could be implemented using one way communication, ie similar to how it's
done now by analyzing EEG). I guess it would also be by far the fastest way.
At least I can think code (both concepts and actual implementation) way faster
than I can output it in any way.

~~~
voyou
I have in front of me a system that uses electromagnetic signals to interface
between my computer and my brain; part of it even works wirelessly - it's a
high-tech system called "a screen and my eyes." In what way is your predicted
"direct brain-machine interface" better than my actually-existing
electromechanical brain-machine interface?

: I can think code (both concepts and actual implementation) way faster than I
can output it in any way.

I doubt you can. You can have the psychological feeling of _thinking_ you've
developed the concepts much quicker, but in my experience it's only when I
start doing the actual implementation that the full level of precision I need
becomes clear, and its that level of precision that slows me down, not the
need to physically press keys.

~~~
stinos
_it 's a high-tech system called "a screen and my eyes."_

I meant the interface the other way around. From your brain to the computer,
without any mechanical inert pieces in between. As for the prediction part,
note that it exists already in a very early stage and it's extremely likely to
only grow. Can't find the exact artical atm, but it was used to move a mouse
on screen by someone who couldn't move anything below their head anymore.

 _I doubt you can_

Maybe not on all levels, 100% sure I can for some. Simple example: C++, I need
a basic iteration over a sequence. I just think 'ok gimme a for_each on
container Foo, calling function Bar'. No matter how advanced the intellisense
or whatnot is, it will take me more time to properly type the needed line.

------
kyllo
Good sell, I agree with all the points the author makes, but as Linus says,
"Talk is cheap, show me the code."

~~~
pcmonk
I'm working on it...

~~~
kyllo
Thanks, it sounds like a cool project. Please do post it here when you have an
alpha version, I'd be interested to see it.

------
bane
[http://en.wikipedia.org/wiki/DRAKON](http://en.wikipedia.org/wiki/DRAKON)

------
jdmitch
Are you looking for others to test/collaborate on what you have done so far?

------
seanmcdirmid
Was the article flagged?

------
Zigurd
There are some general problems with the way people think about graphical
programming. First, they should stop thinking about it as executable
flowcharts and take a step back: There must be a better way than typing
characters and pseudo-words to create programs. But that way isn't necessarily
any of the graphical programming systems anyone has yet come up with.

This article is a good example of thinking that's not even near the walls of
the box, much less outside. And what's that crazy digression about wysiwyg
editors? And then the author drops this one: "Thus, this isn’t really about
graphical programming." Derrrrrr.

We are barely at a point where we admit that evaluating a language syntax in a
vacuum is silly, so people are still surprised that "verbose" Java is popular,
despite the advantages static types and an intermediate language confer on the
editor, compiler, static analyzer, debugger, etc. A less than holistic
approach to graphical languages is probably doomed.

