"Please go to the supermarket and get two bottles of beer. If you see Joe, tell him we are having a party in my house at 6 tomorrow."
It took me a few seconds to write that. Imagine I had to paint it.
in my mind, the best case scenario is some sort of hybrid, where visual and text representations take over on what they're best at, yielding to the other when it makes sense. the way people work on mathematics is a great example of this. drawings, diagrams, graphs, shapes, etc. are used to convey structural information where text and symbols are used to convey more detailed, descriptive information.
Does everyone do that? I think I very rarely do. When I need to convey an idea on a whiteboard, it's still almost completely text, with a few extra symbols like directional arrows that I think are still effectively just symbolic written language.
Literally the only time I can remember sketching a picture to convey an idea is while playing Pictionary. The entire premise of that game is that it is much more difficult to get someone to say a specific word by drawing a picture than by simply writing down the word.
In contrast, the instructions for the Linux version of the same thing will be something like:
sudo apt-get install this
sudo bing bam boom
GUIs are still much better than a command line in this respect, since with a GUI it's actually possible to find what you're looking for by randomly exploring, even if it's not always quick or practical.
One thing that's rapidly improving on the command line side is Search. On my modern Windows computers (8 and 10), I don't search for things by hand in the dialogs any more. I press the Start button and start typing text.
For instance, Start plus "device manager" brings up the device manager, whereas I have no idea how to find it in the menus.
I've also found it very useful when people are asking for things that don't entirely make sense, such as a chart/report that would require 3+ dimensions. By asking them to draw an example, they realize the challenge and we can make progress.
It depends on context. If you want to show how and UI looks, then it's probably faster and better to just draw it, but if you want to show how function transforms a string, then it's unnecessary to draw bunch of stuff.
It's the same as saying that drawing a picture is easier for most people than programming a picture, and it doesn't really have to do with the topic of visual programming, but rather visual design.
It can be, though the type of mockup I find more useful for discussion is much lower fidelity than that.
The feedback and discussion around a mockup is related to the fidelity.
Low fidelity, such as whiteboard or large sharpie drawings, will garner feedback that discusses whether this UI even makes sense at all, if there's a different approach, or how feasible it is to present everything this way.
Mid-fidelity, such as digital mockups, will often focus more on the actual layout, text labels and contents.
If you build a high-fidelity mockup when you really want to have the low-fidelity discussion you're doing everyone a disservice. Even if your team can see past it and still has the appropriate discussion, it's a waste of time.
For that reason, even when my mockup is basically a small modification to something that exists already, I'll often screenshot it, and then freehand draw (MS Paint style) overtop of it with my modifications, using ugly 2-4px wide lines. I've tried both, but consistently found the feedback is better that way -- even when the target audience is a bunch of developers that should be able to see past it.
Seems like you've done quite a bit of this type of work, I've the exact opposite. I guess it makes sense that the fewer data points the clients are given, the more they focus on the broader picture.
Would you often start with a napkin drawing-esque mockup, and add detail to it while cooperating with the client, or would you sometimes start with a finished HTML+JS+CSS prototype and change things up as they request?
That said, some people just can't picture how things look unless you show them how it looks, and in the past I've worked with clients that would think it's literally a joke (and probably fire me when they found out it wasn't) if I come to them with a sharpie drawing.
I think the only times I've gone to a more high-fidelity version up front is when the client/stakeholders had no idea what they really wanted. A "finalized" mockup can start the discussion (even if it turns out to be entirely wrong) and then be used to get into the low-fidelity whiteboard/sharpie mockups. The important thing is to keep the discussion focused on the right thing, and don't stray into fonts/images/icons if you want to talk about the overall structure and what pages even should exist.
I would be hard pressed to discus the operation of any
sort of datastructure beyond a linked list without sketching. For example a binary tree and it's traversal routines are (at least to me) much easier to understand with a picture along side the exact state.
My idea-book (is that a crazy person thing?) is probably a lot like what you're talking about. I use words for my main data, different arrows going in between each sub-idea, and then sometimes more words attached to each arrow ("pub/subs", "queries", "happens once/happens always", etc)
I think that people who think visual programming could be a thing ought to sit down with APL for a few weekends, until they have the epiphany of "oh, the code is the same as the notation for what I'm actually doing" and suddenly they realize that notation and code are interlinked, and each are basically useless without the other, in the same sort of way that x--p--p---- is exactly as valid as 2 + 2 = 4, without context.
I get the allure of NOT having to write code, but it shouldn't be so difficult for people to realize that it's a ridiculous fantasy.
Write fibonacci(N) in ANY visual language, and tell me you couldn't have done it easier, faster, and more coherently in python or whatever. It's obvious.
  [x]
Because pictures are helpful, but neither necessary, nor, in most cases, sufficient. Just look at the use of pictures in science and technology generally (e.g. circuit schematics, Feynman diagrams), and try to imagine understanding what is being presented without reading the words.
I have seen examples where pictures can help a lot (I am thinking of a pictorial demonstration of the Pythagorean theorem, for example), but I am not aware of a constrained pictorial formalism that is both general and expressive enough to do the job of a programming language (directed graphs may be general enough, but are not expressive enough; when extended to fix this, they lose the generality.)
>The complete dismissal of visual programming by text-based programmers is often infuriating.
I, for one, would be thrilled to see an example that actually works. Arguments that it should work don't count for much, otherwise.
There are some hybrids that are pretty useful in their areas of applicability, such as state transition networks, dataflow models and Petri nets (note that these three examples are all annotated directed graphs.)
Basically, painting/diagram is good for more abstract overview, and text/code is for more specific instructions.
>Basically, painting/diagram is good for more abstract overview, and text/code is for more specific instructions.
very well Said ..
Beyond that, though, even when a sketch is a useful tool, I find I seldom use the computer to create it. A whiteboard, notebook, or back of an envelope all seem easier than working with typical graphical input mechanisms that are commonly available. Good touchscreens are a step in the right direction, but still far from perfect and far from universally available, especially on desktops. Aside from any innate problems with graphical programming in general, I think this input problem makes the approach generally unappealing at the moment. It's not an insurmountable issue, but until there are good input mechanisms that are as common as keyboards, I think you will just find that it is hard to reach the critical mass a new representation needs.
I know that input speed is not really a limiting factor in most programming, and maybe shouldn't be the deciding factor, but given how attached to their text editors programmers already are, and how disruptive and frustrating it can be to have to switch, I think it's clear that it is an issue.
Most visual programming efforts focus on representing algorithm logic with diagrams. But I think the difficulty to understand most complex programs (from my own experience) lies in the complexity of data structures, not the code logic. Reasoning about the algorithm is like reading a story, it is kinda intuitive; but for a large data structure (say a big struct with more than 20 fields), it's very hard to keep all those in your memory, you always have to go to its definition to refresh your memory, very frustrating. One of my tricks for reading a large open source project is to first find all the key data structures, draw them and the relationships on paper, then reading the code becomes easier and more pleasant. So maybe if a visual programming UI displays data structures with a visual format always on screen, it can help boost coding productivity.
That still means you need writing to express all the intricacies of programming the details.
Alas hybrid languages where you can link developed blocks visually exists and works (bpel, webratio), but no pure visual language could let you describe the intricacies of taxation in a sane way
If we use text for specific and visual for abstraction, we still need to maintain both and there are no effective tools to do that without creating additional work.
Consider this list as a ranking: 0 and 1 >> alphabet >> Chinese >> picture.
All 4 methods can be useful in some cases. Chinese has tens of thousands of characters, some people consider the language close to pictures, but real pictures have more than that （infinite variants).
Chinese is harder to parse than alphabet, and picture is harder than Chinese. (Imagine a compiler than can understand arbitrary picture!)
(and sane programmers use visual layout to convey structure in every language)
Are there other examples of those cases?
Edit: maybe I was presumptive to assume that they had to be "general purpose". Are we talking about more narrow domain-specific languages here?
But, I get that there is a class of visual languages, often dealing more with interconnected nodes like a directed graph, for which graphics might be better at communicating. But they seem more domain-specific rather than general purpose programming (like most text languages).
The visual language I always think of is LabVIEW. It is very different than a textual language.
Erlang (Prolog) for example doesn't so much. Sure you have parameters, but matching blocks aren't strictly evaluated in sequence.
LabView is more of a data flow graph than a control flow graph, aside from the boxes that represent deviations from the usual rules (For loops, conditionals, sequences, etc.)
This isn't as much pedantry as it might seem... this is one of the big differences between LV and 'traditional' programming... LabView evaluates its data flow graphs in data-dependency order, which can also mean evaluations are done in parallel. (IIRC, for linear execution speed, wires in LV all have a fixed type and blocks can be compiled to machine code with an LLVM back end.)
Are programs text, like a novel, or is it data? And to find out, would you query Moby Dick? Not really -- at least that's not how it's meant to be consumed. You read it from beginning to end.
Would you read your program from beginning to end or would you query it? You query it, exclusively. I submit that programs are data, not text, and visual programming then becomes like data visualization. Text is holding us back.
Accounting uses numbers, which in this context is text. If you replace a spreadsheet with a graph, your accountants will murder you.
Ha! Nice try. You program in a text file. You don't get to somehow call that a spreadsheet just to make your argument sound better.
My whole point is that we need to turn that free text into a spreadsheet (from which graphs can be produced) and "your accountants will thank you".
Is it possible that nearly every professional is right and you are wrong and text is an effective means to represent computation.
I completely understand where you're coming from. I'm saying, "try to imagine how it should be," not, "how it is." Those experiences are horrible because current spreadsheets are for simple functions and nothing more.
Is it possible that nearly every professional is right and you are wrong and text is an effective means to represent computation?
I guess this is where we disagree. I see the software industry as a mess and every professional as a person who was given one part of a Rube Goldberg contraption to be an expert on. The contraption works, to be sure, but it boils an egg in a hundred arcane steps. Given the state of our art, we have no leg to stand on to say what is and is not an effective means to represent computation. Normally a billion professionals would lend clout to your argument, but in the case of our industry, we know that they just staff a million abstruse contraptions. It just means we lack imagination and ownership: we're professionals, yes, but on whatever junk we're handed.
It wasn't always like that. We have seen that Computer Science doesn't really advance, but what is fascinating is that maybe 90% of those pushing for it to advance are of the original, older, generation that brought us that innovation in the first place. Max Planck once said something to the effect of, "Science advances one funeral at a time." I worry that Computer Science retreats one funeral at a time. Slowly we're losing those from the time when our thinking wasn't constrained with current tools; failed efforts were taken as learning points, not barriers; and no one viewed current technology as something the gods handed down to them but as something they created (maybe the day before), so they were constantly evaluating it.
Yes, the future of software development has to be away from free text toward data, and it follows as a logical syllogism. I was about to add that that is my opinion, but it's not so much of an opinion as a conclusion. While it's absolutely fair to wonder what that means, we can't brush it away because we already have the best Rube Goldberg contraption we could ever have.
Consider, how long does it take you to draw an infographic representing the percentage of the United States that has a college education? How long does it take to modify it when you realise it is a mistake
Now how long does it take you to represent/change that data with text?
Right, code is data.
Not now, though. Currently, code is text -- it's not data. That's the problem. And, yes, currently, drawing it instead of typing out the text is a horrible idea. We agree.
But remember, currently, we're failing. Software is a disgrace of an industry, in all honestly. It's mired in undue complexity. It's a Rube Goldberg contraption that we all, somehow, take seriously.
Imagine a better future. Imagine visualizing a complex set of rules and seeing all the places where a change would cause knock-on effects. Imagine being able to manage your code base in a way that doesn't require you to download it all into your head. Hell, imagine a decent IDE. The first step for all of that is to move it from text to data.
While i submit to the idea that programs are "data", and could be represented not just like text, the expressiveness when using a visual representation is small, but the exactness is as high as normal languages.
I've seen large visual-programming programs, and they explode into complexity, and I think there is a reason for that, the expressiveness of a symbolic language using text is so much higher.
Could we increase the expressiveness even further? English have high expressiveness but is inexact.
I don't think it's possible to increase the expressiveness and keep the precision - at least without some sort of intelligent agent that can reject interpretations that makes no sense.
So - when general AI is invented, we can probably soon write programs that are very inexact, highly expressive, but still work.
Until then, we will have to continue wrestle down the computer to obey our if-statements, one by one.
E.g. the image literal could actually be Image.loadFromPath("../.../image.png"), but you don't see the code unless you want to; and if you drag and drop a file into your program, the IDE automatically inserts the corresponding code.
Open up powerpoint, openoffice, or whatever you use to make infographics.
Also open up vim, atom, or whatever you use to edit text.
Time yourself clicking and dragging to make a graph in one and then typing to write a number in the other.
> Imagine a decent IDE
I'm imagining something where you are able to select visual elements without either clicking and dragging or doing an O(n) walk by hitting alt-tab. But I can't imagine how you acheive that, much less efficiently specify how you connect things up and how you write automated tests.
Some of this is my failure of imagination, but its still on you to provide the sketch of a proposal if you want to convince.
I agree that a concrete product is probably going to be required to convince people who aren't good at thinking outside of what already exists, but these kinds of comparisons just feel absurd. Maybe a better way to think about this sort of thing is "how can the structure of code be used to simplify and improve working with it"; the term visual programming is just too loaded in negative ways.
Edit: I don't mean to sound insulting when I talk about thinking outside of what exists. It's not always an easy thing to do. But it becomes an issue when you it causes you to let previous attempts narrow your views on what something is.
Visual programming doesn't have to be, say, exactly what's scratch is today. It doesn't even have to be close. The two keys IMO are utilizing the structure of code as well as the inherent visual processing power humans have.
This doesn't mean no text, it means expanding how we think about operating on and viewing code in a way that makes it easier (and thus more productive) to do.
Actually, our writing system evolved from accounting tokens. Eventually we learned we could save on little figurines by pressing the one into clay a bunch. And at any rate, writing encodes speech and is severely limited to the limitations spoken speech has. Rather than be well-optimized, writing is severely limited by tradition, by the physical limitation of auditory communication, and by the physical limitations of encoding it into paper.
There might be other cases where this is not true, but I think the major gains with a proper visual environment would be on a higher abstraction level. Visualizing the architecture of a project, tracking dependencies, etc.. Basically tasks where your "takes longer to draw" argument is nullified by the fact that it is the computer that does the drawing.
"Joe has been way too busy yesterday. What did you ask him to do in the last week?"
"Yeah, that's too much. Just ask Jessica to do the emails going out next week."
Programming in text, you have to learn, in advance, what the commands are that you're allowed to type, and remember them. That's a heavy up-front investment.
Visual programming generally makes it much clearer exactly what your range of options are -- the goal is for the learning curve to be far easier.
It's obvious that visual programming is much less expressive than writing code, but it's attractive to people that are afraid of code or are too lazy to learn a new language.
I've used many visual tools where every block you laid out had to be properly connected, so in order to refactor it you had to make dummy blocks as input and output and all other kinds of crap. Adding or removing arguments and return values of functions/blocks is guaranteed to give you rsi from excessive mousing.
Here, that was quite fast to paint. Would have been even faster with a proper brush instead of a keyboard :p