- Visual programming tools usually only work at one abstraction level (your first point)
- Often centered around a gimmick and cannot encode "normal" detail-oriented imperative code efficiently
- Only a fraction of a given program is amenable to flow charts, Visual Basic-style forms, etc. Unfortunately, most visual programming environments are all-encompassing.
- So VPs tend to become blunt, awkward tools that lack power of expression, ergonomic editors, scalability to real usecases, etc.
FWIW, I do believe that a better code editor is possible through non-plaintext programming. (Serialize token trees and ASTs, instead of plaintext.) But such an editor would likely be used in text-editor-mode >50% of the time, just due to the high information content and sheer readability of text.
Funnily enough, this is itself another of those mirages, like visual programming, that have been chased for years and failed to gain traction (outside specialised applications).
For my money, the reason for this is that a human editing code needs to write something invalid - on your way from Valid Program A to Valid Program B, you will temporarily write Invalid Jumble Of Bytes X. If your editor tries to prevent you writing invalid jumbles of bytes, you will be fighting it constantly.
The only languages with widely-used AST-based editing is the Lisp family (with paredit). They get away with this because:
1. Lisp 'syntax' is so low-level that it doesn't constrain your (invalid) intermediate states much. (ie you can still write a (let) or (cond) with the wrong number of arguments while you're thinking).
2. Paredit modes always have an "escape hatch" for editing text directly (eg you can usually highlight and delete an unbalanced parenthesis). You don't need it often (see #1) - but when you need it, you really need it.
Bags of characters <-> Unstructured trees of tokens <-> ASTs
(BTW paredit is super cool and I'd like to see more of its kind!)
This is a well-trodden road. It starts with "wouldn't it be awesome if we could manipulate everything as ASTs", then usability intervenes and we fall back to "well, we need to be able to selectively edit as text", which means you need to be able to convert everything to bags of characters and back. And now you've built that conversion, you might as well represent the "source of truth" as bags of characters like everyone else does.
(And heaven forbid you should want to make a checkpoint commit that doesn't parse...)
- Using Merkle trees, we can assign every node a hash-based ID. So now we can refer to other nodes by ID. This lets us store a graph whose vertices are all the tree nodes.
- With the graph, we can now reference bindings not by string literal, but by ID. This eliminates shadowing problems and missing imports.
- There is now one source of truth for the names of variables and functions. As a result, in source control, a commit that renames something is a one-line change.