I used a structural text editor a while back (technically a structural text editing mode for emacs). I went in completely buying the structural text editing propaganda (it's more logical, it will make you more efficient, it's the future...) but I eventually found the experience so excruciatingly bad and stupid that I now think the whole endeavour is a complete waste of time, the idea sounds good on paper but there are some fundamental unsolvable problems with it.
The problem is that, when you want to make a change to piece of source code you know its current state and you know how you want it to look like at the end. With a normal text editor you will simply make a bunch of changes that turn the text as it into what it should be, it will normally go through a series of invalid states but you don't have to think about what those states will be or care about them. With a structural text editor this is impossible, invalid states can not be represented so when you want to make a change to your source code you don't just care about the beginning and end state, you also want to find a series of valid states that takes you from the beginning to the end where each one can be reached from the previous one with an editor command.
For example, let's say you have a function call:
(afunc 1 2 3)
and you want to turn it into a conditional function call based on a flag:
(if flag (afunc 1 2 3))
with a normal text editor you:
1. type "(if flag " before the call
2. type ")" after the call
It doesn't matter the order in which I do these operations or how I move the cursor.
With a structural text editor, I have to:
1. create a new form before the function
2. type "if flag"
3. select the entire function call
4. cut it
5. paste it as the first branch of the if
This is however only possible if the context where the function call currently is allows adding a new node before it (i.e. it's inside the equivalent of a lisp progn). If not I'd have to use a different strategy, for example:
1. select the entire function call
2. use a command to create a node around it
3. type "if flag"
I did this for a few months, eventually I realized that I didn't want to think about the syntactical validity of the intermediate states my program goes through while I'm editing it, because they exist for mere seconds, and I didn't want to memorize the large palette of commands necessary to create those intermediate states. It was all additional cognitive load that accomplished nothing but satisfy someone's esthetic sense that source code should always be in a syntactically valid state.
So of the benefits listed in the article:
1. "No syntax errors" is actually a major detriment of the structural editing model
2. "Better type information" is a lie, all that the structural text editor guarantees you is that the program is syntactically valid, not that it will pass type checking
3. "No keywords" is a lie, it's only true if you use a new programming language and don't store it on disk as text which opens a whole other can of worms (can't even read source code without a special text editor, won't work with github, won't work with any versioning system, won't work with grep, can't copy paste a chunk of it in an email...)
4. "Rich visualization" is an entirely different story from structural text editing but as far as I know nobody has ever shown visual programming to actually provide any kind of tangible benefit
5. "Reduced complexity" is the one I give them, yes if you are making a structured text editor you don't have to write a fault tolerant parser, it does make your (aka the text editor's implementor) life easier
Fans of structural text editors usually mention that they will unlock "more powerful refactoring tools" but actually all you need for those to be "unlocked" is a dumb parser and an automatic code formatter, like "go fmt". Refactoring a function to be inlined into another is still a difficult operation to implement correctly but it isn't like the difficult part is running a parser on the source code, it's dealing with variable shadowing.
I agree with you on basically all points. But to play devil's advocate, I've often found myself doing something like structured editing when intentionally, carefully refactoring existing code. In those instances, it really matters to me that the programs at the start and at the end are equivalent, or at least equivalent up to certain semantic conditions (for instance: preconditions, things assumed by the domain, or cases not meant to be handled in the subject code). Since making a complex refactor in one step doesn't make me very confident that I got it right, I usually carefully perform a series of small transformations such that, at every step, I have something that manifestly behaves the same as the program before that step.
This seems sufficiently like the structured programming model (small edits forming a path of "valid" programs from start to finish) that I wonder if something similar to structured programming might be genuinely useful in this niche. That said, these refactors aren't simple atomic tree edits; they're more like a small cluster of tree edits that together preserve meaning. So there's a difference in content, even if the "small edits that form a path through programs with X property" mental model is the same.
This is a pretty fair comment. I wonder how structural editing figures out things like formatting/searching/diffing and copy/paste across different editors.
Those are all orthogonal concerns to the way the text editor behaves. You could have a structural text editor operate on plain text files and a dumb text editor operate on AST, you can do plain text diffs of ASTs or diff AST parsed from plain text files, etc etc...
Some of these have existed historically (or even still exist).
Paredit-mode is a structural text editor that saves and loads plain text files, Smalltalk was typically implemented as a dumb text editor but then code was saved as compiled binaries (which meant you couldn't save your functions if they weren't syntactically correct but you could have unsaved syntactically broken functions), Mathematica represents its code in a weird format that might as well be binary but copy/paste converts to plain text, there's one git plugin (don't remember the name right now) that does syntax aware diffs even though git deals with plain text...
You missed all the cursor movements (whether by mouse/keyboard) needed for the plain text editor:
1. Position cursor before the call from wherever it is right now (might be quick because your editor already allows some form of structure in it's movement commands: words, parentheses, visual jumps, etc or you might need 10+ presses of the left arrow)
2. Type "(if flag "
3. Position cursor after the call - maybe just a press of "End"/$/shift-A/...
4. Type ")"
Your two ways of structural editing seems to be an artifact of Lisp - I agree that most programming languages have issues when editing structurally - especially when using an abstract/concrete Syntax tree as the structure as compared to a non-syntactic representation (e.g. expression graphs where only types need to match).
So when this weird difference does not exist, you can always use the second strategy you listed. This already includes necessary cursor movements/selections and is one step shorter than the commands necessary for textual editing.
I think that any editor will always have to deal with partial programs - it's just the holes/errors are in different parts. The right UX can make dealing with that possible, just as auto formatting, syntax highlighting, inline type display, etc help somewhat deal with parts already in existing editing systems.
I think there's plenty of ways to make the UX work - for a programming language and ecosystem designed for it. So I think that structural programming should be a complete environment - programming language, editor with a good editing meta-language mapped to keys/auto complete/etc, language server equivalent, structural version control, build system, code sharing/review (diffs!) and legacy interface to textual tools. There'll be a tipping point where one has enough benefits from the structure in each of these to overcome the unavailability of many existing tools like grep/etc. A single structural editor experiment likely isn't at that tipping point unless the UX is really a significant improvement like github.com/cursorless-dev/cursorless is for many folks.
The problem is that, when you want to make a change to piece of source code you know its current state and you know how you want it to look like at the end. With a normal text editor you will simply make a bunch of changes that turn the text as it into what it should be, it will normally go through a series of invalid states but you don't have to think about what those states will be or care about them. With a structural text editor this is impossible, invalid states can not be represented so when you want to make a change to your source code you don't just care about the beginning and end state, you also want to find a series of valid states that takes you from the beginning to the end where each one can be reached from the previous one with an editor command.
For example, let's say you have a function call:
and you want to turn it into a conditional function call based on a flag: with a normal text editor you: It doesn't matter the order in which I do these operations or how I move the cursor.With a structural text editor, I have to:
This is however only possible if the context where the function call currently is allows adding a new node before it (i.e. it's inside the equivalent of a lisp progn). If not I'd have to use a different strategy, for example: I did this for a few months, eventually I realized that I didn't want to think about the syntactical validity of the intermediate states my program goes through while I'm editing it, because they exist for mere seconds, and I didn't want to memorize the large palette of commands necessary to create those intermediate states. It was all additional cognitive load that accomplished nothing but satisfy someone's esthetic sense that source code should always be in a syntactically valid state.So of the benefits listed in the article:
1. "No syntax errors" is actually a major detriment of the structural editing model
2. "Better type information" is a lie, all that the structural text editor guarantees you is that the program is syntactically valid, not that it will pass type checking
3. "No keywords" is a lie, it's only true if you use a new programming language and don't store it on disk as text which opens a whole other can of worms (can't even read source code without a special text editor, won't work with github, won't work with any versioning system, won't work with grep, can't copy paste a chunk of it in an email...)
4. "Rich visualization" is an entirely different story from structural text editing but as far as I know nobody has ever shown visual programming to actually provide any kind of tangible benefit
5. "Reduced complexity" is the one I give them, yes if you are making a structured text editor you don't have to write a fault tolerant parser, it does make your (aka the text editor's implementor) life easier
Fans of structural text editors usually mention that they will unlock "more powerful refactoring tools" but actually all you need for those to be "unlocked" is a dumb parser and an automatic code formatter, like "go fmt". Refactoring a function to be inlined into another is still a difficult operation to implement correctly but it isn't like the difficult part is running a parser on the source code, it's dealing with variable shadowing.