Personally I think syntax-directed programming is the only thing that makes sense. It's 2014, why are we still programming using a batch workflow? Why are we not in a live feedback loop with the compiler?
There are natural answers to this of course - programming is insanely complex. I think history will look back at this time and see that we were struggling just getting programming to work at all. Making a decent syntax-directed programming environment would add so much more complexity to something that's already hard.
We don't even have the tools necessary to get started - what about an exchangeable tree format that doesn't suck?
I've played around a bit to create an AST in XML by hand, and using CSS to make it look like code. It's really interesting, suddenly you see that you're freed from caring how the syntax is formatted.. want curly braces around your blocks? Sure. Want just indenting? Have it your way. Want your function names in a larger font? No problem
But XML and CSS is not good enough. Not by far.
The other issue is that we've gotten really good at making text editors, and we know how to use them well. But tree structure editors? I don't even know of any. I suspect we'll always be editing parts of the tree as text anyway, especially expressions, it's just natural when the primary interface is the keyboard.
We're getting there though. Eclipse, VS and XCode goes quite far towards understanding the syntax and semantics of the code, and I understand that the primary motivator for Apple to go to LLVM was being able to integrate the compiler tightly into the IDE.
I'm sure many here have seen it already, but I would recommend looking at Bret Victors work for an idea of how things could be:
> Why are we not in a live feedback loop with the compiler?
> what about an exchangeable tree format that doesn't suck?
> create an AST in XML
> But tree structure editors? I don't even know of any. I suspect we'll always be editing parts of the tree as text anyway, especially expressions, it's just natural when the primary interface is the keyboard.
I really think it's time for you to discover Lisp ;). Take a good look at the language, s-expressions (your "tree format that doesn't suck"), Emacs with SLIME and paredit (the tree structure text editor with live feedback loop with the compiler).
They are quite familiar with edit and continue (hot code replace). But they want the full Bret Victor experience, not some lesser variation that was possible to implement with 70s-level technology.
We can surely be inspired by the past, but let's not get stuck there.
Sure, but the thing is that problems audunw wrote about were all solved in Lisp and/or Smalltalk. People today keep reinventing things that we had in 70s, except those things from the past are much better, because when they were invented computers were slow and the industry wasn't run by fashion, so they had time to think about what they're doing and what problems they are trying to solve.
I'm not saying we should stay stuck with 70s tech; I say people need to actually look at it. It feels to me like breaktroughs in the industry come from recycling Protean technology, except the Antients who made it didn't live millions of years ago; they were likely drinking beer with your parents.
So before greenspuning a half-assed, bug-ridden clone of Paredit and calling it a Revolutionary New Editor, please, please look at the more geekier corners of the past. Realize, that we already have good solutions for all the problems you're trying to solve (audunw, in this case). Sure they're not perfect. But it's better to learn from the mistakes and improve on something solid than to run in circles.
They weren't better, there is just lots of rosy tinted tunnel vision from aficionados. We had good reasons for not going in the directions tried out in the 70s and 80s (e.g. Smalltalk images), while some of those directions have been nicely subsumed into current IDEs (e.g. Smalltalk's fix and continue).
If you truly think the old systems were better, then prove it! Don't just tell me about some magical system that existed 25 years ago. Show me.
> We had good reasons for not going in the directions tried out in the 70s and 80s
Those reasons have nothing to do with feasability, or usefulness of those ideas, and everything to do with popularity contest our industry is. It's not the best solutions that win, it's the most fashionable.
To give you one example - look at how suddenly functional programming is the hottest thing around. Out of the blue, FRP is the thing to solve all problems. It's 1970s technology. Just rediscovered now. I remember the times when functional programming wasn't hip, and I was getting laughed at for being a "Lisp (and Erlang) aficionado". But time has passed and the same people who were laughing then are now going to every possible Scala meetup and conference, because FP is now fashionable.
But we're getting a bit tangential here. The core idea of my original post: manipulating ASTs and tree structures in editor is a problem with old and good solutions most people don't know about, so, people, please go learn about them. It might save you some work (and spare the rest of us the horror of coding in XML because it's the next hip thing).
> If you truly think the old systems were better, then prove it!
I think there's only one way to do that - you have to play around with them, experience them yourself. Find yourself a local Emacs-using Lisp aficionado and let him show you the tricks, both in the editor and the language.
I've heard many times that syntax direction is too strict and break people flow. The article says it's suited to students, here that wouldn't be an issue. That said, I'm pretty sold on the structure/tree editors vs char buffers, but I think there's need and opportunity for ideas to make building syntactically and type correct programs without slowing too much the user stream of ideas that can be random and at temporarily incorrect.
Indeed, I suspect in the beginning, you would want to convert the tree back and forth between text and structure, so that you can edit parts of it in text. Editing text is a bit like working with a scratch pad, you're jotting down ideas. But once you're down with a block or function, you want it to be correct.
The structure would then be for the editor to give you instant feedback on the syntax and semantics, for navigation, and for refactoring.
In a way, the most advanced text editors and IDEs already do something much like this, but it feels like they are still closer to a text editor than a pure structure editor.
The point is also that the editor is working on the exact same structure as the compiler, so the compiler can give you instant feedback on things you actually have to compile the code to find out. You could have dummy inputs to your function for instance, and instantly see what all the variables evaluate to, how long it takes for the function to execute, and how much space it takes.
Maybe it's CISC vs RISC all over again. That said I don't want to serialize subtree into linear strings to be able to insert/delete a few things. It's information loss to me. I envision something like subtree derivations to "fork" the current code block into a new one and substitute some tokens (not strings). Basically interactive tree rewrite.
Derive tests on the fly, add some visual cues (diffs, types, errors). Be creative, use a layout abstraction like DOM or even higher to experiment freely.
I wrote ALICE -- back almost 30 years ago, so I was surprised to see some traffic on it. Obviously today I would do it a bit differently, I wrote it in the pre-mouse days, and the PC version you can download was designed for a trackball that just worked like arrow keys. To see a later version, run the Atari ST version on one of the many Atari ST Emulators. I've found the source of the ST version and if anybody wants to adapt it to modern WIMP environments that would be cool.
I designed ALICE so you could type to it the same stuff you would type into a regular text editor while coding and it would do generally the write thing. Editing, however, was done on the tree and you had to understand that, though you could edit expressions as text. Later I believe there were SDEs that let you edit just about anything as text if you wanted to and they figured it out -- but I had to write ALICE to run on computers with 128kb of memory.
As far as I can tell, Alice pascal provides the first instance of code completion, which is why it's significant historically. Quite an accomplishment.
There are natural answers to this of course - programming is insanely complex. I think history will look back at this time and see that we were struggling just getting programming to work at all. Making a decent syntax-directed programming environment would add so much more complexity to something that's already hard.
We don't even have the tools necessary to get started - what about an exchangeable tree format that doesn't suck?
I've played around a bit to create an AST in XML by hand, and using CSS to make it look like code. It's really interesting, suddenly you see that you're freed from caring how the syntax is formatted.. want curly braces around your blocks? Sure. Want just indenting? Have it your way. Want your function names in a larger font? No problem
But XML and CSS is not good enough. Not by far.
The other issue is that we've gotten really good at making text editors, and we know how to use them well. But tree structure editors? I don't even know of any. I suspect we'll always be editing parts of the tree as text anyway, especially expressions, it's just natural when the primary interface is the keyboard.
We're getting there though. Eclipse, VS and XCode goes quite far towards understanding the syntax and semantics of the code, and I understand that the primary motivator for Apple to go to LLVM was being able to integrate the compiler tightly into the IDE.
I'm sure many here have seen it already, but I would recommend looking at Bret Victors work for an idea of how things could be:
http://vimeo.com/71278954