Excellent exploration of alternatives to the old 'terminal' application. It would probably help for context to explore the Bell Labs 'BLIT' terminal and the Digital Equipment Corp VT340 (color graphic terminal) and the Tektronix 4010/4014/4064 terminals. These had capabilities that augumented the basic terminal stuff and made it prettier at least to use them.
The humorous thing for me is seeing the discussion which essentially devolves into creating a new windows type metaphor but with a lot of keyboard shortcuts.
It may sound pedantic but consider that the terminal is the terminal because terminals define a keyboard driven user interface. A user interface is a 'vocabulary' which describes actions. Its vocabulary scales from a communication channel that is 300 baud to one which has multi-megabaud connectivity. You can create a different vocabulary (which is what the other terminals above did) and get additional features.
And you can create an entirely new vocabulary if you're willing to change the minimum connectivity channel bandwidth as well (I doubt TermKit would work well over 300 baud for example). There was some good work done on this in the 80's with 'visual shells' on the PC and of course there is a ton of work that IBM did in creating optimized terminal environments (with visuals no less) on 3270 terminals which were in many ways a pre-cursor to todays IDE tools, although perhaps more like Eclipse. Their environment which ran airline reservations was a good example of optimizing the vocabulary (key strokes) against the variety of tasks an agent might need to do.
So rather than calling this a graphical terminal replacement (which it isn't) calling it a visual UI which can be driven entirely by the keyboard might be better. I think that fundamentally the "thing" about terminals that everyone sort of 'gets' is that you don't have to use a mouse, you always know where your 'focus' is, and there are millions of key combinations which you can use to express certain actions. For some, that is simply 'living in emacs' for others it could be something like this.
I bet TermKit would actually work awesome over 300 baud. The two-tier approach -- a backend that executes commands and a frontend that displays stuff -- is going to be pretty efficient (if you are only running the backend remotely). Well... perhaps the chattiness of something like filename completion will be a bit of a problem, but it'll still probably work better that more directly interactive systems because it's working with a more abstracted protocol than a terminal does.