I eventually implemented this as an Emacs package called mugur. With mugur, you only have to define one big list in your init.el file, with all the layers and keys in a single place. No other extra code needed. After playing around with it for a bit, I find it is easier and faster to explore all kinds of usage scenarios to squeeze every bit of potential from these wonderful programmable machines.
I'm already using mugur to generate the Ergodox configuration that I've used to write by hand before. By hand, meaning writing C code. I'm expecting there to still be bugs and little wrinkles, but now, at the beginning of an enterprise, it is the best time to try things out and ask for missing features or point out horrible mistakes. So I'll gladly hear your thoughts on it. I'm only supporting Ergodox at the moment since this is what I own and this is what I use. If there really is a need for other keyboards, it is my belief that that can be fixed in the future. If there is no such need, there is no point for me coding stuff for a distant future that never comes.
I feel like there's an entire subgroup of EMACS users rediscovering why vim is so popular.
Of course, this doesn't account for the motion/verb aspect (where you map commands to verbs, like `dw` (delete word) or `d}` (delete paragraph)), but what you're essentially talking about is a form of modality, albeit universal modality for your keyboard.
If you do get an Ergo, eventually, and you want to try mugur, I'll be more than happy to help if you have problems with it or need new features. Open a ticket or drop me a line.