Further, scaling up nifty ideas to all programs and the entire development development process, both in terms of performance and interactions is hard. One guy on my team (I lead a team at MSFT for a bit) implemented "Omniscient Debugging" on steriods (you could also resume from the previous point in time) back in 2002 that we demo'd around, and it took many years to figure out how to build it in in a way that users could understand - happened long after I left.
We also had a full working IDE in web browser in 2001; more tightly integrated debug/edit/profile/unit test tools; etc. Fundamentally, though, many of these are user interface problems and it's much easier to come up with a spashy demo than it is to build and ship a real project that works with real code.
So, I'd say it's not that the "PL community" has been too focused on static code as separate from dynamic execution of the program, but rather that as IDE designers we don't (or at least didn't ~6 years ago) know how to convey the information to the user without requiring they use an IDE that looks like the cockpit of a 747.
I don't think the Omniscient log-the-world Debugging is practical, but if we just had some form of deterministic replay, we could take snapshots or even cheaper memoizations at certain points in the program. Going backwards is simply a matter of going back to the last checkpoint and going forwards to the point where you want to stop.
I agree that the key is UX, and as Gilad said in the blog comments, they aren't necessarily technological (once we know what the right experience is, we'll build the technology!).
Disclaimer: I work for MSFT, though I study this problem in research (MSR/MSRA).
I have definitely noticed a correlation between developer debugging skills and clean code practices. Obviously, this doesn't mean one is causing the other, but I would love if there was any research looking into this.
It does make me wonder if our time would be better spent finding ways to improve consistency and professionalism in our practices, rather than another new tool to get you out of the mud.
I believe more in the latter school, though of course, they are both extremes and I'm a moderate. The mud is unavoidable and we might as well build winches to pull ourselves out rather than spend time trying to carefully avoid it.
But even this definition feels too confined. There have been some developers I worked with that must have started out this way and then learned 'engineering' practices. They were all exceptional to work with. I wish more of this group could get to that level. Hence my interest in putting bread crumbs on the path to learning, rather than yet another pill to lose weight fast.
Debugging is as close to experimental science as we get in computer science. It is the act of understanding a complex system, and even if we built that system entirely by ourselves (unlikely), it eventually "gets away from us" and takes on a life of its own. It is impossible to understand everything, and debugging is a great way of allowing us to forget details and uncover them later.
So far most of the tools have been created in a very strict, single purpose way. While that's a good engineering approach it also leads to limitations that are hard to overcome. I do remember the early days of Eclipse when they had to build their own java compiler to be able to pull out enough data for the IDE. And that's the simplest example. Or think of the days before JMX.
Obviously this is not a simple problem to solve. I think the only way we could fundamentally change things is to completely rethink the way programming models/environment are thought.
You can have IDE's that don't have quite this problem in environments like Smalltalk. Since you can see all of the source code, and easily tweak and debug the whole system, every part of the system is your API. Unfortunately, this can also result in hard to manage dependencies when people publish extensions.
The way to have the best of both worlds, is to have an environment as described above, then also have a community process that promotes innovative extensions and rewrites them into more conventional APIs. This way, such APIs respond to actual need and no genius system designer has to foresee all the ways people will want to extend a system in the future.