I wish there was an editor with an interface as great as (say) Pixelmator's, but with an optional script window that let you write and edit the code for your vector graphics if you need to. Something PostScript powered, maybe? (PaintCode does something like this, but it's more of a one-way, vector-to-code mapping. Bloom promises to do a lot of this, but I haven't used it in a while.)
Indeed you shouldn't! That's why all serious image manipulation programs have this capability built in.
Most of them also let you write your own plugins for those cases where you need more than what the interface provides.
I think going to all-code drawing is more limiting than freeing. It seems more like a case of having a hammer and seeing that everything can be made to look like a nail.
As for plugins, that's not really integrated into the workflow in the way I'm envisioning. I shouldn't have to write heavy code just to experiment with a quick artistic idea! Think web development. It should be as simple as editing a JS file and refreshing the page.