FireFox already has it built in, it is called XUL and "Mozilla Platform". FireFox itself is written in XUL (you can run 2nd instance of FireFox within Firefox).
What you would need is to build an IE/Opera/Safari plugin that runs XUL code. And possibly add some security system on top of that.
Zed Shaw, if I understood him properly, built a prototype of an application that used this "API": server side was Ruby/Rails, serving XUL UI to a browser. That works even now - only in enterprise environment where you can make everybody to run FireFox.