Everyone's always had this idea, and it has very rarely proven to be terribly practical in reality. The few times that VMs of different hardware tend to be practical are when there is an enormous gap in performance between the emulated hardware and the host hardware (such as with old game consoles and arcade hardware).
I don't think it has been tried too many times in practice, because software binary translation is so proven (eg. like in android x86, or rosetta on mac, or fx!32 on nt, etc etc). Itanium is the only one that comes to mind. Any others?
Then there are several that have lower level "programmable microcode" kind of level in the hardware, ranging from the Transmeta stuff to mainframe-era writable control store...
That's not emulation that's decompilation / recompilation. Which is a very challenging task. TransMeta worked on this under the name of "code morphing". It's a promising technique but I don't think it has the maturity to be able to run cross-architecture code at close to hardware speeds (say, 50% or higher) without introducing new defects, especially x86 on ARM.
What you are discussing about, is a runtime issue.
I was discussing about installation time, which only prevents situations were you make use of code rewriting tricks, which are anyway frowned upon in the day and age of read-only code sections and no-execute data sections.
