I wouldn't say this is precisely true. Look at Dolphin's "ubershaders" (https://dolphin-emu.org/blog/2017/07/30/ubershaders/): they're essentially a Turing machine, running on your GPU, used to emulate a GPU of another architecture... and yet this is still (much!) faster than doing the same on the CPU.
And there's nothing special about emulating a GPU on a GPU; you could emulate a CPU architecture just as easily, at a much higher level than you get from an FPGA, and so perhaps faster than you'd be able to get from today's FPGAs. And, if you're mapping GPU shader units 1:1 to VM schedulers, you'd also get a far higher degree of core parallelism than even a Xeon Phi-like architecture would give you. (The big limitation is that you'd be very limited in I/O bandwidth out to main memory; but each shader unit would be able to reserve its own small amount of VRAM texture space—i.e. NUMA memory—to work with.)
I'm still waiting for someone to port Erlang's BEAM VM to run on a GPU; it'd be a perfect fit. :)
And there's nothing special about emulating a GPU on a GPU; you could emulate a CPU architecture just as easily, at a much higher level than you get from an FPGA, and so perhaps faster than you'd be able to get from today's FPGAs. And, if you're mapping GPU shader units 1:1 to VM schedulers, you'd also get a far higher degree of core parallelism than even a Xeon Phi-like architecture would give you. (The big limitation is that you'd be very limited in I/O bandwidth out to main memory; but each shader unit would be able to reserve its own small amount of VRAM texture space—i.e. NUMA memory—to work with.)
I'm still waiting for someone to port Erlang's BEAM VM to run on a GPU; it'd be a perfect fit. :)