Ask HN: Why can't we replace instructions in a binary for different ISAs?

[monus]

I know this may sound really naive but I'm new to low level programming. I was wondering if the binaries that we build use a certain set of instructions to talk to CPU (amd64, arm64 etc), why can't we edit the binary and change the calls to make the call in the certain ISA we want?

I know that's possible with recompiling from the source code, for example with Go, but I was wondering why can't we do this for arbitrary binaries? Are the calls not apparent in the binary form? From what I can tell Rosetta 2 does a similar thing when you run amd64 app on M1 for the first time[1]. What prevents us to have such a tool for all architectures? For example, I'd like to run an arbitrary amd64 binary on a RISC-V core without recompiling the source code.

[1]: https://www.computerworld.com/article/3597949/everything-you-need-to-know-about-rosetta-2-on-apple-silicon-macs.html

[tlb]

It works for simple cases. Things that make it complicated are:

- Only executable code should be translated, data shouldn't be. Sometimes (like with jump tables) they are ambiguous.

- Different CPUs have different memory consistency models for parallel threads, leading to race conditions when translating

- Special instruction sets like AVX can be hard to translate

- Anything with a JIT compiler (including most web browsers) needs special handling.

That said, qemu works for the great majority of emulated & host CPUs.

[monus → tlb]

Thanks for the explanation!

> Different CPUs have different memory consistency models for parallel threads, leading to race conditions when translating

Does the race condition happen during the translation or execution? If it's former, we can somehow prevent it, right? The translation code is known.

> Anything with a JIT compiler (including most web browsers) needs special handling.

I guess this is something Rosetta 2 does.

From what I can tell, the listed reasons are mostly alongside how hard or challenging it would be but there is no actual technical limitation. It's just no one really made the effort, I guess because of QEMU since it works for most cases albeit with a performance penalty since it's emulation rather than translation.

Do you think it's feasible to implement a translation software that similar to Rosetta 2 but open source and covers other ISAs like RISC-V? I understand that it's hard but it's also surprising that while we see the barrier for different ISAs to be the compatibility but at the same time we settled for only runtime emulation.

[tlb → monus]

> Does the race condition happen during the translation or execution? If it's former, we can somehow prevent it, right? The translation code is known.

During execution.

Qemu is the result of a huge amount of effort to make it fast. It'd take many, many programmer-years to build something faster.

[monus → tlb]

The thing I don't quite grasp is that QEMU does the translation in runtime, i.e. emulation, but why not do it once and use the translated binary?