
Statically Recompiling NES Games into Native Executables with LLVM and Go (2013) - dcschelt
http://andrewkelley.me/post/jamulator.html
======
ZenPsycho
I have actually been rather obsessed with this lately. You'll see that there's
a few problems he runs into that he deems insurmountable, which sounds like a
challenge! Specifically, there's the issue with an instruction which is
effectively a computed goto:

A jump instruction that takes an address and then jumps to the address STORED
at that address. Since there is no way to know at compile time what addresses
are going to stored at a place, you're forced to then dynamically emulate the
whole memory space of the actual NES to accurately calculate it, thus
defeating the whole point.

Is that the only solution though? a head scratcher!

Further are issues with parts of code that, on some level seems to be taking
inspiration from genetics: Jump to one alignment, and the instructions get
interpreted one way, jump to a different alignment and the same sequence of
bytes is interpreted by the CPU as an entirely different set of instructions.
I wonder if that could be resolved by creating a different source code path
for each alignment using flow analysis- A space saving technique effectively
getting uncompressed.

~~~
stormbrew
I suppose it's possible this could be considered "emulating the whole memory
space," though I wouldn't consider it that, but you could just generate an
offset table for all jumpable memory locations and use them to calculate the
correct offset. It's quite likely you could even do that in less than O(n)
space with some time trade-offs.

~~~
ZenPsycho
I was commenting on my memory of the article.

Upon rereading it, it seems the real insurmountable challenge is interrupts!
It always seems like at that level of detail with this stuff, it becomes a
decision to go slower in order to simulate NES hardware accurately. But my
question is: how much code depends on that accuracy, and how much can you
compromise for speed? A further question I have is can you do a cross game
analysis of the whole NES library and find common patterns, reused functions,
that you decompile into a kind of high level conversion that would hopefully
or gloss over the need for specific functions to have that low level accuracy.

~~~
jtolmar
I'm pretty sure Nesticle had nothing resembling accurate timing, and it could
run most games.

You can go well beyond several of the NES's limitations by fiddling with PPU
parameters between scanlines, but that takes careful timing and almost nothing
actually does that. Might only be democoders. If you're willing to ignore the
possibility that someone did that, you should be able to just process a whole
frame's worth of PPU at NMI time.

Super Mario Bros spends most of its time in an infinite loop, waiting for NMI
to pull it out so it can start processing the next frame. If NMI triggers
before this, you get a lag frame. Plenty of popular emulators get the wrong
lag frames, but only speedrunners and TASers really care. So, for this game,
you could possibly toss out the timing entirely and trigger NMI when you hit a
busy loop. Your emulator will completely ignore all lag frames, but most
people won't notice for at least this game.

(It's been over a decade since I was into ROM hacking so my memory could be
faulty on any of this.)

~~~
Drakim
Interrupts need to be fairly accurate. Lots of games use them to have unmoving
status bars, and rely on them triggering on just the right scanline.

~~~
ZenPsycho
I wonder if for things that happen between and among scan lines, how difficult
it would be to statically analyse the code to find out what's running during
that time, and precalculate when the writes would fire.

another approach could be getting runtime information from a running emulator,
and record it onto a file the compiler could use to close the gaps you need
but can't get from static analysis.

~~~
Drakim
The issue is that nothing is stopping the developers from doing all kinds of
crazy loops and jumps in the code that's invoked by the interrupt, making it
into a halting problem.

Unless the interrupt handler is trivially simple, you can't know how many
cycles it will take to finish.

~~~
ZenPsycho
the advantage the compiler writer has in this case is it doesn't have to deal
with _any possible_ code, just this set of well known already released and
frozen in time programs.

~~~
Drakim
I don't think that really avoids the problem. I mean, what if you do a JMP to
an address decided based on a value gained from player input? How is the
compiler supposed to guess that in advance?

~~~
ZenPsycho
the mathematical proof of the halting problem, as I understand it, would in
this context be based on the notion that user input could somehow direct the
program to write a program in ram and get it to execute.

This has in fact been achieved in games like super mario world or pokemon!

It seems to me sometimes that educated programmers call halting problem too
early on situations like this where you don't actually need a rock solid
"provably correct" compiler, just a compiler that does something reasonable in
this bounded set of cases, and we don't mind _that_ much if it crashes
sometimes, since it's not like an NES game is a rocket or a therac.

------
orik
here's the previous discussion:
[https://news.ycombinator.com/item?id=5838326](https://news.ycombinator.com/item?id=5838326)

(i've realized; is this even necessary with the 'past' button?)

~~~
Splines
I never knew there was a "past" button...

~~~
noobermin
Oh dear, I'm coming up on two and a half years on HN (time flies!) and I
didn't know either...

~~~
boulos
Don't worry, it's recent and mostly just designed to help you figure out if
you should flag an article as a dupe.

------
madez
I can’t help it. Reading this feels a bit like hidden political propaganda.
It’s ridden with subtle and not so subtle negative references to gcc, the fsf
and ideals.

Probably it’s me reading too much into it but it makes it hard to enjoy.

------
tibbon
Does anyone have information with how some of the "multi cart" games worked,
like Mario/Duck Hunt/Track Meet? Surely, all 3 games didn't fit in 32k
right?... right?

[http://nintendo.wikia.com/wiki/3-in-1_Super_Mario_Bros._/_Du...](http://nintendo.wikia.com/wiki/3-in-1_Super_Mario_Bros._/_Duck_Hunt_/_World_Class_Track_Meet)

~~~
jimsmart
An educated guess (having coded the NES) is that it's simply a bigger ROM, and
the cart contained some kind of MMC chip [0] which allows different sections
from the ROM to be paged-in - the menu screen contains the code to page-in the
applicable bank from the ROM, and then the game runs per normal, not being
aware of any of this.

[0]
[https://en.wikipedia.org/wiki/Memory_management_controller](https://en.wikipedia.org/wiki/Memory_management_controller)

------
andrewvijay
Too low level for me. But since that I've started go as my first low level
statically typed language, I think I'll just bookmark this article and may be
read after a few years!

~~~
spriggan3
> Too low level for me. But since that I've started go as my first low level
> statically typed language

I don't think any garbage collected language can be called "low level". If you
really want to go low level, learn C and ASM. Manual memory management is the
real deal.

~~~
dkopi
I don't think any compiled or assembled language can be called "low level". If
you really want to go low level, learn Machine code and CPU architecture.

...

I don't think any executed language can be called low level. If you really
want to go low level, learn hardware design, VHDL/VERLIOG

...

I don't think any hardware description language can be called low level. If
you really want to go low level, build your own logic gates out of transistors

...

~~~
daodedickinson
I'm with Nietzsche, the real low-level programming involves feeling new
feelings and somehow communicating/philosophizing/programming them into other
people.

------
gcc_programmer
gcc -Wall -fno-diagnostics-color

If you are going to list architectures supported, also list the ones gcc
supports - it's, basically, _all_ of them.

Llvm+clang is nice, but not better, drink the kool aid. Gcc performance is
still higher, and gcc is _free_ software.

~~~
techdragon
Clang is also free... Just not your preferred definition of Free.

~~~
gcc_programmer
Free as in free beer, not free as in free speech :P

~~~
over
The FSF considers BSD a free software license, it's just non-copyleft. There
was a lot of discussion on Groklaw about taking BSD code, modifying it, and
slapping a GPL license on top. I believe the consensus was this is OK, since
you are still respecting the BSD license terms.

~~~
sgt101
also from [http://www.gnu.org/philosophy/free-
sw.en.html..](http://www.gnu.org/philosophy/free-sw.en.html..).

>Freedom 3 includes the freedom to release your modified versions as free
software. A free license may also permit other ways of releasing them; in
other words, it does not have to be a copyleft license. However, a license
that requires modified versions to be nonfree does not qualify as a free
license.

