The high-level procedure was to load up a level, visually scan the RAM contents (the NES only had 2kb of RAM) for the level layout, and confirm that that section of RAM contained the level tilemap by tweaking values and observing that something changed in-game. Once I knew where the level was loaded into RAM, I set a write watchpoint on that RAM location, then re-loaded the level, which would break at the instruction which wrote that tile value into that location of RAM.
The NES CPU is single-threaded and has only three registers, so it's relatively easy to trace execution. The emulator I used (FCEUX-DSP, although I think Mesen is a better experience today) has a feature where it traces CPU execution, including register values, to a text file. So you can follow the trace of the level loading process and examine the disassembly in parallel and figure out what the code is doing. E.g. reading these values from ROM, performing these transformations on it (i.e. reverse-engineering their compression algorithm), and then storing the tile values into RAM at the watchpoint discussed earlier.
Then there's a similar process for loading palettes and graphics data into the PPU[1], how in-game enemies are stored, various level flags that affect how the tile data is presented, how the MMC3 mapper works[2], etc. Once that's all understood you can reproduce that algorithm in a new C program reading the data right out of the ROM image to create those PNG images you see on TCRF.
Yes, it was a lot of work :) great learning experience though.
Another group has been working on a level editor for M.C. Kids. I think they did their reverse-engineering work independent from mine: https://github.com/pgrimsrud/MCKids_Editor
That's amazing. These insights taught me a lot. Thank you so much.
I asked because I remember reading documentation on the Super Metroid save file format. There's map data in the game state as well but it wasn't documented. Always wondered what it would take to reverse engineer that...
I'm gonna try it with one of my favorite NES games. If they have level dumpers, I've never seen them... I know one which has a complete disassembly though.
I really encourage you to try it! NES is a great platform to start RE. It's a lot of work, but really fun once you start making breakthroughs :) Keep copious notes!!
The X-Men one is really cool, but I don’t get “The message is disguised via a secondary lettering set which shared the same white color as the main text.” The screenshots seem to show the opposite: identical lettering, different color.
that sentence is stating that normally the "hidden text" is white, the same color as the main text. the very next sentence: "Killing 30 special enemies in a stage causes the hidden text to be displayed in red, as shown below (the counter that keeps track of how many special enemies you've killed is stored at $513 in NES memory)."
To elaborate on this a bit in case it's still confusing for anyone:
Until the mid-1990s or so, memory was too expensive to routinely store RGB values [1] for every pixel on consumer-grade hardware, so sprite/texture/etc. assets effectively contained a narrower type "pointer to color" instead of a wider type "color", typically translated on-the-fly by the video hardware by way of a small and fast palette RAM. This meant that by altering the underlying array of actual colors, an impressive breadth of animation and "compression" tricks could be achieved with little CPU time and no manipulation of the underlying pixel data (which, for NES and arcade games, was often stored in a dedicated ROM directly read by the video hardware). The classic example is a halo of torchlight cycling through a handful of red/orange/yellow values, but it's been used for a vast spectrum of effects by a bunch of extremely clever artists, programmers, and artist-programmers.
[1] It happens to be the case that Famicom/NES color is fundamentally based on the phase encoding of composite video and so doesn't strictly map to a single definitive RGB color space, but the basic principle otherwise applies.
I just found out yesterday (thanks, tcrf.net!) that one of the developers on Banjo-Kazooie put some new info on Stop'n'Swop up on twitter this year. Eternal mystery resolved.
tl;dr: turn n64 off, RAM retains data for 10s, load different game, transferring data between cartridges with no memory card and no internal disk! And it worked!
https://tcrf.net/X-Men_(NES) Great little way to hide a secret.
https://tcrf.net/The_New_Tetris Developer rant. Niel Voss really did make great music!
https://tcrf.net/Super_Mario_Bros._3/Unused_Levels Unused levels in Mario 3.
https://tcrf.net/Prerelease:The_Legend_of_Zelda:_Ocarina_of_... A huge wealth of early screenshots and development info for OOT.
https://tcrf.net/Proto:The_Legend_of_Zelda:_Ocarina_of_Time/... ...and a leaked build from around the time many of those screenshots were taken.
https://tcrf.net/Proto:M.C._Kids Because I love this game and I wrote the level map dumper to take those images.