
Color Emulation - bane
https://byuu.net/video/color-emulation
======
byuu
Hi all, article author here.

My background is I wrote emulators for 24 systems and counting (higan and
bsnes), and I want to try my hand at technical writing, in hopes of
encouraging more people to get involved with emulation. I'll hopefully get
better over time and with more feedback.

I'll be writing more about video emulation in the near future. I would like to
cover techniques for interframe blending (the flickering shadow), composite
artifacts (the waterfalls in Sonic the Hedgehog), color bleed (translucency in
Kirby's Dream Land 3), screen curvature, scanlines, color subcarriers (and how
shifting it causes shakiness you see in composite video), interlacing,
phosphor decay, phosphor glow, aperture grilles, non-1:1 RGB pixel layouts
(Game Gear LCD), etc.

I'm intending to write about all things emulation there, in fact. I want to
cover input latency reduction/mitigation, removing audio aliasing above the
Nyquist limit using FIR and IIR low-pass filters, designing thread schedulers
and priority queues, provide mirroring and organizing of PDF datasheets for
all the ICs used in retro gaming systems, etc.

Basically, I want to cover all the stuff you _don 't_ usually find in "how to
write an emulator" tutorials. All the polishing stuff that takes a new
emulator to a well-polished emulator.

It's not ready yet (the site is brand new), but I'll have an RSS feed at
byuu.net/feed in a couple of days if anyone's interested in this sort of
content.

Thanks for reading ^-^

~~~
diimdeep
Writing emulators seems like really hard thing to do. What is in it for you ?
Is it about technical challenge, or something else people from outside might
not know?

~~~
byuu
I was always into RPGs as a kid, and when I found out the US missed out on
tons of them, I got into reverse engineering the games and fan translating
them. I worked on Dragon Quest 5, Der Langrisser, Mother 3, etc.

Around 2004, I learned most of my code didn't run on real hardware, but ran on
emulators. I found out why (the main reason: writing to video memory while the
screen is drawing), and submitted patches to the SNES emulators of the time to
work like real SNES hardware, but they were rejected because too many fan
translations already relied on the old mistakes.

No one back then seemed to care about how accurate emulators were, so I set
about writing my own SNES emulator with the goal of making it as perfect as
possible, and it kind of spiraled out of control from there. Within a few
years we were decapping chips to extract coprocessor firmware, I had to
emulate the Game Boy for the Super Game Boy cartridge, the GBA CPU for this
one Japanese chess game that had an ARM CPU in it, etc.

I guess I just really like the challenge, and never stopped adding more
systems over time. The more systems I emulate, the more I already have most or
all of the chips used in the system emulated. Eg ColecoVision took me an hour
because I already had a Z80 core, a TMS9918 video core, and an SN76489 audio
core. I can definitely see how MAME turned into what it is today now.

These days, the emulation is the easy part, and organizing all of this code
(around 600,000 lines and counting), and getting all of these really unique
pieces of hardware to all work in the same GUI, has become the challenging
part. I have this really elaborate tree-view of the emulated systems to
support zany things like the Genesis "Tower of Power", a complex scheduler
that tracks timeslices in attoseconds, practically a full professional DSP
stack for mixing multiple audio streams, Reed-Solomon decoding for CD systems,
etc. I'm always learning new stuff, and there's always more to improve. I
worry that I won't be able to wrap up higan in my lifetime.

There's not a lot of money in emulation, at most I've been offered $2500 for
commercial licenses, but showing my hobby work to my employers landed me my
last two jobs in software engineering, the latter of which is in Tokyo, Japan.
Emulators literally got me halfway around the world. And I even got to work
with the developers on Stephen Hawking's voice machine software at one point.

There's been some downsides, and I had a lot of maturing to do over the years,
but on the whole, I wouldn't trade this hobby for anything.

~~~
dwild
> the GBA CPU for this one Japanese chess game that had an ARM CPU in it

I was curious about it, thus I wanted to read more about it and evidently, the
source of Wikipedia is your website, but it seems like you took it offline and
Web Archive doesn't have it either.

Are you planning to republish that article?

[https://web.archive.org/web/20120307174933/http://byuu.org/a...](https://web.archive.org/web/20120307174933/http://byuu.org/articles/emulation/snes-
coprocessors)

~~~
thristian
If I recall correctly, the article in question was just "we have now dumped
the firmware from all known SNES co-processor chips".

The first SNES co-processor chip, the DSP-1, was based on a weird NEC DSP
architecture, and if I recall correctly the part-number was figured out from
markings on the chip die, and digging through NEC spec-sheets looking for
things that approximately matched the chip's known capabilities. The
instruction set and encoding was puzzled out by hand.

Luckily, DSP-2, DSP-3, and DSP-4 all used the same NEC DSP core, just with
different firmware, so the same emulator could be used for all of them.

The ST010 and ST011 used a slightly different NEC DSP core, so they required a
little more work, but after handling the DSP-1 they weren't too difficult.

The ST018 was incredibly daunting to begin with, since its firmware was much
larger than all the other co-processors, and there were no identifying marks
on the CPU die and no product sheets to dig up. As a last ditch effort,
somebody just opened up the firmware in a hex editor and tried to figure out
the instruction encoding from first principles... and eventually they said
"that looks familiar", and sure enough it turned out to be the most popular
CPU architecture on the planet.

~~~
byuu
There was also the Cx4, where segher had to reverse engineer the entire
instruction set because it's a totally custom, undocumented ISA based on the
Hitachi HG51BS architecture.

------
simias
Accurate screen emulation is important for many retro games, and not just
because of color issues. For instance the GameBoy didn't support transparency
so some clever devs would make sprites blink very fast to make them appear
transparent. Since the GameBoy screen had a fairly slow response time the
ghosting would effectively blend the sprite with the background. If you
emulate the game on a modern low-latency monitor you just get a flickering
sprite. See for instance the claw's shadow in this gameplay video for Link's
Awakening:
[https://youtu.be/UQlP9sHf5Ho?t=825](https://youtu.be/UQlP9sHf5Ho?t=825)
(although this video is way worse that what you'd see in an emulator because
the framerate of the video doesn't line up with the GB's ~60fps display, so it
doesn't even flicker all the time).

An other example is games using the blurriness of analog video signals to
blend things together: [https://www.retrorgb.com/hd-retrovison-shows-
transparency-co...](https://www.retrorgb.com/hd-retrovison-shows-transparency-
color-depth-restoration-algorithm-to-minimize-dithering.html)

Basically you interlace two images at a high enough frequency and let the
analog artifacts blend them. Once again doing playing that on a modern LCD
screen without post-processing ruins the effect.

~~~
tobr
Has anyone tried to simply point a camera at an old screen, say a CRT screen,
take a photo of one pixel at a time at a few different levels of brightness,
then add all the photos together to render arbitrary images? As far as I can
imagine, it should capture the behavior of colors and the fuzzyness between
pixels very well. You could even set up a small still life around the monitor,
and get accurate reflections and ambient light in the room.

Stuff like the smoothed out flickering that you mention would still need to be
emulated, of course.

~~~
aidenn0
Brightness is a non-local effect. i.e. if you change one pixel, the rendering
of the adjacent pixels can change.

Also, the rendering is different depending on how the signal made it into the
CRT; in particular composite video does not preserve certain things about the
image, and in some cases developers took advantage of this/

Most famously the genesis had a poor composite encoder, which was taken
advantage of by many games; Sonic 3 in particular looks bad _even on a CRT
with a real Genesis_ if you use RGB or component output.

~~~
zellyn
Even color is a non-local effect. Turning two colored pixels on side-by-side
on an Apple II makes them white. I can give you an unbounded amount of detail
why :-)

[https://zellyn.github.io/apple2shader/](https://zellyn.github.io/apple2shader/)

------
t0mek
> The first detail is that most computer monitors run in 24-bit color mode,
> which provides 8-bits of color detail for the red, green, and blue channels.
> But most older game systems do not specify colors in that precision.

> For instance, the Sega Genesis encodes 9-bit colors, giving 3-bits per
> channel.

> The solution to this is that the source bits should repeat to fill in all of
> the target bits.
    
    
      000 -> 000 000 00...
      010 -> 010 010 01...
      011 -> 011 011 01...
      111 -> 111 111 11...
    

This is interesting and efficient approach. Interpolating a 3-bit number (0-7)
into a 8-bit one (0-255) can be done by dividing the first one by 7 (to
normalize it) and the multiplying it by 255 (to stretch it to the full range).
The operation order can be changed to avoid having smaller-than-1 floating
numbers in the first step. So, we basically need to multiply each number by
255/7~36.4.

    
    
      0 -> 0 * 36.4 = 0
      1 -> 1 * 36.4 = 36
      2 -> 2 * 36.4 = 73
      3 -> 3 * 36.4 = 109
      ...
      7 -> 7 * 36.4 = 255
    

So, these multiplications give exactly the same results as repeating the bits
in the byuu's article and bits operations are much cheaper. I have some
intuition on how does it work (we're increasing each "part" of the number by
the same factor), but not a math explanation.

~~~
RetroSpark
The calculation `red = r << 5 | r << 2 | r >> 1` is equivalent to `red =
floor(r * (1 << 5 | 1 << 2 | 1 >> 1))`, which is `red = floor(r * 36.5)`.

~~~
byuu
I thought it would be good to get people familiar with bit-twiddling, as you
will be doing a whole lot of that when writing retro emulators.

This is one of my favorite sites on the internet:
[https://graphics.stanford.edu/~seander/bithacks.html](https://graphics.stanford.edu/~seander/bithacks.html)

~~~
fps_doug
But please keep in mind that today's compilers are (most of the time) very
smart. Hacker's Delight is somewhat outdated today. There's no need anymore
for writing "x << 2" instead of "x * 4".

I recommend Matt Godbolt's Talk(s) about compiler cleverness:
[https://www.youtube.com/watch?v=bSkpMdDe4g4](https://www.youtube.com/watch?v=bSkpMdDe4g4)
(see the 30 minute mark for the multiplication example)

~~~
FreeFull
I'd say it's about intent. Depending on the context, "x * 4" is less clear
than the bitshift (For example, when packing multiple values into an integer)

~~~
kevincox
The point isn't "always do multiplication" but to do what makes logical sense.
Don't pick your operators for performance, pick them for readability. The
compiler will handle the performance aspect for you.

------
jbverschoor
Check out CRT Royale / CRT-Royale-kurozumi which also does some phosphor
fading.

[https://github.com/libretro/common-
shaders/tree/master/crt/s...](https://github.com/libretro/common-
shaders/tree/master/crt/shaders/crt-royale)

Sonic
[https://www.youtube.com/watch?v=u77TUkFHNyI](https://www.youtube.com/watch?v=u77TUkFHNyI)

Streetfighter
[https://youtu.be/xrpm8o17cHE?t=903](https://youtu.be/xrpm8o17cHE?t=903)

Other resources: [https://www.resetera.com/threads/crt-shaders-scanlines-ot-
be...](https://www.resetera.com/threads/crt-shaders-scanlines-ot-
because-1080p-is-all-the-ps-i-got.7235/)
[https://www.youtube.com/watch?v=qBzB0YdHn64](https://www.youtube.com/watch?v=qBzB0YdHn64)

~~~
stelonix
Ok, I gotta say: none of the shaders or scanlines I've tried before ever came
close to what I'm seeing in those videos; to the point I'd think "uh looks
nice but that's not how I remember from the 90s".

Now that crt royale... Color me impressed! Gonna try it as soon as I get
access to a PC.

~~~
DCKing
I've heard people say for CRT Royale to be _truly_ effective you need to use
it on a 4K TV. It apparently needs the additional resolution to truly shine.

Having said that, it already looks really impressive to me on a 1080p TV.

------
RetroSpark
The Sega Master System has 6 bit-per-pixel color, 2 bits for each of red,
green and blue. You would expect values 00, 01, 10 and 11 for each color to
represent 0%, 33%, 66% and 100% voltage on the corresponding pin of the Video
Display Processor. However, I once connected an SMS to an oscilloscope and
found voltage levels much less evenly-spaced than this - and they were quite
different for different console models.

Even if the voltages had been evenly spaced, would it have been correct to use
evenly-spaced values (0, 85, 170 and 255) for each of r, g and b in an
emulator? I believe this would be correct, assuming that the gamma curve of a
modern monitor matches that of a 1980s/90s TV - which should be the case for a
monitor calibrated to sRGB?

~~~
BearOso
Brightness doesn’t correlate linearly to voltage level. sRGB is representative
of the final output, so the brightness would correspond to 0, 33, 66, 100, and
the voltages would be the inverse application of the gamma to those values.

------
1wd
It would be interesting to add three photographs to each example: Original
retro system display, un-corrected on modern display, corrected on modern
display.

~~~
byuu
It is extraordinarily hard to photograph handhelds that lack backlighting, and
I left my CRT back in the US so that's also out ^-^;

It would be a good idea to get some extreme close-up photos of the screens
however, I'll keep that in mind for a future post.

~~~
mark-r
In the US you can't even give away a CRT these days, you have to pay someone
to take them. It might be hard to locate one depending on where you are, but
once located maybe you could offer to take it off their hands?

------
CodexArcanum
One thing I've been thinking about lately with emulation: why can't we
replicate and use the original hardware more easily?

There's lots of knockoff console replicas out there, so it seems like the
chips can still be manufactured. Wouldn't it be fairly cheap to build a
board/chip that contained either the original chip designs from classic
computers/consoles, or hardware-level emulated chips? Seems like you could
stuff several of those retro chips onto a single expansion board. Then
emulation software could tap into this "emulation expansion board" to make use
of the real chips. Is this all crazy talk or does any of that make some sense?

~~~
thristian
There's lots of knockoff console replicas out there, but most of them are
software emulation running on the moral equivalent of a Raspberry Pi, and the
rest are often quite inaccurate approximations of the original hardware, with
many incompatibilities with published games.

Without access to the original designs, hardware emulation is in the same
position as software emulation: you need to do a ton of research and
development. A company called Analogue[1] makes FPGA recreations of a few
classic consoles, but they're pretty expensive, and took a lot of effort to
design. In fact, byuu (the author of the OP) was consulted fairly heavily
during the design of their SNES reproduction.

Basically, if you have to do a lot of research _anyway_ , you might as well go
with the product that has zero marginal production cost (i.e. software) and
save yourself a bunch of manufacturing expenses for the same result.

[1]: [https://www.analogue.co/](https://www.analogue.co/)

~~~
CodexArcanum
Ah, that's good to know! That's a fun fact about byuu helping with the FPGA
SNES that they built as well, really interesting! Thanks for the knowledge
share!

------
nemetroid
The part about color precision is similar to three-hex-digit CSS colors, where
e.g. #5ce is expanded to #55ccee.

------
rossy
I've always wondered if you could just stick a colourimeter on a GBA SP and
generate an ICC profile for it, then use that profile to adapt the colours to
your display. It probably wouldn't be that easy, but there are colour
management tools and libraries available that are designed for adapting
colours between devices, so if you could somehow obtain a device profile for
the GBA LCD, it might be possible to use it to produce very accurate colour
emulation.

~~~
thristian
As I understand it, colorimeter expect the display to output a particular
colour or shade and measure it precisely. You probably couldn't run
colorimeter software on a GBA, but you could write homebrew that let the user
manually cycle to the correct patterns at the right times.

Some issues would be:

\- There's at least three models of GBA display: the original front-lit
display, the GBA SP back/front-lit display, and the later GBA SP backlit-only
display, and they probably all have different profiles

\- It's probably not possible to use a colorimeter on the original front-lit
GBA display, because with a colorimeter clamped on top of it, it would always
look black

\- Colour-correction software that works with ICC profiles is typically built
for print/still image work, so it expects a single high-resolution image with
high-precision output, not a 60fps stream of very-low-resolution images. Maybe
it's fast enough to run in real-time, who knows?

~~~
rossy
Yeah, writing homebrew to get the GBA screen to display whatever a screen
displays under a colourimeter, and getting that to work with the actual
colourimeter might not be so easy. I think getting a device profile for the
frontlit SP would be the most interesting, since I've heard it produces
colours more like the original GBA than the backlit one, but maybe a
colourimeter designed for backlit PC monitors would have trouble with it. I
don't think performance would be an issue though. mpv can do colour management
in real time by generating a LUT from the ICC profile and applying it to each
frame with GPU shaders.

~~~
MartijnBraam
Most colorimeters also have a projector mode for measuring beamer screens. You
can just point the colorimeter to the gba screen bar an angle and shine a high
quality light source at it

~~~
rossy
That's cool. I didn't know that (I've never used one before.) Maybe it would
make more sense to do the original GBA then. The SP frontlight has a bit of a
blueish tinge to it.

------
lanius
Do "official" emulators handle these details? E.g., SNES games in Nintendo
Switch Online

------
spookthesunset
> the Sega Genesis encodes 9-bit colors, giving 3-bits per channel

I love systems that didn’t align everything on an 8 bit boundary. Are there
any modern systems that aren’t 8-bit aligned?

~~~
Dylan16807
Are you sure the Genesis didn't align things on 8 bit boundaries? Either way,
it might be worth mentioning current systems that use 10-bit (30-bit) color.

The N64 is notable for using ECC memory modules with ECC disabled, where the
CPU sees 8 bits per byte and the GPU uses all 9 bits per byte for some
functions.

------
leoziih10
Bag

