
Porting a NES Emulator from Go to Nim - def-
http://hookrace.net/blog/porting-nes-go-nim/
======
fogleman
> As I really liked fogleman's NES emulator in Go I ended up mostly porting it
> to Nim. The source code is so clean that it's often easier to understand the
> internals of the NES by reading the source code than by reading
> documentation about it.

Win!

~~~
AYBABTME
Went to read your code after that comment, it is truly, very clean code! :+1:

------
haberman
Let me get this straight. We have an emulator for 1985 hardware that was
written in a pretty new language (Go), ported to a language that isn't even
1.0 (Nim), compiled to C, then compiled to JavaScript? And the damn thing
actually _works_? That's kind of amazing.

~~~
derefr
The amazing thing about computer software, relative to every other human
endeavour: once you've got one component right, it's _right_ ; you can treat
it like a simple little node in an even more complex design, and scale it up
1,000,000x, and it'll keep working every single time.

Once you've built a C-to-JS compiler, and a Nim-to-C compiler, you've got a
perfectly-functioning Nim-to-JS compiler. There's no game of telephone
gradually confusing things; the fidelity is perfect no matter how deep the
pipeline goes.

(This is also what's amazing about layered packet-switched networks, come to
think of it. The core switch transferring data from your phone to AWS doesn't
need to speak 4G or Xen SDN; it's just coupled to them, eventually, by
gateways somewhere else, and it all just keeps working, one quadrillion times
per second, all over the world.)

~~~
haberman
It's so funny that you say that, because you are describing the way that I
_wish_ the world works, but I have found that in practice it rarely does.

For example, here are a list of ways in which the C-to-JS compiler is far less
than "perfect": [http://kripken.github.io/emscripten-
site/docs/porting/guidel...](http://kripken.github.io/emscripten-
site/docs/porting/guidelines/index.html)

~~~
derefr
That's what I was really trying to get across—the difference between
"imperfect" in other engineering, and "imperfect" in software. In other
engineering, "imperfect" means "will randomly fail 0.00000001% of the time."

In software, though, "imperfect" means "will work perfectly, 100% of the time,
even after a quadrillion tests, within a _well-specified subset_ of all
possible inputs; will fail in arbitrary unknown ways outside of that subset."

It's a difference to being "within tolerance"—even within tolerance of a
physical system, stresses are still going on. Physical systems have a
lifetime, and repair procedures, for a reason.

But in software, you don't have to worry about "stress within tolerance." In
fact, even if someone else builds an "imperfect" software system that accepts
inputs with undefined results, you can just _wrap_ it with an input validator,
and suddenly it's a well-defined-100%-of-the-time system!

(Of course, in implementation, software has to run on hardware, which is the
other kind of imperfect system. But, surprisingly, you can write software to
compensate even for that, with failover and quorum-agreement &c.)

------
allendoerfer
This title covers the essence of hacker news pretty well.

~~~
nicklaf
Also see:
[https://news.ycombinator.com/item?id=7745561](https://news.ycombinator.com/item?id=7745561)

My favorites:

> I decided to re-implement Javascript in Javascript. It failed. Here is my
> story

> ReactOS running IE6 in a JavaScript x86 emulator: we put a browser in your
> browser so you can browse while you browse

> Introducing js.js: a JIT compiler from JavaScript to JavaScript

------
shurcooL
Did you port from Go to Nim by hand, or was it automated in any way?

I thought that Go would be the last language I'd write by hand. Previously I
wrote C++, which was a dead end in that I could never use tools to parse it
and translate to a new language. But with Go it should be much easier to do
that if/when I ever decide to switch to something else.

The performance of the emulator in browser (compiled via emscrimpten) is very
impressive! It felt like solid 60 FPS to me. I wonder how the Go version
compiled via GopherJS would compare? Have you tried?

~~~
conradev
There is also a NES emulator written entirely in JavaScript called jsnes[1]
that would be interesting to compare to. The "solid 60 FPS" doesn't always
translate to mobile. For example, jsnes runs fine on an iPhone 5S (64-bit
processor), but stutters on an iPhone 5.

[1] [https://fir.sh/projects/jsnes/](https://fir.sh/projects/jsnes/)

~~~
Narishma
Forget about mobile, it runs at like 5 FPS on my Core i3 laptop.

------
warmwaffles
Well this is new. I've never seen Nim before. What does it offer that Rust,
Haskell, Erlang, etc... do not?

~~~
def-
Two of my old posts may answer that question:

\- [http://hookrace.net/blog/what-is-special-about-
nim/](http://hookrace.net/blog/what-is-special-about-nim/)

\- [http://hookrace.net/blog/what-makes-nim-
practical/](http://hookrace.net/blog/what-makes-nim-practical/)

Summary by benhoyt:
[https://news.ycombinator.com/item?id=8822918](https://news.ycombinator.com/item?id=8822918)

    
    
      * Run regular code at compile time
      * Extend the language (AST templates and macros);
        this can be used to add a form of list comprehensions to the language!
      * Add your own optimizations to the compiler!
      * Bind (easily) to your favorite C functions and libraries
      * Control when and for how long the garbage collector runs
      * Type safe sets and arrays of enums; this was cool:
        "Internally the set works as an efficient bitvector."
      * Unified Call Syntax, so mystr.len() is equivalent to len(mystr)
      * Good performance -- not placing too much emphasis on this,
        but it's faster than C++ in his benchmark
      * Compile to JavaScript

~~~
jedi_stannis
Thanks, that first post is really interesting. Two things stand out to me:

* Run regular code at run time - does this just error if you try to use const on something that depends on a runtime value?

* Add your own optimizations to the compiler - isn't this very dangerous, especially if optimizations from various sources of code get combined? Even if your optimizations are valid (which there seems to be no guarantee of), computer math is notorious for being different from real math. People unaware of the nuances of integers rolling over or floating point seem like they could easily shoot themselves in the foot here.

~~~
dom96
> * Run regular code at run time - does this just error if you try to use
> const on something that depends on a runtime value?

Yep. For example `const foo = stdin.readLine()` would result in "Error: cannot
evaluate at compile time: stdin".

As for your second point. I haven't used this feature personally but I have
heard that the compiler will tell you exactly where these optimisations are
applied. You can also disable these optimisations very easily during
compilation.

------
mwcampbell
The binary size difference is quite striking. Linux distro packagers are going
to like Nim, I think.

~~~
vezzy-fnord
I couldn't tell if the Nim result was actually statically linked. If it was,
then it almost certainly wasn't using glibc, because the binary would
certainly be much fatter.

~~~
loudmax
If the Nim binary is dynamically linked, comparing the size to a Go binary
isn't always valid. There are reasonable trade-offs to consider when selecting
dynamic vs static linking. Admittedly, when you're running an NES emulator on
your desktop probably isn't one of them, you're not likely to care either way.

The Nim -> C -> emscripten path is very impressive. Kudos.

~~~
ploxiln
Go also dynamically links glibc by default on linux, for the domain name
resolver functions.

I'm not sure what Go does on OS X, but I know that it's not possible to
statically link the c library on OS X.

~~~
sunnyps
I thought gc only statically links and it was gccgo that had the ability to
dynamically link.

~~~
ploxiln
gc only statically links to compiled go objects, and (may have changed
recently? or soon?) only dynamically links to c libraries (via cgo).

------
beagle3
Ubercool.

Question about nim: from looking at
[https://github.com/def-/nimes/blob/master/src/nes/cpu.nim](https://github.com/def-/nimes/blob/master/src/nes/cpu.nim)
, I wonder: is there way to give the no. of cycles and instruction encoding
with the "op" template, so those 256 byte arrays get built automatically?

~~~
dom96
I'm not the author but I don't see why not, you can just have an extra param
to the "op" template.

~~~
beagle3
But how would you put it in the right place in the table? Can you just have
something like

    
    
        cycles[opcode] := ncycles
    

run during compile time in the macro, resulting in a populated 256-byte table
of all the opcode cycles and no anything done in runtime?

~~~
filwit
I'm not sure exactly what you're asking, but I can answer at least part of it.
You can build lists at compile time in Nim via the `static` statement or
`compileTime` pragma. Eg:

    
    
      # using the pragma here, but we could use a 'static' block instead
      var cycles {.compileTime.}: int
    
      # ---
    
      proc doSomething =
        static:
          # invoked once at compile-time (at this definition)
          cycles += 1
    
      proc doSomethingGeneric[T] =
        static:
          # invoked once at compile-time (per unique generic call)
          cycles += 1
    
      macro doSomethingAtCompileTime(n): stmt =
        # invoked at compile-time (per call)
        let ncycles = int n.intVal
        cycles += ncycles
    
      # ---
    
      doSomething() # this call doesn't effect 'cycles', it's declaration does (+0)
      doSomething() # ditto (just here to prove a point) (+0)
      doSomethingGeneric[int]() # this call effects 'cycles' (+1)..
      doSomethingGeneric[int]() # ..but only once (+0)
      doSomethingGeneric[float]() # this call also effects 'cycles'  (+1)
      doSomethingAtCompileTime(5) # this call effects 'cycles'  (+5)
      doSomethingAtCompileTime(12) # ditto  (+12)
    
      static:
        echo cycles # prints '20'
    

I'm not sure this helps solve anything in NimES, but this can be really useful
for meta-programming in general. For instance I'm using it to make an event
system which generates efficient dispatch lists based on the types/procs
defined. It's designed for game logic, to both minimize boiler-plate and
dynamic dispatch. Ie, invocation it's type-specific and usually does not use
function pointers per-instance, so smaller 'events' can be inlined. Plus,
instances are generic (decoupled from object inheritance), and often don't
require any header data. That combination should, in theory (still WIP), give
devs a 'universal' way to model a broad range of game objects from heavy
single-instance types to lightweight projectiles and particles. Here's an
example for a little clarity:

    
    
      # note: 'impl', 'spawn', and 'invoke' are macros
      
      type
        Foo = object
          val: int
        
        Bar = object
          val: string
      
      impl Foo:
        proc update = echo "Foo: ", me.val
        proc render = echo "Rendering Foo!"
      
      impl Bar:
        proc update = echo "Bar: ", me.val
      
      spawn Foo(val:123)
      spawn Bar(val:"abc")
      
      invoke update
      invoke render
      invoke blah
      
      # ouptput:
      #   Foo: 123
      #   Bar: abc
      #   Rendering Foo!
      #   Invoke Error: No event 'blah' defined.
    

Perhaps that's not the best example to illustrate Nim's meta capabilities, but
so far Nim is the only language I've come across that allows me to achieve
this short of thing (at least, directly from user-code).

~~~
beagle3
Thanks, seems like the thing I'm looking for.

If you look at line 531 of the CPU source code (as of today, anyway), there
are multiple 256 byte tables that give instruction encodings, lengths, and
cycle counts.

What I was asking is: "Is it possible to put these as a parameter of the 'op'
macro so that it builds these tables automatically"

The answer might be "No" if e.g. a single op has multiple instruction
encodings. But assume that there is only one encoding per op.

An equivalent question is - can static: sections assign to an array that will
be available at runtime? From your example, the answer appears to be yes.

~~~
filwit
> can static: sections assign to an array that will be available at runtime?

The answer is yes, but it's a tad trickier than just accessing the compile-
time list from run-time code (which doesn't make sense, and is illegal).
Instead, use a macro to generate a bunch of run-time checks against a specific
value. Eg:

    
    
      var eventNames {.compileTime.} = newSeq[string]()
      
      proc defineEvent(name:static[string]) =
        static:
          eventNames.add(name)
      
      macro checkDefined(name): stmt =
        # begin new statement
        result = newStmtList().add quote do:
          echo "Checking for '", `name`, "'"
        
        # loop over every known event name and
        # build a run-time 'if' check for each one.
        for n in eventNames:
          result.add quote do:
            if `n` == `name`:
              echo "Found it!"
      
      
      # add some events to compile-time list
      defineEvent("foo")
      defineEvent("bar")
      
      # define some runtime values
      let eventName1 = "foo"
      let eventName2 = "blah"
      
      # check runtime values againts compile-time list
      checkDefined(eventName1)
      checkDefined(eventName2)
      
      # output:
      #   Checking for 'foo'
      #   Found it!
      #   Checking for 'blah'
    

Note: This will inject a bunch of 'if' statements for each call to
'checkDefined', which might bloat your code.. it's probably better to make a
macro which defines a proc, then just call that to check run-time values.. but
I left those kinds of details out of this illustration for the sake of
simplicity.

~~~
beagle3
Thanks. I'm sure there's a way to promote a compile time seq into a constant
runtime one. Might require some more macro trickery, though.

~~~
filwit
Err... what you said just reminded me of something, and I realized all the
code I just showed you is really over-complicated and that Nim has much more
straight forward options using `const`, like this:

    
    
      static:
        # define a compile-time list first
        var names = newSeq[string]()
        
        # add some values (at compile-time)
        names.add("foo")
        names.add("bar")
    
      # define the compiler vars as run-time const
      const runtimeNames = names
    
      # define some run-time variables
      var name1 = "foo"
      var name2 = "blah"
    
      # check runtime variables against const variable
      if runtimeNames.contains(name1): echo "Has Foo!"
      if runtimeNames.contains(name2): echo "Has Blah!"
    

Sorry about the rather winded (and bad example) replys :| But thanks for the
conversation, it reminded me of this and now I have some cleaning up of my own
code to get too. Cheers!

------
lqdc13
Any plans to do let people call Nim functions from Python with Python standard
objects like strings/dicts/lists as arguments? This would let people write the
fast parts in Nim and slow parts in Python.

~~~
progman
What's the point? Why not write everything in Nim?

If you depend on certain Python libs that's understandable. But it should not
be too hard to translate them to Nim since the syntax of Python and Nim are
not very different.

~~~
jboy
If a company has a significant amount of existing code in Python, no sensible
engineering manager will agree to a complete re-write of the existing code-
base. It would divert resources from more-pressing functionality (new
features, bugfixes, etc) and almost certainly introduce its own new bugs due
to the complete re-write.

(See also this classic Joel Spolsky article, "Things You Should Never Do, Part
I":
[http://www.joelonsoftware.com/articles/fog0000000069.html](http://www.joelonsoftware.com/articles/fog0000000069.html)
)

This would be the case whether you schedule the re-write in a single blocking
development effort (in which case, all forward progress would stop during that
time) or broken into batches over time (in which case, it will be much longer
until the new system is ready, and the old system will be a moving target as
it continues to be developed).

Instead, the chances of a Nim-integration being beneficial to the company (and
thus, your chances of getting approval from your engineering manager) are MUCH
higher if you can simply write NEW functionality in Nim (or occasionally re-
write small, self-contained inner loops in Nim) and the new Nim code
integrates smoothly into the existing Python as a Python module.

This is the approach I've taken at my company (with my engineering director's
approval).

In theory, you could even use skunkworks-Nim in your large Python codebase, as
long as your Nim code presents itself as a good-citizen Python module, much
like the tales of skunkworks-Scala being used in large Java codebases.

~~~
progman
I am not talking about a complete rewrite of big business software written in
Python. I just had small projects in mind which are typical for Python. Big
business usually depends on Java, C# and C++ but not on Python.

[http://www.quora.com/Why-do-large-corporations-use-Java-
or-C...](http://www.quora.com/Why-do-large-corporations-use-Java-or-C-over-
Python-Ruby-and-PHP)

------
mhd
I'm quite impressed about the small amount of code required for a NES
emulator. I thought they'd have to do all kinds of special casing for
cartridge-specific stuff…

~~~
masklinn
The more accurate the emulator is, the less special casing there's a need for:
special cases/hacks are necessary when the emulator takes shortcuts and
doesn't implement the hardware features the game relied on. If the hardware
features are fully implemented all games ought work without hacks. Hence bsnes
(as far as I know) not needing game-specific hacks but requiring a hefty
config to run.

~~~
chongli
The problem with the NES is that games have all kinds of different hardware
inside the cartridges. In order to achieve high compatibility you need to
emulate all of these chips. It's not nearly enough to accurately emulate just
the NES itself.

~~~
def-
Not all games work. Only the most popular mappers are implemented, which
covers about 85% of games.

------
doomrobo
For anyone who was wondering what I was: porting from libSDL calls to drawing
on an html canvas is done automatically by emscripten

