
A simple x86 assembler C++ template metaprogram - chesterfield
http://blog.mattbierner.com/stupid-template-tricks-template-assembler/
======
rayiner
Years ago I made an x86 assembler using Common Lisp macros to expand
instruction descriptions:
[https://github.com/rayiner/amd64-asm](https://github.com/rayiner/amd64-asm).
Meta programming is incredibly powerful, more so when your macro language is
just your host language.

The tests suite is neat (imho). At runtime it walks a data structure built at
compile time to randomly exercise each of the possible encoding clauses of the
instruction and compare the output to Nasm. Writing that was a very "lisp is
for building organisms" moment for me.

As an aside, x86 makes a lot more sense than people give it credit for.

~~~
Keyframe
And I asked about this just a few days ago!
[https://news.ycombinator.com/item?id=10239000](https://news.ycombinator.com/item?id=10239000)

Do you use it? Is anyone aware of a python way of this?

~~~
tptacek
I wrote a programmable assembler in Python on a consulting project (I needed
to write a debugger that used only direct memory access and no other debugger
hooks). It wasn't difficult.

Later, we did one in Ruby, which allowed our assembler syntax to almost mirror
AT&T assembler while still being pure Ruby.

These are easy libraries to write; people should just write them themselves.

~~~
Keyframe
_These are easy libraries to write; people should just write them themselves._

Well, if you say so! Seriously though, isn't x86/64 opcode set gargantuan in
size? Not to mention the maintenance and what to do with parallel constructs.
It doesn't sound easy to me, apart from trivial text emit experiments. I
haven't tried writing one though, so I wouldn't know. Also, what's easy for
someone isn't easy for others. I find image processing and rendering
relatively easy for example.

~~~
tptacek
(a) No.

(b) You don't need even _most_ of the instructions to get to a useful, fun
place.

I was surprised by how easy it was to get to that place. Take my word for
this. Read this:

[http://reocities.com/SiliconValley/heights/7052/opcode.txt](http://reocities.com/SiliconValley/heights/7052/opcode.txt)

and just start coding.

~~~
Keyframe
Ok, thanks for the tip and link. Weekend here I come!

------
kristiandupont
I made something very similar ages ago:
[https://github.com/kristiandupont/rtasm](https://github.com/kristiandupont/rtasm)
\-- I remember struggling quite a bit with machine code generation until I
discovered some deeply buried forum message emphasizing that the x86
architecture was originally octal-based.

~~~
userbinator
_until I discovered some deeply buried forum message emphasizing that the x86
architecture was originally octal-based._

To "unbury" that fact some more, I'll again link to the message I think you're
referring to...

[http://reocities.com/SiliconValley/heights/7052/opcode.txt](http://reocities.com/SiliconValley/heights/7052/opcode.txt)

(Google _still_ refuses to index that link for some reason. IIRC it used to
have a very high ranking, but disappeared from search results within the last
year or so.)

~~~
wolfgke
There is an updated version of this text:

>
> [http://www.dabo.de/ccc99/www.camp.ccc.de/radio/help.txt](http://www.dabo.de/ccc99/www.camp.ccc.de/radio/help.txt)
    
    
      Some bugs were corrected June, the 20th, 1997 by S.Klose (sven@devcon.net)
      (minor bugs in 32bit effective addresses and opcode typoes)

------
userbinator
The syntax vaguely reminds me of HLA:

[https://en.wikipedia.org/wiki/High_Level_Assembly](https://en.wikipedia.org/wiki/High_Level_Assembly)

------
ant6n
I'd be cool if one could leave runtime elements in there, for example leaving
an intermediate value as x, with the C++ template magic reducing everything
down to producing the sequence of bytes with just one integer to be filled in.

Then this could be used for on-the-fly code generation like you'd see in a
jit-emulator like qemu.

------
aidenn0
For use of a subset of C++ as the assembly language for your architecture, see
[https://www.youtube.com/watch?v=LitwrElTCLs&t=11m33s](https://www.youtube.com/watch?v=LitwrElTCLs&t=11m33s)

------
jheriko
i used to use that same function pointer cast trick to execute bytes from
memory a long time ago.

you should find that it fails in most modern execution environments...

in legacy/desktop windows you can virtualalloc things to be executable but the
same is not true in the general case, and this code doesn't do that.

even a long time ago (before windows 8, ios etc.) i wasn't able to just
execute things on the stack or heap reliably and had to write code that
explicitly allocated executable memory.

~~~
lultimouomo
The assembler strings are not allocated at run time, but stored in a read only
section of the program (which apparently is marked as executable, if this
works)

~~~
jheriko
yeah, but the fact that the static data area is marked as executable is luck
rather than reliability.

------
neelm
This type of approach works well in image processing, where you can use C++
for the repeatable, hierarchical patterns and assembly for optimizing some key
loops.

------
tdsamardzhiev
Expected to see brainfuck code. Actually saw a pretty neat collection of
tricks! Well done :)

------
tempodox
fork(2) me, this is one cool application of templates. I wish it were as
comprehensible as Lisp macros :-)

