I've found lots of resources for x86, but considerably less for x64. Pentester Academy has a promising course on it, along with shellcoding, but I'm not sure what else is out there.
If you basically don't know how to code in assembly, learn 16-bit x86 with whichever method you find, it won't be wasted. You can extend most of the knowledge from 16 to 32 bit substituting register names ax,bx,cx,dx,si,di,bp,sp y eax,ebx,ecx,edx,esi,edi,ebp,esp. You can extend that to 64 bit with rax,rbx,rcx,rdx,rsi,rdi,rbp,rsp. Learning 16-bit x86 will have you learn about segment registers, but it isn't wasted either, protected mode consists of quite a lot of complications on segment registers, virtual memory addresses (which you can mostly ignore as an applications programmer as compared to a kernel or driver programmer), and interruptions.
I wrote a book to learn x86 programming and it was published in 1994 back before "real digital age", oriented to be the best didactical text possible (I had to learn the hard way, I wrote what I would have had to read 10 years earlier). Over 20 reprints, recommended in all Spanish-speaking universities (yeah, I know, if I had known then what I know now I'd have done it in English). It was discontinued 10 years ago or so, they asked me to revise it for the modern world, and it didn't really make sense: rewriting it for what assembly language is and is used for in today's world would be a ton of work because it's a qualitative difference, and if I added an extra chapter explaining 32-bit and 64-bit changes and letting the publisher stamp a "2010 edition" logo on the cover would just be scamming people, which I don't want to do. Here is the link to a scanned copy of the original: https://www.dropbox.com/s/sz6rinfhyc8sai6/Lenguaje%20Ensambl... . Could prove useful if you can speak Spanish.
Honestly: I learned Z80 assembly first, in the 80s, and then switched to x86 very easily. Learn whatever assembly language first, what's hard is learning about registers, flags, memory... and if you learn it will, then you can switch to another architecture quite easily.
On this front, I can highly recommend these two resources, preferably in this order for someone totally new to assembly:
NAND to Tetris, a course that will have you build an emulated general-purpose CPU from first principles even with no prior knowledge. You'll learn exactly about registers and memory by making them. I even recommend this to non-hardware people because the way they divide each layer of complexity is great practice even in software. https://www.coursera.org/learn/build-a-computer
Microcorruption, a series of incrementally difficult MSP430 (an easy-to-understand 16-bit instruction set) exploitation exercises in the browser: https://microcorruption.com
These are both geared towards being a gentle introduction to assembly and CPU architecture principles. They don't touch certain facts of X86/64 processors like pipelining or variable-length instructions, but IMO those are best left until you're comfortable with the basics.
Little warning, Microcorruption may be no longer monitored at this time. Account registration is not automated.
A hacker I closely follow tried to play Microcorruption during a Twitch broadcast, and, to my great disappointment, he was unable to due to lack of response.
Luckily their "Hall of Fame" is open for anyone to see. And some of the accounts in there are pretty obvious junk accounts with the usual junk passwords. Would probably take like 5 tries to "brute force" your way in, which given the context of the site would even seem fair game.
I also recommend Xeno Kovah's OpenSecurityTraining courses on YouTube, some of which are specifically dedicated to assembly. The audio quality can sometimes be pretty bad, but the information is good. Though they try to obfuscate things a little bit, these are clearly workshops given to researchers at Mitre, the CVE project maintainer.
I also learned Z80 first, and I can say categorically that learning 16-bit x86 is NOT a good idea. To do anything useful in 16-bit, register starvation is a constant. Segmentation issues are actually hard problems.
I submit that if x64 existed early-on, no one would have bothered with higher-level abstractions. x64 is actually -pleasant- to code in natively.
I also started with the Z80, and then moved to Intel. Due to the history of the Zilog engineers there is a lot in common, and it definitely helped.
One thing I haven't seen people mention yet is actually using assembly language; just people pointing at documentation. To learn assembly, like anything else, you have to use it. I'd suggest writing a toy program to sum string-lengths, or do maths.
One of my own recent projects was to write a "compiler" to convert reverse-polish mathematical expressions to assembly language:
Started with 6502 (pretty much the only way to get any reasonable speed/working was Assembly).
Switching to 8086 was so awkward: it had built in div, instruction, the rep prefix but most importantly so many bits - 16. I routinely used ah/al and the like.
Learning Assembly is quite easy on its right own, ability to write optimal code and routinely being compilers is a whole another story. Writing the inner loops (like grep) is what's usually left to on modern processors.
There's no magic anywhere and 99.1% of it is documented.
IA-32e/AMD64 is basically an extension of IA-32/x86. What you're looking for is to understand the difference between real mode, 386 protected mode, PAE, paging and long mode because the registers, instructions, addressing and sizes of structures differ.
You can learn all of this by reading the Intel software manuals and digging into it yourself.
Yep. It's a FOSS-quality disassembler; good if you don't have IDA (I wish), supports maybe 10% of what IDA does in total but has some other niceities that it doesn't (think: disassembler-equivalent of LLVM). (Ideally, you'd have both.) For those who don't know, IDA disassembles basically every unclassified, commercially-available architecture and executable/library/object container format (think: disassembler-equivalent of GCC if it were an IBM or Green Hills product -:P). It also has a built-in scripting API in Python(?), IIRC.
An interesting semi-abandonware static disassembler that was really good is Sourcer 8.01 from V-Communications. I think it supports Pentium II/Pro/III at most, which would include real, protected mode and long mode DOS/DOS extended/Win16/Win32 EXEs and COMs IIRC. It did a lot of memory typing and clever analysis long before IDA existed, and still interesting for retro computing.
You might be interested in this talk from the creator of IDA, Ilfak Guilfanov, explaining how he used Sourcer and it's shortcomings lead directly to him creating IDA.
It's way better than IDA Pro. I have used the Pro version with the Hex-Rays decompiler. Ghidra is legitimately better. I wouldn't recommend IDA to anyone at this point because I don't see it having much of a future (sorry not sorry).
It's worth learning the Ghidra Scripting API because you can write ad-hoc scripts with Jython syntax to automate tasks/do custom analysis.
I was using ghidra for one of my uhh research projects earlier and I actually found the decompiler from hex rays still does better in some cases. In ghidra sometimes it times out and fails to decompile. Maybe I'm not using it correctly? I haven't begun exploring what kind of plug-ins and stuff it has, maybe that helps?
It depends massively on platform, architecture, and programming language. Hex-Rays is probably still ideal for embedded and obscure platforms. Ghidra's decompiler seems optimized for ARM and amd64 userland consumer-level stuff that the NSA was most interested in hacking.
When Intel was developing Itanium they named the new architecture IA-64 and retroactively named their 32-bit x86 line IA-32.
After AMD released "AMD64" Intel started copying the AMD 64-bit extensions. IA-32e was a short lived name and Intel started using "Intel 64" to refer to the 64-bit extension of x86 and continued using IA-64 for the Itanium VLIW CPU line.
After several years of denying its existence, Intel announced at the February 2004 IDF that the project was indeed underway. Intel's chairman at the time, Craig Barrett, admitted that this was one of their worst-kept secrets.
Intel's name for this instruction set has changed several times. The name used at the IDF was CT (presumably[original research?] for Clackamas Technology, another codename from an Oregon river); within weeks they began referring to it as IA-32e (for IA-32 extensions) and in March 2004 unveiled the "official" name EM64T (Extended Memory 64 Technology). In late 2006 Intel began instead using the name Intel 64 for its implementation, paralleling AMD's use of the name AMD64.
> I wonder where the x86-64, x86_64, and (the most strange of all) x64 names came from.
The "x86-64" name is the original one, and came from AMD themselves: https://web.archive.org/web/20000817071303/http://www.amd.co... (and "x86_64" is obviously an alias for where a hyphen is not an allowed character, like identifiers on many programming languages).
The "x64" name came from Microsoft, probably due to file name length limitations (this was before Windows XP unified the Windows 9x and Windows NT lines).
IIRC, the "AMD64" name came later, probably to distinguish it better from Intel's IA-64 (Itanium).
Wild guess, naming it '32bit extended' allowed Intel to still refer to Itanium as the 'real 64 bit' back in those days.
Intel didn't want the x86 to be 64bit.. they wanted the world to switch to Itanium for that. I figure a lot of the naming mess can be traced back to Intel's marketing.
Here is a book on computer architecture that has a good section on x86-64 assembly language. Please note that I have edited this comment to reflect a change suggested by a cold comment. This book (the third edition) introduces x86-64 assembly very well.
Came here to recommend the same. You can find the 15-213 course videos online. I have done the course and can’t recommend it enough. Do the labs, sincerely. You’ll learn a lot!
Working through this now (again) and its excellent. A few observations- do the labs (google them) they are even better. The lectures are also available on youtube and the recitations on something I found called panopto- they are mostly repetition from the book but nice to reinforce.
I picked up the international edition on Amazon a while back for $20 or so, significantly less than what they’re asking for now. I don’t see that one on amazon today, but I’ve seen that edition on other sites. I’d look for it.
I haven’t finished the 3rd edition, but I made it about 3/4 of the way through the 2nd edition and loved it. I picked up the 3rd specifically for the x64 material.
Honestly? Learn old-style 8086 assembly, then 386 code. The new stuff may be architecturally simpler in many ways, but it sits as an edge case on top of a very thick historical stack that seems completely insane if you look at it a priori.
But the early CPUs were actually quite simple! At the time "CISC" was a good thing because it provided straightforward mechanisms for expressing things (e.g. "push", "call", load a struct field with an offset...) that real world programmers needed to do all the time.
Most 64 bit processors only use 48 or 52 bits for physical memory addressing. So you might have a 64 bit virtual address, but that gets crunched down to your real physical memory which might only need e.g. 36 bits for 64GB RAM.
You rarely actually use 64 bit integers. So much of the processing is on strings which use 8 or 16 bits.
Furthermore, your Intel x64 processor still contains the features first introduced with the first IBM PC using the 8088 which in turn was conceptually compatible with the 8008 and 8080. When you try to understand a modern Intel or AMD CPU, you need to be aware of the legacy of 40+ years of backward compatibility. Once you understand the historical decisions, SSE, AVX architectures start making more sense.
It... kinda didn't though. I mean, yes, it seems complicated when you look back and ask "why can't it just have been a flat space", etc... But the same point persists: you have to look at what came before.
Take a look at the complexities of 8080/Z80 or 6502 addressing modes, or the kind of tricks "big" 16 bit architectures like the PDP/11-70 were playing to stretch addressible memory. The 8086 was a breath of fresh air! Your code could be separate from your stack and from your heap, and all three could be a full 64k without any crazy bank switching or copying! And all you had to do was set up 4 segment selectors and then ignore them. And everything you were used to running did so in a clean, unconstrained environment.
Really, read that 8086 datasheet again (it's like 8 pages), it was great stuff at the time.
It also carries over to OS implementations. For example, many OS designed for the 386 failed to implement demand paging for which the CPU had some great support. I believe Windows continued using segmented addressing due to the backward compatibility with earlier releases of the OS.
If you know how to write low-level C (i.e. with direct Win32/POSIX API calls), that's a good start. If you don't know what that means, you need to master this first. So much of assembly is built to support higher-level programming, things like segments, indirect references, system calls, etc., so it pays to know the WHY of all of it (e.g. do you know what a frame pointer is? why it's useful and can sometimes be omitted)? Learn this stuff first.
Once you have a good handle on C, you need to start learning how operating systems and memory management work, at least the "client side" of them accessible from userspace.
Then you might want to dip your toe into how CPUs are built, with pipelines, registers, caches, all of that.
If you've mastered those things, gcc -S will provide everything else you need (another comment suggested this).
I learned this stuff in a traditional university computer engineering program. It helped a lot. But for context, this question feels a little like, "can someone explain quantum physics"? It's a huge topic but only like 5% is the actual thing you're asking about, the other 95% is the conceptual machinery (calculus, probability, mechanics) it's built on. Mastering all the other stuff is actually the hard part. For all intents and purposes, assembly is just a notational convenience for expressing program structure in the only way a CPU can understand it. At least 90% of the difficulty is understanding how CPUs work, and how to think at sufficiently low level that you can express human-useful work at such a low level of abstraction.
It would also be nice to know why you want to know this. Writing a device driver is going to be different from writing an operating system, which will be different from writing tight numerical loops in assembly. And in any case, dollars to donuts you won't be able to beat a modern optimizing compiler performance-wise.
How can I get started on learning the low level c stuff with direct system calls and such. I learned about assembly, pipelining, cache, and memory allocation during a systems architecture course, so I'm really interested in the low level of the machine. The problem is I don't really have an idea of a project that I could do to learn the level right above. Of course I learned a bit during the course, but I really want to get a solid grasp.
I'd recommend writing something you might have used or be familiar with, in bare-metal C. Maybe try writing a simple web server, which is something most people will be familiar with. Try doing it a couple different ways: single-threaded, prefork (processes), event-based with select/kqueue/epoll. Write your own data structures: hash tables, linked lists, etc.
This sounds simple but will give you a good tour of a lot of different areas: BSD sockets, named pipes/unix domain sockets for cross-thread communication, signals, memory management, threading, basic scheduling, and parallel programming (locks/concurrency), to name a few.
If you really want to understand modern x64 or other ASM, write a simplified compiler for C. I would recommend starting with a tiny subset of the language focused on whatever aspects you want to understand. You can then compare your output with a modern compiler to get a better understanding of the intricacies involved.
Your comment about compilers is most likely true. The exception being if he needs to vectorize or if he had the opportunity use these special-purpose instructions that are seemingly built to accelerate particular algorithms.
On x86 pretty much all of the special purpose and vector instructions are accessible from C or C++ through intrinsics. No need to drop down to assembly for that except perhaps for a very, very specialized use case.
this is hands-down among best that are out there (and it's free). It is known under 2 names[1] which might be a bit confusing but it _IS_ about assembly.
This books is to ASSEMBLY what Richard W. Stevens is to TCP/IP and UNIX programming. Also it IS a beginners books because he makes no assumptions about the readers previous experience. It is very thorough so it's probably the only book you'll ever need on Assembly :)
___
[1] explanation from the author:
> What is with two titles? The book was named “Reverse Engineering for Beginners” in 2014-2018, but I always suspected this makes readership too narrow. Infosec people know about “reverse engineering”, but I’ve rarely hear the “assembler” word from them. Likewise, the “reverse engineering” term is somewhat cryptic to a general audience of programmers, but they know about “assembler”. In July 2018, as an experiment, I’ve changed the title to “Assembly Language for Beginners” and posted the link to Hacker News website, and the book was received generally well. So let it be, the book now has two titles. However, I’ve changed the second title to “Understanding Assembly Language”, because someone had already written “Assembly Language for Beginners” book. Also, people say “for Beginners” sounds a bit sarcastic for a book of ~1000 pages.The two books differ only by title, filename (UAL-XX.pdf versus RE4B-XX.pdf), URL and a couple of the first pages
> Also it IS a beginners books because he makes no assumptions about the readers previous experience
I've just started going through it, and it's definitely made some assumptions about prior knowledge; I doubt I would have been able to get through it if I didn't already have some assembly knowledge.
For practice and user-mode stuffs https://godbolt.org/ is a valueless tool to help you view the output of any C program. Simply write any C program on the left, and add "-O1" in flag inputs. Assembly with color representing line-mappings would appear on the right.
Especially if you need x64 assembly for performance optimization or during profiling a combination of godbolt (https://godbolt.org/) and Agner Fog's incredible PDFs at https://agner.org/optimize/ (especially parts 2, 3 and 4) are helpful.
The other useful step has been to write C functions and compile the code with a good compiler (good experiences with Intel, gcc and llvm) first at -O0 and later at -O2 or -O3 to understand how high level concepts can be translated to assembly.
This requires to have at least a basic understanding of registers and basic assembly concepts but at least has helped me apply the concepts to real code.
Your sources indicate an interest in reverse engineering, but another motivator for low level machine programming is performance.
For me, a great way of learning the latter is to pick a task with some instant gratification (audio dsp, software texmapping or shaders?) and start writing some of that in inline assembly.
I suggest looking at your C compiler output, learning about SIMD instructions and using a pipeline simulator to see why and how it performs like it does. I used VTune and AMD Codeanalyst back in the day, don't know what's the current SoA.
I guess I stand corrected. I was under the impression that C compilers don't tend to use it anymore when compiling for amd64. (Because you can assume SSE support on amd64, unlike a binary that might run on an old Pentium.) Then I misread that situation as it not working.
It does take effort for an OS kernel to save and restore x87 state, so I could imagine somebody dropping support.
> It does take effort for an OS kernel to save and restore x87 state, so I could imagine somebody dropping support.
FXSAVE saves the x87 state with the SSE state, and XSAVE generalizes that to add several optional components, depending on what your processor support (as of right now, AVX state, MPX state, 3 for AVX-512, PT state, and SGX state, IIRC).
It looks intimidating, but there's not actually much to learn. You need to memorize a couple dozen common opcodes. Keep the Intel manual open and flip to the page of the opcode. Also the first few sections go into detail about paging, data types, CPUs, etc. if you don't already know about it.
Try to reverse engineer something you don't have the debug symbols for. One of the big pain points was figuring out the calling convention i.e. what registers correspond to which function arguments on different platforms:
The first one is somewhat written in "weird English" (Russian English?), but it is still readable. It really helped me with x64 assembly as a C programmer. I have used 2 and 3 as reference most of the time and the first was basically my "main x64 assembly book".
I would also recommend getting more proficient in C programming either by studying books such as "Expert C" and the "The C Programming Language" or reading "advanced" stuff somewhere on the Internet, e.g.: http://www.pvv.org/~oma/DeepC_slides_oct2011.pdf
The main takeaway in all this for me was learning about the call stack and the different calling conventions which gives you a clue on how recursion works under the hood.
Also when you are done learning about "practical computer architecture", i.e. assembly language programming, learn stuff about operating systems as well:
Fun fact: this is not really related to assembly programming, but functions such as setjmp() and longjmp() are used for implementing exception handling.
"Assembly Language Step-by-Step Third Edition" 3rd Edition by Jeff Duntemann is a great book.
It starts with the very foundation and builds up, up to interfacing with C and implementing data structures, still in assembly.
Sadly, this book still references 32-bit ISA. My guess is that the author got old enough not to bother updating the book anymore (the first editions of the book were about assembly programming in DOS, then DOS and Linux, then Linux only).
Still a very valid book, as x86-64 is backwards-compatible with 32-bit x86. Also, as somebody else has written, once you understand the basics, you can mostly swap things like register names and/or look to the reference pages of your cpu/linker/assembler.
I would recommend 'Computer Systems: A Programmers Perspective' by Randal E Bryant and David O' Hallaron as an introduction before proceeding on to the official manuals and documentations.
I wrote a Linux general purpose library for x64 assembler: https://2ton.com.au/HeavyThing/ and in recent months I decided to start doing video tutorials about the "how and why" of same. Learning I think is made easiest in Linux by learning how to navigate the syscall interface and how "man 2 write" becomes an important skill. Edit: Videos link as well: https://2ton.com.au/videos/
As others points out NAND to Tetris is a really good entry point to learn assembly. It begins with you designing the hardware from NAND gates. You will make all hardware components like memory, instruction decoder, ALU, and CPU, and put them together into a computer. The next step is to build a virtual machine and then a compiler for a high-level language called Jack. The keystone project is to build a game in Jack and have it run on the computer that you build :)
I have since then ventured into Motorola 68000 assembly coding, by reading an old Amiga course in machine code from the late 80'ties. It's fascinating to learn from an old platform like the Amiga 500.
To keep myself motivated, I have written a lot of posts that chronicles my progress through the course.
Others already have given plenty of useful information, I just have a basic one.
Stay with Intel syntax, nevermind about AT&T syntax, beyond learning on how to read it.
PC world is all about Intel syntax, AT&T is lower level on how instructions get expressed and Assemblers macro capabilities just suck compared with what TASM and MASM were already capable of on the MS-DOS days.
I'm learning Z80 for programming the rc2014 kit (it's really nice kit if your also into electronics). There is a surprising lack of information online about the techniques of programming. For me that's the biggest struggle. You can know the instructions but juggling registers in an optimal way is hard. I'm currently learning I can use the stack inside a routine to have more values in the air at the same time, but then I read some example code and learn a particular instruction can be used in a clever way to avoid all my stack juggling. I am finding it can take hours to write a simple routine. I really get it now when people write it's a non-trivial problem to map C to assembly (especially Z80). Hell, it's fun though! Screw all these high level languages, let's do everything in assembly and damn the horses!!
I learned x86 from Xeno Kovah's course and I imagine his x64 will be just as comprehensive.
It looks like his x64 is just slides instead of videos which may be better, sitting through 16hrs of video when I was learning x86 was ... an experience, but a very useful one.
Anyway, whenever someone asks this question I don't hesitate to bring up his work and trainings I think they're really useful. I know this is for reverse engineering but if you can reverse engineer x64 you can certainly write it.
> One of the best 32bit x86 assembly books out there.
I think it is one of the best books about fundamental computer programming overall. Even if you don't program in assembly afterwards, you will take away a much better understanding of how computers operate. Sure, it is still a beginner book and it doesn't go into much detail, but the parts it explains, it explains very well.
I plan on making some videos on this once I get through web vulns. I've written x64 on Windows and Linux. On Windows you need to get ml64.exe, it gets installed with visual studio. You can invoke it from the command line. On Linux I usually use NASM I think, cause it supports Intel syntax.
The only really annoying thing is the stack alignment for calls. I usually just make a macro to check rsp and align if needed. Other than that I think it is similar enough to x86 just with different registers.
Kip Irvine's textbook on the subject is good. It's oriented towards Visual Studio (good, easy debugging) and windows but a fellow student had no problem using Linux tools to follow along and complete the course, which is not a bad way learn.
It's useful to at least look at such a textbook to make sure you don't skip any of the easy, sort-of prerequisite material.
In 1993, as a broke high-school student pre-cell-phone and pre-search-engine, we didn't even have that (although we did have BBS phone-number/speeds/protocols/login lists). I had to take the nascent VTA Lightrail (when it was even more uncool) to Computer Literacy bookstore (when programming was also less cool) to find decent technical references... and I promptly dropped a few hundred bucks on dead tree carcasses when I was making minimum wage. Haha. X)
- Programmer's PC Sourcebook
- PC Intern
- Undocumented PC
- Undocumented DOS
- Programmer's Guide to EGA, VGA ...
- Michael Abrash books on assembly including self-modifying code and graphics programming
I remember those books. They were all helpful in teaching assembly, bios, and general pc programming like serial ports, etc.
None of them really helped with 386 protected mode programming. GDT, LDT, selectors and descriptors, paging, etc. For that you really needed the official docs. I printed the 300+ page programmer's guide in the school lab which they were not too happy about.
I wrote a book to learn x86 programming and it was published in 1994 back before "real digital age", oriented to be the best didactical text possible (I had to learn the hard way, I wrote what I would have had to read 10 years earlier). Over 20 reprints, recommended in all Spanish-speaking universities (yeah, I know, if I had known then what I know now I'd have done it in English). It was discontinued 10 years ago or so, they asked me to revise it for the modern world, and it didn't really make sense: rewriting it for what assembly language is and is used for in today's world would be a ton of work because it's a qualitative difference, and if I added an extra chapter explaining 32-bit and 64-bit changes and letting the publisher stamp a "2010 edition" logo on the cover would just be scamming people, which I don't want to do. Here is the link to a scanned copy of the original: https://www.dropbox.com/s/sz6rinfhyc8sai6/Lenguaje%20Ensambl... . Could prove useful if you can speak Spanish.
Honestly: I learned Z80 assembly first, in the 80s, and then switched to x86 very easily. Learn whatever assembly language first, what's hard is learning about registers, flags, memory... and if you learn it will, then you can switch to another architecture quite easily.