Blink is like a Tesla sports car whereas Qemu is like a locomotive. I think what may be happening, is Qemu has a lot of heavy hitting optimizations that benefit long-running compute intensive programs. But if you just want to run an big program like GCC ephemerally as part of your build system, the cost of the the locomotive gaining speed doesn't pay off, since there's nothing to amortize over.
Blink's JIT also accelerates quickly because it uses a printf-style DSL and it doesn't relocate. The tradeoff is that JIT path construction sometimes fails and needs to be retried.
Another great example of this tinier is better phenomenom, would be v8 vs. quickjs. Fabrice Bellard singlehandedly wrote a JavaScript interpreter that runs the Test262 suite something like 20x faster Google's flagship V8 software, because once again, tests are ephemeral. It's amazing how much quicker QuickJS is. But if you wanted to do something like write a JS MPEG decoder to show television advertisements without a <video> tag then v8 is going to be faster, since it's a locomotive.
Fabrice Bellard wrote Qemu too. But I suspect his Tiny Code Generator has gotten a lot heftier over the years as so many people everywhere contributed to it. I really want to examine his original source code, since I'd imagine what he originally did probably looked a lot more like Blink than it looks like modern Qemu.
Hi Justine, QEMU developer here. Great job on Blink! You have done a lot of cool work and it's been fun to follow. I enjoyed looking at different choices you made in the frontend, for example flags handling is very different from QEMU.
QEMU's code generator is actually pretty fast and shouldn't really be expensive. It's a handful of passes that are run on individual basic blocks, certainly not optimal when a lot of code runs once as is the case for a very short compile but it's nothing like v8.
I suspect an even more silly reason—startup time might even be the biggest factor, because I think qemu-user's startup has never been optimized. I assume both QEMU and blink binaries are statically linked (or both dynamically linked, alternatively)?
Anyhow these theories should be pretty easy to disprove just by compiling something larger than hello world, so I will do it in case there's some low-hanging fruit left.
Is there a way to run linux binaries on macOS and Windows with qemu-user? As far as I can tell it only runs on Linux (and I guess BSD). Performance considerations are nice, but I think the most interesting thing about Blink is the promise of running small linux binaries on these other platforms.
No, it would be easy to do on Darwin but only if you were willing to trade off emulation accuracy and QEMU wants to be able to run any Linux program.
Even if you remain in the Unix world (so macOS but not Windows), system-call–level translation will let some of the differences between operating systems transpire to the program; and it will cause failures if a program has #ifdefs to distinguish between Linux and Darwin but then sees Darwin behavior under Linux.
On Windows, you would basically be reimplementing cygwin—imagine what a mess it would be for a Linux program to see Windows paths with drives and backslashes.
Even though blink can run any Linux program in principle, the main idea is to use it for the author's "cosmopolitan libc", to run programs that are essentially scripts compiled to machine code. So you'd write these programs portably but still be able to ship a single binary.
> Fabrice Bellard singlehandedly wrote a JavaScript interpreter that runs the Test262 suite something like 20x faster Google's flagship V8 software
Something is wrong here. How did you test this? QuickJS might start up faster on very small testcases but V8 is not that slow; it needs to have very low latency on a webpage too. Did you run a debug build or something?
I have no knowledge of what allows QuickJS to run the tests faster, or if it even does run the tests faster, but QuickJS does have one big speed advantage over V8 in some circumstances: QuickJS allows ahead-of-time compilation of JS to byte code. This removes the need to parse the JS at execution time. It’s a pretty nifty feature.
FWIW v8 does too through its "snapshot" feature. It's used by Chrome and Node.js to speed up the initialization of the standard library. It might even be faster because it includes both the bytecode and the heap.
Fascinating. Most JS code is ephemeral, i.e. rarely is something as intensive as video encoding done in the browser (and even then WebAssembly would usually be preferred).
It seems to me like browsers would benefit from running most code in QuickJS, and then spinning up V8 only for those rare cases of long-running JS?
"Ephemeral" is relative. Most JS code in the browser runs for at least 30 seconds, if not longer, as the user interacts with the page. That's plenty of time to spend spare cycles on JITing in the background to make responsiveness better without worrying about 100s of milliseconds of startup / shutdown latency.
Would it be fair to describe Blink's JIT as more of a "baseline JIT" to QEMU's "optimized JIT", or does that analogy not accurately capture what you mean in the first paragraph?
Both of them are pretty basic but Blink is closer to subroutine-threaded code than to a full JIT. IIRC it has no intermediate representation, liveness analysis or register allocation.
QEMU has all of those but only within a basic block, so it does not do any complicated data flow analysis. It's what would be considered a baseline JIT in the JavaScript world. Rosetta is similar too as far as I understand.
It's also not that easy though, and blink's code generation is comparable to QEMU in 2007 or so. Even if TCG remains relatively basic, 2x better code generation then QEMU is still a strong claim.
I suspect that the reason why blink is faster in this experiment is not related to code generation, as I mentioned in another comment. Looking again at the screenshot, the 28 context switches (vs 0 for blink) might be a clue as well.
Qemu generates pretty good code. I've skimmed through your codebase. Qemu knows all the ISA-specific operation constraints, that lets it work very similar to a compiler. All I'm saying is that Qemu generates that quality code slowly. That's what I mean by a locomotive. A train can go hundreds of miles per hour faster than a Tesla sports car. But which is quicker off the mark? Blink basically just glues together functions that were created by the compiler. So when it runs a program like GCC, which has an enormously long initialization ritual, the JIT path generator is able to plow through it at a speed comparable to memcpy.
While I have your attention, may I ask if you've considered saving your JIT's output to a file, that can be rapidly loaded if GCC is launched a second time? Ephemeral commands tend to be executed independently many times, which would seem to favor an AOT approach. But I don't see any reason why a JIT can't persist its output to gain the advantages of AOT. I'd call it EOT.
> may I ask if you've considered saving your JIT's output to a file, that can be rapidly loaded if GCC is launched a second time?
I don't work that much on QEMU TCG actually, but it would be possible to do so. In fact recent (unrelated!) changes to QEMU might even make it possible to preserve ASLR with such a first-execution JIT compilation.
That said I have rarely seen code generation in QEMU's profiles. In the end what matters is real world performance and I need to redo the test myself to understand what's going on and whether your benchmark is representative of e.g. building a small but nontrivial program (let's say blink itself) with both QEMU and blink. In that case there would be repeated cold-start recompilation, but also the compiled code would run at least once per function so QEMU would have an edge.
To be honest blink is probably more like a bicycle than a sports car. It will start faster than a locomotive, but the cruise speed is definitely lower.
> A train can go hundreds of miles per hour faster than a Tesla sports car. But which is quicker off the mark?
Top speed of most trains is below that of a Tesla let alone super cars. The trains that can go significantly faster (but not really hundreds of miles per hour faster - Maglevs - are capable of similar acceleration). This analogy is not the greatest for the point being conveyed.
I think the point here is that trains go for hundreds of miles faster than a Tesla. If you're going 800 miles, a train might well get there first, though a Tesla's top speed is 4x the trains.
I think this comes down to a difference is perspective of trains. Coming from a European perspective when I first read the analogy I was imagining the incredible fast passenger trains over here (~320km/h).
I suspect if you are coming from the perspective of a US rail user then yes, they aren't exactly known for high speed train travel.
320 km/h is not hundreds of miles per hour faster than the top speed of a Tesla, especially one like the Plaid marketed for speed. I don’t give a fuck if you’re from Europe or the US, high speed trains aren’t faster than high speed sports cars. This is from the perspective of someone who’s not an idiot.
Last time I checked on blink, it couldn't run dynamic executables (or ELFs that run with the dynamic interpreter), and would result in a segfault. How has it improved since then?
Blink really isn't intended to run the binaries that come with your distro, because you're already able to run them. Blink is intended to let you transplant x86-Linux executables onto non x86-Linux systems.
For example, I like to build programs on my x86 Alpine Linux machine and then scp them onto my M1 MacBook, Raspberry Pi, FreeBSD, and Windows machines. Blink lets me run them once I do that. Copying program files only makes sense if the program files are static. Linux distro executables can't be distributed to somewhere other than the distros that created them, unless a tool like Docker is used which recreates the whole distro.
Blink is for people who want to be able to distribute Linux software without having to be in the Linux Distributor business too.
If the binary is linked against a old enough version of glibc, it should run on most non exotic Linux distributions, shouldn't it?
At least I feel I'm regularly installing things from people that are not in the Linux Distributor business and are not huge go binaries either (though admittedly, those seems to be trendy as well :) )
Yes... You can technically spin up a RHEL5 VM and compile your code using GCC 4.2.1. That's what I think the Python community still does for manylinux1. The problem with that is GCC got so much better in the last fifteen years, that you might as well be programming with one hand. We shouldn't need to do that. There's nothing wrong with modern GCC 9+ which can produce backwards compatible binaries just fine. The problem is the policy decisions made by the Glibc developers. So if you can avoid Glibc and dynamic linking, then distributing to many distros and operating systems while using modern tools becomes easy.
Those binaries might also just be statically linked with MUSL (at least that's how I'm building my distro-agnostic Linux command line tools). Same idea as Go binaries though (except that I haven't noticed that such binaries are particularly big versus binaries that link against glibc - of course you need some 'meat' in the tool, not just a plain 'hello world', but the difference is measured in kilobytes, not megabytes).
Blink is primarily a user mode emulator, but it does support real mode BIOS programs. It can even bootstrap Cosmopolitan Libc bare metal programs into long mode. Here's a video of Blink doing just that. https://storage.googleapis.com/justine/sectorlisp2/sectorlis...
KVM requires additional privileges. A Linux container would need privileged rights and access to /dev/kvm to run QEMU with KVM for example, whereas any container should be able to run it in user-mode.
That's not really an issue, as there's a lot of infrastructure around optionally giving device file access to containers. That's why SECCOMP_IOCTL_NOTIF_ADDFD exists.
Nobody's really set it up to do that as it's easier to use Linux's sandboxing features if you're looking to run user code of the same cpu ISA. GVisor has an (experimental last time I checked) backend that uses KVM to run user mode code, but there you have the win of the sandboxing code being written in a memory safe language and giving you a real privilege boundary as opposed to the sieve that qemu-user is. In just about every other instance just running code natively in regular user space (even if sandboxed with seccomp or a ptrace jail) achieves the underlying goals better.
It depends on whether you're more afraid of language bugs or hardware bugs. One potentially nice thing about having a tool like Blink that can fully virtualize the memory of existing programs, is it's sort of like an extreme version of ASLR. In order to virtualize a fixed address space, you have to break apart memory into pieces and shuffle them around into things like radix tries, and that might provide enough obfuscation of the actual memory to protect you from someone rowhammering your system. I don't know if it's true but it'd be fun to test.
Well, not quite 'native'. TLB refills are 4x to 5x as expensive, and anything that needs a context switch tends to be at a minimum twice as expensive, and it's common to balloon even farther from there.
I guess that's mostly if you are running a full operating system inside it, generally in Qemu. It doesn't have to be - could just be a program. Tiny programs running in KVM can use big pages and never cause or require any pagetable changes.
The guest has it's own page tables above the nested guest phys->host phys tables.
> What context switch time? It takes 5 micros to enter and leave the guest. The rest is just "workload".
And then the kernel doesn't know what to do with nearly every guest exit on KVM, so then you trap out to host user space, which then probably can't do much without the host kernel so you transition back to kernel space to actually perform whatever IO is needed, then back to host user, then back to host kernel to restart the guest, then back from host kernel to guest. So six total context swaps on a good day guest->host_kern->host_user->host_kern->host_user->host_kern->guest.
Right, that's very true! It's clear that you know what you're talking about when it comes to KVM and maybe even the internal structure in Linux. However, I/O can be avoided. Imagine a guest that needs no I/O, doesn't have any interrupts enabled, and simply runs a workload straight on the CPU (given that it has all the bits it needs). That is what I have made for $COMPANY, which is in production, and serves a ... purpose. I can't really elaborate more than I already have. But you get the gist of it. It works great. It does the job, and it sandboxes a piece of code at native speed. Lots of ifs and buts and memory sharing and tricks to get it to be fast and low latency. No need for JIT, which is a security and complexity nightmare.
The topic of this thread is about Blink, which happens to be a userspace emulator. Hence my comment.
I work on a C library. Some of the functions I've written, like memmove(), take about 7 picoseconds per byte for sizes that are within the L1 cache, thanks to enhanced rep movsb.
That's a very special case though since it's hardware optimized to work up to a cache line at a time, and not at all related to the syscall cost that was mentioned in the parent comment.
The 5us was the setup time in order to be able to enter the sandbox. A system call is around 1us, but rarely used. So, in general the overhead of using the sandbox is around 5us, as everything else is pure workload.
Very simply. Took three hours to code. Blink just takes a "screenshot" of the ANSI terminal display, gzips it, and appends it to a ring buffer with 64,000 entries. Each screenshot is a full frame. There's no delta encoding or anything smart like that. 64k gzipped frames takes a few hundred megs of memory once it fills up, depending on how big your terminal display is. My favorite part is that, when you press the 'c' button to continue execution, Blink will only transmit 60 FPS to your terminal. In-between frames, it's still capturing screenshots in the background, which get added to the ring buffer. It'll save thousands of frames per second, so you can skim through execution pretty quickly, like a tv show, and then maybe you'll see something happen, smash ctrl-c, and scroll backwards slowly with fine-grained per-instruction granularity, thus illuminating the event you saw fly by earlier. It's so nice. Especially if you're used to tools like GDB TUI, which permits scroll-wheeling down the assembly display, but it won't let you scroll-wheel back up to the thing you were just looking at!
Oops, maybe not that high-level. I meant, underneath the UI layer, how is the state/execution path of the emulated CPU efficiently recorded during execution and later rewound? Any approaches I can think of would destroy the performance. I have a toy emulator project that dumps the entirety of RAM every frame and I'm trying to come up with ideas to improve its efficiency.
Blink's reverse debugging is implemented purely at the user-interface level. It doesn't have anything to do with the execution engine. While Blink does have near-perfect knowledge of memory, I considered doing the thing you proposed, where deltas are stored and reconstructed. I decided that it wouldn't offer compelling benefits beyond just capturing and rewinding screenshots. For Blink's use case, screenshots are arguably superior, since it preserves perfectly the UI states previously observed by the user. I understand this might not be an option for your project, since you might not have a display that's visualizing program memory and execution. In that case, I'd suggest why not build that first?
That's a really slick approach. I reread your first post again and you explained it perfectly, I just didn't grok it the first time around for some reason. Maybe I thought it couldn't be that simple... but it is! Thanks for patiently explaining twice.
Is it actually 2x faster or 2x faster at starting up? QEMU does so much stuff, running cc1 on hello world isn't really a stress of the interpreter IMO as much as all the crap that goes around it.
Blink actually does run the GCC9 CC1 command from start to finish twice as fast. Qemu takes 600ms to run it and Blink takes 300ms. Both Qemu and Blink use a JIT approach. Since GCC CC1 is a 33mb binary, a lot of the time it takes to run it, it stresses the JIT pretty hard. https://twitter.com/JustineTunney/status/1610276286269722629
That's partly what I meant though, how fast is it at a longer running process? C doesn't require all that much semantic analysis so there usually isn't all that much hot code in the compiler, so it would suit a simple-fast JIT whereas QEMU does do some basic optimizations.
I've only ever really skimmed the TCG source code but it wouldn't surprise me if a new-er JIT could smack it's arse given that with these old C codebases (it's probably one of Bellard's few flaws) it's pretty hard to actually make true architectural changes.
The Java/script (I think more Javascript but I'm hedging my bets by including jvms too) JITs are probably the cutting edge but I'd imagine still quite beatable for a few cases.
Rosetta is closer to QEMU than to v8 or Hotspot. Granted, it benefits from an insanely large out of order execution engine, but it shows that there's only so much you need to optimize when translating assembly code.
Looks like it itself is not yet able to be compiled with Cosmopolitan Libc (though it emulates programs compiled with it) but it's planned - very cool!
Absolutely. If you download last year's release of Blinkenlights, you can actually use this software on Windows today. It works great in the Windows 10 command prompt or powershell. https://justine.lol/blinkenlights/download.html
I've wanted that for the longest time. You know the feeling. You mostly write something like Java and Bazel works fantastically at that. But then you find out you need about 100 lines of C code in your build to do some specific thing Java can't do, and suddenly you've opened pandora's box because compiling native code across platforms is a nightmare. Life would be so much easier if we could just vendor a 40kb prebuilt Linux binary that could run on all the other platforms too. All you need now to do that, is a few additional ~200kb blink binaries.
Does it require cosmopolitan libc? I compiled a hello world example in C with `zig cc --target=x86_64-linux-musl` but when running it with Blink, nothing happened.
With memory safety turned on, could Blink be used as an alternative to WebAssembly?
At a glance, the debugger user interface looks much nicer than gdb's terminal ui. How tightly coupled is the debugger interface to the emulator/debugger engine? How much work would it be to plug in a different debugger, say lldb or gdb, into the ui instead of blink?
I think the user experience of cli debuggers is generally somewhat dreadful when compared to their gui cousins -- they seem to display a much narrower view of what's going on. Could the big blinkenlights debugger view be useful outside of blink itself?
Why would you open a blog post about X not with a link to it, but a link to the author's twitter page? That has no use to many of us, who can't even scroll down through the tweets looking for a link should we feel... spelunky....
So this is 600 versus 300 microseconds, and the difference could be in the startup time. Wouldn't it make much more sense to test a longer running program?
You must be from a country like Germany my friend, because the comma is part of the number. It actually reads ~330000 microseconds. I really should switch to using an apostrophe instead.
Blink is a new CPU emulator written in C, made by Justine Tunney. Besides having a really cool name, blink has a lot of impressive features and some of them will blow your mind!
Congrats for the author. Though I wonder: what's the catch when it comes to perf? I can't imagine QEMU devs leaving much performance on the table either.
Blink is currently weak in a few areas, like SSE floating point instructions, which aren't JIT'd very well. Blink is also weak at x87 floating point emulation, which gets truncated to double precision. I anticipate issues like these will improve quite rapidly. It was only two months ago, before I started optimizing this project, that Blink went the same speed as Bochs (i.e. 100x slower). Today, Blink is running slow start ephemeral programs such as GCC 2x faster than Qemu, and Blink now has comparable speed at some mathematical benchmarks, such as elliptic curve crypto.
