
QEMU v4.0.0 released - ingve
https://www.qemu.org/2019/04/24/qemu-4-0-0/
======
idoubtit
Last week, I intended to replace VirtualBox with QEMU for my personal use. The
spartiate documentation made it really hard. I lack the competence to
seriously contribute.

The official documentation just explains the numerous parameters. It also
points to tutorials (wikibook, etc), which unfortunately are very basic and
out of sync. Even Arch Linux does not provide a good documentation (lack of
clarity, some obsolete syntax, "Networking" section is a mess...). Like often,
Debian's wiki is obsolete, and even their QEMU images are 5 years old. I had
to combine the following 3 sources and a lot of try and guesses to build my
VMs.

QEMU Gentoo :
[https://wiki.gentoo.org/wiki/QEMU/Options](https://wiki.gentoo.org/wiki/QEMU/Options)

QEMU official : [https://qemu.weilnetz.de/doc/qemu-
doc.html](https://qemu.weilnetz.de/doc/qemu-doc.html)

KVM Performance : [http://www.linux-kvm.org/page/Tuning_KVM](http://www.linux-
kvm.org/page/Tuning_KVM)

~~~
kashyapc
Launching QEMU directly is not merely "tricky", but incredibly inefficient and
ineffective once you need something that's beyond bare minimal.

E.g. here is a minimal QEMU command-line that gives you access to a serial
console in the guest:

`qemu-system-x86_64 -display none -no-user-config -nodefaults -m 2048 -device
virtio-scsi-pci,id=scsi -device virtio-serial-pci -serial stdio -drive
file=/export/cirros.qcow2,format=qcow2,if=virtio`

Simple, yeah? Now here (it's too long to post in this comment) is a real world
QEMU command-line (as launched by the libvirt[1]):

htttps://kashyapc.fedorapeople.org/Fedora-28-QEMU-command-line-by-libvirt.txt

Compare and contrast the number of devices that libvirt adds.

FWIW, I strongly recommend to use libvirt to launch your QEMU-based guests. It
is far more effective, and more importantly, provides security infrastructure
like running the QEMU process as an unprivileged user, protecting the QEMU
process (and its associated disk iamges) via the SELinux-based sVirt
mechanism, and so forth.

Check out these[2] slides (recording is online, too) on "Security in QEMU"
from one of the QEMU maintainers, Stefan Hajnoczi, at last year's KVM Forum in
Edinburgh.

[1] [http://libvirt.org/](http://libvirt.org/)

[2] [https://vmsplice.net/~stefan/stefanha-kvm-
forum-2018.pdf](https://vmsplice.net/~stefan/stefanha-kvm-forum-2018.pdf)

~~~
antocv
> like running the QEMU process as an unprivileged user,

Anyone can run your example "qemu-system-x86_64 -display none -no-user-config
-nodefaults -m 2048 -device virtio-scsi-pci,id=scsi -device virtio-serial-pci
-serial stdio -drive file=/export/cirros.qcow2,format=qcow2,if=virtio" from
the command line/bash, as non-root user, yet libvirt which requires root to
install, and is expecting root to configure it, is better? How?

Libvirt is one of those softwares which ads a layer of complexity for the sake
of complexity, yet offers nothing of value in return.

~~~
throwaway2048
its not true that libvirt adds nothing of value, it adds easy networking
support, easy snapshots, a nice gui interface, a very nice remote protocol and
clients (that seamlessly support both graphical and serial output, aswell as
on the fly reconfiguration of VMs without restarting them), template VMs
(including one time initial provision setup), easy support for storage pools
including remote ones, an easy way to manipulate RAM usage on live VMs, easy
PCI and USB passthrough, live migration to other hosts, etc etc.

Sure its possible to do all this _without_ libvirt, but it comes as one
integrated, easy to manage package instead of hundreds of disparate parts.

You might not need (all) these features, but to claim libvirt is useless is
extremely parochial.

Its also untrue that it needs root, you can virt-manager for instance without
a root libvirtd instance at all, and it will run VMs without any permissions
needed.

~~~
antocv
> its not true that libvirt adds nothing of value, it adds easy networking
> support, easy snapshots, a nice gui interface, a very nice remote protocol
> and clients (that seamlessly support both graphical and serial output,
> aswell as on the fly reconfiguration of VMs without restarting them),
> template VMs (including one time initial provision setup), easy support for
> storage pools including remote ones, an easy way to manipulate RAM usage on
> live VMs, easy PCI and USB passthrough, live migration to other hosts, etc
> etc.

everything libvirt does can be done without it by talking to qemu directly.
and yes, Ive done many of the above "by hand", it was easier to read man qemu
and other qemu docs than to swallow libvirt.

~~~
throwaway2048
No, not everything it does can be done by qemu directly, plenty of it lies
outside of the scope of QEMU entirely, like managing networks (not just
network devices), or storage pools, setting up permissions for device
redirection, or doing the legwork for VM provisioning and configuration
(including net booting and automatically running appropriate scripts), or
managing access to VMs both locally and remotely.

Its fine if your use cases are covered by plain QEMU, but libvirt does a hell
of a lot more than plain QEMU alone is capable of.

Again all of that is possible via other means, but when other means involve
recreating half of libvirt in shell scripts or other code, id rather let
somebody else do the work for me.

QEMU is designed to run virtual machines, libvirtd is designed to manage VM
server infrastructure.

~~~
fulafel
The alternative is not that many commands:
[https://blog.elastocloud.org/2015/07/qemukvm-bridged-
network...](https://blog.elastocloud.org/2015/07/qemukvm-bridged-network-with-
tap.html)

~~~
throwaway2048
For one static network, what about complicated inter VM networking schemes
(including firewalling) that need to be dynamically adjusted as VMs go on and
offline?

Libvirtd is much better suited to managing that sort of complexity.

~~~
fulafel
I have the luxury of opining that such complicated networking schemes should
be reined in, and not served with baroque management software. Complexity is
the arch enemy of security, because systems must be kept easy to reason about.

------
ktpsns
I'm impressed nobody mentioned libvirt
([https://libvirt.org/](https://libvirt.org/)) here, an open source framework
for managing all kind of virtualization platforms. It is especially well
suited to take off all heavy work of managing QEMU. From the features, Virt-
Manager ([https://virt-manager.org/](https://virt-manager.org/)) can compete
with VirtualBox -- and is probably well beyond, thanks to network
transparency, the Spice protocol, real guest console access, powerful CLI
interfaces such as `virsh` and other goodies.

In Linux distributions, you basically install `virt-manager` and the package
manager will resolve all dependencies to provide you a fully working
virtualization desktop software which is well capable to be used on servers.

~~~
therealmarv
Fun fact: Virt manager (plus the good stuff like e.g. splice) runs also
natively on macOS and makes it easy to control remote Linux VMs:

[https://github.com/jeffreywildman/homebrew-virt-
manager](https://github.com/jeffreywildman/homebrew-virt-manager)

------
slig
LTT [1] did a video recently about running macOS on a Linux host using QEMU.
Anyone here tried that? If so, what's the experience like? Thanks

[1]:
[https://www.youtube.com/watch?v=ATnpEOo3GJA](https://www.youtube.com/watch?v=ATnpEOo3GJA)

~~~
dperfect
I do this using Proxmox (QEMU under the hood), and it's been mostly a nice
experience. I currently run 3 VMs on a single Linux box with 3 GPUs (one per
VM via VFIO passthrough). I've used both NVIDIA and AMD graphics cards with
success. Before Mojave, NVIDIA cards were easier with their web driver, but
Apple and NVIDIA don't get along in Mojave (both companies are blaming each
other) and there is no web driver. So for macOS, I've switched to an AMD GPU
which works mostly out of the box, apart from needing a small tweak in the
Linux host's kernel parameters ("disable_idle_d3=1" to avoid a bug preventing
the device from being used after a VM restart).

To get macOS to see my GPUs with full acceleration and audio over HDMI, I use
the Clover bootloader to inject a small section into the DSDT, mapping the
emulated PCI address to a device that actually looks like a graphics card to
macOS. This same DSDT patch works for both NVIDIA and AMD GPUs (the ones I've
tried anyway).

The best thing about Hackintosh in a VM is that you no longer need to fear
macOS upgrades. Just create a snapshot, try the upgrade, and if it doesn't
work out, you can restore, investigate, and try again.

LTT's video is a nice overview, but his claim that "because this is a VM... it
will run on _any_ hardware" isn't quite true when talking about GPUs. Unlike
the CPU, memory, and some other devices, the GPU is seen (nearly) directly by
the guest VM, so guest OS support _is in fact required_ to see the advantages
of GPU passthrough.

In summary, it's never going to be super easy, but it's not too difficult
either, and the performance (and flexibility) is actually really great.

~~~
goykasi
Thats sounds pretty interesting. Ive been running a hacked together windows VM
gaming setup via qemu with vfio passthrough for the past 1.5 years on my local
linux server. Im pretty pleased with it. Ive got it down to simply executing a
small script and everything comes up. It could be even more automated using
evdev potentially (recognizing mouse/keyboard events).

I was thinking about expanding the setup to include a beefier linux host (more
gpus and better hardware) and more seats -- windows gaming for me and osx for
my wife. But you mentioned several possible osx specific caveats: amd gpus,
clover (?) bootloader w/ dsdt (?), disable_idle_d3, etc. Can you link to some
info or READMEs regarding your setup or how you came across these solutions?

~~~
dperfect
I don't have any specific guides about my setup, but here are a few things
that have been helpful to me:

\- Good starting point for setting up macOS under QEMU/KVM:
[https://github.com/kholia/OSX-KVM](https://github.com/kholia/OSX-KVM)

\- DSDT/SSDT patching: [https://www.tonymacx86.com/threads/ssdt-gpu-graphics-
card-in...](https://www.tonymacx86.com/threads/ssdt-gpu-graphics-card-
injection.183354/)

\- AMD reset bug:
[https://www.reddit.com/r/VFIO/comments/5h351m/gpu_stuck_in_d...](https://www.reddit.com/r/VFIO/comments/5h351m/gpu_stuck_in_d3_mode_cant_initialize_vm/)

\- Kernel extension that can help with some GPU issues:
[https://github.com/acidanthera/WhateverGreen](https://github.com/acidanthera/WhateverGreen)

The DSDT/SSDT patch was probably the trickiest thing. I've posted my
particular patch here if you're interested:
[https://pastebin.com/ngvkVZYN](https://pastebin.com/ngvkVZYN)

Also, a few of my other scripts/config:
[https://pastebin.com/9Nh5rheZ](https://pastebin.com/9Nh5rheZ)

~~~
dperfect
In case anyone runs across similar AMD GPU issues, the "disable_idle_d3=1"
option in the second pastebin link (when used as a kernel parameter) should be
prefixed as "vfio-pci.disable_idle_d3=1". Even then, it can still be hit-or-
miss with some cards (it works most of the time for me, but some VM restarts
still find that the GPU has entered D3 and won't come back without rebooting
the host).

~~~
n42
is this a macOS guest specific issue? would I have similar issues on a Windows
or Linux guest?

~~~
dperfect
I believe it happens for any guest OS (search for "amd reset bug"). Some
people are using a workaround that involves removing the device (in the
sysfs), suspending the host, and then rescanning, but for me, that's not much
better than a host reboot, so when the AMD GPU occasionally does get stuck in
D3 (preventing starting the VM after a VM reboot), I just reboot the host. It
doesn't happen very often, and when it does, it's only when I'm in the process
of rebooting the VM anyway.

Previously, I also attempted using a kernel patch that's been floating around
(DECLARE_PCI_FIXUP_HEADER ... quirk_no_bus_reset), but it just gives me
corrupt graphics when rebooting the VM.

------
LukeShu
Note that this isn't a "major" change from v3.x.x; the major version is year-
based (${year}-2015). This is v4.0.0 because it is the first release of 2019.

------
wyldfire
QEMU's killer feature is user mode emulation [1]. It's never been easier to
test/experiment with cross-target binaries.

[1]
[https://wiki.debian.org/QemuUserEmulation](https://wiki.debian.org/QemuUserEmulation)

~~~
archi42
Exactly! We are using this in our compiler test farm for the quick test runs.
Sadly, the release prior to 4.0.0 broke floating point handling of NaN on (at
least) PowerPC e5500 :( I'm looking forward to test qemu 4.0.0 once I'm back
in the office again.

Edit: Tested it, aaaaaand NaN compares are still broken :(

------
founderling
Anybody here using QEMU? What is your use case?

~~~
1125
I'm using it in conjunction with PCI passthrough instead of dual boot. I've
been running Windows 10 for more than a year now and played all kinds of AAA
titles successfully. The performance-loss seems minimal. Recently I managed to
get a second GPU and spin up another VM, so now me and my partner can play on
the same machine, heaving each a keyboard, mouse and screen.

~~~
scbrg
Do you have any hints on how to set up a similar system? Links, guides,
gotchas, whatnot? I've seen someone mention this setup before (perhaps it was
you?) and I'm sure I'm not the only one interested in running something
similar.

~~~
PudgePacket
You might be able to find some info here
[https://www.reddit.com/r/VFIO/](https://www.reddit.com/r/VFIO/).

~~~
scbrg
This seems very helpful, yes. Thanks a bunch for posting!

------
mikece
I have not upgraded to the latest VMWare Workstation -- should I take a look
at QEMU first or is this something meant for a completely different use case?
(I use VMWare Workstation for contracting/consulting and build out a
workstation to match the client engagement rather than pollute my base OS with
five different versions of Visual Studio; IntilliJ, Eclipse, AND Android
Studio; multiple version of SQL Server/PostgreSQL/Mongo/MySQL... and I can
dump those VMs off to network storage when I'm done with the engagement and
reclaim the storage space on my laptop.)

~~~
yjftsjthsd-h
Yes, qemu can do that.

------
kyberias
Can QEMU be used to run a small (classic) ARM binary with simulated RAM/Flash
and can I somehow plugin my own peripheral simulations? On Windows.

------
rbanffy
I always wanted (but never succeeded) in emulating SPARC guests using QEMU. Is
there a comprehensive list of systems and hardware emulated? I suspect a lot
of people in the HN crowd would be curious about emulating Alphas, even if for
only running Genera.

Having a collection of emulated Unix workstations is an upgrade in terms of
space saved.

~~~
drmpeg
Here's a pretty good tutorial.

[https://astr0baby.wordpress.com/2018/09/22/running-
solaris-2...](https://astr0baby.wordpress.com/2018/09/22/running-
solaris-2-6-sparc-on-qemu-system-sparc-in-linux-x86_64-mint-19/)

I used a Solaris 7 .iso from archive.org.

[https://archive.org/details/solaris_7_1199_sparc](https://archive.org/details/solaris_7_1199_sparc)

~~~
unixhero
Ooh really? That works?

~~~
drmpeg
Yes. Here's a screenshot of Netscape 4.76.

[http://www.w6rz.net/netscape.png](http://www.w6rz.net/netscape.png)

That was with a Solaris 8 disk image that I found here:

[https://github.com/cclark64/qemu_solaris_sparc](https://github.com/cclark64/qemu_solaris_sparc)

~~~
kristianp
Has anyone got a Modula-2 compiler for old Solaris versions? Then I can re-
live the monochrome (2-color) xterm experience from the 90s.

------
NathanOsullivan
Note that the versioning has changed to time-based, v4.0 denotes that this is
the first release of 2019.

------
rstuart4133
I clicked on the story because of the major version number bump, then
belatedly realised they had moved to time based version numbering. The major
version number gets incremented every year.

So the version number is essentially meaningless now - the year of release
(eg, QEMU v2019.0.0) would give you more information. I can't say I'm a fan.

------
tym0
I use Virtualbox on home PC to dev. I ssh into it and use tmux/vim as my IDE.
Is there a better setup for performance on windows? Should I bite the bullet
and get myself a Windows Pro upgrade for Hyper-V? If my memory serves well you
Hyper-V can't have Hyper-V and Virtualbox at the same time.

~~~
pletnes
True. I use vmware and my ubuntu vm is faster than my windows host. This makes
hyper-v and docker on windows incompatible, though. I run i3 inside vmware. I
would expect virtualbox to be ok too, for this use case.

~~~
abdusco
You can use `docker-machine` and its vmware driver to use your VM as the
docker backend

------
shmerl
Did anyone manage to solve the problem of virgl rendering failing with

    
    
        Error starting domain: internal error: qemu unexpectedly closed the monitor
        ....
        libvirtError: internal error: qemu unexpectedly closed the monitor

------
Aardwolf
It's pretty great and has been useful for many use cases for me, except too
bad it lacks good 3D graphics support for modern gaming

~~~
qalmakka
It all depends on how and if someone will implement a Windows driver for
VirGL. Given how complex video drivers are nowadays, it looks like a pretty
daunting task.

~~~
techntoke
Intel has support for KVMGT, but AMD/NVIDIA won't support it. This would
really be the correct approach to get support from the graphics manufacturers:

[https://wiki.archlinux.org/index.php/Intel_GVT-g](https://wiki.archlinux.org/index.php/Intel_GVT-g)

------
smilbandit
I don't use QEMU and I'll always confuse it with QEMM. So every time I see a
QEMU article title I think to myself, What year is it again?

