I've been using this setup for years and glad to see some info here on HN about it. It's very fast but more importantly for development it really does a great job for porting to the real thing. I've never built anything in Xhyve that fails on a real install. Most of the differences I've seen are in vmnet versus like an e1000 device. I don't believe it does full duplex as per the xhyve docs (its a TODO under virtio_net) and on the MacOS side there is no Tap device like with Hyperkit. Despite that web pages can be served quite fast, a lot of ops are a stub to mmap on the host. NFS runs really well as a native format to share between guest and host. Heh, I've left it on for a couple of months accidentally and was still running like a top. My only real gripe is NTP drifts when the host goes to sleep and there is not an out-of-the-box solution for that like say VMWare. Someone has gone part of the way with getting e1000 for FreeBSD with Hyperkit it just needs to be looked into more, but this is a good explanation- and explains the performance differences: http://thebsdbox.co.uk/a-peak-inside-docker-for-mac-hyperkit...
I hope xhyve will become a first class supported feature that gets shipped with macOS. I see recent commits from Apple engineers https://github.com/machyve/xhyve
I've played with xhyve and dockers version of it -hyperkit-, too. Even wrote a little bash script to help with creation, start and edit of VM config files for it.
If anyone's interested.
AFAIK it’s not a port of bhyve but a Port of the bhyve userland tools that tap into the MacOS hypervisor framework.
I wish there was a simpler approach to the hypervisor framework. I normally avoid using any third party software that wants to taint the kernel and third party hypervisors fall into that category. There is parallels desktop lite which can be installed from the App Store but it’s extremely limited (and paywalled) for something that I suspect it’s just a pretty interface to the hypervisor framework.
There was Veertu Desktop, which used Hypervisor.framework, but they have pivoted into CI for iOS. At some point it was open sourced, but it seems to have disappeared from their GitHub account. At least someone seems to have made a copy before:
> QEMU now has ‘hvf’ (hypervisor.framework) available in master as an accelerator, I think it was a summer of code project.
It was not a Summer of Code project, there was (is) a company called Veertu that did the work who sold a virtualization product for running Linux and Windows on macOS on the App Store. Because it used Hypervisor.framework, it could be sandboxed and could be on the Mac App Store.
Google forked Veertu's GitHub repo, adding a few bells and whistles for use in their Android emulator.
Mmmm eGPU + Passthrough would be sweet. Imagine gaming or using GPU heavy applications on a Windows VM with dedicated GPU and without the need for a separate set of monitor and input devices like Looking Glass does. https://github.com/gnif/LookingGlass
Was not aware of this! Thanks for sharing, this could be a great breakthrough for interfacing with hvf machines. xhyve is sorely missing an accessible management cli.
I'm seeing about 15% slower raw speed than the bare drive, dd if=/dev/zero of=tmp bs=1M count=512 is giving me between 68 and 250 mb/sec on a 2013 MBA with a nearly full disk, where in the terminal I'm getting between 78mb/s and 290mb/s.
The hard drive images are literally a qemu image for linux kvm that's been qemu-img converted to a raw format, then the kernel and initrd were extracted, and stored in the mac file system. This image was originally on my linux box for a while, before I needed to go mobile on the laptop with it.
It's a virtual machine, not a container based system. So it's going to perform somewhat like newer versions of VirtualBox I'd imagine. I'm going to take this for a test spin and if I get time I'll post back my benchmarks here.
Not at all. I used xhyve to run FreeBSD on my mac laptop, because I want to have a dev environment that matches production (except for performance wise). Mac OS's BSD userland is sort of similar to FreeBSD, but there's plenty of differences, and although it took a while to get xhyve working (and before it was available, I used virtualbox, which also took time to setup), the gains in being able to try something and know it will work on production, instead of trying something and having to try it again were worth it.
For example: Pre-APFS, imagine running any software with a custom data store using file system holes. Definitely something you'd rather do in a FreeBSD VM than on OS X directly.
I've been moving off Ubuntu in production since it adopted systemd, and a local FreeBSD VM is essential for developing & testing the provisioning and deploy scripts.
Thanks for sharing the scripts. The instruction is intuitive -- well organized and contains a very detailed description of the virtual disk part, very neat comments. As a long-time user of OS X and homebrew, I must say I become very interested in your solution -- running PostgreSQL natively on OS X, or macOS.
However, it makes me so confused that if it is your intention to run something heavy in a consumer-level Operating System. As far as I know, macOS has a newly-added but slowly-performed file system, named APFS. It might be good enough for single disk request but not good for the concurrent requests (like 4; ; ) -- I am not questioning/blaming about the old port design of either xhyve, OS X, or even hardware architecture SCSI, but that's where we can start now.
Anyway, there is a newly-adopted storage technology named NVME emulation which might help a lot in your case -- not only it handles with lots of disk requests at the same time (65535 queues and 65535 commands per queue), but also the architecture makes it possible to program the controller code in software, especially in user world. I don't know all details of NVMe but it works well in my experience -- both of consumer level and enterprise level -- it brings smooth disk latency in the virtual machine even under a very high bandwidth (50Gib transfer in nearly half a minute, can you believe that?). Do you know if it is possible to use NVMe emulation for faster disk access in xhyve and accelerate the PostgreSQL execution natively?
>However, it makes me so confused that if it is your intention to run something heavy in a consumer-level Operating System
I assume this is to build a development environment. Especially if you're working on a laptop, on a client-server app that needs to talk to a Postgress (or other) database, running the database in a VM so it goes where you go is a pretty common setup.
It is true and that's probably the most unclear part of the original blog. I think the author has given some kind of advice -- use a vnet based on host-bridge -- not sure why he just commented out from the script. In my understanding, if I am working on a laptop, I can just set SNAT rules which allow traffic from a private network to go out to the internet. Given that the number of connection won't be large (no more than 400 connections, or 500, a usual number of open file for one process), it is acceptable that the new connections can wait until a period of time.
It would be great if it was possible to buy MacBooks with FreeBSD instead of OSX, which is lately bigger and slower compared to FreeBSD.l, especially due to the useless graphic effects and the bloatware installed (iTunes as example)
I don't think iTunes comes pre-installed these days. It gets updated so frequently they decided to just prompt the user to do an install on first boot. Certainly if you do a fresh install of Mojave over the wire, as I did, it's pretty clean. No iTunes, Photos, iMovie, Pages or any of their pro-sumer stuff by default.
I realized I was being a bit cavalier and did a bit of checking. Been a while since I bought a fresh Mac, but last time I must have done a fresh install off the wire straight away and forgotten about it. My bad.
I'm pretty sure iTunes still comes preinstalled in Mojave. Not to mention Chess, DVD Player, Home, News, Stocks, Books, and a fair amount of other crap.
I have a post-install script I always run that does sudo rm -rf on all the apps I don't like. Has no ill effects in all the years I have been using it, and makes my system feel clean.
For an "average user" Linux, and FreeBSD are absolutely identical. If it's supplied with pre-installed GUI (say Ubuntu as Linux, and TrueOS on BSD side) it's more or less the same point-and-click experience. If it's bare console - well, it would be the same OMG-what's-that experience. Differences will start to appear for a user who wants to dive deeper, but that would make him/her non-average apparently.
If you're using a GUI, they're largely the same, with a slight edge to one or the other depending on your use case. For example, X on TrueOS plays more nicely out of the box with VirtualBox than almost any Linux distro I've tried.
If you're using the command line, FreeBSD has far better and far more consistent documentation.
