
Running FreeBSD on OS X using xhyve, a port of bhyve - whalesalad
https://dan.langille.org/2018/10/02/running-freebsd-on-osx-using-xhyve-a-port-of-bhyve/
======
wolfspider
I've been using this setup for years and glad to see some info here on HN
about it. It's very fast but more importantly for development it really does a
great job for porting to the real thing. I've never built anything in Xhyve
that fails on a real install. Most of the differences I've seen are in vmnet
versus like an e1000 device. I don't believe it does full duplex as per the
xhyve docs (its a TODO under virtio_net) and on the MacOS side there is no Tap
device like with Hyperkit. Despite that web pages can be served quite fast, a
lot of ops are a stub to mmap on the host. NFS runs really well as a native
format to share between guest and host. Heh, I've left it on for a couple of
months accidentally and was still running like a top. My only real gripe is
NTP drifts when the host goes to sleep and there is not an out-of-the-box
solution for that like say VMWare. Someone has gone part of the way with
getting e1000 for FreeBSD with Hyperkit it just needs to be looked into more,
but this is a good explanation- and explains the performance differences:
[http://thebsdbox.co.uk/a-peak-inside-docker-for-mac-
hyperkit...](http://thebsdbox.co.uk/a-peak-inside-docker-for-mac-hyperkit-
wait-xhyve-no-bhyve/)

------
sigjuice
I hope xhyve will become a first class supported feature that gets shipped
with macOS. I see recent commits from Apple engineers
[https://github.com/machyve/xhyve](https://github.com/machyve/xhyve)

------
Ozymandias42
I've played with xhyve and dockers version of it -hyperkit-, too. Even wrote a
little bash script to help with creation, start and edit of VM config files
for it. If anyone's interested.

[https://github.com/Ozymandias42/xhyve-
manager](https://github.com/Ozymandias42/xhyve-manager)

------
ulzeraj
AFAIK it’s not a port of bhyve but a Port of the bhyve userland tools that tap
into the MacOS hypervisor framework.

I wish there was a simpler approach to the hypervisor framework. I normally
avoid using any third party software that wants to taint the kernel and third
party hypervisors fall into that category. There is parallels desktop lite
which can be installed from the App Store but it’s extremely limited (and
paywalled) for something that I suspect it’s just a pretty interface to the
hypervisor framework.

~~~
eksu
QEMU now has ‘hvf’ (hypervisor.framework) available in master as an
accelerator, I think it was a summer of code project.

Wish I could use it, but the current implementation calls the xgetbv
instruction which is not available on my Westemere EP processors.

All MacOS/QEMU needs is PCIE passthrough and you could have an amazing setup
with thunderbolt accessories.

~~~
floam
> QEMU now has ‘hvf’ (hypervisor.framework) available in master as an
> accelerator, I think it was a summer of code project.

It was not a Summer of Code project, there was (is) a company called Veertu
that did the work who sold a virtualization product for running Linux and
Windows on macOS on the App Store. Because it used Hypervisor.framework, it
could be sandboxed and could be on the Mac App Store.

Google forked Veertu's GitHub repo, adding a few bells and whistles for use in
their Android emulator.

Finally, the work was merged upstream to QEMU.

------
xiii1408
What's the overhead of running in xhyve? Is it like a Linux kernel container,
or more like KVM, or what?

~~~
rubyn00bie
It's a virtual machine, not a container based system. So it's going to perform
somewhat like newer versions of VirtualBox I'd imagine. I'm going to take this
for a test spin and if I get time I'll post back my benchmarks here.

------
robertsd247
Isn't running any BSD on OS X redundant?

~~~
mishac
No more so that virtualizing one linux distro on another. Or running a Windows
VM on Windows....

~~~
toxik
And let's be honest, while macOS has a lot of BSD in it, it isn't FreeBSD.

~~~
torstenvl
For example: Pre-APFS, imagine running any software with a custom data store
using file system holes. Definitely something you'd rather do in a FreeBSD VM
than on OS X directly.

------
abacusprotocol
nice nice

------
iMark
It's been macOS for the last couple of years.

------
jjtheblunt
Why run FreeBSD, a sibling of macOS? It's kinda incestual.

------
chilledheart
Thanks for sharing the scripts. The instruction is intuitive -- well organized
and contains a very detailed description of the virtual disk part, very neat
comments. As a long-time user of OS X and homebrew, I must say I become very
interested in your solution -- running PostgreSQL natively on OS X, or macOS.

However, it makes me so confused that if it is your intention to run something
heavy in a consumer-level Operating System. As far as I know, macOS has a
newly-added but slowly-performed file system, named APFS. It might be good
enough for single disk request but not good for the concurrent requests (like
4; ; ) -- I am not questioning/blaming about the old port design of either
xhyve, OS X, or even hardware architecture SCSI, but that's where we can start
now.

Anyway, there is a newly-adopted storage technology named NVME emulation which
might help a lot in your case -- not only it handles with lots of disk
requests at the same time (65535 queues and 65535 commands per queue), but
also the architecture makes it possible to program the controller code in
software, especially in user world. I don't know all details of NVMe but it
works well in my experience -- both of consumer level and enterprise level --
it brings smooth disk latency in the virtual machine even under a very high
bandwidth (50Gib transfer in nearly half a minute, can you believe that?). Do
you know if it is possible to use NVMe emulation for faster disk access in
xhyve and accelerate the PostgreSQL execution natively?

\- SCSI:
[https://en.wikipedia.org/wiki/SCSI](https://en.wikipedia.org/wiki/SCSI) \-
NVME:
[https://en.wikipedia.org/wiki/NVM_Express](https://en.wikipedia.org/wiki/NVM_Express)

~~~
simonh
>However, it makes me so confused that if it is your intention to run
something heavy in a consumer-level Operating System

I assume this is to build a development environment. Especially if you're
working on a laptop, on a client-server app that needs to talk to a Postgress
(or other) database, running the database in a VM so it goes where you go is a
pretty common setup.

~~~
chilledheart
It is true and that's probably the most unclear part of the original blog. I
think the author has given some kind of advice -- use a vnet based on host-
bridge -- not sure why he just commented out from the script. In my
understanding, if I am working on a laptop, I can just set SNAT rules which
allow traffic from a private network to go out to the internet. Given that the
number of connection won't be large (no more than 400 connections, or 500, a
usual number of open file for one process), it is acceptable that the new
connections can wait until a period of time.

------
IloveHN84
It would be great if it was possible to buy MacBooks with FreeBSD instead of
OSX, which is lately bigger and slower compared to FreeBSD.l, especially due
to the useless graphic effects and the bloatware installed (iTunes as example)

~~~
simonh
I don't think iTunes comes pre-installed these days. It gets updated so
frequently they decided to just prompt the user to do an install on first
boot. Certainly if you do a fresh install of Mojave over the wire, as I did,
it's pretty clean. No iTunes, Photos, iMovie, Pages or any of their pro-sumer
stuff by default.

~~~
richrichardsson
Good ninja edit, was about to tell you that you're wrong! iTunes most
certainly does come pre-installed on 10.12, can't say about 10.14

~~~
simonh
I realized I was being a bit cavalier and did a bit of checking. Been a while
since I bought a fresh Mac, but last time I must have done a fresh install off
the wire straight away and forgotten about it. My bad.

~~~
Wowfunhappy
I'm pretty sure iTunes still comes preinstalled in Mojave. Not to mention
Chess, DVD Player, Home, News, Stocks, Books, and a fair amount of other crap.

I have a post-install script I always run that does sudo rm -rf on all the
apps I don't like. Has no ill effects in all the years I have been using it,
and makes my system feel clean.

