
macOS in QEMU in Docker - mjtlittle
https://github.com/sickcodes/Docker-OSX
======
etaioinshrdlu
So this is pretty misleading. It's really a full system emulator (qemu)
running inside Docker, using root privileges on the container that make the
isolation very weak (--privileged).

It also uses hardware assisted virtualization (KVM) which is not going to be
available most of the time Docker is.

You can think of the Docker platform itself as subset of the Linux platform.
With many common features removed by default... SYS_PTRACE, cgroups come to
mind as not allowed within the container. (This "Docker as a subset of Linux"
is also what you end up getting from most "Docker as a service" platforms
offered by clouds, including kubernetes. I'm referring to AWS Fargate, Google
Cloud Run, GKE, AKS, here.)

So don't think of this as macOS in docker wherever docker runs.

What would be a lot more analogous to macOS in docker would be running Darling
in docker: [https://www.darlinghq.org/](https://www.darlinghq.org/) ... if
that could be made to work for the entire system (highly unlikely)

Darling is more like Wine in that it runs native executables for one platform
as native processes on another platform using a compatibility layer. Wine, by
the way, definitely works quite well inside Docker.

Also, one final thought. I wonder if you could get macOS to boot in QEMU
without hardware assisted virtualization. Then you could probably run this in
a fully isolated container again. The performance would likely be abysmal
though!

~~~
hinkley
I don't care though. What I care about is that it's a pain in the butt to do
CI/CD pipelines for an application with iOS/OSX support. So if someone has a
headless OS X contraption on offer, I want to hear some more about it.

The last time I set this up, a manager decided he wanted a laptop like the
rest of us instead of the iMac he got. He asked semi-jokingly if someone
wanted the machine for anything and I said "Yes, I do" before he even got the
sentence out.

There was just enough memory on the machine for me to set up a few Jenkins
agents on it, one for Safari, the rest using the Selenium-maintained docker
images.

~~~
jcelerier
> So if someone has a headless OS X contraption on offer, I want to hear some
> more about it.

you still legally need to run it on an actual Mac due to Apple's terms of
service though... so is there really a lot gained ?

~~~
WatchDog
If a tree falls in a forest and no one is around to hear it, does it make a
sound?

~~~
hosh
If we are going to divert into Zen koans and non-dual philosophies, then it is
definitively "yes it did". The falling away of the perception of an
intrinsicly-existing self doesn't change that there's still perception. It
would make a sound the same way there is a sound made when one hand claps.

Back on topic though: it's too bad Apple doesn't allow licenses for running
things headlessly like this.

~~~
serf
>Back on topic though

You over-philosophized to the point of bringing their kitschy koan off-topic.

How I interpreted the use of the koan : Apple has no history of legally
chasing those who virtualize their operating system; as this is a non-topic
thusfar -- who cares?

The hammer may fall one day, but so far 'why are we worried about a legal
response that doesn't seem to exist?'.

The answer, of course, is that anyone who builds product based on a legally
grey area is at risk when that area begins to crumble.

>it's too bad Apple doesn't allow licenses for running things headlessly like
this.

agreed, but I think Apple wants to drive everyone to a hardware solution.

At one point 'enterprise-ish' hardware was offered, but now it seems that it'd
be in their interest to offer virtualization licenses while trying to smooth
out whatever troubles exist between their software and the major VM hyper-
visor offers out there -- mostly since there are huges holes in their hardware
offerings for those seeking to do 'enterprise-ish' things en masse.

~~~
danieldk
_Apple has no history of legally chasing those who virtualize their operating
system_

Maybe not macOS yet. But the definitely chased people who virtualized iOS:

[https://www.macrumors.com/2019/08/15/apple-corellium-
copyrig...](https://www.macrumors.com/2019/08/15/apple-corellium-copyright-
infringement-lawsuit/)

------
cercatrova
I just want a good CI/CD system for macOS to build iOS apps without needing to
buy a farm of Mac Minis, or even buy a Mac, is that too much to ask?

~~~
Andrew_nenakhov
According to Apple software license agreements, yes, it is too much to ask.
You can not legally run Mac OS on non-Apple hardware.

But don't be sad, their new Mac Pro is _fantastic_.

~~~
benbristow
It's also £5,499.00 in the UK (just under $7k USD).

Not exactly ideal for hobbyists.

~~~
Andrew_nenakhov
I considered including a price tag (... _and comes for only $6990_ ), but
decided against it for being too thick. I also think that using /s markers
kinda kills the idea of sarcasm.

~~~
umvi
Poe's law is especially strong with Apple, /s is definitely needed unless you
go over the top with praising the price.

~~~
411111111111111
> _unless you go over the top with praising the price_

most apple enthusiasts think that the price is spot on and in no way
overblown. (i do not have an opinion on the matter as i neither own nor use
apple devices)

i dont think you _can_ convey any point about apple in a sarcastic manner
while omitting a /s tag. there are always people honestly believing the point
you're stating sarcastically.

this applies to both negative and positive statements

~~~
youngtaff
I use Apple gear and think it's mostly overpriced.

I Need to buy a new MacBookPro and even though I can reclaim the VAT (sales
tax) and it'll come out of corporate accounts the price makes me wince

Oh and I'd like one without a touch bar but screwed on that front unless I go
high end MacBook Air

iPhone SE at £400 is pretty reasonably priced though and will be my next phone

------
miles
More info on the project in this reddit thread from a few hours back:

[https://www.reddit.com/r/jailbreak/comments/gwg3e4/free_rele...](https://www.reddit.com/r/jailbreak/comments/gwg3e4/free_release_dockerosx_run_xcode_on_linux_sign/)

The developer mentions[0] that, like macOS-Simple-KVM[1], this leverages
kholia's OSX-KVM[2].

[0]
[https://www.reddit.com/r/jailbreak/comments/gwg3e4/free_rele...](https://www.reddit.com/r/jailbreak/comments/gwg3e4/free_release_dockerosx_run_xcode_on_linux_sign/fsuz29i/)

[1] [https://github.com/foxlet/macOS-Simple-
KVM](https://github.com/foxlet/macOS-Simple-KVM)

[2] [https://github.com/kholia/OSX-KVM](https://github.com/kholia/OSX-KVM)

------
sjburt
It's hard to see what Docker is adding here since qemu is being run inside
Docker. You could get almost identical functionality out of a bare VM image
and not deal with the hassles of docker.

~~~
qayxc
It's a hammer problem. Docker is the new hammer and now everything starts to
look like nails.

~~~
hinkley
It's the hammer problem because management and IT have _locked up all of the
other tools_ , so you're damned right I'm gonna try to use the hammer.

Spooling up a docker image is low friction at a lot of places. Doing anything
else can take an act of Congress.

~~~
dahfizz
> Doing anything else can take an act of Congress.

Does fiddling with BIOS parameters and installing kernel modules fall into
"anything else"? Because this project doesn't work until you've done that on
your docker host.

~~~
hinkley
Not in this case, but in qayxc's more general statement.

------
benologist
Can macOS virtual machines ever be performant enough to use as a workstation?
So far I have only tried setting it up in VMWare and VirtualBox, the
performance wasn't there but I haven't dedicated a GPU or drive to it yet. It
would be so convenient to decouple macOS from Mac hardware.

~~~
mjayhn
The problem is cocoa performs terribly without gpu acceleration and nobody has
figured out how to get around that, there are some tweaks to get OSX running
in vmware and wherever else but you never wind up with working GPU
acceleration so not only can you not change the resolution once you turn it on
(iirc, may be wrong here), it's refresh rate is horrendous. If you ever used
Windows before it installs the gpu drivers where the window manager is all
weird and unoptimized and glitchy, it's that but worse.

It's been a long time since I played with macos vfio passthru stuff but maybe
that's a way around it nowadays. There's a little /r/vfio community that tries
to tackle it pretty frequently.

Hopefully someone else has more recent details than me, I'm back to using osx
hardware now that the 16" mbp lets me have 32gb.

~~~
schlopper
QEMU/KVM/VFIO has come a long way. If you have a MacOS-supported GPU and
working IOMMU (AMD) or VT-d (Intel), then you can achieve near-native MacOS
performance for your CPU/GPU combo.

------
wdb
I have always wondering how Azure, BrowserStack and such support Safari or
macOS. Do they have custom licensing with Apple to allow to run it virtualised
or are they actually running it on Macs?

If I remember correctly you can only run macOS on their hardware.

------
joshlk
Does this break the user agreement for macOS?

~~~
jdboyd
Only if you do it on a non-Apple comoputer. If you install linux on your mac,
then follow the guide I believe it should be fine.

~~~
ac29
Not a lawyer, etc, but I read this as requiring the VM to be run on macOS
host:

allowed "to install, use and run up to two (2) additional copies or instances
of the Apple Software within virtual operating system environments on each Mac
Computer you own or control _that is already running the Apple Software_ "

[https://www.apple.com/legal/sla/docs/macOSCatalina.pdf](https://www.apple.com/legal/sla/docs/macOSCatalina.pdf)

~~~
mindfulhack
IANAL either, and also I only read that PDF for a minute, but: maybe the Apple
firmware running on the motherboard, a T2 chip etc, can satisfy the
requirement that it is a computer "already running the Apple Software", even
with Linux as the host OS. I think "the Apple Software" carries broad meaning
in that agreement (and not very exclusive in its definition), e.g. clause 1:
"The Apple software (including Boot ROM code)"... ?

~~~
hunter2_
The definition is broad (and self-referential, oddly): The "Apple software" is
defined as "The Apple software (including Boot ROM code), any third party
software, documentation, interfaces, content, fonts and any data accompanying
this License"

If a subset such as "Boot ROM code" on the VM host was sufficient to allow for
using "Apple software" beyond said subset in VM guests, then any other subset
(such as "fonts") would also have to be sufficient, and no reasonable person
would agree with such a thing. Therefore, it follows that the _entirety_ of
"Apple software" must be "already running" on that "Mac Computer" before
booting any VM guests that use "Apple software." This interpretation is
supported by the use of the word "and."

~~~
mindfulhack
Upon second look, it would seem that the spirit of what they mean by "the
Apple Software" is 'the whole set of standard Apple software (that comes pre-
installed on the Apple machine'.

But actually, even more strongly in favour of interpreting that they don't
specifically license macOS for use on a Linux host via their agreement, is the
syntax of 2.B.iii itself (italicisation for emphasis):

> [you are granted a license] to install, use and run up to two (2) additional
> copies or instances of _the Apple Software_ within virtual operating system
> environments on each Mac Computer you own or control that is already running
> _the Apple Software_ "

Their clear syntax of repeating "the Apple Software" in the context of both
guest and host environment indicates that what is used virtually must also be
used on the host.

Additionally, you probably couldn't get out of it by dual booting with Linux
and saying that 'aha, see, I have mac running on the host machine I'm fine',
the grammar of the words "that is already running" indicates that macOS must
be running _while_ using macOS as a guest, under their license.

I can't imagine any serious legal implications that really matter apart from
to major corporation making major money off virtualising macOS somehow. To
anyone else or any other angle relating to it, I don't think there's any worry
whatsoever. It appears the Hackintosh community hasn't been sued into
oblivion...

------
hosh
Does this mean I can run Xcode to do iOS and Unity builds on a Linux host?

~~~
read_if_gay_
Yes. But for testing on actual devices I am pretty sure you need a spare USB
controller to pass through. When I tried a few months ago, there was no other
way to connect USB devices.

------
fellovv
This was done by SpaceInvader for the UnRAID docker community months ago. As
stated, it is just streamlining a process that has been available with KVM-OSX
with various bash scripts for months or years before that.

------
867-5309
on the model of "Hackintosh", I am inclined to propose "Dockintosh"

------
tarkin2
OT: Has anyone Found virtualization really resource in the latest OS X? It was
eating up 25 percent of my RAM doing nothing. Both Docker and Vagrant were so
resource hungry that I ended up ditching them.

~~~
JaggedNZ
The last version of Docker for Mac had some big resource issues, current
latest version seems to be better, but still makes my 2017 MBP into a grill
when running multiple containers.

------
acd
Great work! I would add a disclaimer that you should only run this on Mac
hardware. Ie you can run Arch on mac hardware. Get an old mac and point to it
if someone asks.

------
miguelmota
Has anyone had success running the docker image on Arch? I ran into some
issues but wondering if it's me or the instructions

~~~
miguelmota
Issue resolved [https://github.com/sickcodes/Docker-
OSX/issues/7](https://github.com/sickcodes/Docker-OSX/issues/7)

------
maliker
Now if we could just get a windows docker container running on unix hosts I
could debug my cross-platform support.

~~~
yjftsjthsd-h
I mean, this is just a VM in a container, so you could totally make an
equivalent running NT rather than Darwin in the VM.

------
ChrisRR
How well does this run? OSX runs like molasses in Virtualbox under
Windows/Linux

~~~
paulcarroty
Works much faster with qemu-kvm.

------
joshgel
Does it run iMessage on Windows?

------
ianmobbs
Could this be used to run an iMessage proxy on a server?

------
Dilu8
Lrnix os is best on macOS

------
MuffinFlavored
Does it have to be a Linux host? Can it be a Mac OS X host?

~~~
amaccuish
I'm pretty sure since Docker uses bhyve on mac, so that would be a nested
VM...

~~~
sigjuice
Docker has their own xhyve derivative called hyperkit. Nested VMs are not
supported
[https://github.com/moby/hyperkit/issues/127](https://github.com/moby/hyperkit/issues/127)

~~~
chrisgacsal
If I am not mistaken then xhyve and hyperkit are separate projects utilizing
the same Hypervisor Framework from Apple under the hood.

~~~
lloeki
hyperkit is a fork of xhyve, plus stuff like bridges between inotify and
kqueue/fsevents, or transparent tcp port forwarding.

Hypervisor.framework is an API to execute hypervised code, to build
virtualisation engines that can run unprivileged; you still have to write an
actual virtual machine.

------
mister_hn
Oh yes, finally.

With this, you can do pretty everything

~~~
self_awareness
They should put it on zombo.com

------
sequoiar6868
amazing project

------
thepiwo
my container got the name "silly_torvalds" do you want to tell me something
Linux Kernel?

------
flexvision
Edit: Okay, I understand my assumption was wrong. Thanks

Doesn't docker run a container in a single thread? So this would be running
the entire MacOS in a single thread? Is there a way to tell Docker to execute
this in multiple threads?

~~~
ecnahc515
No, that isn't how containers work. Containers run processes, as many as you
want, each which can use as many threads as they (up to ulimits). Most
hypervisors will allocate 1 thread per virtual CPU core by default, and since
this is using qemu with KVM then that's likely the case.

Looking at the Dockerfile in the OP, you can see it's using
[https://github.com/kholia/OSX-KVM/blob/master/OpenCore-
Boot....](https://github.com/kholia/OSX-KVM/blob/master/OpenCore-Boot.sh) as
the script to start the VM, and you can see the -smp 4,cores=2 in the qemu
arguments which configures how many threads/cores/sockets to assign to the
guest. Though I'm not entirely familiar with the syntax so I'm not sure what
the "4" is for, but I'd guess threads.

~~~
orionblastar
You can read the main Github page here: [https://github.com/kholia/OSX-
KVM](https://github.com/kholia/OSX-KVM)

It only works on Macs, it needs the toolchain and ROM I guess? Docker is just
one step closer to porting it on the Windows and Linux platforms.

Personally, if I wanted to run MacOSX that badly, I'd buy a Mac Mini or the
lowest-priced Mac they have. Much easier and worth it for the AppleCare and
Warranty.

Edit: grammar and spelling.

