Hacker News new | past | comments | ask | show | jobs | submit login
Debugging macOS Kernel Using VirtualBox (klue.github.io)
161 points by lilbunnyfoobar on Apr 10, 2017 | hide | past | web | favorite | 40 comments



[puts on VMware hat].

VMware's virtualization software (ESXi, Workstation, Fusion) has the concept of "CPUID masks", which can be used to mask off features to the guest from the host's CPU for this kind of work. Since it is a very advanced concept, we didn't expose it in the UI for Workstation or Fusion, but the VMX shipped with both products should support it as far as I know (I don't think we've ever disabled that feature for a product anyhow). If you dig into the menus you can set the flag values on ESXi, as sometimes for fleets we have admins that want to normalize their CPU flags across all of their virtual hardware (I believe this was very popular when the NX-bit was new).

However, you're not at a lost for the desktop products. Editing the [vm-name].vmx file and adding the lines to control the cpuid registers is a pretty simple endeavor; the lines are formatted 'cpuid.<cpuid-in>.<GPR> = "value"', and value should contain the mask for the whole 32-bit register, with a dash '-' for no special processing of this flag, "h" for using the host's setting of this flag, '0' for explicit disabling the flag, or '1' for explicit enabling the flag. For example:

    cpuid.7.ebx = "--------------------0-----------"
disables the (if I counted correctly) 20th bit from the return of CPUID in the ebx register when CPUID is passed in '7' as a parameter to eax, which should correspond to disabling the SMAP CPUID flag.

(The VMX file is a bit non-obvious to get to on macOS, as we use 'packages' to make VMs nicer to handle, but simply right clicking the package and selecting "show package contents" should reveal the VMX file.)

[takes off VMware hat].


VirtualBox has a similar feature, via "VBoxManage modifyvm --cpuidset <leaf> <eax> <ebx> <ecx> <edx>", which also appears to be equally as undocumented and obscure. :-)


I've seen these flags mentioned before, for both VirtualBox and VMWare. And it's always without exception been in conjunction with trying to get OSX running in a VM on non-Apple hardware.

I'm pretty sure both VirtualBox and VMWare could create a Apple-friendly template with these settings already tweaked for what OSX expects. In fact it would be trivial.

But for some reason they haven't.

Considering how VMWare Workstation Pro actually supports running MacOS on Windows, but has the relevant config removed in production-builds (needing it to be hacked back in), I'm going to assume there's some kind of political pressure going on here. Maybe even from a certain Cupertino-based company, wanting to prevent the mainline virtualization providers from commodotizing their commodity, run of the mill, standard X86 PC operating system.

That's a shame, because if they didn't, it would certainly make it much easier for me to build a solid and reliable CI pipeline for our MacOS and iOS stuff.


Wow this is great. I can confirm this works, though since SMAP is 20th bit, it should be

    cpuid.7.ebx = "-----------0--------------------"
Inside the VM:

    machdep.cpu.leaf7_features: SMEP ERMS RDWRFSGS TSC_THREAD_OFFSET BMI1 HLE AVX2 BMI2 INVPCID RTM RDSEED ADX FPU_CSDS
I have updated the post crediting you!


Oops, I counted in the wrong direction! I can't believe I missed that one >_<.


Are there any good, up-to-date resources on macOS internals that detail everything from the boot process to the GUI? I could only find [0], and Apple's [1] was last updated in 2013..

[0] http://www.newosxbook.com/index.php?page=book

[1] https://developer.apple.com/library/content/documentation/Da...


Jonathan Levin's MOXiL 2nd Edition volume 3 is out and I recommend you get it. 1st edition is still very good and can hold you off until the other volumes are out.

Amit Singh has a Mac OS X Internals A Systems Approach that is good we well.

There are also various talks at security conferences that detail parts of macOS/iOS.


Is this completely allowed? I'm probably misinformed, but I thought either certain debugging calls or certain kinds of virtualisation were disallowed by Apple. The fact that this kind of tutorial using virtualbox exists makes me think that someone is trying to work around something. I thought you weren't allowed to debug parts of macOS. Are there absolutely no limits, legal or technical?


From https://ssl.apple.com/legal/sla/docs/macOS1012.pdf, you are in fact allowed to run macOS in a VM so long as you do so on Apple hardware:

(iii) to install, use and run up to two (2) additional copies or instances of the Apple Software within virtual operating system environments on each Mac Computer you own or control that is already running the Apple Software, for purposes of: (a) software development; (b) testing during software development; (c) using macOS Server; or (d) personal, non-commercial use.


Author here. Nothing in the post is illegal.

Apple allows 3rd party kernel extensions so naturally you would need to be able to debug the kernel. In my post, I link plenty of official Apple documentation that provide information on how to debug the kernel.


Please don't even use the word "illegal". Nothing in the post breaks the agreement. It's just an agreement


Yes, but that's not the whole OS. Where do your rights end to inspect the software and hardware you bought?

Also, I was careful to not say legal/illegal but allowed/disallowed. I am pretty sure that macOS disables some debugging calls, or at least it used to. What are the limits?


I can think of two different things that could match your description:

- An old call, ptrace(PT_DENY_ATTACH), which prevents the process that calls it from being debugged (with either ptrace or dtrace). iTunes calls this. It's always been rather easy to circumvent, either by attaching before the process has the chance to make the call, or by installing a kext that disables the functionality.

- System Integrity Protection broadly prohibits debugging of system processes, as well as kernel tracing via dtrace. But SIP can be disabled by the user, by design.

Kernel debugging in particular is explicitly supported by Apple in the form of Kernel Debug Kits, which consist of debug symbols for the open source parts of the kernel, as well as variant builds with more debugging stuff enabled at compile time. Peeking at the proprietary parts is presumably against the license agreement but not technically restricted in any way (hard to imagine what such a restriction would look like).


Thanks, the iTunes case was what I was thinking of.


Not to be rude, but why do you care?

If it's illegal, it's up to Apple to deal with it. Why go out of the way doing free work for them?


Because it has affected me in the past. I've had to debug stuff in the past in macOS, thinking it would be similar to BSD, and was thwarted by technical means to disable debuggers. I've found the experience quite distasteful and was trying to remember what the problems were that I faced.


Politics. He's trying to lead us to to the Freedom of the GPL in this Socratic fashion.


My thoughts exactly :-)


So that he looks smart.


Read http://www.apple.com/legal/sla/ to find the answer.

I haven't completely read it (if only because it is 424 pages in multiple languages), but http://images.apple.com/legal/sla/docs/macOS1012.pdf does state:

"M. No Reverse Engineering. You may not, and you agree not to or enable others to, copy (except as expressly permitted by this License or by the Usage Rules if they are applicable to you), decompile, reverse engineer, disassemble, attempt to derive the source code of, decrypt, modify, or create derivative works of the Apple Software or any services provided by the Apple Software or any part thereof (except as and only to the extent any foregoing restriction is prohibited by applicable law or by licensing terms governing use of Open-Sourced Components that may be included with the Apple Software)."

I am not a lawyer, so I don't dare guess at what it really means, but I would guess there are many jurisdictions where most of that doesn't have any implication for persons _buying_ a Mac (a license requires an agreement between parties and cannot be something that one party forces upon another party)

Also, many licenses of open source parts do allow you to do anything with the software, including reverse engineering. I expect that means you are free to debug, say, the Mac OS kernel, once its source has been open sourced.


It's definitely allowed if you're building VMs on your own Apple hardware. On other hardware it's more murky.

Source: maintainer of https://github.com/geerlingguy/macos-virtualbox-vm


You are permitted by the license/EULA to run macOS Server in a VM, as long as it is on Apple hardware.


The server requirement was only true for 10.5 and 10.6. With 10.7 and up, the regular client OS X can also be virtualized on Apple hardware.


Isn't "macOS Server" just an app on the app store now?


The app is more than a regular app. Just like the macOS installer apps, the Server install app installs macOS Server components to the system.


It doesn't really do this much now, because rootless makes it hard to actually modify what the base image consists of (at least, without the "magic double reboot" system updates do nowadays, that uses the recovery partition's kernel to update the code-signing whitelist. And that process is really only set up for a linear progression of system updates; it has no allowance for optional components that add their own whitelists.)

Instead, Server.app does two things:

1. similar to Xcode, Server.app ships with a POSIX env inside it (nothing chroot/jail-y going on; it just has programs ./configure'd with that env as their --prefix, and some stub programs in the base system that exec their /Applications/Server.app equivalents if they exist);

2. it updates [non-rootless-protected] configuration files, that activate previously-inactive parts of the base system, like opendirectoryd and racoon.


It is a bit more than a regular app but I don't think it's installing anything into the system. There are UNIX folder structures inside the app bundle that the system can use. But if you delete the app you don't have any server processes anymore.


Judging by the thriving Hackintosh community and widespread tutorials of how to run macOS in VirtualBox (even on regular PCs) which haven't disappeared due to DMCA or other legal action, I think it's safe to say Apple aren't so concerned. Making a "virtual Mac" in VirtualBox, on a Windows PC, is almost easier than installing Windows.

AIUI their profits really come from the hardware sales, so if people try macOS and eventually decide to buy a Mac, Apple wins; if they don't, then Apple didn't really lose either, because those people wouldn't ever buy a Mac anyway.

I played around with Hackintoshing and even writing/porting device drivers to get my install working, and that was somewhat fun, and I even tried out some of the other interesting Mac-only software(z), but in the end I still preferred to use Windows and Linux.


In case it was illegal. So what?


    Is it allowed?
Legally maybe not.

Really it is on Apple to attempt to prevent this. Not on the hacker. There isn't many ways an OS vendor and prevent their product from being virtualized. If Apple was smart they'd maybe clean up OSX to have a decent cloud offering.

But alias nope.


there are cloud offerings of OS X. Whether you consider them decent is another matter, but it is legal and is available.


A cloud offering of OS X? Who would need something like that?


There's lots of cool stuff you could automate if you could provision OS X on demand. For example, you could offer an "iMessage proxy service" that proxies text messages it receives through your iMessage account by calling the iMessage app as a black box in the OS X instance. If you want to do this yourself, you can do it of course but your OS X will need to be running 24/7.

Fwiw the same is true if you could virtualize iOS.


Something like that could potentially be useful for me to see if the scientific software I write/use on Linux is likely to compile/work on various Mac OS X devices (right now I just count on others to complain when things don't work).

And there are plenty of use cases:

* an iOS development offering (as long as iOS development is restricted to Mac OS X)

* security research

* integration with software not available on Linux (e.g. something Photoshoppy)

* hardcore fanboys who can't stomach using anything not ordained by Apple


I rented one just to make sure my site redesign worked well in Safari. I assume others do it to check OSX compatibility, or maybe to try Swift out in it's native environment, etc.

It's attractive because I can rent it when I need it, versus buying a Mac (or copy of OSX) I'll only use occasionally.


Anyone know if this setup can be used to debug iokit (communication with external monitor over I2C)?

There is 100% method to freeze system while communicating with external monitor using iokit framework over I2C. https://apple.stackexchange.com/questions/61045/does-apple-s...


I've tried this recently, and I can't get 10.11 to build the 10.11 xnu. Parts of the kernel throw deprecation errors, etc. Pretty weird actually. I'm assuming this is due to toolchain mismatch, but I haven't been able to figure it out. I'd appreciate it if anyone has any pointers.


Remove "-Werror".

Sometimes there are too many errors to deal with. I find building on a machine that's on the exact macOS version that you are trying to build does the trick.


This probably does not need a full Xcode install, just the command line tools. The CLT can be installed on a pristine macOS system just by typing cc inside Terminal.


I can't get virtualbox to boot. No matter what I try, I end up with this error:

gIOScreenLockState




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: