I'm not really a developer so maybe I'm just not understanding something, but why in the world isn't Apple making it easier for people to optimize for the M1? I would think it's in their best interest to help developers make the best software possible, by sharing information about how to leverage the architecture. It's bizarre to me that the best sources of information are posts like this.
Apple is helping developers make the best software possible. They have a full suite of incredibly optimized and performant frameworks for compute, graphics, signal processing, machine learning, rendering, animation, networking, etc... That is all available via free download for writing MacOS and iOS apps.
Remember, they are selling computers as a combination of hardware and software. They are not selling processors so they are of course not supporting driver and other low level software for other OSs. That's a bummer if you are into other OSs, but it is not part of their business model so it should not be surprising.
Other OSs are supported via their virtualization framework. My limited tests show about a 10% performance penalty. Not too bad right out of the gate with a new system.
That being said, Ms. Rosenzweig is doing some incredible and interesting work. Really enjoying the series.
In a few months Apple is going to release a successor to the M1 processor, and then maybe in a year or two another revision after that. Apple would like your code to be optimized for that processor as well, in addition to the M1. The way Apple does this is by wrapping their hardware in a high-level API that knows how to use the hardware it is running on, rather than exposing architectural details and having developers hardcode assumptions into their code until the end of time.
Abstractions are both leaky and expensive. There are a lot of things that could have much better performance if they had access to the lower level APIs.
Metal is a good fit for the M1 GPU (since the GPU was essentially designed to run it). There isn't a need for a lower level API than Metal.
Most people will not end up writing code in the optimal way though, since they also want to support discrete GPUs with their own VRAM and those have totally different memory management.
Apple's official graphics API is Metal. There is plenty of documentation for that. Apple considers Metal a commercial advantage over OpenGL / Vulkan. That is; they want developers to develop against Metal.
Why do you think they are not? They are helping developers developing for the macOS platform that they own,develop,support. They are not responsible for 3rd part OSes and the development on those platforms. Why would you expect anything else based on their history and track record for closed/walled ecosystems?
Believe it or not, you're just purchasing rare earth materials packed very tightly into a nice box when you buy a Mac or an iPhone. The operating system is paid for on the backend by developers giving up 30% of revenue that goes through the App Store. The M1 is turning the Mac into an iPhone in exchange for an extremely fast processor and insane battery life, so they're not interested in helping you bypass the technical challenges of running other operating systems on their hardware (much like how they don't help you jailbreak iOS in order to help you install Cydia or third-party app stores).
You are drawing a false equivalency between the Mac and iPhones. iOS devices are deliberately locked so that running your own low-level software on them is not supposed to be possible, and requires breaking Apple's security. If they make no mistakes, doing it is completely impractical (the cost of an attack outside their attack model is greater than the price of the device).
macOS devices are not, and Apple invested significant development effort into allowing third-party kernels on M1 Macs. The situation is very different. They are not actively supporting the development of third-party OSes, but they are actively supporting their existence. They built an entire boot policy system to allow not just this, but insecure/non-Apple-signed OSes to coexist and dual-boot on the same device next to a full-secure blessed macOS with all of their DRM stuff enabled, which is something not even open Android devices do.
You can triple-boot a macOS capable of running iOS apps (only possible with secureboot enabled), a macOS running unsigned kernel modules or even your own XNU kernel build, and Linux on the same M1 Mac.
This pales in comparison to the rhetoric when OS X came out - you see that rhetoric survive today at opensource.apple.com, it's there, just, not in the spirit of the freedom that was promised.
I acknowledge that it’s possible to run unsigned code at “ring 0” on M1 MacBooks but the existence of the DRM restrictions leads me to believe that certain DRM-relevant hardware is not accessible unless a signed OS is running. I’m not exactly sure how the attestation works from software to hardware but I have to guess that it exists, otherwise it would be relatively trivial to load a kext that enables 4K DRM with SIP disabled.
One may consider that not important but I think it’s important to at least note the hardware of these machines are not fully under user software control.
Then again, I don’t think the iOS support requires any hardware so I’m not sure why someone hasn’t released a mod (requiring a kext or not) that enables iOS app loading with SIP disabled.
The Secure Enclave Processor (SEP) is the part that you cannot run your own code on, and it knows what state the device was booted in. However, it doesn't get in the way of normal OS usage. It also serves as a secure element, e.g. we can use it to store SSH keys and authorize use with Touch ID, or to secure a password store, or as a U2F token, just like macOS does. This is much better than most unlocked Android devices, which do this kind of thing in EL3/TrustZone, which runs on the main CPU and means there is always a proprietary, locked "supervisor" over your OS. This is also the case on x86, where the firmware runs code in SMM (ring -2) that the OS cannot touch or control, and in addition there is a management engine running even more intrusive stuff. On the M1, the main CPU is owned entirely by the OS, and doesn't even implement EL3 (only EL2, the VM hypervisor level, which we boot in - we already have the Linux KVM hypervisor working on the M1), making it a much more user-controlled execution environment than almost all x86 machines and Android phones.
In fact, the SEP is re-bootstrapped by the main OS itself after the boot stuff is done (we get the SEP firmware blob in memory passed in from the bootloader), so even though we cannot run our own code on it, we can choose not to run Apple's signed code either, and thus guarantee that it isn't doing nefarious things behind our back. For most intents and purposes, it's like an optional built-in YubiKey with a fingerprint reader.
iOS apps should be FairPlay encrypted AIUI, and presumably that goes through the SEP to only authorize decryption when booted in secure mode. That's my understanding anyway, I haven't had anything to do with that ecosystem for ages. Of course, either way you could load decrypted iOS apps just like you can pirate iOS apps on a jailbroken phone.
The M1 does have other co-processor CPUs that run signed firmware loaded before Linux boots (e.g. for power management, graphics management, sensors, etc), but all of those other firmware blobs are plaintext (only the SEP one is encrypted); we may not be able to change them (unclear yet how much control we have over those CPUs post-iBoot, there might be a way to stop them or reload the firmware) but we can at least reverse engineer them and audit that they don't do anything nasty. Besides, I think most of this stuff goes through OS-controlled IOMMUs to access memory anyway, so it can't do much harm to the main OS.
All the low-level details so far have been reverse-engineered since Apple doesn't provide documentation. Just because m1n1 finds the CPU to be in the EL2 state when its first instruction executes doesn't mean EL3 doesn't exist. An equally valid conclusion is that iBoot dropped from EL3 to EL2 before jumping to the m1n1 code.
Apple's phone chips use EL3 as a "god mode" to silently scan the kernel's code pages for modifications, and panic the processor if any are found:
> On the M1, the main CPU is owned entirely by the OS, and doesn't even implement EL3 (only EL2, the VM hypervisor level, which we boot in - we already have the Linux KVM hypervisor working on the M1), making it a much more user-controlled execution environment than almost all x86 machines and Android phones.
I agree that Apple probably has less random junk running at exceptionally high privilege levels, but your argument is not convincing to me. We have control of the exception levels, which we can check the existence of from the public ISA, but that doesn't mean Apple hasn't added any new stuff elsewhere that can touch the CPU in ways that are not yet known (and, to be entirely fair: I don't even think we know we have control of the exception levels. We have EL2 execution, but GXF exists, and even if we know how to navigate through it who can really say for sure what it does to the processor state?). I think the right argument here is "Apple has no reason to add stupidity to their processor (and many reasons to not add this garbage) so it likely does not exist" and leave it at that, rather than trying to draw up technical reasons why it seems more open.
> iOS apps should be FairPlay encrypted AIUI, and presumably that goes through the SEP to only authorize decryption when booted in secure mode.
Wow is it really possible that iOS apps are encrypted with a private key that is stored within all SEP devices and it hasn’t been cracked yet? If so that’s incredible and would explain why a workaround for using iOS apps with SIP disabled hasn’t been released. Of course I shouldn’t be that surprised since 4K DRM media content would rely on the same property.
Edit: I looked into this and it turns out that each device has its own public key and the server encrypts the app/content with a key derivable from the device public key on-the-fly at download time. This is a simplified explanation but the essential implication is that there is no global private iOS app package key.
> Besides, I think most of this stuff goes through OS-controlled IOMMUs to access memory anyway, so it can't do much harm to the main OS.
Great point. If these other onboard devices have unfettered access to the memory bus and/or can trigger some sort of NMI then you can never really trust these devices. Though as you point out, most contemporary x86 PCs are no different in that regard.
This is the standard content protection mechanism on pretty much every DRM download/streaming/whatever system in the world. Each app/movie/whatever is encrypted with a per-app key (so you can stick it in a CDN). Then each device has some kind of private certificate or key. When the user buys/rents content, you send the content key encrypted with the device key. This is how pretty much every game console, streaming service, etc does it.
There are global keys, which are used for system software. iOS used to be encrypted as a whole (not any more though, but the SEP firmware and iBoot still are) and getting those keys is tricky, as they are baked into hardware and different for each generation. You can build hardware so it lets you decrypt content or subkeys with a key, but not access the key material itself; if done properly (it often isn't done properly), that can mean you can only use the devices as an oracle (decrypt anything, but only directly on-device) unless you spend a lot of time and money reverse engineering the baked-in hardware key using a scanning electron microscope.
