Also, one reason they’re as expensive to maintain as they are is that we don’t test detonate them any more. We have to do a bunch of indirect testing with expensive equipment and supercomputers.
That’s probably a good thing; certainly the people complaining here about costs would not suggest we go back to doing so.
This seems like an early application of the Tim Cook doctrine: Why would Apple want to surrender control of this key bit of technology for their platforms?
The rollout of APFS a decade later validated this concern. There’s just no way that flawless transition happens so rapidly without a filesystem fit to order for Apple’s needs from Day 0.
(Edit: My comment is simply about the logistics and work involved in a very well executed filesystem migration. Not about whether ZFS is good for embedded or memory constrained devices.)
What you describe hits my ear as more NIH syndrome than technical reality.
Apple’s transition to APFS was managed like you’d manage any kind of mass scale filesystem migration. I can’t imagine they’d have done anything differently if they’d have adopted ZFS.
Which isn’t to say they wouldn’t have modified ZFS.
But with proper driver support and testing it wouldn’t have made much difference whether they wrote their own file system or adopted an existing one. They have done a fantastic job of compartmentalizing and rationalizing their OS and user data partitions and structures. It’s not like every iPhone model has a production run that has different filesystem needs that they’d have to sort out.
There was an interesting talk given at WWDC a few years ago on this. The roll out of APFS came after they’d already tested the filesystem conversion for randomized groups of devices and then eventually every single device that upgraded to one of the point releases prior to iOS 10.3. The way they did this was to basically run the conversion in memory as a logic test against real data. At the end they’d have the super block for the new APFS volume, and on a successful exit they simply discarded it instead of writing it to persistent storage. If it errored it would send a trace back to Apple.
Huge amounts of testing and consistency in OS and user data partitioning and directory structures is a huge part of why that migration worked so flawlessly.
To be clear, BTRFS also supports in-place upgrade. It's not a uniquely Apple feature; any copy-on-write filesystem with flexibility as to where data is located can be made to fit inside of the free blocks of another filesystem. Once you can do that, then you can do test runs[0] of the filesystem upgrade before committing to wiping the superblock.
I don't know for certain if they could have done it with ZFS; but I can imagine it would at least been doable with some Apple extensions that would only have to exist during test / upgrade time.
[0] Part of why the APFS upgrade was so flawless was that Apple had done a test upgrade in a prior iOS update. They'd run the updater, log any errors, and then revert the upgrade and ship the error log back to Apple for analysis.
I don't see why ZFS wouldn't have gone over equally flawlessly. None of the features that make ZFS special were in HFS(+), so conversion wouldn't be too hard. The only challenge would be maintaining the legacy compression algorithms, but ZFS is configurable enough that Apple could've added their custom compression to it quite easily.
There are probably good reasons for Apple to reinvent ZFS as APFS a decade later, but none of them technical.
I also wouldn't call the rollout of APFS flawless, per se. It's still a terrible fit for (external) hard drives and their own products don't auto convert to APFS in some cases. There was also plenty of breakage when case-sensitivity flipped on people and software, but as far as I can tell Apple just never bothered to address that.
Using ZFS isn't surrendering control. Same as using parts of FreeBSD. Apple retains control because they don't have an obligation (or track record) of following the upstream.
For zfs, there's been a lot of improvements over the years, but if they had done the fork and adapt and then leave it alone, their fork would continue to work without outside control. They could pull in things from outside if they want, when they want; some parts easier than others.
Historically, it’s more efficient and less of a constraint on growth to take care of those people directly via transfer payments (tax & spend) than by turning over otherwise ordinary business decisions to bureaucracy like the “California Fast Food Council”.
That’s the point the article makes: That some idea is on your mind is essentially always correlated with any number of signals, some of which are visible or inferable by adtech.
I’d guess that nomenclature originates in the world where every small US Main Street had a stationary store carrying all manner of paper sizes and stocks for diverse purposes—none of which involved use in anything more sophisticated than a typewriter.
One particular “standard” that sticks out in my memory was “math paper”, which I recall as being unbleached, about 5” x 8”, and used pervasively in primary education (at least in New England) into the 1990’s.
I interned with the AppCompat team shortly before the release of Windows XP, which was huge for them as it was the first Windows for consumers on the NT kernel.
IIRC, they had a significant lab and tons of infrastructure for exercising and identifying compatibility issues in thousands of popular and less popular software packages. It all got distilled into a huge database of app fingerprints and corresponding compatibility shims to be applied at runtime.
Yeah, would love to see the CPU vendors invent some primitives to let user code pull those kinds of privilege isolation tricks within a single process and address space.
Something like: “From now on, code on these pages can only access data on these pages, and only return to/call into other code through these gates…”
I've had some ideas about avoiding format validation in IPC receivers if the data is encoded by trusted code, which is also the only code that has rights to send the IPC data / to connect to the receiver.
I can't really think of an important problem that it would solve, though. DBus always validates received data, but it's not really meant or very suitable for large amounts of data anyway.
What I’m looking for is a way for a process to de/re-escalate its privileges to access memory, without an expensive context switch being required at the transition. The CPU would simply enforce different rules based on (say) the high-order bits of the instruction pointer.
Imagine a server process that wants to run some elaborate third-party content parser. It’d be great to be sure that no matter how buggy or malicious that code, it can’t leak the TLS keys.
Today, high-security architectures must use process isolation to achieve this kind of architectural guarantee, but even finely tuned IPC like L4’s is an order of magnitude slower than a predictable jump.
I don’t think MPK’s will fit the need I have. Simply: Run some arbitrary, untrusted, non-sandboxed code in the same thread with assurance it can’t read page X. When that code completes and I’m back in code I trust, X is readable again.
Is that something MPK makes possible? The doc I’ve read suggests either your process can flip permission bits or it can’t. Great for avoiding out-of-sandbox reads. But if there’s arbitrary execution happening, why can’t that code flip
the access to secrets back on?
When your comment and the article refer to “physical” addresses, those are physical in the context of the JVM, right? To the OS they’re virtual addresses in the JVM process space?
That’s the fundamental premise of the article: The hyperscalers will consolidate GPU compute exactly as they consolidated all other forms of compute. Including highly sensitive compute like product design and customer databases.
You can argue they won’t, but the “enterprises won’t put sensitive data in Cloud” ship sailed years ago.
reply