IIRC Apple used to do something like this on their installer DVDs (back when they shipped the OS on DVDs). The Mac OS X installer was on an HFS+ volume pointed to by an Apple Partition Map (which would only be recognized by Mac OS X and the Mac's boot firmware), but the DVD also had an ISO 9660 header (that'd be recognized by Windows) pointing to Windows-compatible drivers for the Mac hardware. Completely separate volumes, with completely separate content.
Hybrid discs were common back in the CD-ROM era for games and multimedia discs. There'd be an ISO partition for PC executable installer and a Mac partition for the Mac version, and both partitions would have have pointers into a single copy of the large game data/video files that there wouldn't have been space to have 2 copies of on the disc.
The concept happened way before that too. Even though the Atari ST and Commodore Amiga had different floppy disk formats (and were bitter rivals), because they shared their main processor (MC68000) a great deal of games and software were available for both.
Occasionally, that happened on the same floppy disk, via a horrific sector format interleaving trick invented by Rob Northen (who did a lot of copy protection stuff at the time). Notably, the Future Publishing magazines ST Format and Amiga Format started out as the combined ST/Amiga Format and the "coverdisk" was exactly that - readable by both, with different files on each machine.
This reminds me that one of the filesystems that could probably be added into the polyglot is Amiga's Fast File System, as the superblock is in the middle. Rumor has it that this minimised seek times from anywhere else on the floppy, but I would think that one doesn't need to read the superblock that often.
Microsoft's FAT to NTFS conversion tool has been doing this sort of thing for years, too. It just puts the NTFS master file table and other stuff into the FAT free space, or a designated file. Of course, being a conversion tool the intent was that both sets of data structures described the same files.
Equally interesting are the tools that convert from the "MBR" style partition table to EFI partitioning. Again, this requires that the MBR-style partition table have free spaces into which the EFI partition table can be put. They have to be in fixed positions, though, unlike the NTFS MFT.
Small quibble, the original file system, named ext2_saved, is a subvolume/snapshot. As long as it's kept around, you can revert the conversion while losing any changes made to the Btrfs file system.
The comments explain the strategy in some detail. ReiserFS conversion is supported in addition to ext[234]. Because Btrfs requires only a little fixed space, just its superblocks, in theory any file system could be converted.
The difference here, though, is that NTFS does not have BTRFS's reserved area at the start of the volume. The BTRFS converter has to relocate foreign data structures out of that reserved area, meaning that the volume is not simultaneously both the old and new formats. Ironically, given what I mentioned, this conflicts most heavily with FAT.
Whereas the only fixed-position data structure in NTFS is the BPB in the Volume Boot Record, which it can even mostly share with FAT. In the FAT to NTFS conversion there is a point where (BPB aside) the two sets of filesystem structures are simultaneously valid.
Distro installation images (at least the Fedora ones) are kinda like this. There is a bunch of partitions and weird metadata in the right places, so that you can dd the image on a USB stick (or burn it to a DVD) and it will boot on any BIOS or UEFI PC and on Mac. Yes, there really is a HFS partition on the media just for this. :)
When Apple created HFS+ they did a similar hack that would show the disk with a TeachText read me file named “where did all my files go?” if you tried to mount it as HFS.
That means, Macs with older versions of the OS would deal gracefully with a format they were never intended to run.
I remember thinking it was the coolest thing ever.
I actually didn't try to let it autodetect... until now.
~/cursedfs% sudo mount cursed.img mountpoint/
mount: /home/kuba/cursedfs/mountpoint: more filesystems detected on /dev/loop0; use -t <type> or wipefs(8).
Props to whoever thought to check for the case of fricking polyglot filesystem images, though. Probably intended for something boring like corruption, but still, nice.
I'm pretty sure I managed to create one of these by accident when first using Linux, though with much messier results. Same root cause though: the ext2 and FAT superblocks didn't overlap, and back in the day some of the filesystem creation tools weren't so good at making sure to overwrite conflicting superblocks.
This is also possible with NTFS+ext[2,3,4].
I did work on modifying mkfs.ntfs do create a polyglot filesystem like this, but ran into issues that the windows filesystem check was not happy with the result (though it was mountable as either ext4 or ntfs in Linux).
https://pastebin.com/MujcN5MZ
> It turns out this is surprisingly simple to do: just create a FAT volume with a lot of reserved sectors and put the ext2 into the reserved sectors. This works because the filesystems choose different places to put their superblock: FAT uses the very first sector, while ext2 leaves the first kilobyte unused.
So, is it really two independent file systems? At least when it is read only?
It seems writing to one file system is likely to destroy the other, since they would be unaware of each other's allocated blocks.
> It seems writing to one file system is likely to destroy the other, since they would be unaware of each other's allocated blocks.
The entirety of ext2 stays within FAT32's reserved blocks (that is, between the bootsector and the File Allocation Table), so FAT32 won't write over ext2. The inverse is also true, as ext2 knows its formatted size, so it won't write outside of the reserved sectors.
ZIP files also have inline headers with duplicate metadata that may or may not be the same (e.g. filenames can and do differ).
That's why I've always felt conflicted over ZIP. On the one hand, having a contiguous index is nice from a performance optimization standpoint, and that's why ZIP is more commonly adopted to build upon, such as JAR files. On the other hand, the potential for (and actuality of) conflicting metadata is a nightmare from a security perspective, and it violates several rules of thumb in software engineering, such as that there should only ever be a single source of truth (major), a single parse (major), and to avoid duplicating data or context (minor, but can be sufficient to ensure there's only ever a single source of truth and a single parse), which generally ease implementation and help ensure consistent behavior. Likewise, while combining the container format and compression can improve convenience and user experience (especially if there's a single dominate implementation), it's also nice to keep these layers separate, which not only provides simplicity and consistency benefits, but also permits them to evolve independently.
If the media is read-only (or we pretend that it is), is it possible to have the filesystems share data at least for some files, by reusing the blocks?
Yeah, this is one of the things I'd like to explore when I make the formatter more full-fledged and not a 10 line bash script.
I'm hoping to try marking the datastructures of each filesystem as a bad block or something similar in the others, so that as long as the shared data isn't removed or modified, writing is OK.
Can you add ISO 9660 and/or UDF to the mix? Both do not use the first several kilobytes of the volume (hybrid CDs created by genisoimage use that trick to combine HFS with ISO 9660).
From a quick look at the specs, it would seem like it is possible to add either of those, but probably not both. I'll probably add this in "v2.0" - I'm planning to redo this in a way that actually interleaves the filesystems, which would also allow the ext* part to be larger than 32 MB.
I feel like this name nicely captures the fact that this almost surely has no real-world use and is likely to break stuff (file system autodetection, for one...). I also like to draw parallels to the notion of a cursed idea.
I have yet to meet any filesystem forensic software that wouldn't immediately identify this. Pretty much everything I've ever used would just tell you that it found multiple filesystems and show you both. Recovering files is often possible even without any filesystem metadata at all.
The filesystems aren’t synchronized, though. Quite misleading! I’m not sure there’s a technical reason why they couldn’t theoretically be synchronized (either in real-time via a custom fs layer or as cron task, etc).
I don't know if the title has changed, but "disk image that is simultaneously..." made it very clear to me that this would imply a static state carefully crafted file, not some synchronized dynamic file system driver.
I didn't envision a usecase, I just found the question of whether this is possible interesting. If I had to guess, this will break more things than it will help with - the filesystems are independent, not "synchronized".
Add encryption support for the ext2 part and it's a decent upper "security through obscurity" layer. If the wrong person finds your USB stick, they will likely only ever notice the FAT filesystem. Like for a spy dead drop use case.
This seems about as likely to work as hiding emails in the draft folder and not sending them. Computer forensics have looked at all the bytes on the disk, not just the active files reachable from the root directory, for decades.
> “First of all, I just wanted to say that none of us know [how it happened],” Stone said at the Q&A, when asked about how they came up with it. “[Snowden]’s the only one who knows, and one day he may reveal it. And number two, it was his idea — it was a suggestion that we responded to and ran with.”
I'd imagine they would have done forensics if he was caught with an sd card at a checkpoint, but I'm not sure one would set off a metal detector in the first place. It's more likely a security org would epoxy the sd and usb slots on computers with confidential data, or have audits for any usb devices the hosts encounter.
"He [Snowden] said that the Rubik’s cube was put in for the film and he wouldn’t be divulging how he smuggled out information. However, he added, everyone in the office did have a Rubik’s cube. “So they were floating around and coming in and out all the time,” he said."
Or more pratical: a partition on the SD card of your phone that can boot to a live linux, then store the live data locally in the ext2 part
I did try to do that with my android phone, but apparently it requires some specific kernel modules. I've to figure out which, and how to cross compile them for the phone kernel.
JFYI, in order to try and solve a specific problem some 13 years go we envisioned (and also started as POC) a "OFS" or "Overlapped File System", where a filesystem contained also a (partial) image of itself (mappable as image):
Basically a same file (same extents) was accessible both from the "outer" and from the "inner" filesystems.
Then the idea was abandoned as another valid (and also simple/existing) solution to the actual problem was found through the use of "included in the standard" CDFS graft points:
