Yes, too many times I have seen coders solve problems in O(n) instead of O(1)… But that’s nothing compared to not having any sensible, provable theoretical foundation for the design of a piece of software, most of which look like being cobbled together based on a consensus that “it might work”, a demand of “the highest paid person in the room”, or the opinion of a “subject matter expert”. The situation is truly sad.
MDRAID is good for availability and fault tolerance but no good for integrity.
For example, in a RAID-1, if one of the drives has a silently corrupted block, MDRAID will happily return that if you're unlucky enough for it to decide to use that drive to satisfy that read request. If you have error detection at a higher level, you might start pulling all but one drive from the array at a time and re-issuing the read request until it gives you bad data again (then you know which drive is bad).
If you have an 8-drive RAID-6 and one of the data blocks in a stripe is corrupt, again, it will happily return that (it won't even read the parity blocks, because every drive is present). Again you would have to pull one drive at a time and re-issue the read request until you get back good data, assuming you have a way to know that (e.g. a Zip archive with a CRC32). If you're still getting bad data, you didn't pull the bad drive; re-add it and pull the next one. This would happen when you pull the drive with the corrupted block, because then it would calculate what that block was supposed to contain based on the parity in that stripe.
Most distros have something akin to a monthly scrub job, where MDRAID will confirm mirrors against each other and parity against data. Unfortunately this will only tell you when they don't agree; it's still your responsibility to identify which drive is at fault and correct it (by pulling the corrupted drive from the array, nuking the MDRAID metadata on it, and re-adding it, simulating a drive replacement and thus array rebuild).
Worse still, the RAID-5 and RAID-6 levels in MDRAID have a "repair" action in addition to the "check" action that detects the above. This doesn't do what you think it does; instead it just iterates every stripe in the array and recalculates the parity blocks based on the data blocks, writing new parity back. Thus you lose the above option of pulling 1 drive at a time because now your parity is for corrupted data.
You need a filesystem that detects and repairs corruption. Btrfs and ZFS both do this, but Btrfs' multi-device RAID support is (still) explicitly marked experimental and for throwaway data only.
In ZFS, you can either do mirroring (equivalent to an MDRAID RAID-1), a RAID-Z (equivalent to an MDRAID RAID-5 in practice but not implementation), a RAID-Z2 (RAID-6), a RAID-Z3 (no MDRAID equivalent), set the "copies" property which writes the same data multiple times (e.g. creating a 1 GiB file with copies=2 uses 2 GiB of filesystem free space while still reporting that the file is 1 GiB in size), or some combination thereof.
ZFS checksums everything (data and metadata blocks), and every read request (for metadata or data) is confirmed against the checksum when it is performed. If it does detect checksum mismatch and it has options for recovery (RAID-Z parity or another drive in a mirror or an extra filesystem-level copy created by the "copies" property being greater than 1), it will automatically correct this corruption and then return good data. If it doesn't, it will NOT return corrupted data; it will return -EIO. Better still, checksums are also metadata, so they are also replicated if you have any of the above topologies (except copies=). This protects against corruption that destroys a checksum (which would ordinarily prevent a read) rather than destroying (meta)data. A ZFS scrub will similarly detect all instances of checksum and (meta)data mismatch and automatically correct any corruption. Better still, a ZFS scrub only needs to operate on filesystem-level allocated space, not every stripe or block.
tl;dr: Don't use MDRAID on questionable storage media. It won't go well.
Off-topic, but just wanted to note that using floating-point numbers as keys may be generally a bad idea (unless you use a custom comparator that takes into account the error that can accumulate during calculations).
I’m only a C++ amateur, but IMHO C++ vs C#/Java isn’t really a fair comparison here—the latter doesn’t have template shenanigans and so types are much more transparent to the reader (by which I mean that you don’t have to execute a dynamically-typed program in your head to get from the term on the right-hand side to the type on the left).
reply