Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> I've seen dozens to hundreds bit rot on each photo sync service (Amazon, Google, Apple).

Hold on a sec. How did this happen? I am assuming

1. Bit rot isn't a thing on NAND. Or a probability much much lower than HDD.

2. I am also assuming this wouldn't happen on their end because they will have all the protection in place.




One would assume.

But something happens (used to happen) where a file gets corrupted on cloud end, on my end, or in transfer, then it seems a sync detects a difference and assumes it must be an update, so replaces a good file with a corrupted file, and then, well, that's it for that image.

Tools like this can try to help, but usually fail:

https://betanews.com/2015/03/24/fix-corrupt-photos-with-jpeg...

I shoot on redundant CF cards, I back those up to long term RAID backed by S3/Glacier, so someday I can re-import the ones that are "lost", but determining which those are is surprisingly difficult when what you see in the UI isn't the file on disk any more, it's a generated thumbnail, and when the corruption doesn't affect the JPEG or RAW file format integrity, it's just a few bits flipped here and there.

I haven't found any CLI or GUI tools that correctly identify all of them. I haven't yet tried using vision LLMs, discouraged by the cloud LLM cost or local LLM compute time that would take.

NOTE: This hasn't happened in years. Perhaps entirely coincidentally and anecdotally, hasn't happened to me since all Apple devices are on Apple Silicon with none running any Rosetta-requiring software (I don't let Rosetta install at all), implication being the entire ecosystem is now code from within the past 5 years.


Bit rot is very much an issue on NAND. Reading causes disturbances. Flash chips have to compensate for drift in the voltage levels of cells that occurs as things age. While there are layers of protection using error correcting codes, in my experience corruption is more common with SSDs than HDDs. I've retired at least 3 SSDs from various generations over the past 14 years because of data corruption, and 1 SSD that simply refused to show up anymore. In comparison, the HDDs I've had have almost all failed without returning corrupt data, mostly due to mechanical failure.

Sadly, Apple doesn't believe in using ECC memory on their devices. The photos might get corrupted in RAM on the phone, or on an Apple laptop or who knows where. Without end to end checksums, there's no way to know when and where the photos are getting corrupted. Modern filesystems with built in checksumming are definitely worth the overhead.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: