HN disclosure: I’m the author of Photos Backup Anywhere, but this thread mirrors the exact issues that pushed me to write it.
One thing that surprised me when digging into Apple Photos is how much state isn’t represented by just files-on-disk. Albums, Live Photos (paired assets), bursts, slo-mo, edits, and even “simple” things like adjusted capture dates are all tracked separately, and most export/backup tools end up flattening or partially reconstructing that on restore.
The approach I took was to treat Photos as the source of truth and verify restored items against it, rather than assuming filesystem metadata is enough. As far as I know, this is the only tool that restores albums and correctly round-trips all Photos item types while preserving location data, creation dates, and modification dates when restoring back into Photos.
My current process for offloading photos off the iPhone is to copy them in subsequent batches of '0-9999' from the 'Image Capture' app.
This is because I usually have far more than 10K photos and apple starts renaming the files after 9999 as 00001(1) for the rest. This is pretty undesirable.
Is there a way for me to export unmodified raw/jpeg/live/videos off the iphone to an external drive without a macbook with a large enough ssd, and wanting to use icloud as an intermediate bottleneck?
Do you take into account the iPhone not holding the original images of every photo? It will offload originals and just keep thumbnails if the library is too large.
Mine is approaching 1.5TB, I’ve got no hope of keeping that all on an iPhone, and also no guarantee that any given photo is fully available locally.
Aren't there hooks on the filesystem layer that downloads them when you access them? E.g I can browse via terminal to my iCloud Drive somehow and cat etc works on files which aren't local (after locking to download them first).
I don't know about space-optimized storage on-phone. I know one setting for transfer to mac or pc - I have it set to "keep originals" instead of "automatic". There might be other settings I'm not aware of.
EDIT: actually, there are other directories (under /mnt but outside DCIM in my example) that seem to have other photo stuff, maybe edits? ymmv
> My current process for offloading photos off the iPhone
I'm not sure about Linux, but my workflow on Windows and MacOS is to frequently back up my iPhone locally (which you should do anyway because few incorrect PINs can security lock your phone [1]) and use utility like backup extractor (e.g. [2] but there are many others) to extract all photos from the backup. This effectively removes the need to use iCloud.
Does the PhotoSync app permit that? I use it to copy files to my NAS but it has some USB-related options I never explored. I used to use Image Capture but heard of PhotoSync and have never looked back.
With Photosync I have our photos export to my NAS and have it update the file names with the timestamp + original file name, which makes it so much more sane to sort through.
I use and like PhotoSync but I thought it doesn’t export unmodified originals but your edited versions. Personally I like this behavior better but that might not be what you want?
I’m not sure Apple allows any third party app to access the unmodified originals. Imagine you crop a photo to remove some embarrassing part. A third-party app can just recover that? What a privacy risk.
Of course this won’t matter if you don’t do any photo editing on iOS.
It's right to question this. I had taken https://www.photosync-app.com/support/basics/answers/does-ph... ("PhotoSync transfers the original, unaltered images including all EXIF and GPS data") on face value without thinking of all of the iCloud behavior. Even limited copies would suffice in my case, but that's insufficient for others.
> As far as I know, this is the only tool that restores albums and correctly round-trips all Photos item types while preserving location data, creation dates, and modification dates when restoring back into Photos.
My free open source tool osxphotos [1] does this. In addition to an "osxphotos export" command to create archival exports with all metadata, it also has an "osxphotos import" command that can restore all metadata including albums (with exception of named persons/face regions, though persons can be converted into keywords on import). It can also read XMP sidecar files to restore metadata from those. It's a CLI and definitely a power user tool.
It also includes an "osxphotos sync" command that can sync metadata between two libraries. Some people use this to sync metadata like favorites from iPhone library to the Mac library. (can't go the other way unfortunately)
If you manually edit the date on a photo, is that also stored separately from the image file itself? Wondering because I've noticed photos I've backed up to Immich from iOS photos don't respect that edited date and reflect whatever the original date was.
I've been thinking about looking into a fix for this since it's bugged me a bit.
Yes — in Apple Photos the manually edited date/time is not written back into the image or video file. It’s stored separately in the Photos library database, which is why tools like Immich usually fall back to the original capture date in EXIF / QuickTime metadata.
Photos Backup Anywhere does handle this case: it reads the adjusted date from the Photos library and stores that modified timestamp in its own SQLite database, linked to the backed-up file, so the edited date isn’t lost even though the file itself isn’t rewritten.
Thanks for the reply, I appreciate it. Does Apple provide any APIs for interacting with this metadata or was this something you had to implement yourself with lower level DB lookups?
I have taken time to slowly extract photos from old androids and its such a nightmare, and if you cant get a meaningful interface to load you have to resort to tooling that scrapes the whole drive and hope it grabs everything.
Yeah but for some phones the storage is full ;) I had to figure out how to remove just enough to root it and then just adb in for one phone then I learned that if I extract too quickly it gives up the ghost so I have to add a delay
Awesome! Thanks for posting, I've been looking for exactly this kind of project that takes end to end pristine restoration seriously.
The FAQ states you back up the originals as plain images files in YYYY/MM/DD format, which is great for integration with other tools. What about metadata? Is it in a format that lends to integration with other tools, say if you stop developing/supporting your app for some reason?
Metadata is stored in an SQLite database alongside the backed up files. The format is not documented, but should be self explanatory if you examine the schema.
This is a great app! Thanks, I bought it. I'm backing up to a network share over SMB. Just wanted to let you know that the error message in the event that the share becomes unmounted during a backup is a bit obscure: it just says "pdb.sqlite" doesn't exist. Stopping and restarting the backup fixes it. Might be helpful to provide some clearer error handling / messaging and guidance for disappearing network shares?
You’re right about that error message. What’s happening is that when the SMB share gets unmounted, the app can no longer access its SQLite database (pba.sqlite), and the resulting error isn’t very user-friendly. I agree it would be much clearer to explicitly detect a missing or unmounted network share and provide guidance.
I’ve added this to my list of things to improve. Thanks again for taking the time to report it — feedback like this is super helpful.
My pleasure - much appreciated and thanks for being so responsive.
I’ll just add in here a sneaky humble request for graceful automatic pausing and restarting as network shares (or other volumes?) (dis-)appear - although I imagine that one’s a bit trickier to implement.
All the best! Very satisfied user. Great replacement for the script + cron job I was using before.
Pretty severe -- ADP allows users to store data with e2e encryption.
Disabling ADP is a pretty serious* thing to do -- and pretty disappointing since I was interested in the product. Since it's OSS though this might be something that can be worked around in the future.
That’s a good question. To be completely honest, I don’t know much about Advanced Data Protection myself, and I didn’t do anything specific in the app to detect or interact with it.
I’m actually curious: how did you discover that it doesn’t work when Advanced Data Protection is enabled? Was it through an error message, incomplete backups, or something else you noticed?
The app that the person you're replying to ('gerdemb) created (Photos Backup Anywhere) is different than the open-source CLI tool that is the HN submission (icloud_photos_downloader).
Yes — Photos Backup Anywhere supports Apple Family Sharing
If the app is shared via Family Sharing, each family member can install and use it on their own Mac with their own iCloud Photos library. The app works with whichever Photos library is signed into macOS on that machine, so you can absolutely use it to back up your partner’s images as well.
Yes — that’s exactly what Photos Backup Anywhere is designed to do.
The app creates file-system-level backups of each individual photo and video from your Photos library. Every asset is exported as a real file on disk (HEIC, JPEG, MOV, etc.), not a database blob or package, so you can browse, copy, verify, and back them up with standard tools.
In addition, the app keeps a small SQLite database alongside the files to track metadata (including edits, albums, and relationships like Live Photos), which allows accurate restores and verification while still giving you transparent, plain-files access to your media.
Yes — that’s exactly what it’s designed for. You can back up photos from one Photos library and later restore them into a different Photos library. The restore process recreates the items in the target library using the data stored in the backup.
Backing up “mercurial” Photos data is only half the problem. The tricky part is restoring it in a way Photos actually recognizes as equivalent to the original library state. Photos Backup Anywhere restore works by re-importing items while explicitly reapplying Photos-level attributes: paired assets for Live Photos, burst membership and picks, slo-mo metadata, edits, locations, adjusted capture dates, and then reconstructing albums after the items exist again in the library.
In other words, the filesystem copy isn’t treated as the source of truth. The restore verifies items against what was backed up and only then rebuilds higher-level structure like albums. That’s the piece I didn’t see addressed elsewhere, and what originally motivated me to build it.
Very cool! I've transitioned from using Google Translate and DeepL to ChatGPT for translations. In general I think the ChatGPT translations are better, but the default UI for is a little tedious. Looks like your tool solves this problem.
I tried translating some Japanese, but the result is not correct.
兄や弟さえも憎むようになりました.
I have come to hate even my brothers and sisters.
It should be "I have come to hate even my older brother and younger brother."
Are you using ChatGPT 3.5? I've found that you need to use version 4 to get quality translations as 3.5 makes a lot of mistakes.
Yes, we're using ChatGPT 3.5 for now and we will certainly switch to GPT-4o mini in the near future. It already works with ChatGPT 4 but we noticed that the cost increase was not worth the minor improvement of quality.
I tried your example with ChatGPT 4 and it's somewhat better but not perfect: "I've even come to hate my brother and younger brother."
We hope that the quality will improve with the cheapest (and also fastest) models available.
While that might be Apple's "official" solution, there's a major caveat: the Photos app doesn't support storing the Photos library on a networked disk. This forces you to either fit your entire library on your Mac's internal storage or use a directly connected external disk with no network involved.
To overcome this limitation, I developed a MacOS app that creates a full backup of your Photos library. You can safely store this backup on any external storage like a NAS, network drive, internal disk, or external disk. Check it out here: https://ibeni.net
Compared to the "iCloud Photos Downloader," which scrapes data from iCloud, my app uses the official Apple Photokit APIs to back up your photos. This method is more reliable and efficient.
The library works at the database-level and stores databases modifications as binary change sets in a separate table that models a graph similar to a git repository. Capturing modifications is as simple as wrapping the transaction with a handler provided by the library. The graph of database modifications is stored locally on each device, but can easily by synced with an online repository.
The library detects conflicts, and provides a handler to the application for conflict resolution.
Thanks for your comment. Let me know if you end up implementing this on Android. I think this concept could be adaptable to other platforms or languages. I just happened to be working in Swift and so I started there.
It took about a week to prove the idea was viable and then another week or two to get the code into a state where I felt it would be useful to share.
Nice to hear from someone else with a huge iCloud Photo Library--we've got about 1.2 TB! After a lot of experimentation with Google Photos, Synology's photo APP, etc. we finally settled on going all in on the Apple ecosystem (we're an all-Apple family). The game-changer for us was when Apple finally supported shared photo libraries between users.
The ability to search the photo library has been steadily improving (in my experience, Google still has an edge over Apple in this area) and with the advent of new AI technology I only expect it to get better. Except for special occasions I don't even attempt to "organize" or "tag" my photos. It reminds me of how Google Search surpassed early 'library catalog' style indexes like Yahoo.
The biggest downside of iCloud is the lack of an API to access the data programmatically for backups etc. I've experimented with `icloudpd` and it worked OK, but I'm not sure about the long-term stability as it's basically just "screen scraping" iCloud.com to download the photos from the web which is not officially supported by Apple and is sensitive to any changes to the website as well as the (remote?) possibility of getting your account banned. The performance is also bad as it has to download all photos even the ones that are already stored locally.
There’s also `osxphotos` (https://github.com/RhetTbull/osxphotos) which works by reverse engineering the undocumented sqlite database in the photos library that Apple uses for storing the photos metadata. This avoids the performance problems of accessing the library through iCloud.com like `icloudp` does and enables many more features like searching by faces, places, etc. and they have an export function with a ton of options to backup your library.
Finally, I have a side-project working on a simple set-it-and-forget Mac app for backing up your entire photo library (including iCloud Photos) to a local disk, network share, NAS, etc. Unlike the above options, it uses the official Apple PhotoKit library to access the photos and doesn’t require setting up a Python environment, running command-line tools, etc. If you’re interested check it out here: https://www.ibeni.net
Thanks for the feedback! The app started as a personal project exactly to solve the problem you mention of the iCloud library exceeding the available disk space. (My personal library is more than 1 TB in size.)
The app doesn't require an account or login and the backups are stored only on your device or chosen drives and not on any external servers.
I need to add a privacy policy to the web page to make the policy clear and I believe Apple also requires a privacy policy before the app can be officially released.
One thing that surprised me when digging into Apple Photos is how much state isn’t represented by just files-on-disk. Albums, Live Photos (paired assets), bursts, slo-mo, edits, and even “simple” things like adjusted capture dates are all tracked separately, and most export/backup tools end up flattening or partially reconstructing that on restore.
The approach I took was to treat Photos as the source of truth and verify restored items against it, rather than assuming filesystem metadata is enough. As far as I know, this is the only tool that restores albums and correctly round-trips all Photos item types while preserving location data, creation dates, and modification dates when restoring back into Photos.
Project page is here if it’s useful: https://photosbackup.app/
Happy to explain details if anyone’s curious — there are a lot of sharp edges in Photos once you go beyond “export originals”.
reply