I prefer SyncThing on both mobile and desktop devices. It's open source and mature, the server only makes devices findable between each other. It allows 1 or 2 way sync. And it has advanced settings for keeping removed files (e.g. trashbin that cleans anything older than X days.)
I found that SyncThing is not a great solution for the "mobile photo reel" use case. I want my entire photo collection to be easily searchable & accessible from my mobile device, but I don't need a complete copy of my photo reel to be saved to my device. I want to be able to delete things from my phone to free up space, but I want them to still exist in my photo collection that is backed up to the cloud.
I have the same need... I use Nextcloud for many things (including photo upload) but I can't stand their photos app. I use photoprism for the photos app and I love it.
I just have the same directory mounted to both my Nextcloud & Photoprism containers. They both operate with a normal directory of files on the file system, so I just have them both use the same directory to store data.
+1 syncthing too. Over the years it has gotten rock solid for me. I use it between all my machines and phone. With various 1 to many, 1 to 1 and one way or two way sync.
One of my machine is always on and is backuped to the cloud everyday. Effectively making a backup of all my devices at once.
Better yet, since syncthing can also save staggered copies of modified files, a rm -rf * will be synced but the backup machine will still have copy from about 10s ago. This saved me a few times.
Once I realized that I had deleted a file a few months later. And I found it into the syncthing backup, itself within the cloud backup.
I just finished setting up an automated backup system with syncthing and restic, and now my phone, laptop, desktop and server are all backed up in several directions and to a cloud storage area, every night. It's glorious and works like a dream. Happy days!
I would be very careful when setting up any kind of replication on backup repositories. I much prefer to actually backup to multiple destination, since when replicating, any error will propagate to the synchronized repositories as well, rendering all of the equally useless.
Great advice thanks - I have taken some steps to mitigate that (snapshotting, and a manual weekly USB backup) but you're right, this is an area I should look at more closely.
Since this is upvoted so much, I wonder what people's complete setup is for something like this to replace, say, Google Photos. So once the photos are on a "server", what do people recommend for albums, sharing, metadata, geotagging, search, etc?
I'm not sure it covers all your features listed, but I use PhotoStructure [1] for the 'album' side of things. It's been mentioned a bit on HN, which is where I found it. Sharing is very open for me since I'm just sharing wholesale with family, but when I need to share specific images or albums to people, I usually do it via some other way that suits them -- so if they use messages, email, google drive, dropbox, or just want to download from a webpage (eg. caddy), I'll enable that.
I want an easier way to edit the comments on photos (as embedded metadata) that I haven't found yet. Any image browser where I can hit a convenient hotkey, type "Summer 2023 at the river with cousins X and Y" and move on would be great. If anyone has a suggestion I'm listening keenly! If this is built in to something on MacOS or a linux default desktop (like KDE or something I'm less familiar with), or is a small paid non-subscription app, I would buy it.
I use https://github.com/jpsim/AWSPics , which takes care of everything. It's great IF you're comfortable with your whole photo solution being cloud-based (but you still own and control it, you're not just handing it all off to a SaaS), rather than being self-hosted. Personally I prefer the former these days, but I know that I'm in the minority here on HN.
With AWSPics, sync your photos (just a simple directory tree on your local device) to S3 (I just do it manually from time to time from my desktop, but no doubt it can be done automatically at regular intervals, and/or it can be done directly from a phone), then Lambda functions generate thumbnails and browsable galleries (as static HTML), then you can view it all via CloudFront (password-protected using Origin Access Identity).
AWSPics isn't really being actively maintained these days. But it still works fine for me. The setup is cheap, you basically just pay for the S3 storage (currently costing me a bit under $1/month - and I guess you could use a cheaper S3-compatible alternative like Backblaze and reduce your costs further). No server to set up or maintain. Durable backup built-in. Fast reliable CDN built-in.
I don’t want it to be synced. I want my pictures and videos to be backed up.
I have 2TB on iCloud, 2TB on Google Drive, unlimited back up of photos on Amazon Drive and 1TB on OneDrive. When my OneDrive account gives out of space, I’ll just use one of the other 6 accounts that come with an Office 365 subscription.
I guess for an archive solution, I would use my personal AWS account that’s only used to store my videos that fell off the back of a truck once I took my Plex server off line.
I have 2TB of videos stored in AWS S3 Deep Glacier archive. I’m charged like $2 a month for it.
Yeah, I get your intent, but I think there's some conflation of backup and sync.
Here's a hypothetical - imagine you deleted a photo from your phone, and then modified (in place) a different photo.
Is your expectation that the first would be deleted from the various places you have it backed-up? (If not, then what's your process for fully destroying a photo?)
Would the second be propagated to each of your SaaS storage providers overwriting the existing, or renamed and situated adjacent to the original? And if you modified it in-place a second time?
If your phone were then to stop working and needed replacing, would the restoration process restore your phone to before, or after, the above deletions / changes were made? ie. would the deleted photo exist on your phone, would the second photo be pre, mid, or post modifications?
(I draw the distinction between backups and archives whereby backups get you to where you were most recently (pre incident), and archives let you restore to something like 'Tuesday afternoon, two weeks ago'. I'm talking here purely about backups vs sync though.)
It gets backed up to all of these? On iOS I have tried all except Google and they are all finicky. Even iCloud/Photos.app seems to have a brain of its own.
And on the Mac Apple engineers have made sure the photos are not synced in a simple easy folder hierarchy that can be backed up elsewhere.
Photoprism and piwigo seem to be the most used for self hosting. Haven't tried either but slowly making the self hosted move and these were on my research list.
You don't. I currently run this setup and it's more for people who want to "definitely degoogle" more than they want feature parity with photos. This app is awesome, I intend on contributing if they need it, but it looks solid.
I don't want to degoogle, but having all my photos hinging on a Google account that can more or less arbitrarily blocked by Google gives me anxiety. Google Takeout effectively destroys your data by stripping all metadata from the photos and putting them into a json file. I haven't found a good solution for regular backups of Photos, has anyone else?
Call me old fashioned, but I just don't feel confident in placing my valuable data only with a provider like Google. One day there will be an "Ooopsee, we found a bug in our hash function (refactored by summer intern) and um...data for 'a very small number' of users was backed up to the /dev/null shard..."
I'm very satisfied with PiGallery2. I'd love to try PiGallery3, but I can't find it. Could you share the link for the project, or was it a typo? (hope not!)
I have a Synology NAS and it has a Photos app that is essentially a Google photos clone. It has face recognition, tags, albums, auto backup from phone, etc.
Works pretty well, although I wasn't seeking it out or researching other options
I personally still use and pay for Google photo. Mostly because I am lazy finding a better solution. As google photo UI is becoming worse overtime, I wouldn't mind an alternative though.
how do you get the photos out of Google Photos (and delete them from there I assume?)? I started on a CLI that could track photos across different accounts and make it easy to move then between them i.e. from GP to s3 or to a local drive etc. Was going to include duplicate detection. But if there is already another tool...?
Syncthing + Tailscale. I have it set to only accept syncs from the tailscale addresses. I use this to do voice recordings that fairly quickly sync home for records of conversations.
I'm unclear if Syncthing inherently encrypts transfers, but layering it within Tailscale would add that. No?
> All device to device traffic is protected by TLS. To prevent uninvited devices from joining a cluster, the certificate fingerprint of each device is compared to a preset list of acceptable devices at connection establishment.
So yeah, transport is encrypted. I do believe they need to put that fact front and center, though. It took me a few minutes to find out. (Thanks for making me find out, though! I use Syncthing heavily and it never occurred to me to even question this.)
Thanks for finding that. I had the impression syncthing was focused on efficiency only, and not necessarily privacy/encryption. I had the impression a synced copy would be rebuilt from several sources at once, over the syncthing discovery protocol - and may not be encrypted in transit.
+1 for SyncThing. It (now?) works effortless on my phone and having your screenshots and snapshots automatically on your NAS and PC to process them there is fantastic.
Didn't use the "rolling trashbin" feature yet but need to look into that
And to avoid mishaps, you can also set the phone side to be send only.
Do remember that as you delete photos on your phone to save space, it will delete them on the PC. You would need the PC side to move the photos to a different directory (a cronjob would do).
I use the File Explorer Pro App on iPhones and iPads to occasionaly backup all the albums to a NAS. It also has many other useful features i use from time to time.
This doesn’t answer your question directly but iCloud is definitely not the way. Not only it’s unreliable and unusable largely, it is also designed in a way that discourage interoperability.
On Mac (I am aware you have said Windows) So if I were you I’d just dump app the photos locally in normal folder hierarchies (yes drag and drop; not going hunting in Library folder) and then backup wherever you want them. Here’s something https://support.apple.com/en-in/HT205323. Googled it. Absolutely no idea whether it works or how.
Then put something like Dropbox on the job on iOS and it’ll keep uploading your pics and videos which will show up inside one folder on your Windows as well.
My point is — you can’t rely on iCloud after you setup your backup up sync strategy. So get all your data at one point then start your parallel backup/sync setup (and maybe leave iCloud on as well if price is not a concern).
I use https://github.com/icloud-photos-downloader/icloud_photos_do... running on my NAS to regularly download photos from my wife's iCloud account. The photos are stored in full resolution on iCloud with the EXIF data, unlike Google Photos, so that's nice. The only annoyance is that you need to reauthorize the tool every three months. But it sends you a reminder when the time is coming up.
Beware that edits, portraits and some custom photo modes aren't currently downloaded. And modifying an edit twice or more will not sync. (no modtime or chksum)
I was able to backup my iPhone photos (stored on device) to Window using Dokany, ifuse and rclone. This have one caveat though: if you are using iCloud, some of your photos are actually stored as low resolution copies of the original to save space on your device, so original will be not backed up.
I wonder if these services will ever reach mass adoption.
I consider myself to be technological advanced, but other than niche photographer, i wonder if 99.99 % of population would ever go through the effort of setting such a service ?
I dream of selling a Box that plugs into the wall that backs up all of your photos and videos and maybe acts as an ActivityPub server, and you can add your friends who also have a Box, and your friends' Boxes back up all of your photos and vice versa.
The added benefit of integrating federated social media is that if you want to share a file with your friends, there's zero load time because the file is already backed up on their Box, or it's striped across multiple Boxes and downloads quickly.
It would have to be dirt cheap though, like a Chromecast.
Honestly I think Apple has a good model for this. The value would be in making the box plug-and-play. Apple has proven people will pay a premium for devices that Just Work. It could still be built on open software. Some kind of a cross between Apple’s opinionated approach to defaults and RedHat’s pay-for-support model of developing open source software.
The problem being solved here is data locality. This idea predates the TV show by literally decades.
Apple doesn’t want to sell us a box. They want to sell us a subscription. My assertion is that Apple’s model of selling hardware would work for home servers.
I repeat myself: focus your mind on the value, or the problem the customer is looking to solve. Data locality isn’t a problem, it’s an implementation choice or detail.
The problem is something like: “I need to keep my family photos/files safe” or “I need to store customer orders and data” or “I need a way to protect my data when I drop my phone in the toilet.”
Even giant companies with eye-watering IT budgets see the appeal of having someone else manage their physical hardware and software infrastructure.. I haven’t worked at a company that owned its own servers in nearly a decade.
Apple doesn’t want to sell us a box and coincidentally nobody wants to buy a box.
Home hosting protects from institutional threats like governments, TLAs, and LEO. Cloud providers can’t offer that. They’re a huge target and it’s an open secret they are compromised. Apple even tried to pitch that as a feature. And they’re the leaders in privacy!
It’s easy to reason about data locality. I can have a literal social network of just my actual friends and no tech company gets to spy on it.
Cloud backups are still good for disaster recovery if they are fully encrypted.
Wow! Those institutional threats sound pretty sophisticated. I’m sure the average self-hosting solution is prepared for adversaries like that, and when agents come knocking on my door I’ll just tell them “no hard drives here!”
You know, with such a scary and complex network of online adversaries and government agencies, I wonder if I could pay someone else to manage that risk for me. My business is selling online greeting cards for pets and I don’t really specialize in network security and data protection.
Okay, crazy idea. What if you didn't even need a piece of hardware and the service included online storage? It would simplify it even more for the end user so they didn't have to plug in hardware and maintain it. We could call this service something like eOnlineStorage or iBackupPhotos, maybe charge a varying price up to $9.99 a month for it.
Box would be a one-time purchase. There's no maintenance involved - if it breaks, you just buy a new one and log in, and redownload all of your data from your friends.
They used to have the AirPort which was pretty much iCloud at home. And the early iPod was intended to be a portable personal profile you could plug in to any Mac. We’re not on the best timeline.
I use synology myself, don't have experience with qnap but I think it will be pretty similar. plug the box, activate the service, download the phone app and your photos are automatically synced and browsable from the device. You can add other services but the basic functionality is there.
Because nobody outside of HN is going to install software on an old phone or laptop. But they'll buy a $30 Box.
By all means the software will be open-source (and mostly off-the-shelf if possible) so anyone can install it on whatever device they have, but the killer product will be the Box.
Yeah but it doesn't support syncing photos from the camera roll. So you have to use something else to copy photos from the camera roll to the mobius sync folder. I use the PhotoSync app for that, but it's not free and it's not that reliable. And the extra step adds even more latency to the whole pipeline. iOS allows these apps to run in the background only once per day generally, and so it can be several days before photos get all the way synced.
If this app can do background sync of the camera roll directly from iOS to an Android device in one step I may switch to it and stop using both Mobius Sync and Photosync.
AFAIK you can use SyncThing-like utilities, like Möbius Sync, on iOS, but not for photos, because… iOS does not consider a photo a file. Which is baffling (probably for some security reason), so I have to stick with a USB cord and ifuse + heif-convert on Linux.
Apple provides access to photos just fine, using a different API.
Having a separate API for photo access is annoying while developing this use case, but it's appropriate for a couple different reasons: for one, if apps could access the raw files, that could reveal metadata like location. For another, filesystem access wouldn't properly handle cloud-synced photo libraries.
Perhaps photos on iOS are persisted into some sqlite-style database on the device rather than as individual files...
The Dropbox iOS app would be a good example here. It acts as a ReplicatedFileProvider so apps on the device can access your Dropbox files, but it also provides an optional config flow where users can ask for their photos to be backed up. Then, the Dropbox app fetches the photos through PhotoKit and writes the .JPEGs or .HEICs to a folder of your choosing into your Dropbox files.
That’s what I mean, there are technical reasons why it doesn’t necessarily make sense to represent photos as individual files: they might not live in a file system, they might have to be downloaded, they might have to be transparently converted from one format to another, they might have to have metadata transparently stripped. Any applications are going to need to use an API to get the right form of photo that the user wants anyway. Having a dedicated API that users can call as they need makes more sense than having the operating system hackishly pretend that your photos live in a bucket of JPEGs somewhere
You can use it for photos, but for those contained within the photos library that's handled separately through the PhotoKit now. It should be technically possible to backup your photos library through syncthing, I only think the automation of such and the permissions required would probably be annoying to deal with. Not sure if Mobius developer(s) looked into that yet.
If you don't use the photos app, you can keep them all in the files app and it syncs just fine. Annoying to use since the files app is nowhere near as nice for photo browsing/management.
What I've done is to leave a Windows VM running which has both the iCloud Photos app, as well as Syncthing, both pointing at same directory. This then works as an iCloud/syncthing bridge, letting my iPhone sync with my linux machines.
Yep, backing up phones (without using cloud, if you don't trust it, and I don't) is a pain... syncthing solves this, by syncing all the photos (and other files you want/need) onto a pc/server, that you can then back up using other solutions.
Something is absolutely unavailable on iOS. All these other apps I’ve tried just don’t work. I think it’s that mobile operating system’s problem. Probably the devs don’t want to play a game that they know is rigged.
