Hacker News new | past | comments | ask | show | jobs | submit login
Syncthing graduation day (syncthing.net)
282 points by kim0 3 months ago | hide | past | web | favorite | 114 comments



When I started using syncthing some years ago it worked, but it was still a bit brittle. I ran over time into several situations where the sync would just stop or not start, be slow, some error happened somewhere and the GUI did not show them properly, there was no built in reacting to file modification, just a regular sync interval. It was good enough to be used, but certainly not done yet.

Yesterday I installed it again on my main PC (part of a migration from a different linux distro) and I was impressed: The notifications when adding new devices arrived with a very small delay, syncing just worked, file monitoring is now built in, the error message related to that (inotify limit) was helpful.

Syncthing came a long way and is great software, congrats for reaching 1.0!


I switched to Syncthing from Dropbox a few months ago. While there are a handful of warts, the actual syncing seems to Just Work (TM). I am massively impressed with this, as building even a centralized file hosting service is quite the undertaking. I donated and I'd encourage others to do it as well, link on the homepage: https://syncthing.net/


> handful of warts

What type of issues?


What I'm thinking of are synced ignore lists of some sort. I think I read in the issue tracker that this is something they plan to change eventually.

And related, sync % reflecting ignored files. For example I've ignored a bunch of transient things like node_modules so my backup server sync is permanently at 70% even though it's actually up to date with all the files I want synced.


I do this in two steps:

1. An "stignore.txt" at the root of a shared directory. This gets automatically synced to every device.

2. "#include stignore.txt" in '.stignore' that I need to manually write on every new device.


I've been using Syncthing for a few months, and it has been bulletproof. Truly set and forget. I was extremely apprehensive when I started, but it convinced me. I wholeheartedly recommend it.


I started shortly after Dropbox deprecated BTRFS and told me to move to a "better" FS like FAT.

After the initial setup, which wasn't that hard after all, but took more effort than doing nothing and continue on Dropbox I was amazed. My setup became better as now my phone is a real client that syncs files, not just a file browser that can "star" and cache some files, and there's more granularity on what's shared.


Funnily enough, the Android app not being (or not having the option) to be a browser and cache is one of the things holding me back from wholeheartedly embracing Syncthing. Different needs for different users.

I'm aware of https://github.com/syncthing/syncthing-lite but the consensus seems to be it is not quite there yet. I should perhaps give it another go.


This is why I run both Syncthing and Nextcloud. As you say, different needs.


What's wrong with the android 'app'? It works fine and besides initial setup I have openned it like maybe 3(?) times in the 3 years I have used it.


For those who don't know (including me) could you explain what you mean by deprecating BTRFS? I don't understand how the file system is relevant to a file syncing app like Dropbox.


Neither does.. well anyone. Dropbox decided to drop support for BTRFS and pretty much any other Linux file system that isn't ext4, and to my knowledge still hasn't explained their rationale to this day.

See: https://www.dropboxforum.com/t5/Error-messages/Dropbox-clien...


Sharing the reason is either a legal risk, or a loss of competitive advantage risk (some optimization that only works when their whole pool is in particular format).


As of last November, Dropbox only supports ext4 on Linux, NFTS on Windows, and HFS+/APFS on Mac. Couldn't tell you the technical reasons for the change, but they're listed here:

https://www.dropbox.com/help/desktop-web/system-requirements...


Probably i use syncthing in the most odd way.

I've around 70 servers with syncthing on them and i am using it with Route53 for use as CDN.

One thing i am missing is that, i want to sync in a way that it's aware of the network topology and syncs from the nearest network first.

The downside is that all your data lives on every server.

Is there a smart way to sync only needed data using p2p for CDN application?


I'm using syncthing to sync three computers. Two slaves connected to one 24/7/365 online master. Unless I'm behind a very aggressive firewall, I can connect to master to sync everything.

It creates a big peace-of-mind. Debian's version was built with iNotify for a very long time, so I never got big CPU spikes. I think iNotify is merged into master last year or so.


I don't remember enabling iNotify earlier than it was merged in master. Did I miss a build flag that enabled it earlier than I should have?

Anyways, I'll use this opportunity to say that it has always been great to work with the guys at Syncthing. They responded quickly to every issue that we have ran into. For example, the switch from FontAwesome to ForkAwesome was done for due to a Debian issue:

- https://github.com/syncthing/syncthing/issues/5236


IIRC, the package description said something similar to "this is the iNotify version of the syncthing binary", or I may be completely wrong. I need to get previous versions and look to the descriptions of each :)


You might mix that up with syncthing-inotify, which was an external small inotify watcher that then pinged syncthing via its API. It was the recommended way to get inotify support till the functionality got merged.


Maybe, however all my Debian installations are very strict in their repository configuration, and I'm failing to see syncthing-inotify package in Debian snapshots server [0].

...or maybe I'm making all these up from all the things I read while researching syncthing, who knows :)

Regardless of all, it's working very well and it's dependable now, and it's the point I think :D

[0]: https://snapshot.debian.org/package/?cat=s

Addenda: Actually, forgetting all little details about it is a reliable sign of an application working well and just disappears to the background IMHO.


How reliable has it been? Do you mind sharing the number of files in the shared directories and their total size? I am asking because when I was searching for a cloudless Dropbox replacement, some tools would lose some files (or truncate them to zero).

Then I found Resilio Sync and I've been using it for about 3 years now without any problems. Now I have a feeling that Bittorrent Inc will abandon the project after the acquisition. Syncthing is looking promising.


SyncThing has been completely reliable for the past two years across my Android phone, Linux laptop, and a Linux NAS.

I use it to backup photos / videos that I capture on my phone, as well as to sync my KeePass database and Org Mode files which I edit on both my phone and laptop, often concurrently. My largest folder is 46 GB across 9,416 files.

The SyncThing Usage Data dashboard at https://data.syncthing.net/ has really interesting statistics: apparently at the 50th percentile, users are syncing more than 5,300 files and over 12.3 GB of data.


Wow, either I'm reading the stats wrong, or someone has 84.6 TiB in Syncthing. That's impressive.


I don't think this is too crazy as long as it's made up of big files (video perhaps?)

I'm using Syncthing for misc files but accidentally dropped a large git repo in it about a month ago... All the tiny files caused it to really chug.

Not that this is really Syncthing's fault.. file system operations over many files just tend to be way slower.


Unless I kill the service accidentally, its uptime is equal to the system's uptime.

I sync documents and wallpapers accross my computers, I didn't see any corrupted or truncated file, it has been completely reliable from that point of view.

Also, as I said before, if the firewall of the place I connect is not extremely hostile, I can connect to it from anywhere in the world (it also has a auto-fallback to Tor). If you want to go completely dark, you can run your own tracker too. :)

My file stats are: 641 files, 85 folders, ~980 MB size.


> also has a auto-fallback to Tor

1. That is neat.

2. Could you please say how you did this?


I didn't do anything. When I checked the local page (localhost:8384) to see how syncthing is doing, one of the clients had a tor address in the list of "last known" addresses :)


I've been using it with SyncTrayzor on Windows for a few years and have found it extremely reliable. Currently syncing 3 computers with ~ 10,500 files, 600 folders, and 280 GiB. It is set up for LAN only, as I don't have a use case for syncing on the go.


Resilio is already a separated spinoff from Bittorrent Inc. You don't have to worry about Bittorrent abandoning it ... just Resilio itself going under! (Which might be quite likely, since it's so small on its own.)


I sure hope they don't go under (Resilio). Why do you say it's "quite likely"?


Because their only revenue is selling licenses of Resilio to corporations, and my hunch is that's a hard sell.

I might be wrong. Maybe they can compete with dropbox by being cheaper!


I liked Bittorrent Sync and later Resilio Sync a lot, but they have been very unreliable when it comes to pricing and licensing, changing both several times.

Their license structure is currently worthless to me. I have a license, but that is only for home use. They only offer subscription licenses for business. Subscription licenses are harder to purchase in some organizations, plus everyone that I want to offer files to needs to buy a license as well. This is a large downgrade from e.g. Dropbox, where one can just hand out download links, open file submission requests, etc.


I've been using syncthing for a few years because I was sick of missing files when going from my laptop to desktop and vice versa. I wanted to sync most folders in my /home directory (documents, photos, videos, code...) While ignoring node_modules folders.

It has worked surprisingly fantastically. I thought I would need a server to use as a "master" but syncthing actually supports merging folders automatically. It does not need a static IP address for either computer and it will opportunistically sync my laptop and desktop when they are both on the same network.

I had no idea before I tried using it that syncthing could support such a setup so well. High praise.


I've seen Syncthing before but I don't think it is exactly what I need. If I'm not wrong, it is a secure p2p protocol and a suite of apps implementing them to synchronize folders between devices, in the way you would do with Rsync.

That means if I have several terabytes that I want to sync between devices, I'd need to have enough space in every device, is that right? (and each device would get a 1:1 copy of the folders). I read that when there are conflicts, the apps create a new "filename".sync-conflict file so you can manually resolve conflicts.

What I'd like is a secure way to backup all my files to some cloud provider (S3?) across all my devices, and have an MRU cache in a folder where I can configure the size (for instance, keep at most, say, 512mb of the files I most recently used). For things like photos, it would be better if the sync thing would get thumbnails from the cloud, so I would not need to download the whole thing just to find photos.

GDrive / GPhotos is the thing I use now. The combo has a lot of bells and whistles and I'd be fine having a simpler product where I could have more (perceived?) control on how things display, when syncing happens, etc (in lieu of other things like the nice A.I. photo search, integration with GSuite of apps, etc.). Kinda like using a file explorer on my local desktop.


Do you know https://github.com/s3ql/s3ql? It mounts remote cloud storage like a local filesystem and promises a bunch of optimizations against network latency, including a local cache. Apart from the thumbnail feature that should fit perfectly to what you are searching.


Thanks! I did not know about it, I'll take a look.


Also rclone


Nice! rclone seems better documented and I could even find a GUI frontend [1]. Will need to give it a try!

[1] https://martins.ninja/RcloneBrowser/


You don't need to sync every folder to every device. Each instance can choose which folders it wants to sync.

Plus there are several folder types, particularly send or receive only.

See https://docs.syncthing.net/users/foldertypes.html

However,it's not a cloud backup service. If you need online storage then best option is to get an online service.


The biggest problem I have with syncthing is that it can't be used with iOS, and there are no plans to ever support iOS. I don't personally use iOS, but my wife does, so I can't get her phone to automatically sync pictures of our family with my phone and computers.

I suppose the people involved don't use the platform, and I realize iOS is more annoying to develop for than Android, but it's a pretty major downside and it's tempting me to use a cloud provider instead. I'd much rather just pay $15-$20 or something for an iOS app.

Maybe now with the 1.0 release it will be stable enough that a third party can step up and make an iOS app?


If you can find other people who share your desire and present the developers with a clear case for making decent money that way, I'm sure either they or a third party would take up the challenge soon enough. Many of us who have no personal interest in the Apple ecosystem won't support it just because someone asks because of the effort and investment required - being meaningfully paid for it changes the equation.


I can certainly see the need for something like this, though I've seen a lot more folks on another forum talking about ownCloud and Nextcloud as alternatives to Dropbox business plans.

I really dislike that the Syncthing website appears to have mastered modern "we won't tell you what it is" marketing speak. Did I miss something and this is now a game with scoring for speaking around topics?

"Syncthing replaces proprietary sync and cloud services with something open, trustworthy and decentralized. Your data is your data alone and you deserve to choose where it is stored, if it is shared with some third party and how it's transmitted over the Internet."

Oooooooh, new shiny with buzzwords! What's it do? Is it a replacement for Chrome's cross-system syncing? Firefox Sync (which I think can already be self-hosted actually)? Password managers? Maybe hosted Exchange? Wait, it's not?

Oh, wait, it's a multi-system file syncing package. Well why didn't you say so while you were telling me I deserved to have control and blah blah blah. Cripes, it even says "a large number of features have already been implemented" then provides a list that implies that running on Windows, OSX, Linux, the BSDs and Solaris is one of those features.

There are several places where what it does is implied, but to actually get to a statement of "For this guide let’s assume you have two machines between which you want to synchronise files." I have to go to the bottom of the site, click Docs, then either click on Introduction and Getting Started (in the documentation structure) or click on the "getting started guide" link within the first paragraph of the Introduction.

grmbl get off my lawn

Edit: I should note, I'm not talking about the 1.0 announcement on their forums - that's for folks already actively involved with it. I'm talking about the very spare https://syncthing.net


Lately, I find myself looking for the github icon before anything else in a website. I find a github readme much more readable than any landing website.


Wow, no kidding.

The first two sentences of the readme are:

> Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers.

That copy should absolutely be somewhere on the main website.


> That copy should absolutely be somewhere on the main website.

Yes. Specifically, it should be in a large font, roughly centered, at the top of the page. You should load the page, and immediately read those words.


Agreed.

Also: top-left logos should link to the root (syncthing.net) not to a sub-root (forums.syncthing.net). When I land on your forum site and I want to know what syncthing is, I expect to find the answer by clicking the top-left logo in the header.


Seconded. I was just about to say exactly this.


They appear to be using Discourse, which appears to have copied phpBB/Vbulletin's decision to do the same thing for the past 20 years.


Jesus Christ do I know what you mean. It's like actual things don't even fucking exist anymore and everyone is just trying to sell buzzwords themselves.

I used syncthing to implement part of a project at work and it does a pretty good job, it deserves a website that actually tells you what the hell it is.


I know, right? It seems like a great project, but just saying "P2P open-source Dropbox alternative" would convey as much information as the rest of the home page combined.


Yeah, same. I had to look at Wikipedia to figure out what it was.


Yeah, as somebody who loves using Syncthing, I would be very hard pressed to know what it is by looking at the front page.

I cancelled my YouTube Premium, and started seeing ads on YouTube for Android; and one of the first few was an ad for some app called Killi or something. It jumps straight to how to find it on the app store and google play, and then explains how to set it up, all the while you're wondering what on earth it is.


I feel like I missed my best opportunity (pre-social media) to combat this.

IIRC, Nexium (antacid of some sort) when it was first introduced had ads everywhere full of people talking about the "Little purple pill" and how much it had changed their lives. There was nothing in the ads that indicated anything about what it did, just "ask your doctor if Nexium is right for you."

Had I been a bit more connected at the time I think a real push to start talking about "The purple pill that cured my X" would have been amusing, with X being something like uncontrollable flatulence, incontinence, explosive diarrhea, etc. Unfortunately, that was back in the days when social media was LiveJournal and Usenet was already well on its way into decline.


Strongly agreed.


I've used syncthing to keep my music, books and docs all in sync between my home server and my laptop. It does a great job. I have also used it for a few years now in place of plex sync. I have plex keep certain unwatched shows optimized for mobile and I just sync that directory to my laptop and phone so I always have my unwatched shows with me. Plex sync was totally unreliable for this so I moved away from it and syncthing has never disappointed.


Syncthing is great. I use it to sync my password database and some other stuff across my devices and have not run into a single issue. I set it up 3 years ago and haven't had to touch it since. I had no idea it was technically in "beta" all this time.


Syncthing + KeePass. Amazing convenience and Time saver.


Does it still consume vast amounts of CPU and memory? A few years back I used it on a few hundred GB and it almost constantly used 1-2 CPU cores. On the server it racked up thousands of CPU hours, i.e. it cost real money to run.

Apart from that I don't recall any major problems - which is very positive for anything related to file synchronization.


I've used syncthing for a few years to synchronize 100k+ files and 10k+ folders. I used to regularly get CPU usage spikes from syncthing whenever it scanned folders for changes. The default operating mode of syncthing used to be "do a full scan every N (configurable) seconds for changes," and you needed to run a separate program (syncthing-inotify for me) to use filesystem watches instead. Those full scans would be cheap if you're synchronizing just a few folders, but it's CPU-expensive to regularly check tens of thousands of folders for changes (it might have been stat-ing every file for modtimes, too, I dunno).

Over the past year or so, file watching (via inotify/etc) has been built into syncthing itself. I enabled watching on all my computers and syncthing's CPU usage has gone down considerably as a result, while user happiness has increased; I spend less time banging on syncthing's pipes and more time doing stuff with the files it synchronizes.


I tried it a few years ago on a mix on Linux and Windows systems and quickly abandoned it for Seafile because of the CPU issues. Any idea if it now supports (reliable) real-time file watching on Windows?


Windows and Mac support is much better these days.

However, the case insensitive file system in Windows makes it annoying to use with *nix systems.


It never has for me, and I've got some large sync'd dirs (~70GB being the largest). Been using it for several years and it's great.


I'd love to start using Syncthing, but the absence of storage encryption is absolutely a deal breaker. As I understand, it was a planned feature quite a while ago, but now it isn't even in the backlog anymore. A pity, really.


That problem is easily solved at other places in the FS stack. Use an crypto overlay like EncFS and sync only encrypted data. Alternatively, even stream encrypted updates to a directory via a gpg job and sync the encrypted blobs. Another layer, but the script would be so simple it'd essentially write itself.

While it would be convenient to have configurable encryption on endpoints, syncthing is composable, and thus this issue can be worked around. If you have a platform that doesn't allow you to compose tools easily like Android, that's limited technology.


I wrap my cloud storage with encfs by default:

https://en.wikipedia.org/wiki/EncFS

EncFS is not perfect in that if your adversary has the opportunity to watch your encrypted files updating on the regular they can (allegedly) break the encryption, but they can't do it from a single snapshot.

It's a decent protection against an airport sniffing your phone traffic while you're on the public wifi, it's not going to stop the NSA if they decide you're a person of interest. In other words, if this is a concern for you EncFS is an acceptable baseline protection.


It runs on your own devices, which should either be trusted or have their own encryption (it's built into every major OS). Why would additional encryption be useful?


The point is exactly running on untrusted devices - get a el cheapo server at some VPS provider and have it backup all your data + serve it from a always-on decent datacenter connection.

I've also been waiting for that one feature :/


I see. So it would intentionally not be a live usable copy. I've been using Duplicity going to Wasabi for that purpose - encryption, versioning, etc. One of my Syncthing nodes is an always-on PC at home, so the files sync with other devices whenever/wherever they have a network connection, which is what your encrypted node would do.


https://github.com/syncthing/syncthing/issues/109 is probably the issue you want to follow. Yes there hasn't been any activity on it for some time unfortunately.


How does this compare with Perkeep (previously Camlistore)? They seem to be very similar, though I haven't made the jump to try either out.

What I'd like is integration with on of Perkeep or Syncthing from my freeNAS box to my local devices.


IIRC, Syncthing is decentralized P2P and synchronizes standard tree(ish) filesystems (like most similar projects). Perkeep does away with the filesystem and instead uses a blob-based merkle-tree-like system with good properties for very long term archival (and as a side effect, can also use P2P and multiple encrypted cloud backups easily). So Syncthing can copy preexisting file structures and fits in usual workflows, but Perkeep has some data storage benefits. I don't know how their performance compares.


When you say P2P, it's only in a technical sense of not having centralized servers, right? I don't want to interact with nodes I don't own, for my particular use case.


If your nodes have direct network access to each other, no.

But if two nodes that have added each other's keys cannot reach each other directly (e.g. are in separate private networks/behind firewalls/etc) but both can reach an external relay server, it may act as a dumb pipe bouncing encrypted traffic between them.

https://docs.syncthing.net/users/relaying.html

There is a public pool of such relay servers (https://relays.syncthing.net/), and by default syncthing will reach out and connect/announce its itself there so other nodes looking for it could contact it via a relay.

You can host your own relay server and configure your clients to use it exclusively: https://docs.syncthing.net/users/strelaysrv.html#client-conf... or if you only want syncing when your nodes are on the same local network you should be able to just not configure any relays at all.


That's correct. It only syncs between nodes you configure.


Been using syncthing for a few years - I had used BitTorrent Sync until my default search provider became Yahoo! and I could no longer trust anything BitTorrent Inc was doing...

Syncthing is great - a folder that syncs, I have a VPC in the cloud, so for the data I really care about, it's mirrored offsite more or less in real time in case my house burns down.


Congratulations to the team!

I like a lot the announcement: it's very easy to stay perpetually in beta to escape commitment. Their reasons for graduating are sensible and I wish more developers of software on version 0.x since a long time would replicate it.


Been using Resilio-sync but have been really excited about switching to Syncthing, glad to hear about the 1.0.0.


same, the ST-WebUI is just not doing it for me. I paid for Resilio now and I'm quite happy with it. But I also really wish for Syncthing to become a lot more mature and widely adopted to have as an alternative in case resilio discontinues.


I didn't know Syncthing and after installing it on a couple of machines I'm very impressed!

However, I'm surprised too see that by default the whole interface is available on 127.0.0.1:8384. It is possible to set a password but it's a bit hidden in the settings and the documentation does not insist on it.

This makes me wonder, how bad is it for a malicious program running locally to have access to this API?


It's also possible to move the listener to a permission-protected unix socket https://docs.syncthing.net/users/config.html?highlight=unix%...

Pity that I don't know of any browser that readily supports HTTP over a unix socket (nginx can proxy_pass it, but then you're back to a TCP listener).

But a multi-user install could reasonably lock things down in that way, though the defaults definitely favor "works out of the box on a single-user machine".


So? Local programs also have access to the filesystem, which includes the synced data, config files and procfs.

What exactly is your threat model?


On a multiuser system it could be an issue. Other users with access to make HTTP requests could add a malicious endpoint. e.g. on a shared server of some sort. I could see this in academia or some business settings. Threat model would be an attacker with a foothold on one machine gaining access to the documents of another user.


Could be, theoretically, but this would probably be inconveniencing all syncthing users for a negligible fraction of the userbase.

On a shared computer where you perceive your peers as a threat, adding a password to syncthing web UI is probably a minor entry in a long list of things you have to do to secure your processes and data, and to ensure the kernel and programs you're running are trustable (in practice, you can't verify that with 100% confidence; they can be backdoored, out-of-date with known security flaws, etc etc).

Ensuring security in such a malicious (where your co-workers/peers are apparently a nasty bunch) shared environment is so hard that to begin with that I personally wouldn't put any sensitive data there at all.


It defeats user-privilege separate on localhost - I (or an unprivileged daemon) can't get at parts of the filesystem belonging to their user instead of mine, but I can get at TCP ports on localhost that belong to other user's syncthing daemons, and reconfigure their syncthing to send all their data to me.


You assume a machine with a single all powerful user. These days computers don't really work this way, for our own good. A huge amount of effort is put into keeping programs as isolated as possible (unix permissions, chroots, cgroups...). Most systemd units include isolation features enabled by default.

Having an API that can reconfigure syncing being available without authentication for anything running on the machine feels backwards to me. More so for a project that focuses on security.

If anyone from the project reads this, may I recommend either encouraging users to set a password or mentioning the implications on the page about Security Principles[0]?

[0] https://docs.syncthing.net/users/security.html


The real reason behind isolated systemd services is to confine the damage if a world-accessible daemon gets breached.

If your threat model is you can't trust the programs running on your computer, then maybe you should switch to a different OS, for example Debian, and stick to the package manager.


It's not great but not particularly horrible either. It takes the interesting combination of "this machine is locked down to the point it does not allow outbound connections by default" and "this machine has a default syncthing install with no password set and an actively running piece of malware that can't create an outbound connection otherwise".

Pretty rare someone manages every outbound connection but doesn't configure a password on the tool handling syncing all of their data across the network. Still, would be better if it prompted for a password the first time a user visited the page to set up the client.


As others here have noticed, Syncthing has improved by leaps and bounds over the last couple of years.

I use it for syncing my phone (everything!) to a pc, and keepin several ps's up to date with each other. These days, it just works. Works so well, in fact, that I have begun experimenting with up-sync to a production server. So far, things are looking real good.


I have nothing but applause and kudos to add here, they are really deserved. Syncthing is fantastic.


Is syncthing already able to do client side encryption (so you can host your files on a server without the server knowing the files you are storing there)? I remember some years back it only used encryption during the file transfer.


It's still an open issue: https://github.com/syncthing/syncthing/issues/109

But you can combine it with an encryption layer yourself by using any of the projects listed here: https://nuetzlich.net/gocryptfs/comparison/


Not yet, see https://github.com/syncthing/syncthing/issues/109

That's pretty much the only feature I miss with Syncthing.


One of the forum posts says:

> I really love to use syncthing as a anti-bit-rot check system (i already do^^).

Bit-rot is something I want to (manually) protect against, without setting up a NAS running ZFS. How would you use Syncthing?


I assume if bit-rot happens then a conflict between versions happen (as if the file was changed manually), so you know a file got changed and can recover the previous one ? I doubt there is any self healing capabilities

By the way if you're on linux btrfs has self healing capabilities (and any of the cheap 2/4 bay 64bit synology nas can use the btrfs self healing and coupled with raid protect your data from rot). Not a very big expense for a lot of safety.


How could inotity notice that though? Bit rot happens at a different level. Detecting it would require regular full data scans, right? And then I guess all files must be write-protected to actually trigger a conflict instead of syncing the corruption. Hmmmmm.


Windows unfortunately - love it most of the time but when it comes to file systems...

Think a NAS will be a future investment.


If going Synology, make sure to go for their 64bit models (I don't think they still sell 32 bits, but unsure); it's needed to support btrfs. Other good brand is QNAP. good quality 2 bay can be found below 300€, 4 bays below 500€.

As the other commenter said you can also built it yourself but frankly configuring Samba and its permissions and updating your mdadm config when changing a drive and all the other things gets boring and borderline dangerous really quick, just know that if you buy a pre-made NAS what you're paying is not really the hardware (you can get better for cheaper if homemade), it's the software.

I like DSM (synology's software) because it doesn't get in your way, it lets anyone use easily yet still offers the advanced options. If you've never had a NAS or don't want to spend hours in config files every now and then, I recommend it.

(as usual the question for devices like this is not so much "how easy is it to set up" but "how easy it to recover if say a drive died and you need to repair your array")


A NAS of course could be a rather cheap system running Linux/FreeBSD with three drives in a RAID-Z1 configuration, with SAMBA server. Then you're not dependent on what fs Windows supports (though I recall there being some work towards supporting zfs on Windows, so perhaps eventually).


Use something like snapraid (http://www.snapraid.it) on the master, perhaps? That's probably the lightest solution you can put in place


Is this a good thing to use to, say, keep two git repo checkouts in sync between a laptop and dev server, to avoid the need to do "deploys" for hacking/testing? Or is that overkill?


You can certainly do that but I ask why not just use git and set up the same remote? Unless you don't have one.


Fwiw mirror might be a good fit for that sort of "super real time while editting source files" use case:

https://github.com/stephenh/mirror

Disclaimer it's my basically-well-functioning hobby project. And obviously wouldn't recommend it for doing actual deployments with it.


Last time I used syncthing was couple weeks ago v. 0.14.54 and it's still hit or miss for me in terms of upload speed. Whenever I'm sharing something with friends the upload speed is completly random from 1Mbit to 20Mbit(my max upload speed). I didn't figure out how to debug this issue but I've tried even running my own relay just in case but that didn't solve it either.


For those, like me, wondering what Syncthing is, Wikipedia describes it as "a free, open-source peer-to-peer file synchronization application available for Windows, Mac, Linux, Android, Solaris, Darwin, and BSD. It can sync files between devices on a local network, or between remote devices over the Internet. Data security and data safety are built into the design of the software."


Or in other terms - it allows you to host your own dropbox between your own devices using a protocol similar to torrent.


Not quite. Dropbox makes it very easy to share (publicly or privately) your data. Seafile or Nextcloud I'd say are more similar to Dropbox


Ah yes, minus public sharing. :) My bad.


Syncthing is amazing, been using ut for years. So flexible, nothing beats it for filesync.


I switched from Dropbox->Seafile after Dropbox stopped supporting BTRFS. Seafile is minimalistic like syncthing, but also has an iOS app that will do photo syncing which is important for me.


I love this because it's fast and has a way to ignore files which dropbox lacks


How's this different from Unison? The documentation on GitHub seems sparse.


Unison does one-to-one synchronization (local or remote). It puts more emphasis on reconciling sync conflicts. It can handle symlinks. It isn't really maintained, is it. It has problems on windows. It's written in Ocaml which had its heydays back then for a short period of time. Unison rather is a conventional tool which you call and then wait until it's done.

Syncthing does many-to-many remote servers synchronization (but you can also do 1:1 or 1:M). It leaves reconciliation of conflicts to the users. If you're lucky, a copy of the file in question is created that has as "sync.conflict" suffix. It ignores symlinks (or copies them as is). It is actively maintained & has a much wider user community. It works on windows ... as well as android etc. It's written in Go. Synchting can also be run in the background to watch folders and keep everything in sync as files change. (This actually is the main use case.)

You can use syncthing for almost everything unison does when you want to sync servers, workstations, phones etc. You cannot use syncthing to keep a memory stick in sync. IMHO unison is still better at syncing two directories with frequent merge conflicts. It's a pity it got trapped in the ocaml hole. I personally replaced unison with syncthing for most use cases.


Thanks, I misunderstood that Unison had any sort of 1:M sync - seems like that part is a hack using multiple pairs in a star topology.

Also good point about OCaml and not having iOS/Android support.


I (still?) use Sparkleshare. Is it worth switching?




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: