
Syncthing Usage Data - GutenYe
https://data.syncthing.net/
======
okket
And it looks like two of the last really big tipping stones are being
addressed:

1) Advanced NAT traversal via UDP, which is is less efficient than TCP, but
orders of magnitude better than relying on relays. More about this here:
[https://kastelo.net/2017/03/08/syncthing-
kcp.html](https://kastelo.net/2017/03/08/syncthing-kcp.html)

2) Internal filesystem notification watch facility for near realtime sync
(currently possible via an external service -> syncthing-inotify). For more
see pull request
[https://github.com/syncthing/syncthing/pull/3986](https://github.com/syncthing/syncthing/pull/3986)

The future looks bright!

Addendum: Have a look at the community forum
[https://forum.syncthing.net](https://forum.syncthing.net)

------
jcoffland
I have one suggestion for the authors. Make it clear on the front page exactly
what Syncthing does. There's lots of information about Syncthing's qualities
but if you don't already know what the software does, it's not obvious.

~~~
adrianratnapala
Hear Ye!

But the same suggestion could be made about almost any product with a web-site
these days. Indeed Syncthing is doing much better than most; they have a box
at the top of the page with a one-sentence explanation/plug.

~~~
bedros
happy syncthing user here; I agree with comment above, syncthing purpose
should be explained better, probably something along the lines

similar to dropbox

many separate sync folders

many computers to many computers syncing.

Personal storage, no data in the cloud

------
Nutomic
Note that usage reporting is disabled by default, and has to be enabled
manually. The Android app alone has around 30.000 active installs according to
Google Play statistics. Usage reporting only shows about 2.000 of those. Based
on this data, I would estimate that there are between 100.000 and 300.000
active installations of Syncthing.

Source: I'm the maintainer of the Syncthing Android app.

~~~
voltagex_
Thanks for your work.

Is there any way to get additional information about why a phone and a desktop
client on the same LAN can't see each other? (or take 10+ minutes to sync?)

~~~
canton7
Post on the forum ([https://forum.syncthing.net](https://forum.syncthing.net))
-- that's where you'll get the best level of support.

------
dragly
It amazes me that Syncthing handles rapidly-changing files better for me than
Dropbox. Where Dropbox has left different versions of files on different
computers even after they have changed multiple times, Syncthing happily
synchronizes them without issues. And the fact that Syncthing is open source
and allows me to keep files on my own machines is a big plus. However, I would
love to see a paid option for easy backup of selected files to a machine in
the cloud. Some people want their files backed up and accessible through a web
service and I wish I could recommend Syncthing to them as well.

~~~
jamwt
Just a heads up: that almost certainly means SyncThing is failing to recognize
some classes of conflicts and concurrency races, and is destroying some
changes. Doing this right is a hard problem. I know they can be a pain to deal
with, but we don't spin out conflicts for no reason!

Disclosure: I work on the Dropbox sync engine.

~~~
growse
Do you have any specific bug or design reference to the Syncthing codebase
that illustrates this, or do you simply believe that it's impossible to do it
faster than Dropbox does it?

Fwiw, i frequently get notifications from Syncthing that it's found a conflict
it can't resolve and asks me to resolve manually.

~~~
jamwt
Sorry, not super familiar with Syncthing. It's definitely not impossible to do
better than Dropbox, but to some degree, as in all things, it's a function of
engineering resources, telemetry, and usage.

We've likely put 25-100x more resources into solving this problem over the
last 10 years, and we just have a lot more data b/c we have 100s of millions
of users on lots of platforms using Dropbox with every application you can
imagine. So we're able to tease out the "long tail" of weird file system and
application behavior in a way that's very difficult for smaller projects.
Truly durable conflict management in the face of arbitrary mutations by user
applications on the filesystem ends up being a really, really hard problem to
cover exhaustively. The Dropbox client handles literally hundreds of special
cases.

So, yeah, I believe it is (in general) safe to assume that Dropbox is probably
doing A-more-correct-thing for a complicated (and admittedly confusing) reason
when it comes to sync behavior. But we're not perfect--we do still find
surprises from time to time, so feel free to contact support if you see
something that looks wrong!

------
canton7
One of the common complaints about Syncthing is the lack of native UI (you use
your web browser), and the difficulty in setting it up for non-technical users
(copying an executable into a directory, running it directly, setting it to
automatically start on login aren't something you'd ask your parents /
grandparents to do).

The homepage recommends Syncnthing-GTK
([https://github.com/syncthing/syncthing-
gtk](https://github.com/syncthing/syncthing-gtk)) (Cross-Platform) and
SyncTrayzor
([https://github.com/canton7/SyncTrayzor](https://github.com/canton7/SyncTrayzor))
(Windows only) to help with these pain points. The provide installers, more
native-feeling UIs, tray icons, etc.

There are also a large number of other Community Contributions
([https://docs.syncthing.net/users/contrib.html](https://docs.syncthing.net/users/contrib.html)).

------
mduerksen
Syncthing is sorely missing iOS support. I know they have their reasons not to
implement it[0], but I feel that feature might be worth the hassle.

Photo sync on iOS without iCloud is a huge pain atm.

My backup strategy involves a NAS folder that stays up to date (i.e. also
deletes photos that I delete on my phone), and which I occasionally sort into
final location. After that I can free the space on my phone.

All other options (including Resilio Sync) are a add-only "sync"[1], which
makes it annoying to sift through all the photos including those I already
weeded out a week ago.

[0] [https://docs.syncthing.net/users/faq.html#why-is-there-no-
io...](https://docs.syncthing.net/users/faq.html#why-is-there-no-ios-client)

[1] In theory, Resilio has a real sync, but for some reason, not for the iOS
camera roll.

~~~
Nutomic
Syncthing is entirely made by volunteers, and so far no one has had the time
or interest to work on an iOS app. If you (or someone else) wants to work on
an iOS app, that would definitely appreciated.

------
vocatus_gate
As a heavy user of Syncthing for the Tron project (
[https://reddit.com/r/TronScript](https://reddit.com/r/TronScript) ) I'm
excited to see how well Syncthing is doing, and really appreciate the hard
work from the devs. We spun up a Syncthing node to distribute the project
after Resilio Sync (formerly "BT Sync") put a hard cap on the number of nodes
that were allowed to connect.

Keep up the great work, team!

~~~
josephb
For those with some spare CPU cycles and bandwidth you can give back to the
project by running a relay server.

[https://docs.syncthing.net/users/strelaysrv.html](https://docs.syncthing.net/users/strelaysrv.html)

Eg that VM running a DNS adblocker, from another front page article :)

~~~
pricechild
What's the drive or motivation behind people running syncthing discovery &
relay servers?

Do they generally then run their own clients to only use their own servers?

With Tor I'm convinced over the "speech should be free, I'm making the world a
better place" type argument.

With syncthing I'm incredibly impressed that this optional (but incredibly
useful) infrastructure is being run by people for free since it doesn't quite
make the world a better place in the same way? It just saves a bunch of other
people setting up their own relay/discovery server on a cheap linode/aws
instance?

Impressed!

~~~
Nutomic
I think people just want to help others, and do something useful with their
spare resources. I run a relay on a server on my home internet connection, so
it doesn't cost me anything.

~~~
2bluesc
> I think people just want to help others, and do something useful with their
> spare resources. I run a relay on a server on my home internet connection,
> so it doesn't cost me anything.

Same. The devs the contribute their time to build syncthing, the least I can
do to help their (our?) cause is to spend 10-60 minutes to setup a relay and
spend 5 minutes to babysit it every other month.

------
jepper
Brilliant piece of software. Perfect to gather data from different devices to
backup in the cloud or to sync documents or a massive music collection.

The only thing missing is an encrypted external sync node so I can put it on a
VPS somewhere

~~~
sinatra
This is the blocker for me too. Support for a zero knowledge server where even
if someone gets access to the filesystem on the server, they can only see it
encrypted.

Other than that, Syncthing is absolutely amazing!

~~~
qznc
I would like to mix it. Then my desktop could be a mirror for my friend
without knowing his files and vice versa.

~~~
keksberg
Is there any other solution that is capable of exactly that?

~~~
brtw
Crash plan does something like that for backup

------
darrmit
Syncthing is awesome. It is more difficult to setup than something like
Dropbox (though, scanning QR codes is pretty easy IMO), but that has gotten
exponentially better over the last year or so via the community and it seems
to only be getting better.

------
mouldysammich
syncthing honestly is one of the best pieces of software i use. It manages a
fairly often changing music library, a lossy copy of it, and my phones photos
really handily all on a raspberry pi 1. One of my favourite programs i use
daily.

------
fiatjaf
Since no one is complaining: I find Syncthing too complicated to set up. You
must create a folder, then get a number for a computer (which you end up
writing in a pastepad or something like that because the two computers don't
communicate), then go to the other computer and place the number in, then do
the same for the second computer to the first.

Repeat it for 5 or 6 machines.

If any machine is switched what do you do? Start over again establishing the
link with all other machines.

~~~
okket
This is the price you have to pay for a (secure) decentralised system, but
syncthing tries to help where it can. E.g. you can set a syncthing instance as
an 'introducer', so you only have to establish a connection to this device to
get offered connections to all others devices. You still need to authorise
these introductions on the devices, but you do not need to type in the crypto
verification strings.

~~~
danieldk
_This is the price you have to pay for a (secure) decentralised system, but
syncthing tries to help where it can._

In general, Resilio Sync's 'key per folder' model seems to work better. I can
give a key to other people and they can join the swarm without extra work on
either sides. Also, it has a logical extension to encrypted-only peers: they
get a derived key that can be used to sync in the swarm, but cannot decrypt
the data.

If there was a stable open source program using Resilio Sync's sharing model,
I would switch in a heartbeat. (There is Librevault, but it still seems to be
mostly in development.)

~~~
ohnoesjmr
Yeah, the only problem is you can never "un-give" a key, which means you
either have to rebuild your cluster under a different key (having to re-give
key to everyone else apart from that one guy you want to kick out), or deal
with the person having access to the data forever.

~~~
brigandish
If the data is encrypted using a session key and then the session key
encrypted with a master key and stored with the data (concat them together)
then it can. You can't revoke access to the data they've already got, of
course, but as changes/additions are made they lose the ability to decrypt the
new data because you've used a new session and master key.

Arq uses this method when encrypting blobs.

------
thewhitetulip
I have been a happy user of syncthing, it is brilliant Piece of software, I no
longer have to run around looking for data cable, I just open my laptop, and
it syncs the complete file system!

------
gravypod
Is there something like Syncthing that works like a RAID system? I'd like to
use multiple computers in different locations and use them to backup and
expand my storage size.

Basically redundant but distributed storage with N node fail tolerance.
Preferably interfaceable via a "/mnt/cloud" directory so I can use it from my
laptop and easily write backup scripts and store data into it.

If this existed I'd have something big to work on setting up next week.

~~~
metalliqaz
This is an interesting idea, but I don't think it could work the way you are
describing it (like RAID). Remember the limitations of RAID, and then add
intermittent failures and huge latency to the communication between the
"drives". It would not be able to remain coherent in real time.

Now, I think you could definitely set up a system that could handle the
distributed data like this and allow you to access it, but I am skeptical that
you could construct such a system that would be able to
read/write/update/delete files on the fly and retain N-node failure tolerance
at all times. You would probably have to allow the system to replicate data
slowly.

There is also the problem of network splits. When they rejoin, how do they
resolve conflicts and such?

~~~
gravypod
This system could largely be non-volatile. Each node would just be writing
data like...

    
    
        <node_name>_<year>-<month>-<day>_<hr>-<min>-<sec>.tar.xz
    

into the filesystem and another server would be reading through everything and
processing it (not in real time, but as a batch system).

It's ok for things to be left over for collection at the next cycle.

Also netsplits aren't much of an issue if you're largely just doing writes and
reads on seperate files. It only becomes a problem for short-term and
currently processed files (that are being read and written to in real time).

This is more of a long-term cold storage system and not nodes talking to each
other.

TL;DR

    
    
        "It's not a bug it's a feature"
    
        Patient: "Doctor, how do I stop the pain I get from doing this"
        Doctor: "You stop doing that"

------
odabaxok
Can anyone share their experiences with Syncthing compared to Resilio Sync
(aka Bit Torrent Sync)?

I am using Resilio Sync since its beta version, but they have added a lot of
restrictions since then and made some functions only available for paying
users.

I am concerned about ease of use (setup and everyday usage) on Windows and
Android.

~~~
vocatus_gate
Oh boy can I ever.

We used Resilio Sync (aka Bittorrent Sync aka BT Sync) for a while for the
Tron project (
[https://reddit.com/r/TronScript](https://reddit.com/r/TronScript) ). It
worked great for our use-case: distributing a large number of files to a large
number of nodes who required read-only access. Anytime I changed a file on the
master node it would blast out to everyone else while simultaneously
preventing any other node in the swarm from propagating changes.

Unfortunately with all the restrictions they introduced, it stopped working
reliably. Internally they introduced a hard-coded peer cap of something like
32, while our swarm was already over 500.

We switched to Syncthing and it's been working well. There is still one major
issue and that's that anyone can mark their copy of the folder as "read-only"
(formerly called "Master"), and then they will attempt to propagate changes
out to the rest of the swarm. So, there's that. But strictly on a distributed
file syncing level, it works great.

~~~
pricechild
That's interesting... so even the original 'master' client is treated equally?
Other clients can decide to send/receive amongst themselves?

How do you deal with that?

~~~
vocatus_gate
I basically post a warning that says "be aware that someone might attempt
this." There's unfortunately nothing I can do about it with the way Syncthing
is currently architected. We haven't had any malicious users yet but I'm sure
we will eventually.

Fallback is just to provide a static .exe on the mirrors and leave the Resilio
Sync node running alongside it.

------
Nux
I'm using Syncthing to distribute files in a web cluster of a few servers
instead of using shared storage - thus avoiding the spof.

Works quite well, with a few gotchas (ie make sure you don't sync cache files,
session files etc).

Good software!

------
pedrocr
Has anyone been able to sync large folders with syncthing? Last time I tried
when syncing 500GB you'd end up using >500MB of memory which made it unusable
for home NAS setups. Hopefully it has since improved?

~~~
noja
Are you running an official build?

This page indicates that your usage should be around 80 megs (with the gui
closed): [https://forum.syncthing.net/t/huge-ram-usage-2gb-on-
synology...](https://forum.syncthing.net/t/huge-ram-usage-2gb-on-synology-
ds1513/8845/9) and that an unofficial NAS-specific build had a problem with
high ram usage.

~~~
pedrocr
I always ran official builds, my NAS are just standard x86 servers. Last time
I checked was January 2015:

[https://github.com/syncthing/syncthing/issues/468#issuecomme...](https://github.com/syncthing/syncthing/issues/468#issuecomment-70808104)

Since that bug was closed with no solution I'm not too hopeful but maybe I
should try it again.

~~~
Nutomic
If you can reproduce the issue, be sure to post on the forum so we can figure
out the cause.

~~~
pedrocr
See the bug report at the time. It was closed without being fixed and it was
100% reproducible. Maybe it's fixed now but no one was too worried about it
then and I saw similar reports on the forums for a while before I lost
interest. I should either try syncthing again or finally test infinit
completely. This is the last piece of the puzzle I'm missing to have a fully
workable personal data management solution that allows me to bring up a
completely new laptop with my full config and data with just a clean Ubuntu
install and a few commands.

------
ohnoesjmr
If you look at the relays page[0], that shows number of users currently using
relays, which is 44k atm.

[0] [http://relays.syncthing.net/](http://relays.syncthing.net/)

------
aerique
This goes against one of the points of Syncthing (but I'm more interested in
its other features): are there any good 3rd-party hosts so I don't have the
hassle of maintaining on myself?

~~~
mouldysammich
It's paid but seedboxes sometimes support syncthing. Seedhost.eu i know for
example supports it.

------
problems
Interesting that the mobile usage seems to be so low from this, I sync my
desktop, NAS and phone using Syncthing constantly.

I also reject any sort of statistics though, so I'm probably not helping.

~~~
swiley
I've run both the mobile and desktop version on my phone and it just drains
the battery way too fast. I ended up just using an rsync cron job to sync my
phone over ssh while everything else uses syncthing (and yes, I know this
means I can't delete anything but I very rarely do anyway.)

~~~
problems
You can set it to sync only on wifi and/or only when charging, which has
always done the job for me. Depends on your use case, but things like password
DBs need to be update-able from both directions, which rsync doesn't really
do. On the other hand, I can rely on the fact that I don't need my home to be
sync'd up unless I'm actually at home, on wifi.

------
therealmarv
I loved the concept and hated the battery consumption on Android devices. Well
it works totally different than a normal cloud storage so there is nobody
really to blame but I cannot afford that much battery on my small 5" phone.
Also did not liked that dates are not correctly on Android (but again this is
not Syncthing fault). Why we cannot have date correction on external storage
(without root) like on desktop file systems on Android? :(

------
baldfat
I would love to use Synchthing but Bit Torrent Sync, now known as Resilio
Sync, works behind firewalls so much better. Hope to one day come back to
syncthing.

~~~
noja
Syncthing works fine behind and through firewalls. Which year did you last use
it?

~~~
baldfat
Last year. I am in different Public School networks and SyncThing still got
caught. I worked for hours to figure it out and just couldn't :( Did they beef
it up since last year?

~~~
noja
There are public relays which you can use (automatically unless you disable
the feature), so NATing and firewalls shouldn't be an issue.

~~~
pricechild
I think they're upset that the Network Administrator is blocking the system
intentionally, not that it didn't make it through in the first place.

Don't upset your Network Administrators!

~~~
baldfat
Well I am in about 10 different school districts and they all have different
policies. I just won't do anything that won't work as a default so I can claim
ignorance.

------
phaemon
This looks pretty cool, and I'll definitely look at it as a Dropbox
replacement.

On the negative side, your stats page doesn't have a link back to your main
page, so you won't even know how many folk had a look based on this HN post.
It's a good idea to ensure that _all_ your pages have a link back to your main
page.

~~~
canton7
Yeah that page which was linked to isn't really part of the website: it's a
separate site entirely, which is a frontend to the data-collection service.

People aren't really supposed to link to it directly as a means of introducing
new people to Syncthing...

~~~
phaemon
Indeed, but you never know what page someone might submit to a site like HN,
so it's usually a good idea to have a link back to your main site on _all_
pages.

For example, they could like to your Forum page (which doesn't have a link to
your main page). Or to your Github page (which has a link to the Forum, but
not to the main page). Or your Docs page (which...well, you get the idea).

Sites that have witty and inventive 404 pages sometimes get _those_
publicised. You really never know what someone will choose to submit.

------
interfixus
Nicely, nicely crafted software, and getting better by each bump. I have been
running it like forever between various Android devices, day to day PC, backup
server, and whichever relevant devices have been passing through the
household. Just works, can't recommend it enough. The GTK gui is a handy
extra.

------
ferongr
Hopefully some day an implementation in something other than Go will emerge.
The current Go implementation is mind-numbingly slow and uses huge amounts of
memory.

~~~
ycmbntrthrwaway
I think it is unlikely because major part of it is not protocol
implementation, but peer discovery and things like that.

Have a look at librevault [1] instead, it is written in C++ [2].

[1] [https://librevault.com/](https://librevault.com/)

[2]
[https://github.com/Librevault/librevault](https://github.com/Librevault/librevault)

------
ymhuang0808
Thanks the authors and contributors. I use Syncthing for sync my photos on
Android with my desktop. It works perfectly.

------
robinson-wall
Interesting drop in the numbers when version 0.12 was released. Does anyone
know what that was about?

~~~
Nutomic
Iirc there were some changes to usage reporting (additional data reported), so
users were prompted to enable usage reporting again.

------
j3097736
It's pretty impressive to see more arm64 than i386

------
sneak
I had to stop using syncthing on my MacBook and MacBook Air because it was
using far too much ram/cpu.

I wish it could be made more efficient somehow.

~~~
Nutomic
You could give sync thing minority a try:
[https://github.com/syncthing/syncthing-
inotify](https://github.com/syncthing/syncthing-inotify)

