
Unison File Synchronizer - _o-O-o_
https://www.cis.upenn.edu/~bcpierce/unison/
======
rgovostes
From the changelog:

> Further improvements to the OS X GUI (thanks to Alan Schmitt and Craig
> Federighi).

Here are Craig's commits:

\-
[https://github.com/bcpierce00/unison/commit/48f8e1b27edbe2df...](https://github.com/bcpierce00/unison/commit/48f8e1b27edbe2df0a6da272b6694f0d522c710b)

\-
[https://github.com/bcpierce00/unison/commit/6645d1793ce843f6...](https://github.com/bcpierce00/unison/commit/6645d1793ce843f6fe163c7ad68dc1932ad89500)

Around the same time, Dave Abrahams, now of Apple's Swift team, makes an
appearance on the unison-hackers mailing list.

~~~
rcarmo
Wow. I've been using Unison for ages and had no idea Craig had had a hand in
it.

------
jacobu9
I've been working on a spiritual successor to Unison for a few years called
Mutagen [1]. It aims to provide more flexible synchronization and tighter
integration with filesystem watching, enabling ~real-time remote editing with
your local editor of choice. It also adds SSH-style network forwarding, so you
can forward network traffic to/from remote systems to access remote
applications without exposing ports. It currently supports SSH and Docker
containers, but additional transports are coming.

I'd be really happy to receive feedback if you have a chance to try it out!

[1] [https://github.com/mutagen-io/mutagen](https://github.com/mutagen-
io/mutagen)

~~~
sooheon
Haven't dug into it yet, but quick Q: does this allow the use case of using a
local IDE to interactively develop/run code against a docker environment
running in a remote server? (so two "jumps")

~~~
jacobu9
That's actually one of the primary use cases at the moment. The way Mutagen
works with Docker is by using `docker cp` to inject one of its "agent"
binaries into the container and then communicating with that binary over
`docker exec`.

You can also orchestrate your setup a bit (e.g. to work with Docker Compose)
using Mutagen's new orchestration infrastructure. You can find an example of
this at the bottom of the Mutagen homepage:
[https://mutagen.io](https://mutagen.io)

------
metalliqaz
Is this project still going? I used to use it a lot for duplicating my project
directories between my lap PC and my home PC. It was so powerful and so useful
at the time. Honestly I thought it had died.

I use Syncthing now. It's really useful.

~~~
comex
I recently switched from Syncthing to Unison, after getting tired of Syncthing
either taking forever to sync anything or getting stuck altogether, with very
little visibility or control over what it was doing. So far Unison (with
fsmonitor) has ‘just worked’, with files updating immediately on save as I’d
expect.

(That said, one of the problems I encountered with Syncthing seemed to be that
it wouldn’t properly reconnect when I switched networks, which is necessary
since it runs as a persistent daemon. With Unison, so far I’ve been starting
it up in the foreground in a terminal as needed, so it hasn’t had to provide
the same functionality. But I expect I’ll eventually set up some type of auto-
reconnect, which will hopefully work better than Syncthing did.)

~~~
StavrosK
Huh, I've had the opposite experience with Syncthing. It's basically just
worked, it's been bulletproof. I'm not sure why the disparity.

~~~
jwr
Similarly here. Very happy with syncthing.

------
avhception
I can't praise Unison enough. I've been using it every day for over 10 years
and is easily one of the most useful pieces of software I've ever come across.
I've donated multiple times over the years.

~~~
rwmj
I'm a maintainer of the Unison package in Fedora and I'd love to know which
version(s) of Unison you use day to day.

In Fedora (and I think this applies in Debian too) we have to maintain 3
versions because Unison isn't interoperable across minor releases. For this
reason we package 2.13, 2.27 and 2.40, and I think there is discussion about
packaging the latest release too. Keeping these ancient (esp 2.13) versions
going is a pain to say the least.

~~~
LeoPanthera
This incompatibility is one of the reasons why I reduced my dependency on it,
though I still use it for minor things.

The last straw came when it turns out that unison was incompatible with the
_same_ version, if that version had been built on a different system. I can't
remember the details but there was some library version difference that
results in a unison that has the same version number but wouldn't talk to one
from another system without crashing.

Drove me crazy.

~~~
mjn
That can happen because unison uses OCaml's built in serialization, which
isn't guaranteed to be stable across OCaml versions. It doesn't often change,
but the representation of arrays was changed in OCaml 4.02, which meant unison
compiled with 4.01 couldn't sync with unison compiled with 4.02. Debian bug
discussing the issue:
[https://bugs.debian.org/802919](https://bugs.debian.org/802919)

~~~
rstuart4133
> which isn't guaranteed to be stable across OCaml versions

Oh, so OCaml is a toy language from academics. I've read so many good things
about it over the years, but no one mentioned that aspect of it. There should
be a law about putting a warning in Big Red Letters on the box so someone
doesn't make the mistake of using it in a large, long lived project.

~~~
ernst_klim
> Oh, so OCaml is a toy language from academics.

What? Why did you conclude so?

> but no one mentioned that aspect of it.

Really? Did you open the documentation? It's stated explicitly.

[https://caml.inria.fr/pub/docs/manual-
ocaml/libref/Marshal.h...](https://caml.inria.fr/pub/docs/manual-
ocaml/libref/Marshal.html)

> There should be a law about putting a warning in Big Red Letters on the box
> so someone doesn't make the mistake of using it in a large, long lived
> project.

It's documented, and it's not intended to be used for serialization, you use
it in very particular cases, for example for faster object sharing with IPC
for multicore. We use Json or S-expressions or other stuff for that purposes.

I really don't know why unison people don't switch to a proper serialization.

------
yawaramin
A few years ago, I was working in a company that used a shared drive on a
local Windows network. Several analysts and people from other departments
would use the drive to store shared price lists, tables, and other
documentation. Part of my job was to keep these data files updated with info
on a daily basis.

The problem with trying to edit these files was that everyone logged in on the
network had access to them and could change them arbitrarily. Or they could
even just inadvertently lock the files if they left them open on their PCs.
There was no control or change management.

I decided to keep the 'canonical' versions of the files on my PC at work and
use Unison to sync them over to the shared network versions of the files once
a day. Unison would instantly tell me if anyone other than me had changed the
files, and I could investigate further. It was a huge relief knowing that I
had proper control over those files.

------
moviuro
Unison's cool, but I hit some major roadblocks with it, namely: it can't run
on an heterogeneous network [0,1]. As many others have already suggested,
syncthing[2] is the new best tool out there for multi-directional sync.

[0]
[https://wiki.archlinux.org/index.php/Unison#Version_incompat...](https://wiki.archlinux.org/index.php/Unison#Version_incompatibility)
, [https://groups.yahoo.com/neo/groups/unison-
users/conversatio...](https://groups.yahoo.com/neo/groups/unison-
users/conversations/topics/11439)

[1] [https://try.popho.be/byeunison.html](https://try.popho.be/byeunison.html)

[2] [https://syncthing.net](https://syncthing.net)

------
eridius
Unison has been an indispensable tool for me for many years. I keep my desktop
and laptop in sync by using an always-on Linode as a sync server.

Something I love about Unison is the fact that I can sync files from anywhere
on my filesystem without having to move them or replace them with symlinks.

I can also sync subsets of my files by defining multiple profiles. I have
Unison set up to default to a "common" profile that includes nearly
everything, but I can explicitly request a sub-profile if I want. I also have
a separate profile that syncs the profiles themselves, so I can make profile
changes on one machine and run `unison profiles` to upload them.

------
sdfjkl
Unison efficiently and reliably syncs my website (static blog mostly) between
OS X and FreeBSD. It's one of those extremely underrated pieces of software
that just quietly does an excellent job, yet most people never heard of it and
continue to struggle with rsync or similar (syncthing fulfills a somewhat
different role). Perhaps because it's written in an oddball language, or
because the website isn't flashy enough.

I often evaluate a wide range of software before choosing what I consider best
for a job, and many years ago, Unison came out way ahead in such an
evaluation. It never failed me.

------
eridius
The Development Status section of the manual mentions a follow-on project
called Boomerang¹. It looks like the GitHub repo² only had activity for about
2 years, then stopped. Is that a dead project?

¹[https://www.seas.upenn.edu/~harmony/](https://www.seas.upenn.edu/~harmony/)

²[https://github.com/boomerang-lang/boomerang](https://github.com/boomerang-
lang/boomerang)

------
ufo
I use Unison every day to synchronize my work computer and my home computer,
using an USB thumb drive as the intermediary. The thing I like the most about
this arrangement is that I don't need a direct SSH between home and work, that
there is no cloud service involved, and that the GUI lets me double check what
files are going to be written to/from the usb drive before it synchronizes.

There is one issue that I have run into though. If I format the USB as fat32
it can't properly store permissions. And if I format it as ext4 then I need to
make sure that my numeric user ID is exactly the same in both my home and work
computers (otherwise the permissions get messed up).

Is there a way to make my arrangement more robust so that I can sync files
through and USB drive, without needing to use the same numeric user id in all
my computers?

~~~
Fnoord
> Is there a way to make my arrangement more robust so that I can sync files
> through and USB drive, without needing to use the same numeric user id in
> all my computers?

$ chown -Rf ufo.users /media/usbdrive && unison

~~~
ufo
Would I need `sudo` for this chown command?

~~~
Fnoord
Yes, you very likely need to be root. You could also perhaps add in fstab that
you mount the USB drive as a specific uid. See man mount and man fstab and
search for " uid" (with the space, else you search for uuid as well).

------
shoo
I used to use unison to periodically sync files between a host filesystem and
a VM filesystem as it was a lot faster - in terms of filesystem performance
when not syncing - than using the shared folder functionality provided by the
VM.

------
rcarmo
I've been using Unison since... forever.

A hint regarding running the latest Mac binary on Mojave: if the GUI crashes
on you, it's likely to be an issue regarding syncing file _permissions_ and
not just files.

This has been biting me for a couple of months now whenever I sync my
development tree between Dropbox and OneDrive, since Windows' WSL tends to set
wonky 0x777 permissions on entire file trees, and it is usually those that
cause Unison to crash.

Otherwise, I've been using it to sync 400K+ files without incident.

------
logandk
Unison is great for keeping multiple machines in sync. I eventually switched
to Dropbox for reasons I don't recall, but it might be a good time to
reconsider. Also, I'd forgotten, but apparently I even made a (poor) attempt
at writing a Unison UI of OS X, some 11 years ago...
[https://github.com/logandk/autoson](https://github.com/logandk/autoson)

------
the-peter
I used and loved Unison for years, but I found myself wanting something that
just worked invisibly with no admin effort so I switched to Resilio
(Bittorrent Sync). Both work for my purposes. Anyone else try both of these
and have feedback?

------
sansnomme
How difficult would it be to build a Dropbox clone on top of this?

~~~
JoachimSchipper
Pretty hard. Unison is nice, but...

\- Unison is written in OCaml, which is (probably) a perfectly fine language
but not commonly used

\- Unison synchronizes files to files, but for a Dropbox-like system you
really want deduplication for space savings (i.e. server-side storage is a
bunch of pointers to content-addressed blocks.)

\- in general, it's not clear to me that the client is really the hard part of
Dropbox. Note e.g. the part where Dropbox now runs its own data centers for
cost reasons.

\- there's a million fiddly things to get right, and Unison hasn't had _that_
much usage

~~~
willtim
> there's a million fiddly things to get right, and Unison hasn't had that
> much usage

Unison is the _only_ bidirectional sync tool that I trust to get the details
right. It is backed by a formal model with various proofs of correctness. Such
models are also easily representated in OCaml; which I can assure you is more
than a fine language, especially if one cares about correctness. Dropbox has
struggled to get these details correct in the past (see the paper "Mysteries
of DropBox", by Unison's author Professor Benjamin Pierce). TLDR, Pierce
teaches these Python hackers how to fix their broken code.

~~~
coldtea
> _Unison is the only bidirectional sync tool that I trust to get the details
> right. It is backed by a formal model with various proofs of correctness._

That's just about the sync process/stages, the easy part that can actually be
formalized.

The "million fiddly things" are about OS and filesystem issues,
incompatibilities, and so on, and Dropbox has a hugely larger test base for
those things...

~~~
willtim
I disagree. POSIX, although somewhat dated, has provided a good enough
abstraction layer for filesystems and OSs. My proof of this is the number of
different and successful filesystems for Unix/Linux. If the abstraction didn't
work, everyone would be forced to use the same filesystem.

The issues and subtleties are with bidirectional sync. It is not "the easy
part". Dropbox didn't get it right in the past, we have no proof they have it
right now.

~~~
coldtea
> _POSIX, although somewhat dated, has provided a good enough abstraction
> layer for filesystems and OSs._

First of all, POSIX semantics are not what Windows support. Second, even where
available, POSIX is a tiny part of the possible issues. Adequate for naive
apps that need to open or write some files, not for a reliable sync tool.

[https://danluu.com/deconstruct-files/](https://danluu.com/deconstruct-files/)

~~~
willtim
The issues discussed in that link, such as concurrency, lack of atomicity and
failure are all factors that make bi-directional sync a hard problem. Yes
POSIX provides few guarantees and so that burden rests on the application code
above. The formal model must consider the race conditions and failure modes,
it is not "the easy part". I do acknowledge there is a modelling gap to
overcome and some problems with POSIX, but I disagree that its the hardest
part of the problem.

------
patsplat
What a fantastic tool!

A long time ago used unison to keep file uploads on 2x primary web servers[1]
in sync. It worked like a charm. With IIS on Windows!

------
BenGosub
How does Unision compare to Rsync, is one a part of the other or both are
different tools for syncing files?

------
conqrr
Daily user since beginning of this year. Help keeps my laptop in sync with my
remote dev environment.

------
throw0101a
Not to be confused with Panic's Usenet/NNTP client:

* [https://en.wikipedia.org/wiki/Unison_(Usenet_client)](https://en.wikipedia.org/wiki/Unison_\(Usenet_client\))

~~~
trevyn
Nor with the Coda replicating filesystem, which is not to be confused with
Panic’s code editor of the same name.

~~~
extempore
Nor with a certain super cool language of the future
[https://github.com/unisonweb/unison](https://github.com/unisonweb/unison)

------
pgen
Unison does not support hard links!

