
Rsync, GUIs, power, control, design, and decisions - luu
https://bits.ashleyblewer.com/blog/2019/06/29/rsync-guis-power-control-design-and-decisions/
======
theamk
> Is it better to make up common rsync “recipes” for people to use, or is it
> better to let folks have access to ALL of the flags [...] and pick’n’mix
> what they want?

I think both. This is the hard part about GUI design -- it is pretty easy to
make a list of 100 command-line options listed alphabetically on the man page,
but for GUIs, people expect much more.

There should be some sort of logical grouping. One area of the screen would be
options related to what is transferred, a second area is metadata, the third
one is speed-only optimizations, fourth is logging and so on. Having
"hardlinks" all the way on the left and "symlinks" all the way on the right
does not make a good GUI.

There is a very wide variety of input elements, so checkboxes are are not the
right solution all the time! For example, while having separate "\--verbose"
and "\--quiet" options is fine for CLI, one would expect to see a drop-down or
a slider in the GUI. This will decrease cognitive load because there are less
things to read and there is no need to worry what happens if you check both.

Another great GUI feature is dynamicity -- take the metadata for example. A
good design might have a dropdown: "everything", "nothing", "x-bit only",
"custom". Most of the time, first three options would be fine, but "custom"
will show entire new group of checkboxes.

... this was just my subjective design, based on how I use the program. If I
were trying to do a popular product, I'd have to spend some time asking people
/ looking at the forums to determine which sets of features do people use. For
all that I know, there is a super common case for "\--group" without
"\--owner", and this is just me not knowing about it.

~~~
hzhou321
I would think there maybe a "command-line" version of GUI design: supply
limited default buttons for common options, but allow user customization. For
example, I would love to have the button from author's design but tune it
specifically to my need including running a specific set of "command lines".

~~~
e12e
Note that even rsync itself has some limited support for "profiles", like
--archive.

~~~
sirn
rsync do actually support profile via option aliasing. rsync uses popt(3) to
parse its command line options and popt allows you to define aliases, so you
can put something like this in ~/.popt:

    
    
        rsync alias --sync -rpcLDvz --chmod=D0755,F0644
    

A call "rsync --sync foo@bar:baz/ baz" will expand the command into "rsync
-rpcLDvz --chmod=D0755,F0644 ...".

The "program alias newopt opt" syntax is actually popt's thing (see Option
Aliasing[1]) and it works with everything that uses popt(3) for its command
line parsing.

[1]: [https://linux.die.net/man/3/popt](https://linux.die.net/man/3/popt)

------
lordleft
'People complain about Electron a lot, but the primary complaint just seems to
be that its bloated – carrying an entire browser engine along with something
small. But like, all of our computers are already bloated, and there’s at
least a 40% chance that right now I am bloated, and it doesn’t mean you should
love me any less, and I think the above points I’ve raised override the lack
of svelt-ness of the program. People don’t care that an application is a
streamlined 3MB or a big round 200MB boy – they care that it works for them
and it makes them feel good about using it.'

I think this is true in some cases, but the sorts of folk who run rsync may be
more interested in performant hardware that can run on older machines than
average. Then again, they'd probably opt for sticking to the command-line
tool. Still a very cool project!

~~~
mbrumlow
I avoid electron apps because it is almost anti web.

I would use apps built more like electron if they simply removed the electron
part and replaced the app with a name for a namespace and simply shipped tht
js/html/images along with a api service.

Installing would be binding tht app to tht name space and using the browser of
choose and navigating to namespace.localhost. In this case rsync.localhost.

The system would be a bit more complex for handling auth and a few things but
let's not get bogged down in solvable details.

Then I could access my apps from my phone. Another system (if I chose to
expose them publicly).

~~~
mattnewton
This is more or less how Jupyter notebooks work, and I agree this experience
is pretty great. Make it a local web server and the browser becomes your UI.

~~~
mbrumlow
Like maybe we can build this out. A common daemon for registering with.

The concept would be very simple to build. I might start a github project.

\- Global daemon to register apps with. Apps could be installed system wide or
per use.

Think plopping a zip/tar/ or directory structure in something like
/usr/local/webap.d/(namespace) -- might be abetter directory. Then a manifest
file that describes some things about the service, such as how to start the
API service, and the default entry point (index.html).

\- The global service is responsible for bringing apps on line by way of
instance activation on say visiting a name space, or just keep them running.

\- Services once launched pick random port from range to bind to and system
service acts as a reverse proxy and common auth for validating the local user.
Maybe even ensure each of the API services are run per user and jailed to the
home directory

IDK, just thinking off the top of my head, I have written bits and pieces of
this for various work projects, but never glued the idea into one single
system.

~~~
spirodonfl
It's not that hard, actually, I've done this with Node. Since you can install
Node anywhere, include a binary, or whatever, you can have Node run as a
server and it can run with elevated privileges, if needed.

You have it scan a directory structure and auto generate a path that you can
use in a browser, any browser, to view your newly install app.

You can do this with APIs too so you can have internal tools with APIs via a
Node server that do all kinds of things.

Some simple port forwarding rules (ymmv) make this available to your LAN or to
the WWW. Add a DNS layer on top and you have something really cool.

I do this for all kinds of stuff, mostly PWAs, but it lets me have my own
notes, todo, systems management and more.

~~~
johannes1234321
It is easy, till you get into the trouble of local Firewalls, Double use of
port numbers and all those details, where the daemon runs unexpectedly or
stops without good information to the user etc

~~~
mbrumlow
In my minds eye we would give one dedicated port to the main service. The apps
would pick a random unused port, and report back to the main service to which
would then become a reverse proxy from the main services port to the bound
port.

So if the rsync app bound to 5392, it would report back to the main service
so. And it would ensure all (or some subset of request) to rsync.localhost
were reverse proxied back to localhost:5392.

For the most part when I say some request I mean most if not all, but it might
be nice to have a system level rsync.localhost/.info uri that would give
system level information about the app.

~~~
yjftsjthsd-h
If we're on the local system anyways, why not just use an actual socket in the
file system?

~~~
johannes1234321
Since requirement was "works with any browser"

~~~
yjftsjthsd-h
Yeah, but we're just communicating with the local proxy, right? So it'd be
browser > HTTP over TCP > proxy > socket > application.

------
pixelbath
> People don’t care that an application is a streamlined 3MB or a big round
> 200MB boy

At the risk of beating this topic to death here on HN...Some people care; I
know I do. I think the audience for rsync would care more than a nontechnical
user, for sure. If I downloaded a program with an extremely basic UI that
essentially is a string builder, I probably wouldn't bother if the final
executable was over, say, 10MB. My preference would be to hack together a
basic CLI menu that could do the same thing and I can stick it in a folder
that contains a bunch of other scripts.

Am I the target audience, or even close? Probably not, but I don't know many
people who want what rsync offers who aren't programmers.

From the linked ServerFault page asking why Windows users don't use rsync:

> I would say that rsync is just too freaking complicated. Anyone I know who
> uses it regularly has a pre-set group of flags that generally does what they
> want.

This is followed by more reasons Windows users can't handle or don't want a
CLI, along with a few mentions of Robocopy on Windows. Which, honestly, is the
reason I don't use rsync on Windows. Windows (both Desktop and Server) ships
with Robocopy, not rsync. To run rsync, I'm (or rather, was until WSL) forced
to use a Cygwin-like environment to run it, which is overkill unless you're
already using an environment like that (i.e. not on a Windows Server instance,
most likely).

------
teknopaul
If ever there was a tool that didnt need a gui its rsync. Idea is to keep two
folder structures in sync. Exactly the sort of thing you want to script, or
trigger when one side changes, or cron.

I've use rsync all my computing life but i've rarely wanted to have to
manually push a button to make it work.

A good cli interface is clearly the correct interface to rsync.

Embeding rsync in an app that does something else makes sense, eg edit locally
auto push changes, but embeding rsync in an app that does nothing but rsync
without any automation or integration options makes no sense to me.

You have to learn a new interface that you cant automate. Its dead end
interface. The whole point of cli interfaces is that they are infinitly
hackable and pluggable.

~~~
oehpr
I think the foundation of this comment is incorrect. This helps you build up
the rsync command and test it. It gives you the rsync command that it runs.
You're supposed to take the command it generates and use another tool to
automate it. You're not supposed to automate the interface. It fulfills the
conditions you expect of it.

And personally, I loath constructing rsync commands. Every 2 months I have to
parse that 2300 line rsync man page. And every 2 months google sees another
search query "rsync GUI" Because I am sick of building these things.

~~~
verall
I appreciate this comment, because at first I didn't understand why I would
ever use this program, and now I want to use it. Rereading 'man rsync' for the
10th time is really is annoying.

But it just will never be worth a 200mb download. Gosh. It's like bloat on a
whole 'nother level.

~~~
aeorgnoieang
Interestingly, the useful parts (for you, and likely lots of others) are those
that would be readily adapted to a regular web app/site.

------
klodolph
> Successful UNIX tools (or any tools, for the most part) are the ones with a
> simple concept manifested very thoroughly, and rsync is certainly that.
> [...] There are over 100 flags in rsync that you can select to do different
> things. So, this is what I mean by doing something simple, and doing it very
> well and very thoroughly.

Rsync is certainly not simple. It certainly doesn't do one thing. It's
"extraordinarily versatile" to the point of madness. Here's what it does:

\- Copies files from A to B

\- Mirrors directories

\- Computes diffs

\- Applies patches

\- Evaluates filters

\- Runs as a daemon

\- Runs over various remote shells

\- Applies filters to filenames

And then it doesn't give you a good progress bar or let you resume interrupted
operations, or gather errors at the end. I think this is a good case where the
tool may have grown too large and unwieldy, and it may be better to replace it
with an API or a collection of small, UNIX-like tools. For example, if I just
want to copy files from point A to B, I can do this:

    
    
        tar c . | ssh host tar xv
    

If I want a progress bar, I can use pv,

    
    
        tar c . | pv | ssh host tar xv
    

Usually I find this easier to use than rsync.

~~~
gchamonlive
What is simple in rsync is the idea of synchronising stuff from A to B. That
is all it does in all the cases you mentioned.

It is hard to emulate what rsync does with archive mode enabled, which speeds
up transfers, so tar isn't exactly a replacement for rsync.

There are better ways to show progress, for example using the flag
`--info=progress2`. It doesn't behave well in automated scripts though

~~~
darkpuma
fyi, combining something like _`--out-format= "file: %f %l"`_ with
_`--progress`_ results in reasonably parse-able progress updates.

Keep in mind however that the format of --progress has changed in the past.
Depending on your version of rsync it may or may not insert thousand
seperating commas into the number of bytes written, e.g. 2,384,908 vs 2384908.
The former can be supressed with --no-h, an option not available on at least
some older versions of rsync that default to no commas.

------
nixpulvis
Nothing beats a good CLI with a proper man page in my book. I like having all
the power at my fingertips, and a good way to search for what I want to do
without using the internet.

~~~
jimmaswell
I always end up having to google how to do stuff with CLI tools but GUIs let
you discover how to do things intuitively. Tried using man pages but they're
always slower to find stuff in and a stackoverflow answer is more likely to
target the use case I'm going for.

~~~
F-0X
> Tried using man pages but they're always slower to find stuff in

Man pages are presented through less, so you can begin a (regex) search with /
and hit n to go to the next result or N to go back.

If you knew this, I am actually very surprised you don't find information
near-instantly.

~~~
jimmaswell
I'm aware, but this involves guessing the right word/phrase and sifting
through multiple matches, and there often aren't quick examples given.

------
rsync
Very interested in this GUI for rsync that you wrote (for obvious reasons) ...
one thing that jumps out at me right away is that none of your examples show a
remote in the form of user@host:path ... I also did a text search of your page
for "ssh" and it is not mentioned ...

Does your GUI have the ability to specify a remote in the form of
user@host:path ?

This gets a bit complicated - not only do you need to allow password
interaction with ssh but further, upon initial connection, you'll be presented
with a host key dialog and need to confirm the key fingerprint, etc.

------
zonidjan
> our computers are already bloated, therefore it's okay to make them more
> bloated

No. Don't do this. The only reason it's not already a problem is that most
apps still _aren 't_ Electron.

------
gchamonlive
If anybody is interested, I have written a simple python wrapper for rsync
called sysrsync that abstracts the idiosyncrasies of using trailing slash.

It executes or compiles the rsync command if you want to proof it before
executing

You can install it with

    
    
      pip install sysrsync
    

[https://pypi.org/project/sysrsync/](https://pypi.org/project/sysrsync/)

~~~
ape4
Yes, the trailing slash is key

------
fitzroy
I really wish there were more GUI wrappers that literally wrote the CLI
commands out on screen as the GUI was modified, to help with bridging the
GUI/CLI gap for people.

Also, I miss arRsync.

~~~
F-0X
The problem with this is that it misses the point of most cli programs, which
is that they are to be composed with other programs.

Besides, when colleagues of mine decided to demystify the command line and
learn it proper, they're always surprised how much easier it is than they
expected.

The GUI/CLI gap, as far as I have seen, is one of willingness to invest some
time to learn.

~~~
erikpukinskis
> The problem with this is that it misses the point of most cli programs,
> which is that they are to be composed with other programs.

That's not true, CLI programs would still exist even if they could not be
composed. And I would guess that if you counted how many of our commands
include 1, 2, or 3 executables, the 1 column would be the biggest by far.

One of the major headaches of CLI programs is that they're hard to learn
interactively because there's no starting point. You have to read the docs.
The point of a GUI command builder is that it gives you something interactive
to play with to start to learn how the CLI commands work. The point is not to
do everything through the builder, the point is to bootstrap people to the
point where they no longer need it.

Also, there's no reason a GUI command builder couldn't also show you how to do
composition.

------
noonespecial
There is a gui for rsync that's quite popular. Its called Dropbox.

A lot of successful projects seem to be of the form 1) Consider old-school
unix tool. 2) Aim for one aspect of its functionality that serves a mainstream
need. 3) Put friendly face on it so less technical users can reap the
benefits.

------
tombrossman
I recommend also trying Grsync (released in 2005), which is mostly the same
thing and is available with multiple languages support.
[http://www.opbyte.it/grsync/](http://www.opbyte.it/grsync/)

------
writepub
> People don’t care that an application is a streamlined 3MB or a big round
> 200MB boy

Assuming all else to be equal (UX, functionality, etc), people will absolutely
gravitate towards the [5MB storage, 20MB RAM, 0.5% CPU] combination to
electron's [300MB storage, 200MB RAM, 5% CPU], especially when they are
running multiple such programs.

But, what electron offers is getting from zero to product across all desktop
OS-es in arguably the least time possible. So, the real choice is between
[PRODUCT_NO_SHIP] to [300MB storage, 200MB RAM, 5% CPU], and we all know what
the world chose.

~~~
klingonopera
This is actually an issue in app development? I thought with SDL, QT,
wxWidgets, et al., writing native apps was basically a covered topic...?

~~~
writepub
It is a problem. Here's [1] a talk from fossdem on the various issues, and a
couple solutions.

[1]: [https://youtu.be/ANcU1g7ZWdU](https://youtu.be/ANcU1g7ZWdU)

~~~
klingonopera
It seems to me, that the issue here is mostly web developers who are used to
coding for browsers, are slowly moving into the desktop application domain,
and instead of them applying the traditional languages used there (C and C++),
are circumventing these by creating frameworks that allow them to continue
using their web langs in a desktop environment.

If that is the case... they'll never get rid of the megabyte bloats. That's
basically just baseline overhead from a browser engine itself.

Crazy idea: Reverse the entire process? Use C for everything, including web
tasks, i.e. FastCGI? Correct me if I'm wrong, but with WebAssembly around, is
it possible that C programming might make a comeback into web development?

~~~
writepub
I think the central problem with native UI is how much better looking web-
based UIs are, and the talk I cited spends time making this point. HTML/CSS/Js
built UIs have much better, modern, reactive styling options & when done
properly, look much better than, say GTK.

I'm personally a huge proponent of C and would love to have access to WebApis
[1] via Webassembly. That would still necessitate a runtime capable of
interpreting HTML/Css into views, but projects like webview [2] have
demonstrated that this access needn't come at a huge cost. For instance, in
[3], for a UI in HTML/CSS with business logic in C, system resources clock in
at [RAM: 6MB, CPU: 1.3% Storage: 10MB], which is completely reasonable.

[1]: [https://github.com/WebAssembly/webidl-
bindings/blob/master/p...](https://github.com/WebAssembly/webidl-
bindings/blob/master/proposals/webidl-bindings/Explainer.md)

[2]: [https://github.com/zserge/webview](https://github.com/zserge/webview)

[3]:
[https://fosdem.org/2019/schedule/event/godesktopapps/attachm...](https://fosdem.org/2019/schedule/event/godesktopapps/attachments/slides/2994/export/events/attachments/godesktopapps/slides/2994/slides.pdf#page=19)

~~~
klingonopera
Wow, thanks, those resources will hopefully keep me busy later... I'm building
a cross-platform C engine, and had the usual suspects of "Windows, Linux,
maybe Mac", but then recently also "discovered" this ancient FastCGI
technology, and just day or two ago skimmed the WebAssembly wiki, and I'd like
to explore possibilities of implementing my engine in web applications, too,
afterwards.

But if I understand WebView correctly, wouldn't this be HTML in C/C++, i.e. is
WebView... "just" a browser? Or... is it "just" spawning a minimal OS-Window
on the target system with a system-built-in browser engine? For my intents and
purposes, that's kinda the reverse of what I'm aiming for with my C project,
which rather would be to bring sprites and raw data formats to your browser
window.

In essence, what I meant with "reversing everything with C" is basically
programming webpages like C programs. Taken to the extreme, what I mean is for
example compiling Windows to run in a browser-context instead of a physical
x86-PC.

------
pmlnr
electron gui for rsync when all one needs to do is read a very good man page
for rsync. Sigh.

Yes, yes, it's not "intuitive". But maybe, just maybe, learning and/or
teaching is just as important, and remembering "rsync -avuz [from] [to]" is
really not that hard.

------
kstenerud
There are reasons why it will be difficult to write a GUI for rsync:

1\. The 80% use case for rsync is a carefully crafted pre-set script that runs
in a crontab, not as a one-off thing that you select options for and hit
"run".

2\. There are so many switches and options that you're not going to be able to
model them all in a coherent manner without overwhelming the user.

3\. The price of failure is data loss, which means that people will either be
afraid to use it, or foolishly lose data and then be afraid to use it. A dry-
run mode helps, but is not a panacea.

What you need is a multipart system:

1\. A set of vetted recipes with good, succinct descriptions of what they do,
why they're useful, and where the cliffs are, and a program to apply them for
the specific user.

2\. A design GUI that helps guide you through the process of building your own
recipes.

3\. A runner for running your recipes from the command line, so you can
crontab it (if the recipe system doesn't generate self-contained executables -
scripts or whatever - that you can just call directly).

4\. A runner for running or testing your recipes in a GUI environment, for the
5% of times you actually need this.

Trying to mix more than one of these parts into a single program is a UX
mistake.

------
carapace
Great article.

Check out Unison.

[https://www.cis.upenn.edu/~bcpierce/unison/](https://www.cis.upenn.edu/~bcpierce/unison/)

> Unison is a file-synchronization tool for OSX, Unix, and Windows. It allows
> two replicas of a collection of files and directories to be stored on
> different hosts (or different disks on the same host), modified separately,
> and then brought up to date by propagating the changes in each replica to
> the other.

> Unison shares a number of features with tools such as configuration
> management packages (CVS, PRCS, Subversion, BitKeeper, etc.), distributed
> filesystems (Coda, etc.), uni-directional mirroring utilities (rsync, etc.),
> and other synchronizers (Intellisync, Reconcile, etc). However, there are
> several points where it differs: ...

~~~
aidenn0
Unison is terrible in that you must have exactly the same version (down to
micro version) of unison on both sides. That makes it very hard to use in
practice.

~~~
StefanKarpinski
While that is true, it was so frigging handy that it was well worth setting
this up to get it to work.

------
ehonda
Thanks for making one, I only recently learned about rsync and was wondering
if there was a handy gui for it :), then it magically appears before me on HN.

I'm weary of this "electron" thing though. Do I have to download a 500mb
package to run this?

------
sysrpl
I wrote my own rsync gui utility recently as well. Here is a comparison:

[https://www.getlazarus.org/apps/backer/](https://www.getlazarus.org/apps/backer/)

------
skybrian
It seems like to write a tool that's genuinely helpful for non-experts, you
need to become something of an expert in how people use the tool: what are
they usually trying to accomplish? What are common mistakes?

It's similar knowledge to what you would need to write a book about rsync.

And the way to do that is with lots of experiments and user testing. A good
start would be to find one person who really wants this and make sure it works
for them. Designing things by yourself, even with lots of sympathy for the
user, doesn't seem like enough?

------
andrewshadura
Tcl/Tk is much a better choice of a toolkit for this job.

------
peterwwillis
I agree with the other users who have a pre-set of options that work for them.
I have used _-avzP_ for 99% of my Rsync uses over a decade and a half
(exceptions being adding _-e ssh_ or refactoring for backup solutions). By
putting all the options in an 'options menu', keeping the defaults sane, and
check boxes off the screen, you end up with a much simpler and easier to use
tool.

------
optimuspaul
I couldn't get over "add source" and "add destination", set would be a better
verb... or just have no verb

------
mikorym
Slightly off topic, but is it possible to use Rsync with Dropbox (or Box, G
Drive, OneDrive, pCloud, Mega, etc.)?

------
enriquto
Wait, this is a parody, right?

~~~
_zachs
I'm not sure. Don't want to be mean, and I'm sure building this was a great
learning experience, but it seems more like and excuse to practice Electron
than make rsync more usable.

