
Ask HN: How do you manage large files across multiple laptops/desktops? - kaiku
My wife is a graphic designer and works on both a Mac desktop and laptop. The assets she generates for a single project can reach into gigabyte territory – multiple versions of a set of print layouts, RAW photos, etc. One problem she has is how to reliably sync large files between computers. She uses Dropbox currently, but sync over network can takes a long time or sometimes just doesn&#x27;t finish depending on the file. And when she runs into space limits on one computer, she&#x27;ll need to adjust Dropbox settings to selectively sync items.<p>I&#x27;m curious what people do in this case. I think she needs:<p>1) A large amount of storage, expandable as time goes on
2) Files accessible from multiple computers
3) Ability to access files securely when working from a coffee shop, traveling, etc.
======
kop316
Have you looked into Nextcloud?

If you primarily do it at home, the speeds of Nextxloud are typically limited
by the network bandwidth, and the storage is limited by what server you have.
I use it and manage 5 TB of data on it without problems.

~~~
atoav
I am using Nextcloud for precisely thw usecase OP outlined and I am quite
pleased with it I must say.

~~~
mekster
NextCloud is decent but not technically polished enough. Web UI is not
performant, not too user friendly and mobile apps aren't really that good. I
do have 4 instances running but I'll welcome alternatives if I find one.

~~~
kop316
Heh, I would say the Web UI works extremely well and is very friendly, and I
really like the Android App. What issues do you have with it?

~~~
mekster
Here are my gripes.

\- Pages load pretty slow over mobile networks. Every app page requires like 5
seconds to load, as it loads the page in 2 steps, first the page and then then
the content. Not talking about navigating folders in Files but apps like
Calendar and News are especially slow.

\- Bookmarks app silently stopped working few weeks ago when I thought I kept
adding Bookmarks through Android mobile app, so I lost 2 weeks worth of them.
Not sure if the server or the client app went wrong but I switched to Pocket
even though I run everything self hosted where I can.

\- Contacts app previously had a problem where you couldn't save more than one
information per page reload until recently when used through CloudFlare which
was very confusing to see some saved data disappear.

\- News app's UI is a bit of a joke in terms of speed and usability, so you
need a mobile app to consume it than from the web.

\- Frankly all of the apps need mobile apps because pages load slow and it's
reluctant to go through them from the web but the mobile app quality varies so
much as they're all dependant on third party developers.

\- NextCloud talk, I had a high expectation of it but no matter how I tried,
it never worked reliably and client apps were nowhere near the quality to
recommend to non tech work people.

\- Using an external storage for Files makes the performance and reliability
quite low and when you think about using it for work, you need maximum
reliability but I ended up using local storage as that seems the only way
that's tested enough.

\- You can't tell how much of a file is uploaded reliably at all. The "seconds
remaining" display jumps all over every second as it's probably calculating
from the speed of the moment instead of averages of the last minute or
something and you can't cancel the individual files queue.

\- I have basic auth on the NextCloud instance but the official NextCloud app
refuses to work through it, even though the web version allows you to have
yourself logged in via basic auth.

\- Email app acted like an alpha version, so I instead use integrated RainLoop
which is fine but I couldn't get people to login automatically using the
NextCloud logged in user even though I have that option turned on, instead I
had to edit some source code to make it happen.

\- Logging out takes forever.

I am impressed that the look and feel is good and the caldav being so easy to
use but I'd say it's close when it comes to overall quality to have work
people use it than just for yourself.

------
mceachen
There are already a lot of suggestions to use an externally attached storage
(or "sneakernet"). I think it's the simplest solution with best performance. I
dual-boot off an EVO SSD in an external thunderport/USB-C enclosure, and
performance is similar to the built-in HD. The enclosure is the size of a pack
of gum. Make sure the USB-C to USB-C cable is actually USB 3.0 certified.

She should also verify her Time Machine configurations on both Macs are set up
to back up her sneakernet drive fairly frequently. ... You only seem to need
backups when you don't have them...

------
jedieaston
Something that isn’t uncommon in some shops is using a big system/VM where the
person does their work and just remotes into it from their mobile system as
needed, so the file only actually is in one place.

Of course, it only works if you’re online, and VDI-ish solutions aren’t really
a thing on OS X, so YMMV, but I remote into my desktop while I’m at school
because my laptop doesn’t have much ram.

------
ogou
One thing Dropbox does well is transparently resuming a stalled or failed
sync. The other solutions I can think of are more manual or the UX is very
basic or obtuse. You could set up your own storage space on AWS and rsync
(which has GUI tools available), but that's essentially roll-your-own Dropbox.

I work on large video files that I transport between locations and had an
external drive that I used for a while but would end up forgetting or not
wanting to unplug everything. I have settled on a high speed 64gb SD card that
fits in any pocket or even my wallet. Out of all the fancy network tools I set
up, this option has proved to be the most enduring.

Doesn't help with synchronizing which files are most current though. You might
look at Transmit 5 from Panic. It has lots of useful syncing features now.

------
codingdave
I use an external USB drive. You can get anywhere from the standard thumb
drives, up to multi-terabyte larger drives. So when travelling I just take
whichever drive I need and whichever laptop I need, and plug them together.

It isn't exactly syncing, but it solves the problem, at least for me.

------
bradknowles
If they’re reliably on the same network, then I’d probably use syncthing. Or,
a NAS device and mount the volume across the LAN. Or rsync or syncthing to the
NAS.

If you have to share files off the LAN, that gets more involved.

We currently use a mix of syncthing and NAS for our internal purposes, and
iCloud for external syncing.

------
WClayFerguson
Use rsync to make one folder structure match another by copying only the
changed parts of files. You can sync many gigabytes VERY quickly this way.

You'll want to have an external USB3.0 drive to get from one machine to
another IMO, because you would have wanted it backed up ANYWAY right? ...so
there's no need to try to copy directly from one machine to another machine,
without using the USB drive.

Here's an example command:

sudo rsync -aAXv --delete --force "/home/clay/ferguson/"
"/media/clay/BAK/ferguson"

~~~
mekster
I'd hate to recommend a command line to a designer.

~~~
WClayFerguson
It really seems like there should be some kind of "SyncGUI" app built into
Linux that does this sync with a nice GUI, and maybe there is one, I've never
looked that up...

because all it needs to do is prompt for source and destination folders, with
a stern warning about what it DOES. ha.

------
therealmarv
Buy a large SSD like 2-8TB (just avoid the Samsung 860 QVO 4TB, it sucks more
power during write than USB A standard allows) and buy a nice case like
[https://www.amazon.com/dp/B07D2BHVBD](https://www.amazon.com/dp/B07D2BHVBD)
and mount it there. Work from the external SSD, it's fast enough in my
opinion. Use the cloud only for backup purposes whenever the drive is attached
(I use Insync for that but this is Google Drive centric).

~~~
anonsivalley652
A Mac Mini expanded with two SSDs with syncthing is superior to sneakernet
because it provides replication. And, if you really want to, add a 14 TiB
HC520 placed in an external USB 3 case.

Add an rsync server to get backup. Forget the cloud, most solutions are
overpriced and insecure. Use TarSnap if you have absolute have to backup
reasonable quantities of critical data and call it a day.

------
throwaway12757
Second the use of high speed SSD or SD Card, at night just make sure it backs
up to the cloud just in case you lose it.

------
EvanWard97
If just a 2 terabyte cloud service faster than Dropbox would be enough, pCloud
is nice.

------
anonsivalley652
Try them all and then pick one (or two):

\- SpiderOak One

\- Mega (very cheap)

\- Syncthing (open source)

------
pid_0
icloud, otherwise external storage

