How do you manage large files across multiple laptops/desktops?

Question

My wife is a graphic designer and works on both a Mac desktop and laptop. The assets she generates for a single project can reach into gigabyte territory &ndash; multiple versions of a set of print layouts, RAW photos, etc. One problem she has is how to reliably sync large files between computers. She uses Dropbox currently, but sync over network can takes a long time or sometimes just doesn't finish depending on the file. And when she runs into space limits on one computer, she'll need to adjust Dropbox settings to selectively sync items.I'm curious what people do in this case. I think she needs:1) A large amount of storage, expandable as time goes on 2) Files accessible from multiple computers 3) Ability to access files securely when working from a coffee shop, traveling, etc.

kop316 · Accepted Answer

Have you looked into Nextcloud?If you primarily do it at home, the speeds of Nextxloud are typically limited by the network bandwidth, and the storage is limited by what server you have. I use it and manage 5 TB of data on it without problems.

mceachen · Answer

There are already a lot of suggestions to use an externally attached storage (or "sneakernet"). I think it's the simplest solution with best performance. I dual-boot off an EVO SSD in an external thunderport/USB-C enclosure, and performance is similar to the built-in HD. The enclosure is the size of a pack of gum. Make sure the USB-C to USB-C cable is actually USB 3.0 certified.She should also verify her Time Machine configurations on both Macs are set up to back up her sneakernet drive fairly frequently. ... You only seem to need backups when you don't have them...

jedieaston · Answer

Something that isn&rsquo;t uncommon in some shops is using a big system/VM where the person does their work and just remotes into it from their mobile system as needed, so the file only actually is in one place.Of course, it only works if you&rsquo;re online, and VDI-ish solutions aren&rsquo;t really a thing on OS X, so YMMV, but I remote into my desktop while I&rsquo;m at school because my laptop doesn&rsquo;t have much ram.

ogou · Answer

One thing Dropbox does well is transparently resuming a stalled or failed sync. The other solutions I can think of are more manual or the UX is very basic or obtuse. You could set up your own storage space on AWS and rsync (which has GUI tools available), but that's essentially roll-your-own Dropbox.
I work on large video files that I transport between locations and had an external drive that I used for a while but would end up forgetting or not wanting to unplug everything. I have settled on a high speed 64gb SD card that fits in any pocket or even my wallet. Out of all the fancy network tools I set up, this option has proved to be the most enduring.
Doesn't help with synchronizing which files are most current though. You might look at Transmit 5 from Panic. It has lots of useful syncing features now.

codingdave · Answer

I use an external USB drive. You can get anywhere from the standard thumb drives, up to multi-terabyte larger drives. So when travelling I just take whichever drive I need and whichever laptop I need, and plug them together.It isn't exactly syncing, but it solves the problem, at least for me.

bradknowles · Answer

If they&rsquo;re reliably on the same network, then I&rsquo;d probably use syncthing. Or, a NAS device and mount the volume across the LAN. Or rsync or syncthing to the NAS.If you have to share files off the LAN, that gets more involved.We currently use a mix of syncthing and NAS for our internal purposes, and iCloud for external syncing.

WClayFerguson · Answer

Use rsync to make one folder structure match another by copying only the changed parts of files. You can sync many gigabytes VERY quickly this way.
You'll want to have an external USB3.0 drive to get from one machine to another IMO, because you would have wanted it backed up ANYWAY right? ...so there's no need to try to copy directly from one machine to another machine, without using the USB drive.
Here's an example command:
sudo rsync -aAXv --delete --force "/home/clay/ferguson/" "/media/clay/BAK/ferguson"

therealmarv · Answer

Buy a large SSD like 2-8TB (just avoid the Samsung 860 QVO 4TB, it sucks more power during write than USB A standard allows) and buy a nice case like https://www.amazon.com/dp/B07D2BHVBD and mount it there. Work from the external SSD, it's fast enough in my opinion. Use the cloud only for backup purposes whenever the drive is attached (I use Insync for that but this is Google Drive centric).

throwaway12757 · Answer

Second the use of high speed SSD or SD Card, at night just make sure it backs up to the cloud just in case you lose it.

EvanWard97 · Answer

If just a 2 terabyte cloud service faster than Dropbox would be enough, pCloud is nice.

anonsivalley652 · Answer

Try them all and then pick one (or two):- SpiderOak One- Mega (very cheap)- Syncthing (open source)

pid_0 · Answer

icloud, otherwise external storage

How do you manage large files across multiple laptops/desktops?

Have you looked into Nextcloud?
If you primarily do it at home, the speeds of Nextxloud are typically limited by the network bandwidth, and the storage is limited by what server you have. I use it and manage 5 TB of data on it without problems.

I use an external USB drive. You can get anywhere from the standard thumb drives, up to multi-terabyte larger drives. So when travelling I just take whichever drive I need and whichever laptop I need, and plug them together.
It isn't exactly syncing, but it solves the problem, at least for me.

Second the use of high speed SSD or SD Card, at night just make sure it backs up to the cloud just in case you lose it.

If just a 2 terabyte cloud service faster than Dropbox would be enough, pCloud is nice.

Try them all and then pick one (or two):
- SpiderOak One
- Mega (very cheap)
- Syncthing (open source)

icloud, otherwise external storage