Transfer Large Amounts of Data

Question

Hi HN,My friend has folder on his computer with over 1.100 videos, in total it's about 300 GB.He wants to hand me over the folder but I have no idea on how he should transfer everything over to my computer in the most efficient way possible.My idea so far have been,1. Just grab his harddrive and put it in my computer2. Bring a portable hard drive over to his house3. Rsync4. Airtar / Wormhole5. Torrent from his computer to mineDo any of you have any better ideas?

janalsncm · Accepted Answer

If you have access to a portable hard drive that&rsquo;s pretty doable. Looking online [1] it seems that they can usually transfer at 30 mb/s which means it&rsquo;ll only take about 3-4 hours.[1] https://askubuntu.com/questions/41397/external-usb-hard-driv...

imagetic · Answer

#1 https://massive.io while pricey per gb ($0.25/GB), you get the first 150GB free, so you'd be paying about $35 for the file transfer. Would be the least amount of effort of any of those scenarios, no real special tools required. Could even drag and drop the folder into the browser.
#2 would be portable hard drive. 300gb of video isn't enough to justify any syncing tools. Just drag and drop.
#3 rclone to Cloudflare R2 maybe, somewhere with no egress fees. Be about $5.

anenefan · Answer

I found torrenting over even a local network was much slower than directly copying to a portable hard drive.
Most likely you want to settle on a process that verifies the copied files. If your friend uses M$ you could take a simple path and use a program like Terracopy [1] to be sure and just copy them all to a portable hard drive. (I've used it since working on a Win 10 system (whatever version it was) where quicker file copy was achieved by stepping back to iirc win 95 era and not verifying files.)
In linux there are many options - personally I tar folders up to 20 or 30 gigs using --verify (-W) ie tar -cvWf folder1 'folder1'/
... in your case you would need to direct the write of tar to the portable drive.
[1] https://codesector.com/teracopy

wazokazi · Answer

Take a gigabit network switch over to your friends house and connect both the machines to the same switch. Use windows to discover the shared folder on your friends computer and copy it to yours.

version_five · Answer

What kind of constraints are you under, or what are you trying to optimize for? In many circumstances that wouldn't be considered a lot of data. Is low bandwidth the issue? If so, physical sharing the hard drive, or even mailing a usb hard drive makes sense. How valuable is the data and how much is your time worth?

sterlinm · Answer

Portable hard drive is probably the most practical. Another option not on this list would be rclone [1].[1] https://rclone.org/docs/

Transfer Large Amounts of Data

If you have access to a portable hard drive that’s pretty doable. Looking online [1] it seems that they can usually transfer at 30 mb/s which means it’ll only take about 3-4 hours.
[1] https://askubuntu.com/questions/41397/external-usb-hard-driv...

Take a gigabit network switch over to your friends house and connect both the machines to the same switch. Use windows to discover the shared folder on your friends computer and copy it to yours.

Portable hard drive is probably the most practical. Another option not on this list would be rclone [1].
[1] https://rclone.org/docs/