My current methodology for our immediate family is aligned with the common back up advice - one local copy, one off-site copy (at grandma's house,) and one in cloud storage. We're using Google Photos for cloud storage. The easy integration with Nest Hubs makes for nice digital picture frames around the family homes.
What is your system for backing up family photos and videos to stand the test of time? Is it adequate to put everything in cloud storage and forget about it? Do you reassess every couple years and adjust to the new landscape of storage services? Is it unavoidable that we'll be paying $100+/year forever for a [presumably increasing] few terabytes of cloud storage?
Is there a good solution for posterity? For example, once I die, and if my family were to become unable to pay the hosting bill, is there any way to guarantee these heirlooms remain intact and available?
I tweeted about this a while back and several people had good replies: https://twitter.com/aarondfrancis/status/1445408384472211464
For context, this was after facebook was down for a day. I think the odds of Google deleting all my photos is pretty low, but the odds of me losing access to my Google account for legitimate or non-legitimate reasons is much higher.
2. buy a new drive every year and copy it forward
3. store a copy with relatives
Note that a cloud provider:
1. can shut off your access at any time, by accident or on purpose, and you have no recourse
2. will sell your data to anyone at any time
3. will give it to any government for the asking
4. will leak it to any hacker
5. will allow employees of said cloud provider to browse your data
6. will scan it to use it to sell you ads
7. will scan it looking for anything they don't approve of this week, and take "corrective action"
8. if you've got terabytes, good luck trying to download it
I've found that the greatest benefit of cloud storage is not as a backup, but as our primary place to keep and look at photos. We started uploading and organizing everything in Flickr a few years ago, and now we spend more time enjoying our photos with Flickr's web and mobile apps than we ever did when they were scattered across our laptops and backed up to an external hard drive.
I still have the photos stored locally on my laptop and an external hard drive, and I fully expect that I'll need to upload them again to another service (maybe Google Photos, if that still exists) when Flickr finally fails as a business.
Back up the media to that disk, and then take it down to your safe deposit box. No absurd monthly fees, no Terms of Service, it's off-site so if your house burns down, you're covered.
"Your what?" Yeah, the banks still have those. Mine costs $15 a year. If you die, your next-of-kin will have to bring in a death certificate to get access to it. They'll have lots of other hassles, believe me.
Forget being trendy and having it in the cloud. How often are you going to access it? "Never", right?
You do have to go in every few years and replace it with the latest & greatest, and you'll probably have more media to backup anyway.
I did used to use Mylio [1] which is a really lovely solution that uses a sort of internal P2P approach - your images don't go near a cloud host if you don't want them to. It just keeps your "vaults" updated with each other - a vault can be on a NAS / desktop / mobile. The issue I had with it was that the mobile scenario is a bit shonky - required you to have your phone on at the same time as one of your other vaults in order to sync mobile photos, and that proved too fiddly to maintain.
Unlike other mentions in the comments what you don't want to have is custom scripts, git, backups tools, strong encryption, cronjobs running on some server that will go away eventually. A labeled, unencrypted HDD in a closet in two different locations will most likely do the job just fine. If you want to throw in your cloud backup that you take care of that will be a nice bonus but I wouldn't rely only on a "tech" solution.
Posterity is something we've been thinking deeply about. Our infrastructure is designed to support successors, similar to GitHub[2]. But since cloud storage is expensive, successors will have to choose between exporting a local copy[3] or paying for the newly accumulated data.
We are still designing this "feature". If you think we can do better, please let me know. We would be grateful for any feedback. :)
[1]: https://ente.io
[2]: https://docs.github.com/en/account-and-profile/setting-up-an...
My personal backup is the usual 3-2-1: 3 backups, 2 places, 1 offline. I have one copy on my local harddrive (that I work with), one automatically synced copy via seafile on one of my dedicated servers (which also maintains a few months of history in case I accidentally delete something) and I have one external, offline harddrive at a relatives house, that I sync to every half a year or so. Since I'm paranoid, my dedicated server is backed up to an external storage every night as well via borgbackup. If you don't want to spend a few bucks a month on backblaze or another service, just use a local NAS - as long as you have one harddrive offline and external as well (in case of a ransomware attack that crypts all files).
Important: My files and backups are fully encrypted and it's imperative(!) that you backup all documentation, all config files, all settings, all cronjobs, all executables that have something to do with the backup and restoration process unencrypted with every backup - in the desaster case, nothing sucks more than trying to find the right settings again.
Case in point: I originally used a custom shell script and encoded the files with openssl. However, the default hash scheme was changed between openssl 1.0 and openssl 1.1 (or something like that) and when it came to restoring after a harddrive failure, this took me like a weekend to sort out.
As for posterity: it's up to you if you encrypt the external drive at a relative - if you're fine with a burglar having the images and you cannot be ransomed with them (e.g. due to nudes), just write what is on the harddrive clearly and you're fine.
It's raid 0 (2x6tb) with 2 separate backups done nightly/weekly, one in the house and one done in the garage.
The speed of access is unmatched to any cloud system, in the long run it'll be cheaper than online (assuming £10 per month for 6tb of storage).
I made the plunge after trying a few cheaper options plus cloud. I'm happy with the investment and haven't looked back, it's been rock solid. Synology.
The same system could probably be had for less now, the nas box also acts as my docker server which has been running without issues for 2 years. Most importantly it's not a full time job maintaining it, I spend maybe an hr a month if that checking up on it.
Comes with duplication detection and everything you'd need without learning shell can be done using the nice gui. It's just nice having a system anyone can use, no shell scripts in place at all. It just works.
I store all my photos on a desktop computer and use BackBlaze to automatically keep everything backed up.
It violates the 3-2-1 rule but in the 10ish years I've been using this approach I've had tons of harddrive failures, had to restore some backups due to my own mistakes, but I haven't lost a single image.
I used to do complicated sync stuff but I found that the more moving pieces there were the more I would lose photos for various reasons.
My reasoning is that I don't trust Google to not lock me out of my account at some point, so having both a local and a remote backup gives me piece of mind. I periodically check the offsite backup to check that it's still all working. Total cost for about a terabyte of files (it's not only photos and videos) is about $6/month, which is pretty reasonable.
It is just ZFS snapshots with replication. I wrote my own shell scripts for this that takes daily/hourly snapshots. This scales well with multiple TB's, bandwidth is no issue as only changes from last snapshot(s) are replicated.
After 10 years, I've replaced only 2 hard drives of about 30. I do upgrade FreeBSD once every second or third year. Nothing else, it just keeps going.
This project has probably costed me max a week in configuration & setup & maintenance over the 10 years.
For pics that don't fit neatly into an album theme I'll group them by date, "Family pics 1995-1996" based on their quantity.
Then I'll print photobooks, adding info where possible about location, date, and people. Basically, recreating physical photo albums. I'll print 2-5 copies of each album, depending if I expect to send copies to other family members or not.
Having physically-printed photos works both as a last-ditch backup and as easy access for anyone to browse. I nostalgically remember spending time as a kid flipping through my parent's and grandparent's photo albums - sometimes with them, but many times on my own. I want to ensure my own kids can have the same experience.
Online: A big BTRFS "raid 1" (block-based mirroring) array. This is what Plex runs off of. If I lose a drive, I just replace it and rebalance. I run btrfs scrubs weekly.
Offline: I have a WD external hdd with a plain old ext4 hdd that I rsync all files to weekly. If my btrfs array shits itself, I just copy back over from this hdd.
Emergency: If the above process doesn't work (say: the external hdd is dead when I go to restore from it, or I can't physically find it, or my house floods and everything in the basement is lost, etc), then my fallback is:
I sync all truly critical stuff that can't be replaced (so, basically just family pics and videos) to S3 as deep archive storage class. Costs pennies. I don't do any fancy tar/archiving, encrypting, or compressing (since video and image files are generally already compressed). I figure the risk of people snooping at our family pics is smaller than the risk of me needing to restore from a backup and not being able to because of stupid reasons (lost gpg key, some kind of problem with tar, etc).
This is small, since most of my disk use is from things like rips I've made of BluRay/DVD's, FLACs from bandcamp or rips of music CD's, etc. Things I can recover with appropriate effort.
I have a raspberry pi with a USB drive attached. The OS is ubuntu, so I can use ZFS. The drive is backed up to backblaze S3 using restic in a cron job. The backup and ZFS health is monitored with healthchecks.io
The data sync across all the devices including the raspberry pi is done using resilio sync.
I view my photos when I am home using an app written by myself: photograf (https://github.com/ptek/photograf) It runs on the same raspberry.
You have got me thinking though about my backup of family photos/videos. My backups are all too clever, encrypted and inaccessible at the moment. I feel inspired now to make sure photos/videos are easy for my family and future generations to access.
The ability to search photos easily over time and the weekly "this week X years ago" reels are great value.
My family are all taking photos on their own devices during a vacation, holiday, gathering, etc. and then someone creates a shared Album on Photos (iOS) where everyone just dumps all the photos taken. It's a complete mess in my opinion. WE have a shared album "Christmas 2018", "Holidays 2019", "Xmas'20", etc. depending on who named it.
Everyone else seems to like the ease of upload and ability to find past pictures. Getting the photos out of the apple app seems like a pain to me. I feel like there has to be a better way, but it has to work for the lowest common denominator (Grandparents, etc).
We have a couple people on Android and are essentially cut off from the family photo stream. When that happens everyone's mentality is, "get an iPhone".
It's cheap, easy and ransomware proof.
This means obviously importing all photos taken via my DSLR into Lightroom, but also syncing all photos taken on our iPhones via the Lightroom Mobile App.
Lightroom Classic keeps all the (compressed) photos in the Adobe cloud for easy sharing and browsing, but also writes them out (unaltered) to a directory on my Local NAS.
This NAS gets automatically backed up via Arq Backup (https://www.arqbackup.com) to an encrypted Amazon S3 bucket. Additionally, once or twice a year I create a versioned copy of the NAS via Carbon Copy Cloner (https://bombich.com/) to an external hard drive. This hard drive is stored offsite somewhere safe.
In a nutshell, for around $12 a month + a NAS + a hard drive, we have all the convenience of the Adobe Lightroom cloud combined with a local copy on the NAS, a cloud copy on S3 (in case Lightroom cloud gets corrupted) and an offsite copy (in case our place and the whole internet burns down :-)).
In short, I have an old desktop-turned-NAS with ZFS that will tolerate 2 physical drive failures and access my photos over a network share. It runs a weekly cron script to copy an iterative, encrypted borg backup of the photos offsite over sshfs to my VPS. I can mount specific date's snapshots over sshfs back to the nas/another machine if I ever need to recover anything without having to enter the password on the VPS.
It's a good question on what would happen if I died. Thanks for getting me to consider this. My spouse wouldn't have a clue with the VPS or how to fix the NAS but could pay someone to easily copy the data off. I should leave a note taped inside the case. For now I'll plug an external ntfs drive into the NAS and if needed she could plug that into any other device.
Life is already complicated as is.
You’ll obviously need to leave keys, instructions etc for your next of kin. Or put a hard drive in a safe deposit box.
Then I have a custom script that encrypts and replicates all the borgbackup repositories on Google Drive (where I have unlimited space because my previous university still hasn't deleted my account).
Ideally I would also replicate everything on a drive at some relative's house, but I haven't had time to set up this yet.
Overall I am rather pleased with this setup. This is for general backup of everything. Specifically for photos, I have a script on both my and my wife's Android phone that copies all photos and videos to a shared directory on the MacBook, and she periodically organizes all the photos there. Then they fall in the main backup.
If I were you, I'd delete 90% or more of those photos and videos to save running costs. I don't know how enthusiastic you are about watching photos of yourself when you were shitting your pants as a toddler, but, speaking for myself, I'd consider being urged to watch even a small fraction of what you have at family reunions some form of torture....
In addition to the other good answers, a box full a printed photos will stand the test of time.
2. Encrypted external offline USB drive onsite
3. Encrypted external offline USB drive offsite
4. Upload of an encrypted archive to the cloud storage
It's all manual so sometimes some backups are months old. Haven't lost a single digital photo for almost 15 years now, and have gigabytes of obsolete photos...
Then I share my Amazon Prime backup with my wife. All our photos are backed up there but not videos.
For videos, I upload edited videos on YouTube as unlisted or private. And some of better videos as public on Youtube, Facebook, and Instagram.
And finally I try to share as many photos and videos in emails, Whatsapp, with friends and family. Sort of distributed back up in the worst case scenario.
(Also never delete any photo or video unless you really don't want anyone to see it ever. I used to extensively prune my library but wish I had not. Storage will likely keep getting cheaper. And AI will get better and it will be able to create better slideshows and edited videos.)
We use Apple Photos as the primary, of which I export a backup once a while (at-least yearly for sure). Because Apple has’t solved what everyone ask — a way for two or more users to merge Photos, I use Dropbox[1] on the other devices (wife, daughters) to collect into “Camera Uploads”.
I split the content into “Screenshots” (PNG), Videos, and Photos with Hazel[2]. Photos to imported to the Primary Photos Library, Screenshots, and Videos are either deleted or sorted weekly (few minutes), monthly (30-min-ish), quarterly(30-min-ish), and yearly (about 3-5 hours on a day around the end of the year, with the other digital clean-up task) during my digital chores that I do. My daughters produces a lot of them that I need to throw out. She can position her iPad and video-shoot while she talks her way cleaning her desk for an hour.
The photos and videos are now in my primary device iMac (for now), with a second backup on a 2012 MacMini running many other digital errands. The 2TB iCloud+[3] is good enough for now.
I would definitely love to clean this steps and move to a more non-proprietary format (read non-Apple) to Open Source tools/process.
I tried Google Photos because it was easier to combine and I even had one of their large plan but it turned out to be one of the messiest thing I tried. Stopped using them[4].
I will be reading and learning from the other interesting comments in this thread. Thanks to all.
4. https://brajeshwar.com/2021/how-to-delete-all-photos-and-get...
The price of existing cloud storage is too high, and some of the companies(Shoebox, Canon Irista) doing the business gradually shutdown the services, this is a money losing business, it’s not the efficient way to manage huge amount of assets centralized (flicker CEO’s open letter sent last year confirmed this), they have to either make it more expensive, or make you the product. Cloud service is convenient for the user, people don’t need to buy expensive hardware, don’t need to be the professionals to maintain that, don’t need to worry about the energy fee to keep it run 24x7, but things are changing, single board computers are getting cheaper, more powerful and more energy efficient, storage are getting cheaper with larger capacity, software are getting more intelligent, people are having more and more concerns about the privacy, it’s now viable to host the Photo service, your private cloud, at your own place.
There are tons of open source alternatives there, but non of them provide competitive features like google photo, even though there are some backend services like photoprism that does a very good job on indexing the photo with AI, however the mobile APP is missing, and google also has the ecosystem to show photo/video on google home and chromecast.
We've been building Lomorage (https://lomorage.com) trying to fill the gap, it's easy to setup a self hosted service(Still need a lot of work to make it easy for non tech users), cross platform, has iOS and Android mobile APP, can setup multiple accounts, some basic AI search(not polished and need more work), there are other features missing, but it's stable enough for daily use. Would appreciate you try it out and give some feedback.
1. a cronjob on my android phone (via termux) does an rsync to a VPS
2. the VPS has a cronjob to sync everything with S3
3. the frontoffice tool for people to access those photos is my open source Dropbox like frontend that is bring your own backend: https://github.com/mickael-kerjean/filestash My wife and I got an account and family members can access it through shared links.
The S3 bill goes to a shared account so that If I die, the VPS will probably be quickly removed but S3 should stay in there with my wife paying for it.
I keep all my photos in the "native" format, which is basically JPEG and whatever my iPhone is spitting out these days. They are currently editable (which for me is mainly crops, with occasional other fix ups). I scanned all my old negatives, APS, and slides and just had them all issued as the lowest compression (highest resolution) JPEG (I didn't have a huge choice of formats).
But I have a bunch of videos from film tape, and cards. What's the best equivalent (supporting simple crops and edits) that might reasonably be editable in future?
If I were being properly rigorous I'd be burning three copies and putting two of them in different physical places.
Printing out the occasional photo book of our favorites is also a part of the strategy - if our grandkids want to see photos of our lives after we're dead, they won't need to worry about lapsed hosting bills or ancient storage technology. They can just look at the book.
Obligatory reference to my rant about data preservation:
https://videohubapp.com/en/ - MIT open source: https://github.com/whyboris/Video-Hub-App
Non tech solution: I just print everything. I refill canon cartridges, making the cost of ink negligible and buy cheapish 4x6 photo paper, and I just print like crazy. After some basic calibration, the prints are quite good enough. I guess longevity of the aftermarket inks is still to be seen. Wife enjoys just picking up some prints to send friends and family. She snaps a pic of whatever she sent, and I reprint those.
The chance of all of these versions becoming unavailable is low enough for me not to worry. If I end up under a tree some day nothing changes in the short term, the data will be available to those who have not yet met their tree. In the longer term I'd expect someone to either pick up where I left off or move the data to some storage which befits their level of expertise and needs. I don't want to force them into any contract with any provider, nor can I decide what they deem to be worth keeping.
https://github.com/prettydiff/share-file-systems/tree/master...
I'm glad my backup and storage setup has stood the test of (thus far a relatively short) time.
Most videos are from mobiles, and I use syncthing on each device to automatically back them up to NAS. The NAS has an external USB drive to which back ups are directed, plus another NAS in the shed as a secondary on-site backup - which also has an external USB HDD that gets backed up to. All of the backups other than mobile device to NAS via Syncthing are done with crontab'd rsync scripts.
I then irregularly rotate portable USB-caddied HDDs from my parents and in-laws houses and backup to them as well. This rotation is well overdue, thanks for the reminder.
I don't do cloud. It feels too ephemeral in the context of my history with online services - I change my mind and switch around a bit, and lose interest here and there. I'm the kind of person who would host their own email server if I wasn't already gigabytes deep in a grandfathered free custom domain plan from Google. (I do actually host my own email server using Citadel, but it's a domain that's very rarely used and entirely non-critical).
I have a primary SSD for my OS and applications, a secondary SSD where I store my latest/active Lightroom catalogues/files and a large 4TB (spinning disc) hard drive for local backup.
After a shoot I'll import the files onto my secondary SSD and then manually copy them over to the local HDD once I've done my picks and culled things a bit. That HDD is automatically backed up to Backblaze so there's always an original copy of the files either local or in the cloud. Any images that I edit/pick/export will be manually uploaded to Google Photos (usually in it's own album) and I try to annually pull down files from Google to go into another 'jpg/exported' folder on my HDD that's backed up to Backblaze. My wife and I also have our iPhones backing up to Google photos automatically plus we have Apple storage/backups - so we tend to have double coverage on most things.
Some of it is a bit overkill probably but I'm particular about having access to all my photos and I've definitely gone back to look at and edit old photos/shoots so it's worked out ok for me so far.
I sometimes wonder how much of the way things are is because that's just how the US legal system is, and how much of it is because that's the worldview of the childless twentysomething singles who built these systems. (No judgment: I know when I was a twentysomething childless single building things, I had a person-centric world view, not a family-centric one.)
(An extreme example of this problem is the oculus quest, a device that is begging to be shared between family members, but it requires a Facebook account to use. Further, it is a breach of the terms to have a family facebook account. Nobody in our family with a facebook account - i.e. the older people - wanted to sacrifice theirs to the quest.)
It's weird/sad that a family doesn't have any sort of legal personhood, but a corporation does. A joke solution here might be to incorporate my family and have all these digital assets belong to the family corp. I'm not even sure if I'm joking.
For the cloud backup, I think that you will be saving money by using backblaze [1].
As for the solution for posterity, if you're worried about the possibility of next generations not being able to pay, the only solution I see from the top of my head is to print the photos. Given the amount that you have this would be a daunting task, so only print the ones that you find the most valuable and look for a "safe and stable" (whatever this means... :D) place where to store them.
The cameras have Wifi cards and sync to the desktop that syncs to Backblaze.
On new years day each year, the desktop gets archived to a external HDD and put in a safe.
That partition is backed up daily to two separate external USB hard drives using rsync.
That gives three separate copies. One of those copies is also rsynced to a separate laptop, so a total of 4 copies.
I don't store archival stuff like videos, photos, music in my personal directories and scatter them all over the place. An archive is an archive and needs to be centralised and then distributed outwards from there.
Is there a good solution for posterity? For example, once I die
Unfortunately, you can't control posterity from beyond the grave. What you consider priceless is something that your descendants maybe can't wait to get rid of. I learned this lesson when my brother died. I started off trying to keep as much as possible of his stuff, but as I sorted through it I found that it was worthless to me so I ended up discarding about 95% of it. I saw that when I died, somebody else would do exactly the same thing with all of my carefully-hoarded information.
- But what about searching... That moment when you're looking for a specific moment or latest photos of your dog? or pool to find a specific pool.
- Dupe detection.
Those unique features that for example, Dropbox is an inferior tool for the job.
Then there's Dropbox. It automatically uploads all my raw data, and syncs it across multiple devices. So at least I have local and cloud backups of all my raw photos/videos. A picture taken on my phone camera will immediately appear on my PC desktop hard drive, as long as it's on. The downside is searching through photos can be a slog compared to GPhotos, but that's why I use both.
I used to use Picasa many years ago, but haven't reinstalled it on a newer computer. I've been meaning to run it through my Dropbox collection again to make it easier to sort through locally. IMO, it's still the best photo management software. As far as I know, it's still the only one that does facial recognition locally and without a subscription.
They have a decent enough Photos app for browsing / sharing photos.
For offsite backup, I send to B2 which is ridiculously cheap, but I pay for, so I’m not the product. I haven’t put Cloudflare in front of it, but that’s something extra folks do.
Recent Local + Recent in Cloud(DropBox) --> Copy on Home NAS (Synology) + Local NAS weekly Backup (External USB) --> Continuous Cloud Backup (BackBlaze) --> Offsite Copy Annually
Copy all important and historical data to a cloud storage solutions. Currently using DropBox
Export photos to home NAS (Dual Drive) and NAS Backup to external drive weekly. Daily backup all local drives to BackBlaze
Once a year, copy all photos and important data to a larger encrypted USB HDD for one year storage off site at parents home. I swap the drives about once a year. This is my I need it quick recovery copy should the house burn down.
For encrypted storage, I use only local OS encryption. Got burned by TruCrypt/VeraCrypt not supporting new OS.
My wife is the one who takes most of the photos in our family. For her computer (MacOS):
1) Back up her computer to USB hard drive via Time Machine every day or two.
2) Back up her computer to a USB hard drive in her office (via Time Machine) when she takes it in, about once per week.
3) When her drive starts getting full, move photos to a larger USB media drive attached to my computer. (Every year or so.)
For my computer (MacOS), including USB media drive:
1) Back up to a "main" and "redundant" USB backup drives continuously (every hour or so) using Time Machine.
2) Back up Mac hard drive, but not media drive, to "main" USB backup drive every night using SuperDuper.
3) Back up to the cloud continuously using BackBlaze.
4) Swap "main" USB backup drive with an identical drive stored in a safe deposit box (every 3 months).
This provides a nice combination of convenience, redundancy, and recent off-site backup.
1. Pictures are either manually copied to a staging folder on my storage server (for DSLRs) or automatically synced there via Syncthing (for smartphone cameras). For the latter, Syncthing is set up to preserve deleted files in case of accidental deletion.
2. The storage server runs zfs and takes 10-minutely snapshots of all datasets, which are replicated to 2 servers using zrepl: one sitting upstairs and another I rent from Hetzner halfway across the world. Replicated snapshots are kept for 2 years. Everything uses zfs' native encryption, but the replication targets do not have the key.
3. For the really important pictures and other documents, I create 2 additional backups: an encrypted backup to Backblaze B2 via rclone and burning them to M-DISC blu-rays. This is sort of a last ditch thing in case a zfs bug renders the primary backups unreadable.
From then on, I backup to Google Archive Cloud Storage over restic [2]
[1] https://olimex.wordpress.com/2020/03/13/bay-hdd-sdd-is-easy-...
[1]: https://www.microcenter.com/product/634744/wd-4tb-red-plus-5...
Our local "copy" is basically our personal laptops. This has required manually upgrading our SSDs over the years though because we have around 1TB of photos and videos.
Our off-site copy is backblaze. We don't keep a non-cloud off-site copy.
After that I decided to migrate as much of my hardware as possible to 19" racks. So next, I upgraded to a Synology NAS RS819 (4-bay, 4x4 HDD = 16 TB raw capacity, 12 TB in a RAID5 setup). This lasts for quite some time, as I'm not much into movies. Importantly, the NAS is normally physically _not_ connected to any networks except when I transfer a new batch of photos for archiving.
Some historic material is on a DAT tape in a fireproof safe but that is close to its expected end of life, which is fine as there are HDD copies.
To view the photos, I access a subset of commonly viewed media on another 19" server, so the NAS is truly for archival use only.
What would be nice, but is not yet implemented is a second site to add redundancy and protection from disasters (of course a backup should be offsite per definition). The two sites could synchronize over rsync. But I am still too concerned of the attack surface when putting archival machines on a network. A solution could be the following: some friends recommend Google's "cold storage" as a tertiary cloud storage, which is affordable, and they use scripts to encrypt their data; I think that is a good idea.
The question regarding mechanisms to manage things in case of one's death is an excellent point raised by the poster - I bet a lot of valuable information assets have already been lost due to people passing away when their relatives are not IT savvy to rescue them (or the deceased didn't take precautions to leave passwords and instructions behind).
For an offsite backup, I should start printing 2 and storing one copy at a relative’s house…
Nightly update to Backblaze B2 using Synology Sync.
Considering a third local copy stored at a family / friends and updated yearly.
This requires almost no work on my part... just the occasional copy over of pics from phone / camera to my network drive.
I use SFTP to upload new pictures from phones etc. to the pi. There are many apps that can do SFTP, like e.g. Foldersync on Android.
I have a shell script on the pi that runs once/day and uploads new files to S3 storage, after encrypting them with gpg.
I occasionally get a new USB stick and retire the old one, keeping it as a backup copy.
The whole thing actually works quite well and means:
- Local, unencrypted files I can access easily
- Encrypted files in the cloud, that no external parties can read
- Physical backups that I can put in a safe, or whatever
backup server galvanically disconnected, manually plugged in to synchronize, then scrub and shutdown, small window for catastrophe, chances of both servers getting killed at the same time is so small I'd probably be dead too.
main and backup servers are in two different locations (different power substations).
I just use rsync for doing the backup.
Family is connected to main fileserver. It snapshots once per week, so if they destroy something, I can get it back, it keeps two years of snapshots.
Once every 5 years I buy a new fileserver, the old one becomes the backup (assuming it is still healthy and not experienced any problems during it's service).
TrueNas is configured to automatically backup to Backblaze B2 (which is off-site).
After editing, all the exported JPGs are stored on Google Photos and shared with family. I like Google photos because of the content search and face detection features.
So that's 1 on-site medium and 2 off-site mediums. I used to burn the photos to archival disks and place them in the bank, but it got tedious.
Edit: I use Backblaze because it's the cheapest. Most Photo storage providers don't have support for Raw files. (If anyone has a recommendation let me know).
2. Periodically I take photos and videos off our phones and store them on my desktop PC, imported into Lightroom. They are stored on 5400rpm HDDs in mirrored RAID config
3. I get terabytes of OneDrive space with my Microsoft package (I can't remember the name of it right now!), and it automatically uploads all the media from my desktop PC
4. I also have Seafile installed on the desktop, and it automatically uploads everything to my Seafile server, which uses Azure Blob Storage
I'm not close enough to the end to worry about posterity. If I die tomorrow then my keys are lost and all of my data is entropy. I expect to be motivated to have my data pass on in legacy but that will only happen if I have a kid who is willing to accept maintenance of familial digital archives. We're the first generation.
So at any one time there are almost complete 3 copies of everything.
There is no off-site backup. The safe is probably not super theft-resistant nowadays but it should provide some protection in case of fire.
I'm reaching near capacity on 4 Tb and don't exactly know where to go after that. I could upgrade all 3 disks to 6 or 8 Tb but I resent discarding the 3 4s that are in perfect condition. So IDK.
For the rest (e.g, snapshots, non-sense ) I have them copied between devices thanks to sync tools. I also do cold copies when I feel I need to do that.
I hope when I am 80 or 90, I would be still able to find my images on telegram then...
I still have Dropbox as a backup this way so it's a cloud backup of sorts.
Not foolproof but it's super simple and good enough for me.
I don't use Google Photos for privacy concerns.
Now that I use Tailscale, I could consider setting up a NAT at home and make files easily viewable using NextCloud, but I worry I won't backup properly to Backblaze B2. Need to figure out how to schedule backups before I make the switch.
Photos are imported from SD card using a script. This script creates a new directory, captures some metadata, copies the photos in and creates thumbnails. This directory is rsynced to the NAS then encrypted, compressed and sent to s3. Nightly, from another location a raspberry pi with a large encrypted disk rsyncs the entire NAS.
All viewing and editing of photos is done against the NAS, and any changes are picked up nightly by the pis. The s3 copy acts as an immutable original.
While NAS has mirror on it, I do not consider it enough backup. I assume it will not save me from ransomware or a family member stupidly deleting a lot of data or from fire or from theft. It is mostly to give convenient access to it for my family.
HDDs are in rotation and one is always offsite with my family that I visit regularly. Whenever I travel there I make a backup on one of the HDDs and then bring the "old" drive back home to be reused on one of future trips.
People are such hoarders when it comes to family photos and videos. You are never going to look at 99% of those photos and videos.
At some point I will buy a simple Raspberry PI (or cheaper system) with one multi TB drive using USB. A friend of mine will do the same. A cron job will run a sync on the drives at night. 50% of the drive will me mine, 50% of the drive will be his. So the system has redundant storage and it is off-site.
The NAS server has apps that offer backing up to various cloud services, but I choose not to use them.
I'm currently using returned/refurbished Samsung T5's at about $100/TB.
[edit] only time I lost some image was because I have waited too long to copy it from my camera to the computer :)
I would consider it the minimal defense against certain technical malfunctions. I should add another offline copy at some relatives, to protect against the 'house burns down' scenario.
Using an online service could help against the time between the backup cycles, but that is the least of my concerns and online services have their own set of problems.
As I have the MS Office subscription, the cellphone photos are also backed up in the OneDrive.
Once we're close to filling up available storage, I turn on my local server which downloads it into its RAID disks (then I turn it off afterwards). I wipe my & my wife's onedrives and we repeat the process.
Once a year, usually around Christmas, we're sitting together and choosing photos that we want to print and put into family album.
Copy 1: Desktop Computer
Copy 2: Backup 8 TB USB Hard Disk, always online
Copy 3: Backup 1 TB USB Hard Disk #1, always unplugged, rotated with #2
Copy 4: Backup 1 TB USB Hard Disk #2, always unplugged, rotated with #1
Copy 5: Microsoft Onedrive (Pay for 1.2 TB Storage)
Copy 6: Amazon Photos (unlimited photo storage for free with Prime)
Main purpose of the unplugged offline storage is in case of ransomware attack which could conceivably wipe out my online and cloud backups.
I have the Dropbox app installed on a desktop computer which is always running and syncing (Local Copy 1).
Once a week I hook an external drive up to the desktop and take a backup via windows-built in backup utility (Local Copy 2).
I have Backblaze installed on the desktop continually backing it up (Remote Copy 2).
We keep the digital photos backed up on an external SSD and on iCloud (auto-synced from several devices), too.
Bottom line: we stopped trying to obsessively 'preserve' digital hoards. It's such a waste of our time, and is a questionable goal anyway.
Every 3 or 4 years I get paranoid and I make a backup in a random HD drive that it takes me days to find, which for a backup it is not ideal.
- Local server processes items and provides UI for family members
- Pics and videos get backed up to Wasabi (S3-compatible) storage using restic (incremental backups, no need to upload hundreds of GBs every night)
- Once in a while I back everything up to external HDD and store it in a drawer at work
- Local copy on desktop
- Google Photos (with upgraded storage) for "everyday" use (sharing albums with family, viewing on phone, etc).
- The "proper" backup is done with Kopia to B2 and Wasabi. I don't have a large amount of photos, so B2 is really cheap and its just a redundancy for Google Photos and Wasabi.
Are they actually heirlooms? If the goal is to keep these safe so generations can continue to not look at them, what's the point? If you haven't looked at something in years, odds are nobody wants to look at them.
However I recently realized that if I wanted to get the images back out of Glacier it would be ridiculously expensive. I’m looking for another option, hopefully something that is cheap and reliable.
Everything automatic, grouped into folders in yyyy-mm format
The sad part is that nobody will ever look at all those terabytes ...
Could be worth it if you have more than 1 TB. Or even more than 200 GB at Google.
It costs next to nothing. The redundancy levels of S3 are way better than anything I can make.
Periodically file/classify photos on NAS. Periodically make physical backup NAS to a remote site.
This approach is low cost and convenient.
After the fourth photo service I had migrated to went out of business (and Picasa was then cancelled), I realized I needed something that wouldn't go away on me, and started working on PhotoStructure, a self-hosted photo and video "digital asset manager" that does automatic organization and deduplication to sweep everything into a single, neat, timestamped pile, and, critically, _keeps my original files intact_.
The point isn't to use PhotoStructure, specifically, but to use an application that you could walk away from and not be heartbroken, because that app used a standard filesystem hierarchy, and used standards to store any metadata changes.
My beta users asked me "how do I keep my stuff safe?" so many times, I ended up doing a bunch of research and wrote up this article, which discusses file integrity, why "3-2-1 backups" isn't really what you want, and how to go from there:
https://photostructure.com/faq/how-do-i-safely-store-files
Know that Google Photos is a great _secondary_ backup, but even in "original mode", what you upload aren't always the same bytes as what you download--their API strips off GPS metadata, I've seen changed captured-at times and exposure information changed after the round-trip through GP. (iPhone uploads, in "original" mode may survive the round-trip, but the point is that it's not something they guarantee).
I'd recommend using one of several mobile apps that will backup your original bytes directly to your computer at home. There are three I've tested and listed here: https://photostructure.com/faq/how-do-i-safely-store-files/#...
I'd also recommend using a filesystem on your home server that can detect media errors. I discussed that, and what cloud storage backups I recommend in the above article, as well.
-=-=-=-=
OK, so, now that you've got your stuff safe, let me tell you a story.
My Mom and Dad passed away almost a decade ago, and left several boxes filled with albums, shoeboxes and loose photos.
Almost none of the photos had dates on them. The albums had no writing in them, and most of the photos were of locations or people that I didn't recognize.
It took a while, browsing through these images, to realize that these images had relevance to only my parents: and maybe only the parent taking the photo.
Without additional context, these boxes of memories were almost entirely irrelevant to the next generation.
It was a punishing realization for me. I know these boxes were important and relevant to my parents, but all but a handful of photos had relevance to me.
Digital photography at least has a modicum of metadata automatically.
But you should still consider you "heirloom" of hundreds of thousands of photos and videos to be _irrelevant by default_ to the next generation.
So, how can you add relevance to your corpus of imagery?
I think that can be helped to some extent with
1. software (PhotoStructure shows random "samples" of years to deal with browsing through gigantic libraries, and displays "streams" of related photos using common metadata attributes to browse across hierarchical trees)
2. rating and pruning (so people can browse only the "best"), but
3. I think the real answer to avoiding irrelevance is for you to tell the story behind the image to give it context and relevance. It doesn't have to be a novel, but even a couple words can inoculate the album from irrelevance.
Good luck!
Digitally I keep a few different backups but they are raw and would take a while to sort through.
- I hardly ever take pictures or shoot videos. Not owning a smartphone helps.
- I don't care at all about old family pictures, so if family members want to do something "for posterity", I hope they don't give me a giant pile of media and expect me to keep it.
Also, I feel like everyone's always creating content, not deleting it. Won't the result be an unmanageable pile of cat videos where you can't see the forest for the trees?
The most important ones (e.g wedding pictures) are also stored on a local disk.
so, requires a home nas, and two backup drives, and an offsite person,
When I first set up that photo solution I feared that no one would like to use it, but everyone liked the idea to have one central storage for all photos where everyone can also see and download the photos of others.
To make sure this is not the only place where everything is stored, I have another, older NAS sitting at my mom's house that is used as a remote backup solution. If ever comes the situation that both my and my mom's NAS are destroyed, photos probably don't matter anymore, so that's totally fine for me.
Of course, two NAS with much storage is quite expensive, but I guess it's cheaper in the long term, than paying for a multi-TB cloud storage every year.
TLDR: two NAS at different locations, one has a shared folder where all family photos are stored, one is backup only.
Same on Windows
Manual copy sync to OneDrive on Linux
Boom done
As a number of people have pointed out there are several problems with cloud based solutions:
1. Will the solution continue to be available
2. Will the company properly take care of your data, and protect your privacy vs. the companies own self interests.
3. There is an ongoing recurring cost.
With home based solutions you also have a large number of issues:
1. A lot of work that requires technical knowledge (making backups, regular testing, etc...)
2. You have to keep copies in multiple locations and maintain your own disaster recovery.
3. Obsolescence of your hardware, backup solution is a real problem.
4. This takes real time that you would probably prefer to spend doing something else, which means it will probably get neglected at some point, and therefore still highly susceptible disaster issues from hardware failure or much worse.
A number of people have suggested that the old way is better (Have paper copies of the photographs). This is also a bad solution
1. When converting modern digital photos to print you will loose data that is available in the digital photo.
2. Printed photos will degrade over time (Whether in a book or printed individually)
3. These are highly susceptible to destruction via disaster
4. There is no way to search your data
As far as the ongoing cost of storing in a cloud based solution, by the time you figure the cost for any of the other options when done right, The cloud based solution is probably cheaper depending on how you value your time in each of the other options.
As a number of people have pointed out in the comments as well, the photographs are only valuable if you have useful Metadata of the photos. With modern photos that is much better than before as you get a timestamp and often a GPS location with the photo. Ideally you want at least the date, location, event(if applicable), people in photo, description/story. That will take a photo from being useless to others besides the person who took it, to becoming highly valuable. The more information the better. Collecting that metadata is much harder to accomplish with old analog photos that have been digitized as they often have none of that metadata handy.
Storing, searching and using the metadata is solvable with good software. I tried solving many of those issues with my software. the issue that is not easily solvable is the where/how to store the data. My software allows storing on our service or you can import from Google Photos or Dropbox, but as noted those all have issues. If anyone has ideas on how to solve those issues and actually come up with a solution that is good, I would love to hear it.
NAS that backs up to Backblaze
For family photos, you might view them once every couple years, just to browse randomly. Every once in a while you might access the data with the intention of finding a specific photo. The data is of course never modified and only expanded once every five years or so.
So, low bandwidth, infrequent access and expansion and no modification. The winner by a mile is Blu-ray.
A hdd will de-magnetize over a very short period of time even if it’s stored in ideal condition. Less than a decade. You need to plug it in and let it refresh those tracks. If you keep it plugged in all the time then something else will die. Any way you slice it, there will be maintenance.
The data on a Blu-ray just sits there. It will outlast you.
Some Blu-ray Discs can have problems. This is because of improper materials and poor manufacturing. If the layers of plastic that sandwich the medium become delaminated or damaged in some way, this will lead to the oxidation of the data medium and loss of data. A properly manufactured disc like a Sony archival disc will never do that. And they make discs with data mediums that don’t oxidize so those ones can hypothetically last for thousands of years. Despite what people might say, a proper Blu-ray Disc is a great way to store data.
So here’s the deal. Get rid of duplicate pictures and pictures you don’t really care for. Compress as tightly as you can. Burn a copy of those files to a set of discs. Then another set of discs from another brand. And then another and continue however many times makes sense for your budget/ value of the data. Send each set to a different family member starting with the most remote. When it time to update the database, send out the latest discs.