My issue is specific to OneDrive. When I reconcile a backup set against live data, files are missing. I've had it happen with both Veeam Backup for MS365 and Synology Active Backup for MS365. Neither system reports any issues when backups run. I don't know the cause and I can't reproduce it, but it happens consistently for at least one of my tenants and seems to get worse over time. I've seen the issue on more than one tenant, so I don't think it's anything tenant specific and the only (untenable) solution I've come up with is to restart the backup from scratch.
The tenant that has the most issues is a business with about 200k files. The business owner owns the files and everyone else has access to a few shared folders near the top of the hierarchy. They have about 350GB of data which ends up being about 1TB of quota after versioning.
I originally ran into the issue with Veeam by randomly spot checking 1-2 files every once in a while and running into missing data by random chance. That made me realize I needed to do some kind of bulk reconciliation on a regular basis. I gave up on Veeam because they append the OneDrive file version to all restored files and it makes it difficult to reconcile. For example, locally restored files end up with ' (ver 2)' or similar appended to the file name.
I switched to the Synology system because it's ideal to reconcile since an up-to-date backup set can be shared via SMB. That makes it possible to have an up-to-date OneDrive sync and a mapped drive to an up-to-date backup set on the same machine. After that, it's a matter of comparing two folders as long as care is taken to get a consistent point-in-time for both sets of data.
The only noteworthy thing that I think plays a part is how frequently the tenant reorganizes their data. They're always renaming and moving files and folders to keep things organized. I'd frame it as being frequent, but not unreasonable. The reason I think this is noteworthy is that in cases where I'm able to track the life-cycle of missing files, they seem to "disappear" after being impacted by a directory rename or move operation.
I can't engage with support for this particular tenant because the data is supporting documentation for government work. I can't even share examples or screenshots with the file names AFAIK.
I've seen people complaining about similar issues, but the complaints are from years ago [1]. This [2] caught my eye.
> The root cause for the missing data was due to incorrect representation of the changes from the SharePoint API side. Veeam RnD team performed an investigation and found that sometimes the SharePoint API mechanism of tracking changes did not track the changes inside the Child files or Folders inside the Site`s list.
Does anyone here reconcile their OneDrive backups well enough to say you're confident you can reliably restore your data with 100% consistency?
Is it possible there's a change tracking bug lurking in the SharePoint API? I don't know anything about how it works, so any insight would be useful. For example, would the OneDrive client and backup clients use the same change tracking? That seems unlikely based on what I'm seeing, but, again, I have no idea how it actually works.
1. https://forums.veeam.com/veeam-backup-for-microsoft-365-f47/...
2. https://forums.veeam.com/veeam-backup-for-microsoft-365-f47/...
Your purchase department should have access to their Microsoft-side account representative to get you access to support without breaching legal requirements.
A year or two ago I lost data backing up my OneDrive files to my local HDD. I was told my backup had succeeded, and so I deleted the files on OneDrive. later, when I tried to extract some large files from the backup, I saw that they had been replaced with some text files, containing a message that the download of those files failed.
What the heck... Was I expected to extract the archive and go through all the files one by one (there are thousands of them) to check if every file was properly backed up?
What happened was that onedrive was stuck trying to upload my VMs (I had those in my _local_ documents folder, which onedrive claimed ownership of during reinstallation without telling me). Onedrive never showed my excel file in the list of files it was uploading, so it could have either tried to do the VMs first and never got to the excel file or quietly errored out on that without ever uploading it.
In any case, I'm still dealing with the fallout of the fatal mistake I made: Trusting a microsoft product to do the absolute fucking basics.
Maybe I'll learn this time.
They designed it to look like a differential backup model, but it's not. In fact the only backup you can trust as to its integrity is the first full one. https://helpcenter.veeam.com/docs/vbo365/guide/retention_pol...
“When a retention policy is applied in backup repositories with the Snapshot-Based Retention type, Veeam Backup for Microsoft 365 removes versions of an item, but not an item itself. Data removal from backup occurs every time the restore point of an item's version in a backup file goes beyond the retention coverage. Eventually, if no more changes were made to an item, Veeam Backup for Microsoft 365 will remove all versions of an item except the latest one. The latest versions and items that were never changed stay in a backup repository with the Snapshot-Based Retention type forever.”
I cannot personaly recommend anything usefull to you sadly. Stuff just drifted in completly wrong direction imho. Linus showed how that stuff could be handled by introducing GIT. I went into that direction myself, coding simple DVFS repo manager to handle all my mutable files. But to 99% of people working with computers today its too hard to handle. Heh, we are going backward I think.
Same with Teams. It’s easily the worst piece of SW on my computer, closely followed by OneDrive.
I am convinced that there's a major issue with iCloud Drive and iCloud Photos losing files and photos. I've got more than one folder on iCloud Drive I know had files in them; but nothing is there. I've got old photo albums, from say 2012 or so, that have 6 pictures in them; but the identical album in Google Photos has 9 (I always manually replicate uploads, albums, etc to Google. A backup, I guess).
The one good thing I'll say about Google is; I've got Hangouts chat logs from like freakin 2007 or something in my history. In fact, their data retention may be too good; about six months ago I logged into Google Drive (Workspace) to find a bunch of folders I know I had deleted, years ago, and I mean years, just chilling in the root folder. Stuff from 2010? 2011? It was actually a trip down memory lane, and I'm not un-glad to have recovered some of the stuff for my archives, though its unclear why it happened.
Dropbox as well. I don't use it regularly, but I signed into my Dropbox account from the 00s one day the other day, and everything was just there. For 14-15 years, never paid them a dime.
The fascination with Microsoft in the corporate world is a collective psychosis that we'll probably never recover from. Everything they make is shit. Its insecure, unreliable, doesn't do half of what it advertises, and does the other half so much worse than the worst competition that their products' adoption in organizations could by any reasonable logic be correctly viewed as intentional sabotage.
It still showed links under recent files but the actual files were gone.
Customer support said they can’t do anything about it. 6 years of photos, personal files, university work and co gone.
I happened to have an old notebook with _some_ files on it.
You _cannot_ trust 365/OneDrive.
Ah, there we go! It's their won fault for assuming a filesystem can be used for basic file and folder changes. /s
On a serious note, it turns out that replicated filesystems as seen in OneDrive or DropBox are hard to engineer. Astonishingly difficult to get right. "Formal proof verifiers" level of difficult, otherwise customers will lose data in everyday circumstances. It's happened to me, it happened to all of my coworkers, and it has happened to your customers.
Data loss with a poorly engineered replication system isn't some rare corner-case, it's the typical case, and it is a miracle if data isn't lost!
A few years ago DropBox rewrite their sync engine and wrote a great blog article about all the complex scenarios they had to handle: https://dropbox.tech/infrastructure/rewriting-the-heart-of-o...
OneDrive has done none of this. NONE. I guarantee it.
It loses data if you look at it wrong. If you call Microsoft Support they'll tell you not to look at it like that.
I do all of my work on local C:\ paths and copy files to SharePoint or OneDrive only if forced to collaborate with colleagues. This rare interaction with OneDrive causes data loss about 5% of the time, often enough that I notice.
- When saving a file, it's sometimes not possible to not have access to the file immediately after saving. This is super fun when you're writing software and you save and then can't compile it.
- The setting to keep every file locally doesn't stop files being ejected and then not kept locally.
- Files that are not local take very long to download (e.g. an 85kb file just now too ~60s on a 1GB fiber connection)
- One drive creates duplicate folders and files during syncing errors
- Some bug in making changes to a word doc and then renaming it made the new changes inaccessible by either the old or new name. This was an old issue, but was particularly stressful when I was working on a ~50 page document with a large amount of edits across the entire doc, that had a hard deadline for submission.
- For a period of time I was unable to login with my account to OneDrive using the Mac software. Support could not give me any indication about why the error was occurring and were unwilling to continue diagnosis - the suggestion was to effectively wipe my user profile and start over
- Eventually in some update the sync client started working, and promptly deleted every file in my OneDrive folder before pulling it down again (100s of GB). A stressful few days of wondering which files would be kept / lost at the end
- Sync speed is reduced to 10s of kbits/s when you have millions of files (on a 1Gbit/s connection this is frustrating)
- Putting any git repo or a nodejs project in OneDrive is problematic due to the millions of files issues, generally results in locking up the process
- No tooling to understand how much syncing time is left and see the overall status of large syncs
- No user accessible user interface / logs to help understand why things are syncing / what is happening under the hood (this wouldn't really be necessary if the rest of the software wasn't so buggy).
There are many more examples I could go into all of little things that OneDrive does that a buggy, broken, or terrible. But my general impression is that the quality of the product (at least on the Mac side of things) is not sufficient to be fit for purpose. OneDrive really has 1 main job: make the data I save available when I need it. If fails at that regularly.
In summary, I wouldn't trust OneDrive to not lose data (but haven't yet moved off it due to inertia and the effort required). I would recommend that other users choose anything other than OneDrive if you care about any of your data long term.