Have any of you done this? What has your experience been like? Have you found a way to make this work with multi-player? What storage provider has the best API for this?
OpenDAL was on HN recently and would be a pretty decent abstraction to use for this: https://github.com/apache/incubator-opendal
Yes, using a cloud storage provider might work instead of running your own database...
> Have you found a way to make this work with multi-player?
... But this is where you will run into serious problems.
"Cloud drives" aren't designed for highly concurrent multi-user access. That is when you need an ACID-compliant database; (or otherwise understand this issue well enough that you know what you're doing.)
In our case, our API was based around light usage of files: If two people edited the same file at the same time, we couldn't resolve the conflict automatically, because these were files: We had no knowledge of the files' schemas.
Likewise, our APIs weren't designed for "live" high-performance usage. In our case, if it took under two minutes for an update to sync, we were happy. That's an eternity in a multi-player game.
In general, if your application is going to use a cloud drive as its data storage, you should target light usage of storage needs: IE, the user's data is generally private, the user is generally generating light changes, multi-user usage doesn't require heavy "live" data.
IE, this would work great for a password manager that allows sharing passwords via the cloud drive's existing sharing system. It would also work great for an online image editor, sheet music editor, anything where you could envision a desktop app that edits files.
- Having the entire product be reliant Google is just too big a risk. Google has changed apis, and restricted access to third parties.
- Lots of added complexity around search, pagination, etc.
- Due to the last point, all the files need to be tracked in an external database. This adds a requirement to keep everything in sync.
- Now that the files are not stored in Google Drive, I’m working on adding an option to support auto backups to Google Drive.
A while back I looked into the feasibility of accessing various APIs from the browser directly, and I ran into a dead end. Everything I found has CORS restrictions that prevent this from working. If anybody knows of anything, I'd love to take a look at it.
- http://scripting.com/2015/06/25/dropboxCouldBeKingOfTheOnepa... ('Dropbox could be king of the one-page app', 2015)
- http://scripting.com/2023/08/05/130423.html ('Identity as a product', 2023)
Technology-wise it is just a Java app running on my raspberry and using Google's libraries for Drive [2] to create the spreadsheets and the actual Sheet API [3] to modify content. Authentication was a bit hard to figure out mostly because of the poor documentation. Each query to these APIs takes about one second, so it is definitely not suitable if you require low latency on the client's side or lots of concurrent accesses to the same spreadsheet. Not sure about pricing since my bot is not so popular and I am still in the free tier (:
[1] https://t.me/gsheet_notes_bot
[2] https://developers.google.com/drive/api/guides/about-sdk
[3] https://developers.google.com/sheets/api/guides/concepts
The huge advantage is that most people already have some kind of syncing solution set up, and they can just use that. This is especially important for corporate environments where IT has to evaluate what's allowed and what isn't.
It comes with some drawbacks. Your code needs to be resilient enough to deal with inconsistent syncs, you have to use file system observation, stuff might be delayed etc.
But on the plus side, it seems to work very reliably, I've not gotten any complaints yet, and if something does go wrong savvy users can just fix it themselves. It's just a bunch of JSON files on disk, after all.
Most sync tools offer a way to share folders with other users, so if you do it right you get multiplayer for free.
If you have a suitably robust "middle ware" layer this should be doable with any type 1 storage back end.
If you want to save your game progress, you just login with OneDrive.
I previously made a single-page EPUB reader using a similar technique with GDrive: https://h5reader.azurewebsites.net/ (that one does not work with 3rd party cookies disabled though because it uses iframe to login instead of redirects, and I never bothered to update it).
My experience with Drive OAuth is sketchy Colab notebooks asking me to authenticate with Drive, but in such a way that I'm apparently granting access to everything inside of it. I've never been sure if that's because of limitations on the OAuth API, the Colab client specifically, or just sloppy developers.
Is it user-privacy focused? Product cost-savings focused? Something else?
The privacy discussion would be interesting. On one hand, users "own" their data in the sense that they can delete it themselves at any time, but you also allow a product (hopefully limited) access to your own storage provider.
Dropbox has fallen so far in my eyes over the past ~decade. It used to be rock solid file storage, a USB drive in the cloud, now it's so busy trying to be everything to everyone and failing at most things. The macOS is hot garbage, literally one of the worst pieces of software I've had to work with. It's buggy, doesn't give useful info, gets stuck syncing, and has alerts that you can't act on. My mom's laptop yells at her that some files might not be openable and she should upgrade to their new file provider API. Ok fine, let's do that, oh wait you can't because you have a personal and a business account, try again later. It's been this way for a while now (like a year+).
I have Maestral on my laptop but honestly I just need a tiny push to leave Dropbox completely after being a paying customer for over a decade.