HACKER Q&A
📣 kvdr

What is the current state of art in distributed databases/datastores?


I had gone through a survey course while in grad school a few years ago. Riak, HBase, CouchDB were the shiny new things. I kinda lost track of things after school, but want to check back in again on what is the latest.

What is the current state of art? Is there a book that I can read up on this or better yet, some academic course/offering that covers this? Mostly looking for what design decisions/algorithms/data structures used by the databases. Is the Klepmann book (DDIA) slightly out of date now or still very much relevant?

Thank you!!

  👤 brudgers Accepted Answer ✓
The big change is Jepsen, https://jepsen.io

CAP tradeoffs are better documented.

And there is more to go on than marketing claims.

Also, SQL is the new NoSQL.

👤 diehunde
For internals and technical aspects, checkout CMU's Database Systems lectures on Youtube. They also invite developers from new databases to explain their main ideas.
👤 ddorian43
Check out NDB Cluster (Rondb) and the blog of the creator, example of a thread pipeline to be more efficient than Redis/ScyllaDB per-core sharding: http://mikaelronstrom.blogspot.com/2021/03/designing-thread-...
👤 max_hammer
`snowflake` is pretty good MPP database. Buts it's a managed service.

Few advantage over traditional MPP

1. You can clone prod DB for testing with no additional cost.

2. Time travel. No need to take manual back.

3. Good integration with AWS S3

4. Can scale horizontally and vertically on demand

👤 marklit
YouTube runs an in-house database called Procella. Its feature set is pretty amazing. Some of the devs behind it came from the Hadoop world. Google published a paper on its architecture.

Klepmann's book is still a good read. A lot of the concepts are evergreen.

👤 randomopining
Apache Ignite