What has been your experience with CockroachDB?

Yeah I’d be glad to talk about it. Was part of a team that was asked to design a whistleblowing software platform for large corporate and government customers. The rest of the stack was RDS Postgres but this whistleblowing app had super different requirements. We would need to run in basically every region and also not exclusively in AWS.

After trying some different options it became clear that Cockroach was the best option by far. The strongly consistent nature of Cockroach made it a good choice for a SoR like whistleblowing unlike say Mongo. The ease of deployment and the stability of the Raft consensus protocol for managing ranges of data was better than expected.

For me having done a global rollout of it, I’d recommend it with very few caveats. You do need to train developers on what queries it can and cannot support, since it does have Postgres client support but obviously cannot handle the full range of queries. You also need to have lots of alerting around the Raft log, since that is your bread and butter in terms of “is this working or not”. However in aggressive testing, removing nodes with no warning under load, simulating pretty severe networking problems, etc, it has held up remarkably well.

Maybe people who answer/read this also evaluated yugabytedb, which has a much friendlier license (apache 2)? We are evaluating different solutions and we depend on some Postgres compatibility as we have a lot of code to that effect. Yugabyte and cockroach come up all the time.

(note the username!)

Check out Aerospike. It's the most robust and truly "zero effort scale" real time data platform I've ever seen. Properly configured and used, it whups everything in terms of TPS and performance/machine aka TCO. By a huge, huge, factor.

Downside is ... everything else. :/ Less polished developer experience, docs, &c. There are so many huge customers, but very few who are talking about it. Requires more of a time investment -- because you're learning a new platform, not using a drop in replacement for what you already know. But the reward is performance beyond anything else.

(the VLDB 2016 paper is a good high level overview of the architecture)

If you do check it out, be sure to use version 6.1 or later, in particular see what can be done combining secondary indexes with container data types.... A, uh, birdie told me that. :P

edit: Same birdie told me Aerospike has customers who replaced CockroachDB, when they found out it has trouble scaling past a point.

I really hope they can get their act together in terms of communication and marketing -- it's sadly "the most advanced solution you've never heard of".

I know of two scaleups that have left it. As usual, this might just be poor implementation choices rather than product or support issues.

Launched it on a t3.nano (0.5GB RAM) in a single node configuration for two extremely small tables - got oom kills in couple of hours.

Launched on a t3.micro (1GB RAM) - same.

Launched on a t3.small (2GB RAM) - finally it works and doesn't crash.

So, min 2GB RAM to serve about 10 (as in: ten, not thousands) records queried every other second.

Why don't you try swapping your existing PG server with CockroachDB and see how it goes.

Being a die hard fan of Postgresql for more than 15 years, I think time has come for me to select CockroachDB as new default.

I miss Table Partitioning (only available in enterprise edition ) and Full text search (yet to be built). Also minimum hardware I had to employ was quad core and 8 gigs of RAM.

But other than that everything is so smooth including installation, online documentation. It made me lazy.

We tried it for a payment system, the concurrency issues where an absolute nightmare to deal with.Especially in combination with Hibernate. In serializable isolation level some queries would just never finish, no matter how many retries you gave it.

Haven’t used them in production because their license doesn’t allow operating as a service.

I have used an alternative.

All these comments and still no one really talking about their experience... For what it is worth, the biggest put off for me was that they no longer allow to export data in PostgreSQL compatible way, i.e. You are locked in when you start using them.

Yes. AMA and I’ll do my best to answer.