HACKER Q&A
📣 atombender

What do you use to run complicated, multi-node integration tests?


We run big integration tests that stand up multiple services (Postgres, etc.) plus our own code in a Kubernetes cluster, then run lots of tests on them. Some of tests are stress tests that put a bunch of load on them, some are benchmarks, and some are pure integration tests.

We currently use Argo Workflows (to express the pipelines) plus Argo Events (to start automatic tests on GitHub webhooks), which allows us control over the Kubernetes aspect, but it is a mess of YAML, and it requires us to first put the workflows into Kubernetes as "workflow templates" that are then executed to generate the running "workflows". Argo is also quite flaky and buggy. I've been thinking of using Jsonnet to make this less of a mess, but that doesn't solve all the complaints.

We also use CircleCI for some stuff, but it's much too limited; no support for running multiple nodes, for example. We've decided to just use it for normal building, unit tests, and linting.


  👤 verdverm Accepted Answer ✓
Jenkins is the most battle tested and flexible build / CI system. In the end, you can have your (any?) build system create VMs outside of the build system. This could get expensive in CircleCI (having a long running build which is just waiting), maybe more feasible if you are self hosting Argo. You could of course make it a more manual thing. Typically it becomes overdone to do this thorough of testing on every commit

Yaml mess? See https://cuelang.org