Why do some companies build their own CICD orchestrators?

Checkpoint is a deploy system, not a CICD. A quick Google finds a nice description of it: https://slack.engineering/deploys-at-slack/

That said, once you step away from webapp world, a lot of times all existing CI/CD systems are pretty terrible. For example we have pre-merge tests that need a few tens of cpu-hours spread across a dozen machines, and running natively (no docker). This rules out pretty much every CI/CD system out there except Jenkins and Buildkite, and both of them a pretty horrible at it unless you spend lots of time with custom scripts. At some point it will become easier to rewrite entire system rather than try to fight existing one.

Maybe the question is really "why do some companies build a new in-house technology rather than use an existing one"?

1. Some companies are so slow and some teams so agile that creating a new working project takes less time than submitting a ticket for permission to install and demo software and get the required permission to purchase it. 2. Developers hate to deal with salespeople, so unless the CICD platform is OpenSource / free the developer might choose to build something simple. 3. Developers might be unaware that a solution exists to solve their specific problem. 4. The commercial solution to a small developer's problem might be too expensive.

The developer estimates of time and money for building something in-house often neglect the "hidden costs" ... (including the long tail). The hidden costs often include : 1. opportunity costs - a developer can't do two things at once, so if they are writing a tool, they are not releasing new features or fixing defects. 2. Maintenance costs - keeping the thing running (when the host OS is upgraded, the network changes, the repository credentials expire, etc) 3. support costs - if this touches any other outside teams or customers, someone will have issues and need support. Maybe a new team has to be created to handle the Zendesk support tickets, provide Slack help, etc.

CICD can get out of hand pretty quickly. If an off-the-shelf solution works for you with minimal changes to your code and workflow then that's fantastic. Ofttimes they're simultaneously clunky and full of unnecessary (to you) features while not supporting the few things you care about. Writing a general purpose CICD probably isn't a good idea for most organizations IMO, but writing a CICD that handles just your use cases is super easy and can take less time than writing the workarounds necessary otherwise. Plus, when things inevitably go wrong the code is right there for you to fix.

The second you notice your engineers are writing anything like custom batching logic to get around CICD limits, custom change detection because the general-purpose solution can't appropriately recognize transitive dependencies, ways out of the CICD sandbox to interact with some raw hardware device with vendored blobs that only work on XP, or anything else fancy then that's a strong indication that you would have been better off doing it yourself from the start.

I built one for a client. It was a rocketchat bot taking commands for making a release of a specific tag or latest master branch, a revert, showing what is currently deployed and printing a commit log vs previously deployed commit. Worked like a charm. Was a set of ansible scripts called by node. No need for anything fancy.

Oh and it always expected a confirmation.

There's an article here explaining one reason why they put Checkpoint in front of Jenkins: https://slack.engineering/circuit-breakers/

It has been tried, for example making diagrams in Rational Rose and generating C++ code from that. Then I assume filling in the code "details" manually. Probably someone else can comment on how that has worked.

That was before agile or TDD were such big things.