Is there any big project trying to make VM APIs standards?

Question

I wrote these three files a year ago when I needed to create VMs at various cloud providers (linode, vultr, digitalocean) through their apis:https://github.com/andrewarrow/wolfservers/blob/main/linode/create.gohttps://github.com/andrewarrow/wolfservers/blob/main/vultr/create.gohttps://github.com/andrewarrow/wolfservers/blob/main/digitalocean/create.goNotice how each is slightly different and they don't all agree on the image name for some standard image and size. I'm wondering fellow hackers, has anyone taken this type of project on and made progress to have a standard way to make a VM? i.e. I should be able to spin up a VM at digitalocean setup just the way I want and then, boom, do the same thing at vultr without any extra work. The end goal being to run my code on these VMs directly, avoiding the extra layer of a container running inside them.

dimitar · Accepted Answer

There isn't a standard, each cloud tries to lock in engineers to it, so they don't compete on price. Any abstraction that is build on top of the different APIs will leak as the implementations are not exactly the same. That is why tools like Terraform just provide a consistent way of working, but then you steel need to learn their APIs as Terraform providers. I think tools like https://jclouds.apache.org/ have attempted some abstraction, but they also have to allow vendor-specific code, so I don't know how much of it ends up being used.One way you can avoid this problem is by running Kubernetes - then you need to only have code to set it up for each provider and write k8s config that is common to all. Kubernetes is pretty complex though and might not be a good fit for all applications.

dijit · Answer

Compared to how I do things: You're thinking of it in too few steps.
I have always written my deployments in stages:
Stage 0: Set up accounts, API's, access (usually terraform with a little bit of seed information)
Stage 1: Create any VM's necessary (or managed services, like databases) [terraform]
Stage 2: Push any config files that might be necessary, this is provider agnostic [Ansible/Puppet/Chef/Salt]
Stage 3: Push your business application to the machines.
---
The reason I divide things up into these stages is because they operate at different cadences (you don't enabled API's or create VMs at the same speed as you develop new code) and with different specialised tooling (making ansible do what terraform does is a lesson in frustration, and vice-versa).
The benefit is: You can substitute all of Stage 0 and Stage 1 and keep Stage's 2 and 3 relatively unchanged.

wereallterrrist · Answer

I think pulumi has a but of that but i also kind of think there's a reason it doesnt exist. I mean even a "cross-blob-store" client is hard or impossible to come by.If you're writing go though, you can probably leverage docker-machine which i would have to guess has such abstraction internally.

meowxiik · Answer

Vagrant or Hashicorp's Terraform?