What does HN think about this? Is there a way of looking at "cloud agnostic" that still allows you to look at the cloud as an application platform and not just as a platform for running Kubernetes? Is there another way to think about this?
As an example, let's say you can deploy your app as containers orchestrated with SystemD on an Amazon EC2 instance. Switching providers will require you to rewrite your whole deployment procedure but not the containerized application code. Your application is cloud agnostic, but not your deployment pipeline.
Now, let's say instead, you have an Helm chart to deploy to k8s, you can move from Google GKE, to Azure AKS, to Amazon EKS, to baremetal k8s, without modifying your Helm chart. Your package is cloud agnostic, but not your infrastructure.
If your application requires a PostgreSQL database, you can provide it with Amazon RDS, or by self-hosting it with KubeDB in a k8s cluster, or any other solutions. To your application, it would be changing the `DATABASE_URL` environment variable and running a migration script from the old DB to the new one. There is no true lock-in in that specific case.
In other words:
- yes, using Kubernetes can facilitate the deployment of your app on your infra in a "cloud agnostic" way
- no, using Kubernetes does not mean your whole infra is "cloud agnostic"
- it's ok if parts of your infra is not "cloud agnostic", that does not mean you're "vendor locked"K8s is the closest you are going to get to putting your entire application on a USB stick and plugging it into any computer to run. It is currently the de facto app for modern applications. It’s missing a lot of stuff, but it’s a good way of packaging things and with CRDs you can bolt on the other things you need and make them portable as well.
Whenever the majority of the industry agrees on an abstraction level we are able to make lots short term improvements. I say short term because obviously platform hegemony is often a bad thing, but it frees us up to focus on one area of the stack. So if you can build a great app with k8s and move it between clouds, you have sufficient lock-in protection for a provider. When someone improves on k8s they will create a mostly clean interface so you don’t have to do a ton of things to port your app.
Look at Docker as an example. Everyone thought that was the greatest thing in thr world, but once they have everyone a clean abstraction layer they setup a ton of other container runtimes to slide/shim in and now the runtime layer is being well commoditized. The value for most people and businesses is at the top of the stack.
Here's what makes sense to me: 1. Design your system as components that communicate via interfaces you can implement easily anywhere. In other words, don't design your system to use a proprietary system to communicate. If you can't easily reimplement dynamo or s3, don't make that an inextricable part of your system between your services.
2. Use the cloud services that make implementation easier inside your services. The idea is to reduce the implementation effort/investment as much as you can. Sure, porting to something else could be costly later, but you are saving real effort now. When and if you do need to reimplement on another provider, attempt to do the same thing.
3. If your business doesn't really need or want cloud, don't do cloud. Doing cloud half-way is worse than a well thought out alternative.
Kubernetes is a waste of computer resources if we are speaking about production deployments. What is your container distro? Alpine? Just run your app on alpine! It’s not that complicated. Networking will actually function properly and safely and you don’t need Kubernetes to have declarative infrastructure as code. Addicted to yaml? Ansible uses YAML, can reproduce your build steps and doesn’t leave detritus like an agent nor waste any target resources.
Avoid containers unless you have a specific reason for using them. It’s a silly trend, IMO
Kubernetes will not get you either. You still have cloud specific to stand it up and more.
The short of it is that you cannot be cloud agnostic without significantly more code and complexity. Best to have a few configuration points. Even with Kubernetes, you will still have cloud specific pieces. Without, you can still get many of the benefits with Packer + Terraform.
A good question is what is the purported benefit of going cloud agnostic. If it's to switch clouds, that is not a good one. You won't do it often enough to get an ROI. What is the cost to develop and maintain this posture? (significant)
Run k8s on bare metal with metallb, and something like ranchers longhorn. Then, when you've got that working, move it into the cloud. Run it all on VMs and not cloud-managed k8s. Then you probably still aren't quite there, because IAM and S3 and so on.
In short being cloud agnostic is dumb. Just know that you will have some vendor lock-in.
Pick how much you can tolerate, and live with the rest.
IMHO, it's a stupid constraint. Just pick a provider and go all in.
If not, get the definition from management and skate as close to it as you can.
Our level of abstraction is containers, we don't use services that cloud providers sell, for example, Lambda, as they are very cloud provider specific. We prefer to use technologies like RabbitMQ which can be deployed on any cloud or on prems easily.
This is probably the minimum. What else does your solution include? Does it include hosting, infrastructure as code is it a managed service, something that clients run themselves or a SaaS?
Generally if you keep deployment simple you don't have to use docker and kubernetes. For example if the app code is delivered as a monolithic .jar file or native executable and you don't have a large number of 3rd party services that need to be deployed.
Docker and Kubernetes do help be cloud agnostic for a particular type of solution architecture - many microservices and potentially many other requirements like queues, caches, 3rd party including open source applications. If you had to create this distributed system on different providers it would be a lot of duplication of effort. In k8s there is still some - you most likely have to configure docker repositories, storage, load balancers, ingresses for different providers but configuration is better than reimplementation. And you still have to manage the cluster. So depending on your architecture and application requirements you may find it much simpler to use Docker and Kubernetes or them to be a needless complication.
There are also certain best practices that help with portability - the 12 factor is a good guide.
If you're looking for alternatives, or something lighter weight than Kubernetes, I've used Nomad (plus Terraform and Ansible) and some shell scripts to get repeatable clusters deployed and migrated between cloud providers: https://www.nomadproject.io/
OpenStack is somewhat EC2 API compatible, but the reverse is not true.
What's the reasoning for that requirement, though?
I understand the fear of vendor lock-in, but if AWS decide to put their prices up tomorrow, it's non-trivial to just move everything to Azure (for example). Sure, you have some container images which are portable in theory.. but it's still a large undertaking.
I'd be interested to know if anybody has built with 'cloud agnostic' in mind, and actually needed to migrate to another provider.
Unless you expect to migrate between cloud providers relatively often, and/or have built your application to be "cloud native" up-front (implying low cost to move to the cloud), it may be cheaper and swifter simply to perform a one-time rewrite and redeployment once you've chosen a desired deployment platform.
However, it is one of the only technologies that pretty much is available on every major cloud provider in the world as a managed service with guarantees of compatibility
- Terraform
- Ansible
- Systemd (sometimes Docker)
This way the only thing that changes between providers (e.g., moving my whole infra from digital ocean to linode) are the terraform files.
Do I need a load balancer? That’s nginx on a droplet (terraform apply + ansible-playbook). A database? That’s mysql on a droplet (same commands to deploy and provisioning)