Excluding hosting providers (e.g. AWS for EKS, Google for GKE, etc) how many Kubernetes clusters does your company operate, and what information can you share about them?
Example:
- How many k8s clusters do you have?
- How many VMs in total or typical cluster size?
- How many containers do you run?
- Why so many? (e.g. lots of customers, lots of internal developers, cell-based architecture, etc)
- Do you run on EKS, GKE, AKS, OpenShift, bare metal etc?
- How do you manage them (Rancher, Crossplane, Terraform, etc)?
- Is the platform growing or shrinking?
- What's the split between development & production?
- If you're able to share the company name, great... if not, can you explain the space you're in (e.g. SaaS provider, AdTech, etc)
- For those at scale (more than 500), what have you found that a) works well b) doesn't work well?
To get things started, here's some information I can share about my company in relation to the 2021 CNCF annual survey...
https://www.cncf.io/reports/cncf-annual-survey-2021/ (about ~2,100 responses)
About 6% of respondents had more than 5,000 nodes (we have over 50,000 - over 10x that)
About 9% of respondents ran more than 5,000 containers (we have over 500,000 - over 100x that)
About 6% of respondents ran more than 50 k8s clusters (we have over 2,000 - over 40x that)
Thanks for sharing!
Then each "application" gets a namespace. It can be tricky to decide which containers fit into which application exactly. It turns more into logical organization at that point.
If it's a startup / small company you can throw everything into one cluster and divide it up by namespace. Not ideal but not the end of the world.
Our other cluster houses QA/testing envs alongside our DataSci/ML workloads. That cluster can be running anywhere between 8-200 nodes, with 50-2000+ pods at any given time.
We have run into IP address allocation issues, but this mainly an issue with how we set up our VPCs.
Adtech.