Does anyone use a fully Kubernetes-native stack at home/work?

Question

Forgive any potential misuse of terminology (not my primary field of expertise), "fully Kubernetes-native stack" here is meant to mean that all services and infrastructure (or as much as possible) are deployed within the cluster itself.For example, rather than using any AWS/Google managed databases, queues, or caches, deploying open-source varieties using standard Kubernetes objects instead.If so, does anyone have any useful guides/manifests/charts or advice for setting up and configuring something similar? I have a bunch of old devices I've been steadily repurposing as compute/storage for experimentation at home, and it'd be cool to develop closer to a real world distributed environment without footing as large a bill. I would guess I'm not the first person to have been interested in doing this, so at this point just trying not to duplicate any work if this is already a solved problem.

epelesis · Accepted Answer

The keyword you're looking for here is Operator [1]. Kubernetes Operators are custom resources (e.g. Database, Load Balancer, ACME Certificate Entry) that manage themselves via a reconciliation loop. If you're looking for a specific piece of infrastructure, search for ` + "operator"`, (e.g. "Postgres Operator", "Kafka Operator", etc...). Once you find an operator for the piece of infrastructure you need, just follow their instructions on how to set it up and configure it. Learning about how operators work is pretty neat and writing your own is fairly simple if you don't find an operator for your use case.
Personally I run my personal cluster on K3s [2], secure it with cert-manager [3] and route traffic via the Traefik Operator [4]. Right now I use Litestream for all of my DB needs, but I've heard good things about the Postgres Operator by CrunchyData [5].
1: https://kubernetes.io/docs/concepts/extend-kubernetes/operat... 2: https://k3s.io/ 3: https://cert-manager.io/ 4: https://doc.traefik.io/traefik/providers/kubernetes-ingress/ 5: https://github.com/CrunchyData/postgres-operator

geoah · Answer

I guess that would really depend on your setup. It would differ significantly if you&rsquo;re doing virtualised nodes ontop of vmware/proxmox/qemi/etc, if you&rsquo;re doing single node with storage, or if you&rsquo;re doing something more exotic like raspberry pis with either one node with storage or external nas. I have a couple of different setups (mostly for fun), which are pretty simple. Main things I use are a csi driver to allow me to use my trunas scale as storage (https://github.com/democratic-csi/democratic-csi), cloudflare&rsquo;s argo tunnel to expose my clusters to the outside world (https://github.com/cloudflare/cloudflare-ingress-controller), an operator for managing external dns with cloudflare (https://github.com/kubernetes-sigs/external-dns/blob/master/...). Everything else is pretty much they way any other cluster would look.In regards to databases I mainly stick to kubedb (https://kubedb.com/) as my needs are not that great.