services:
proxy:
build: ./proxy
networks:
- frontend
app:
build: ./app
networks:
- frontend
- backend
db:
image: mysql
networks:
- backend
services:
app:
build: ./app
environment:
DB_PASSWORD_FILE: /run/secrets/db_root_password
secrets:
- db_root_password
db:
image: mysql
environment:
MYSQL_ROOT_PASSWORD_FILE: /run/secrets/db_root_password
secrets:
- db_root_password
secrets:
db_root_password:
file: ./secrets/db_root_password.txt
If you would express db as a dependency/resource of app, could you infer that you could put it in a seperate network and have the credentials automatically link? 'As a developer' I'm not really interested in the network specifics or which credentials, I just want them to talk securely and minimize any attack vectors and keep any configuration to a minimum. With tens of apps, their databases and transversal connections, how to do you keep the configuration to a minimum?
Googling around I found;
humanitec: https://docs.humanitec.com/integrations/resource-types/mariadb They express something as 'resources', a dependency type that can be provisioned automatically with inputs and outputs that then can be injected in the application env vars: mysql://${externals.my-db.username}:${externals.my-db.password}@${externals.my-db.host}:${externals.my-db.port}/${externals.my-db.name}
you're limited to a limited set of drivers ofcourse and how would you express an app1 to app2 dependency?
juju: https://juju.is/
Each app is packaged in a charm which seems to be a yaml declaring inputs, dependencies and other meta data and optional python code that can respond to certain lifecycle hooks
https://discourse.charmhub.io/t/implementing-relations/1051
name: my-node-app
...
requires:
database:
interface: mongodb
provides:
website:
interface: http
Does anyone have experience with these tools or others like it?
How do you stop others and yourself from drowning in credentials, certificates, env vars, jvm params and k8s yaml(times every environment) How do you do (not the configuration management) but manage an inventory of what configuration is needed to run your environment (or a subset for ci)?
I'm using docker swarm/compose extensively for multiple years and configuration can be a bit verbose sometimes but it's very simple system to maintain and write config for.
I think, splitting relatively small configs into smaller bits and using additional abstraction can actually increase complexity of the system compared to having everything in a single file as it's basically write and forget and not something you modify heavily once system is mature.
> “you can solve every problem by adding or removing an abstraction layer”
Have you considered generating your YAML/JSON config with something that composes?
If you are open to it you might be interested in dhall [1] as it’s a config language with variables, functions and imports.
I have used it for pet projects and I could see how it could offer some tidy encapsulation patterns for larger, more complicated production applications.
While Docker Desktop / Podman / Rancher Desktop combined with stuff like Skaffold aren't exactly a drop-in replacement for docker-compose, it does do a much better job at bringing up and tearing down entire compositions while re-using existing packaging and access controls.
If you are running docker-compose for non-development things, it might be a different story; it might be suitable for non-GitOps things, but as posted elsewhere, at that point you're better off using something like systemd.
When composing really small setups I either do this with a shell script (think 10 lines including trapping exits) or a systemd unit. Whenever it needs to be bigger I nearly always end up with an actual scheduler (K8S, Nomad) and GitOps because you can't really deliver something maintainable, available and durable anymore without it (well... I suppose if you have only 1 project to deliver, forever, you could manually manage it).
It does get a whole lot easier when you have a common foundation you can re-use. Spinning up an entire stack with security, monitoring, alerting, automatic rollouts/rollbacks for even the smallest project is just single-digit minutes work now.
Pulling in some other factors: how sharable/collaboratable is something these days if it is not built on similar enough technologies and modules? A solo yolo project might not care much about this, but when was the last time someone asked for software that is risky and not durable?
This will bring you closer to the deployment stack if you are deploying to Kubernetes. Then also let you leverage tools like kustomize to dry out your configurations.
There are some great projects like tilt, devspace, skaffold, etc that help facilitate deving on a local or remote cluster.
As far as configuration management that can be as simple as cascading kustomize configs or helm. Then leveraging something like vault. The point really is, if you start with Kubernetes you have way more flexibility with tooling and options to do whatever the heck you want.
Shameless plug I recently started a series on local Kubernetes development. It covers some of this with tilt and more. If you would like a specific thing covered here I can add an installment to it. https://youtube.com/watch?v=Nj55RDVwrIE&si=EnSIkaIECMiOmarE
https://docs.docker.com/compose/environment-variables/
- `${VARIABLE:?err}` exits with an error message containing err if `VARIABLE` is unset or empty in the environment.
- `${VARIABLE?err}` exits with an error message containing err if `VARIABLE` is unset in the environment.Arion docs: https://docs.hercules-ci.com/arion/
Nix language: https://nixos.wiki/wiki/Nix_Expression_Language
I then disable iptables in /etc/docker/daemon.json: "iptables" : false to prevent docker from poking a hole in my firewall. (This should be on by default..) I also enable User name spaces with this: "userns-remap" : "default" so that the docker image runs isolated. I can disable this in my docker-compose with userns=host Actual instructions are slightly different, but: https://docs.docker.com/engine/security/userns-remap/
I also found I could disable networking all together on some services, and just expose a port. Some services don't need to connect to the net.
Then I run a cloudflare argo tunnel to the application. But you could also use a nginx reverse proxy, some other tunnel, or a one page express.js reverse proxy with pm2. There's a cloudflare argo tunnel docker-compose setup which allows you to type in your tunnel information in the config. I thought that was cool but I did not try it.
The first solution I happened upon was serverless. Specifically SST, which is written with AWS CDK, but you must develop on live services and I just can’t justify paying to develop.
Then I found Serverless Framework, which is an abstraction on CloudFormation, but the offline solutions like localstack get a lot of flack for being buggy and localstack charges for some services. I also looked into Architect but the documentation is abysmal.
Then I figured serverful might be the easier way to go. I found that docker compose has a built in integration with AWS ECS where it transforms your yaml into Cloudformation to provision the right services. However, it seems to just be missing key parts like custom domain and SSL certificate provisioning which seems to defeat the IaC ethos.
Then I figured I might go with Terraform and I found some seemingly good starters like https://github.com/aws-ia/terraform-aws-ecs-blueprints https://github.com/cloudposse/terraform-aws-ecs-web-app https://github.com/turnerlabs/terraform-ecs-fargate but the examples are just lacking. They don’t have any examples for multiple containers that can access each others’ resources that I can find. Reading these templates has at least given me a better idea of the resources I need to provision in AWS but the networking and configuration still frighten me. Like do I need to configure nginx with a reverse proxy myself? How do I orchestrate that container with the others? And apparently services can crash and just not restart? And I need to make sure to configure volumes for data that needs to persist. And setting up the CI/CD seems daunting.
I’ve also heard about docker swarm, kubernetes, pulumi, AWS SAM, etc but it’s a lot to learn. When I go on Discords for web frameworks, mostly everyone including the devs of these frameworks use 2nd tier managed providers like Vercel, Fly, Netlify, Supabase, Cloudflare, etc. But many of those are just not as reliable as core cloud providers, the cost is way higher, and now you’re setting up a local stack that probably wildly differs from how it will work in production between those services. Glad to see I’m not alone in a very reasonable expectation of a simple way to orchestrate multiple containers on AWS, what must be the most common use case web developers have
I was hoping something like https://dagger.io/ might solve this but they currently don't support compose files, or maybe https://www.acorn.io/ ?
It does feel like there is a legitimate space for a multi-container service application standard that isn't just kubernetes, unless perhaps things like k3d and docker desktop end up so good at running tiny k8s that docker compose dies.
Also I'm not sure it really matters; again this would help to understand your question better, because I think you're trying to solve a problem that may not exist in your threat model.
[0] https://medium.com/@kinghuang/docker-compose-anchors-aliases...
No, sorry.
The way it works in the cloud is, you build some infrastructure, you give that infrastructure the ability to contact a secrets management service, and you either A) configure a container orchestrator to look up a secret when your service starts and inject it at start time (either as an env var or as a file), or B) your service itself looks up the secret using permissions inherited from the environment it has started in (such as an instance metadata service).
You could fake all of that in a crappy way by running an additional service which is just a web server with a complicated random URL for each secret you want, configure each service to declare an environment variable which is the complicated random URL, and have each service curl the URL at start time to retrieve a secret. But that's not less configuration, it's slightly more. The only benefit is that you can kill the secret service after the URLs have been grabbed, leaving the secret only in memory (and assuming the URL was rotated every time this whole thing started, would make them temporary, sort of).
But, as a hacker, that would all only present small challenges which I would eventually work around once I find an attack to read from memory or execute code. So unless you need super duper extra security for some reason, just use what you've got.
Secure from what? What is your threat model? It's hard to imagine 'default' security because everyone is worried about different things.
> I just want them to talk securely and minimize any attack vectors and keep any configuration to a minimum
Your application is more likely to expose a SQL injection vulnerability than someone gaining unauthorized access to your network. I'm not saying you shouldn't secure it, I'm just saying that it is probably pointless since your application has access to :all-the-things: and it is probably the weakest link.
Further, if you're doing this on a developer machine, you're just making it harder to debug issues. As the user of the host machine, you have access to everything by default.
> How do you do (not the configuration management) but manage an inventory of what configuration is needed to run your environment (or a subset for ci)?
Documentation, documentation, documentation. Basically, have a markdown file with each config item, it's value in each environment (not literal values, but ex: 'github token for production, with X,Y,Z scopes'), why it exists, and who is responsible for it.
" Dependency Management
Standing up a service typically also means standing up a series of related services. If an application has dependencies on other applications, wouldn't it be nice if those dependencies (and any transitive dependencies they may have) were automatically instantiated by the cluster-management system? "
As docker compose is only for single node deployments, how is your workflow from dev to production? If your containers are deployed in something like Kubernetes in production, isn't inconvenient to use docker compose for development?
The gist of it is Traefik is a reverse proxy that learns about your containers via configuration in the container only, so you can stand up additional services without rebooting everything. Still config heavy, but maybe less than some other approaches.
You can also dynamically build a compose file for example:
export COMPOSE_FILE="$(ls -1 services/*/compose.yml | paste -sd ":" -)"
In general I don’t have many issues with the docker-compose.yml format.
It sounds like a simple straightforward task.