This idea you should use docker or other container solutions when there is no compelling and logical problem they are solving is just a time suck. As a solo founder your time is super critical to manage closely, optimizing things that are not high paybacks is a bad idea.
Docker can be great, just IMO and especially for a solo founder you need a compelling reason(s) to add any complexity and/or extra time to your system. Especially if your solution is a simple project that you deploy on a single VM (or maybe multiple for reliability) there is no reason to use docker.
I've been a solo founder more then once, guard your time carefully. With some exceptions, only build things that have an immediate payback, use only enough tools to get it done and don't add any complexity or dependencies that can cost you extra time to debug, understand or learn. As you add people you can do more and be a little more aggressive on things that don't have immediate need/payback.
It's better to religiously build simplicity, repeatability and fast automation into everything you do without it. That has a much better ROI on a product build because you have to do all those things anyway to make docker and kubernetes effective.
The key thing to do is to build on technology that is portable, compact and repeatable which means at the moment something entirely self-contained (Go, .Net Core, Java, Python with some caveats come to mind)
Docker isn't a complex tool and it doesn't take 200 hours to learn enough to be proficient. I have to imagine all of these folks tried to learn Docker 5 years ago and got burnt and refuse to try it again.
At its core, you create a Dockerfile which builds on top of a base image. You tell it what files to copy over and which commands to run to get the system in the state you want it. Then you build the image which runs through all those steps and gives you this reusable snapshot. You can create containers using this image locally or you can push it up to a registry and let any of your systems run that exact same service.
You absolutely don't have to use Docker, if you're happy using plain VMs why switch?
Kubernetes certainly takes longer to learn so I can understand why there is more frustration around it. Though once you learn it, it feels like you've got a super power. Being able to abstract yourself away from the underlying infrastructure is freeing.
My co-founder and I originally tried to cut costs by using Terraform + Ansible to provision and maintain all of our infrastructure on basic VMs rather than using Kubernetes which is always more expensive. But we quickly decided it was worth the extra expense to use K8s for all the nice things we get with 0 effort.
All that being said, all of us techies get way too in the weeds on all of these things that don't matter. Don't listen to what others are telling you that you need to use. If you prefer to install apps directly on your VMs, keep doing that. If you find Docker gives you a better QoL like it does me, use that instead. All the time you spend on infrastructure work is time that you could be working on new features!
Having said that I can tell you for a solo cofounder some of the benefits of using docker.
- Documentation of what environment you have to setup is time consuming / error prone and you will tend to less of that as single dev . That Makes it harder to hire and train new people or remember for yourself what exact version of something you used 6months back. Dockerfile is a great way to do that
- HA and orchestration tooling with k8s or managed container platform services is not easy without defined environment like docker. That means autoscaling, automatic recreation/reboot etc can be handled 3 am in the morning without you waking up.
- Scaling 1-2 is harder than scaling 0-1 . Setting up your environment to be treated like cattle and not Pets from day 1 goes a long way in that.
Growth article by pg: http://www.paulgraham.com/growth.html
2. Time is the most precious resource at your disposal. Don't spend it on learning a new tool/tech unless you don't have any alternative.
3. Why fix something that's not broken?
In webdev, most of the complexity I find is either in the app itself or the aspects of deployment _beyond_ Docker.
I usually set my Dockerfile up at the start of a project and use the image it produces across dev, testing, and prod. It takes me only a couple of minutes to set up and only a couple of minutes to (infrequently) modify. Even when I got "ambitious" with my Docker setup, the time investment was still same day.
Cloud providers also have a lot of easy-to-use products which work with containers right out of the box. The same image would work on Heroku, Google Cloud Run, and AWS Fargate. I don't need to make any changes to my app outside of providing different environment variables, just provide the image and we're good.
On the complexity side, there was definitely some learning to be done, but I think learning docker enough to get some work done takes 3 hours at most. Understand the concept, how building works, how networking works, and you've got a working knowledge.
I don't use k8s or any composition software, I just run my images as if they were little processes with their external dependencies (e.g. wsgi server) included.
Also, if you want to use a PaaS, many of them accept a docker image, so that makes things way easier to deal with and lets you use anything you want, compared to having to conform to a project format.
All that said, if you don't think you need it, don't bother. It's not a game changer, but it's handy for some. If you have a good way to deploy your applications consistently (there are countless), then go with that. If you're interested in docker, learn it. In the end, choosing docker or not choosing docker won't be the thing that makes your project successful.
The catch is that you have a working system you are happy with, whereas using Docker introduces a learning curve (a time sink) and a configuration/testing cycle (at the least) that will not benefit you at all in your current use case. I say that authoritatively, as if it were not the case then you wouldn't be 'happily using generic VM' and would, instead, be asking about whether Docker would solve your ongoing issue.
---
On a more expansive and general note about Docker, it has its place.
I've done Docker, but I've also done stuff ranging from the simple Git push-to-deploy, through third parties like Buddy/Codeship/Azure, cloud containers like AWS Fargate (with Docker), VMs with Terraform and Packer, and more. All have their sweet spots, but your question is specifically about Docker.
- If you have low/zero code dependencies or can deploy pre-built binaries (eg Go, Net Core) then you probably don't care about the OS beyond choosing an LTS version you can redeploy to at will. For that situation you don't need Docker; just blue/green onto existing or new images.
- If you use interpreted/scripted languages (eg Python, Ruby), or you build binaries in-situ (eg Go, Net Core), then you probably need full control over your OS configuration and installed tooling. For that situation you have options, including Docker.
All of that is overridden by considering the same question, but about dependencies in relation to your scaling methodology. If it supports scaling with standard images you may not need Docker. If it needs specific configurations then Docker is a good choice.
And finally, if you plan on using larger deployment tools or cloud provisioning then there may be non-Docker options to choose from - but at that kind of scale introducing Docker is probably the easiest way forward as it is acceptable to pretty much all related tooling.
A reason to ignore their advice would be that you genuinely know better and know what you are doing. Another reason might be that you have plenty of other/more interesting things to do. I've certainly sacrificed devops in favor of feature development a couple of times and it was usually the right choice. It's important but not when it starts taking double digit percentages out of my time. But the difference between an informed choice and just winging it is knowledge.
Docker is kind of easy to get and learn. So, read up and then decide if you have a use for it. Much better than mucking around with vagrant, vms, and stuff like ansible, puppet, etc. which was the fashionable thing to do before Docker was a thing. If you like your vms still, look into using packer to make creating them a bit more repeatable. A packer built vm image (with ansible or whatever) is a nice substitute for Docker. More work but you gain a bit flexibility. I've used it to build vms that run docker.
There are clear advantages in having a production system that's expressed in terms of a text file that you can read instead of a bunch of stuff that's accumulated over time which you forgot. Especially if you're a solo maintainer of a system, because you're not a full-time SRE, you're not going to invest into making the system clearly documented and you have lots of other demands on your time, and that will bite you one day (e.g., do you have a plan for how you're going to test an upgrade to a new version of your OS or a new release of a language?).
But there are plenty of ways to do that other than Docker. Principled usage of Ansible or even a hand-made shell script will work. If you're on a VM, you get 80% of the benefit of a container - you can spin up a new VM and try to provision onto it and switch over and see what happens. Or, as another commenter mentioned, if you hire a second engineer and need to get them set up on a test environment of their own, they can run the Ansible playbook or shell script on their VM.
If Docker is the tool you understand for how to get replicable environments where everything is documented, use it. If it's not, use something else. But you should pick some tool.
That being said, I still use docker to create a production image. I think Docker is great for deploying a single "entity" to a remote machine. No more forgetting to install dependencies, copy assets, configure auto restart etc. on remote machine.
Adopting Docker means learning a ton of tiny details to get your development experience working well, to iron out deployment quirks, inevitably slower development experience, and other various trade offs.
Sometimes the trade offs are more than worth it (and we happily use it on our team of 5), but if your deployment strategy is already reliable and efficient it seems unlikely to be worthwhile. For us, the ability to deploy around the world instantly and with reliability is the feature we wanted most.
The best piece of code is the code not written, and it applies to Dockerfiles to. Don't waste your time and mental space on something you don't really need.
But docker (swarm mode in particular) is extremely handy and convenient for deployment.
Once you learn the basics you barely need anything else for deploying solo founder scale projects.
Just dont step accidentally into kubernetes black hole. Unless you want to suffer.
A reason, for not using docker would be when using any application written in Go (Golang) since Go static links almost everything and does not even rely on Linux GlibC (GNU C library), so an application built with Golan can even run in old distributions.
Docker is, for most people, just a more simplified and consistent build and deployment strategy. Rather than having your deployment process have to include downloading and configuring dependencies, they'll just be baked into the image you're set to deploy. Rather than having potentially different operating systems on a host and build machine, it's all just container config. Rather than having to deploy a Ruby service and a Go service differently, you just build their containers and send them to a container registry.
Containers bring simplification. A lot of that simplification is ruined by Kubernetes, but if you're currently on a generic VM, just switching to Docker might save you some headaches. It also might not, and I would say if you're finding success without it, what is the reason to switch?
Sort of like:
1. Do you take 2 hours to spin up a server with a few web apps. Maybe you'll have to do it 1 or 2 more times within the next year.
2. Take 200 hours to learn Docker, in order to be able to spin up that same server in about 10 seconds. You'll only have to do it 1 or 2 times within the next year.
Whereas the real use case is #3, where you tack onto #2 the following:
+ where you have 20 developers
+ meanwhile 4 developers per year churn through your team
+ you have dev ops experts who can take advantage of all the connectivity & modularity of Docker & its ecosystem (including Kubernetes)
In this case, it's useful to have people who know Docker.
Whereas if you're not doing any of the things above (and/or other container-related activities), then no need to dig into it (unless it's for fun, which Docker is pretty fun to learn and can teach a dev new things).
Mostly: The cost vs benefit does not exist at your scale.
Its economics: Cost vs benefit.
The cost of time to learn it and put it into practice (and understand its quirks), vs benefit of it on a prototype or small project is a negative relationship.
Docker is cool, it's useful, but it's just an extra layer of stuff you won't need unless you need its extra dev ops benefits.
That said, I don't need Docker, but I still find it fun & interesting to learn Docker. I've used it for tangential purposes: - Personal projects: such as experimenting with something new: to run Grafana or a new type of database that I don't want to install directly but want to play around with and even integrate into a project. - At work: to connect to a database
At the end of the day, it’s just another layer of complexity that could be a liability.
Do you need to use it? No. Like with everything else, it is a tool that requires time to master.
Yes, it adds a layer of complexity but it also helps maintain a disciplined approach to infrastructure concerns.
2. Scaling vertically first. I’m not even joking. Don’t like build things so you can’t scale horizontally but scale vertically first... this also means you will like be able to count your servers using your digits. Everything about Docker will be more difficult to maintain for such a small number of servers. Systemd unit files and some disk images are a lot easier to manage than k8s.
3. I think you have this backwards. Ask yourself, what are the reasons you have to use docker? Because I honestly can’t think of one unless you’re getting into the business of like supporting or running other folks containers.
I'm old and I've been using chroot jails since 2005. So I was never really impressed by docker. But if I ever needed to use it for scalability or dev reasons I would.
There's always another incremental improvement you can make to your tools. After you start using docker, it'll be something else.
Just make a decision to put that aside, because it doesn't matter. Almost anything is good enough, just build your product!
I wrote a blog post on this theme you might enjoy: https://davnicwil.com/just-build-the-product
Without knowing anything about your application, I'll guess that there's a time when it will solve certain problems for you, but that time isn't today. The same can be said of many technical solutions that small projects may not need. As such, you should be somewhat aware of what the options are without committing to those options today.
Assuming you’re just starting out, you want to invest your energy in understanding your customers’ problems and how to solve them. They don’t care how your code runs, what language you use, etc. so stick to the things you know.
Some other commenters are taking about problems with dependencies and your production environment getting config drift without docker - this need never happen if you just use scripts to build your environments and never modify your servers manually, always do it via updating your scripts in vc. That’s just infrastructure as code, it doesn’t need docker. A few bash scripts and environment variables can do the same job
Docker is just one way of doing things, and recently it's the most common way. I used to be more cautious but so far I've not run into any bigger problems, so I'm using it for some things in my private infra. But I don't think it's actually better per se. Where it shines for me is spinning up a local database, or some service I need. Ephemeral stuff that I can throw away again, less so for long-running production deployments. (Unless you need/want k8s anyway)
- It's easy to expose yourself to security issues - never ever run a "docker -p 27017" on a machine facing the Internet or your MongoDB will get hijacked. Only publish ports you want published as in "visible to the Internet". If you want to have separate containers for databases and application, use --link!
- Docker's financing and business model isn't exactly well thought out IMHO. Docker itself is open source commodity tech by now, but it's well possible that Docker-the-company goes belly up.
(E.g. if you deploy to production in 20 min people think that is fast, if your build cycle in development is 20 min that is slow.)
For a small system the ratio of dev to deployment is on the side of dev.
Docker’s approach to I/O saves space if you have a lot of containers that share content, but it costs time for a small scale system.
Also docker doesn’t subtract complexity. A dockerfile takes all the work (most importantly the understanding) it takes to write a bash script that can install your system in a VM but you also need to know docker’s quirks.
I hosted my own registry to have as many images as needed in docker too.
Serverless all the way, because I don't have the peoplepower for such low-level things.
My backend is an Express app. Locally I just run with it Node. In production, I let Heroku run it. I also use AWS Lambda functions and Cloudflare Workers, neither of which use Docker.
I have no need to introduce Docker into my workflow.
If our app was web-only we would go with a simple VM or web host.
Docker has some handy build tools, but those also exist for VMs (e.g. vagrant).
Last year I started using the Docker at work. Before docker, all services were installed manually on physical/VM server machines.
Getting my head around Docker took some time, but I feel that it was time well spent. I started with composes, but moved to docker swarm (single node) for having configs/secrets. Currently I manage everything with the Portainer, that allows me to control all the swarms and services with relatively easy way.
After the working stack is created and tested on the development server, installing it Into one or multiple production servers is quite easy. Depending of the case ugrading services is relatively safe as you can rollback to previous images if you or someone else messed up.
The nicest thing for me, is that all dependencies are inside the image. No need to worry about conflicting versions or setting up same thing multiple times. Sadly you still need to be aware of any security related issues that might exist within the image and maintain them. But this same issue does exist outside of Docker as well.
However, if you don't understand or think what you are doing, it is still possible to screw yourself in many ways. But I feel that this is always possible in our field, no matter what tools you are using.
While the Docker is not a silver bullet, it has made my life a bit less stressful. And I think it is important to understand what you're setting up and how you are doing it. Personally I would not want to go back to my old workflow.
Ps. Sadly the swarm is deemed by many to be dead. It is relatively easy to manage by a single person and I am not getting the same feeling from alternative solutions, such as Kubernetes.
1. DATA DESTRUCTION
Docker is unsafe by default. If you don't take specific measures by setting up bind mounts etc, then it is very easy to get into a state where shutting down the container deletes all the files your server wrote inside it.
This is especially nasty when the program inside the container has generated a private key that was then authorized to do something, and you lose it. Yes, I've seen this happen. What a mess.
2. PERFORMANCE
Docker on Linux is a relatively thin layer over kernel features and is reasonably fast. Docker on everything else is excruciatingly slow to the point that it can break things. In particular, doing anything with Docker on macOS can be so slow that it yields an unusable workflow for developers. It's very common in startups (and these days even bigger firms) for devs to be able to choose their own OS, meaning it's easy for someone to Dockerize stuff in a fit of enthusiasm and then people on Mac or Windows discover that it's no longer pleasant to develop on, or may not even work properly at all. Filesystem bridging is a particular pain point.
3. DISK SPACE LEAKS
Docker likes to download lots of very similar operating systems even when you already have one that works fine, and is very poor at deduplicating files. This can lead to Dockerized systems that appear to work for a while and then one day just go bam because they ran out of disk space.
4. BIZARRE AND UNINTUITIVE CACHING SEMANTICS
What's the difference between these two Dockerfile snippets?
RUN apt-get update
RUN apt-get install xyz abc
and RUN apt-get update && apt-get install xyz abc
It looks superficially like there's no difference because a Dockerfile is sort of like a shell script that's setting up an OS image. But the first approach is subtly broken in ways that won't become apparent for a few months. The problem is that each and every command you run creates a new snapshot, and Docker assumes that every command is a pure functional transform of the prior state. This is incorrect. So what happens is the apt cache will be snapshotted and when you add another package to the install line, it will try and use an out of date cache, failing because the mirror operators have removed the old versions.5. SECURITY PROBLEMS
5a. Docker requires fairly complex networking configuration especially once you get into containers talking to each other. It is easy to screw this up and accidentally expose your unprotected database socket to the whole world, whilst believing it's firewalled. For example Docker has been known to actually remove firewall rules in the past, from live production systems.
5b. Docker images snapshot an entire OS meaning it won't get any security updates unless you are careful to continually rebuild them. It's easy to screw this up such that no updates are actually applied (see point 4), but in practice hardly anyone does do this constant upgrading so stale Docker images are such a large problem that whole startups exist to try and tackle it.
6. LACK OF SERVICE MANAGEMENT FEATURES
Docker is metadata poor. It has no good way to express startup dependencies between containers, cannot express sandboxing rules and many other problems. People often think containers are sandboxes, but they aren't actually designed to be so, and this has also led to nasty problems in the past.
7. USER/GROUP FLAKYNESS
Docker containers have their own user/group namespace, although this is rarely what you want. It's easy to end up in a situation where software fails in subtle ways because the uid/gid database inside the container is wrong or missing information, because home directories aren't aligned, or because uids/gids outside the container aren't the same as those inside but they're sharing a filesystem.
Example: the JVM queries the home directory of the user at startup. It can be the case that the user doesn't have one, when using a container, so it sets the system property to '?' which no real Java software checks for. Cue lots of files and directories named '?' appearing. Have fun tracking that one down!
---
A common theme in the above is that Docker makes assumptions about UNIX (really, all operating systems) that don't reflect how they actually work. This is a good sign that it will blow up in some obscure way. So if Docker has these problems, what's the alternative? Well, for what you describe I'd just use systemd with tarballs or if you want a bit more support, DEBs. This is what I do and it works fine. Why, well:
SystemD is conceptually simple and clean. It is metadata rich, so you can do things like say one service depends on another. SystemD sandboxing is not an accident, it's an explicit design feature with plenty of knobs to give you what you need, but it's also very easy to bring up a simple systemd service in a normal UNIX environment as a starting point. Because SystemD is designed to work with normal software it won't do things like delete files your service created, just because it shut down, and it won't delete firewall rules just because it can.
For software distribution, rsync works fine, but DEB/RPM add support for running scripts on upgrade which can be convenient. They can also pull in dependencies, and the operating system can keep those dependencies up to date for you automatically if you want. Or you can pin all your dependencies to specific versions and take complete control. Although automatic upgrades can make the environment less reproducible, if you are small and only have a few machines in the first place it doesn't really matter because you'd be upgrading one machine and letting it soak for a bit before doing the others anyway, so keeping your machines identical is not going to happen.
Again, depends entirely on your product.
The answers were varied and complex. I don’t want to use a product if so many intelligent people simply can’t agree on its purpose.
Docker in itself is not necessarily easier or more difficult to manage - that depends on your orchestration solution. If you're using it to deploy to Heroku, that's probably fine. But if you intend to use something like Kubernetes or AWS ECS, I'd rather not to as they come with significant overhead and lots of moving parts.
I'm currently managing a Kubernetes cluster in GCP for a big client, and I couldn't be more convinced that this is definitely not the kind of solution I'll use for my own projects.
Finally, although it goes contrary to what is usually recommended, at the beginning I'd personally avoid Terraform and/or AWS CloudFormation as well, as they come with their own overhead and sometimes make small changes more difficult. And it's a context switch.
Docker specifically allows you to simplify and control the process of matching a dev environment, to a staging env, to a production environment - while leveraging simple scripting the cicd build / deploy processes that is repeatable. Repeatable code, repeatable deployments, being the consistency that a developer is supposed to want of their code and application.
Yes, if you have a VM with 48 dedicated cores and 192GB of ram you can just go back to building a monolith like its a retro 90's throwback party - but it would be far more efficient to aim for a zero downtime blue/green deployment, so you can roll out your code without having to shut any service down -- which you can achieve very easily, incorporating Docker and even minikube into your VM.
So, if you don't want a consistent build/deploy process that is repeatable, don't use docker If you don't want to take advantage of lessons learned and best practices by developers and sysadmins who have been doing this for 30+ years, don't use docker.
Spend a few hours to learn docker -- there is a zillion simple copy/pasta docs and medium articles on how to get up and going with Docker and minikube.
20 years ago, the similar question would have been "Why would I use a VM instead of a physical machine?" with these same sort of answers like: "I already have a huge sever", "load balancers are stupid", etc, etc Today, VM's are the default go-to for people.. Containers are the next step in properly abstracting the development->Deployment process.
There are massive advantages to be able to create VM Images and handle delta differences between them to track changes overtime and deploy and scale at a level that you cannot achieve with even a few physical servers. If you have never scaled an application multi-region, or deployed an application for 100,000+ people - you might not be aware of the need to make this process of patching, code deployments a consistent rolling-update process. I highly suggest you learn this now, since if your business is successfully, and you hit a trend spike and your single VM / one-service crashes, you can loose a lot more than spending a few hours to learn Docker and build a container that could be uploaded to AWS ECR and scaled up to 11 anytime you need...
Docker -- or any type of container -- is the current best-in-class tech that we have for repeatable software development->Deployment. Don't shun it because of the popularity of it. Read about why it is advantageous to use it instead of finding an answer to validate why you shouldn't.