I've been trying to convince my CTO that we would gain more benefits from a dedicated hosting solution (Hetzner) but haven't managed to convince him yet.
Right now we have around 5 small dedicated VMs for each dev (2 cores, 4gb ram) and 2 production VMs (4 cores, 16gb ram).
His arguments are that things are easier to manage in the cloud and he doesn't want to have the extra burden of manually configuring infrastructure since none of us have any real background in systems administration and in the future when we will scale to more production machines it would be easier to just start another VM than wait for a new dedicated server. We also use Azure managed MySql which helps us to see performance easier than if we rolled our own.
Do you agree with me and if so what arguments should I use? Would you recommend something other than Hetzner?
Spinning up a new VM takes a few seconds, just like on Azure, and ordering a dedicated server is only a click away if you really need it. So you get the best of both worlds.
I don't believe that you can be "too small" to save on infastructure. Hosting our setup on Google Cloud would probably cost 10x what we currently pay for, maybe more considering that our largest server has 32 physical cores...
[0] Blog article about our setup: https://pirsch.io/blog/techstack/
None of your dev team has a background in sys admin or managing a DB? You're going to be in for a period of difficulty while you gain those skills and that's going to be felt quite heavily on a team of 5. Now if only one of you picks up that skillset now you've also got a high Bus Factor (1) in the event something goes wrong with your self-hosted infra.
It's entirely possible that you're right in the long term but in the short term it doesn't make sense because your team is too small and the cost difference, while probably proportionally large, is almost certainly tiny in absolute terms.
I would explore the possibility that you want to be able to easily move off of Azure in the future. This may or may not be worth planning for but it likely makes more sense than going to self-hosted right now. If you can get buy-in that you shouldn't be on Azure long-term then you should avoid Azure specific services that will make it harder to switch. This would mean keeping your stack simple - compute and DB where possible. Avoiding things like Event Grid, their PubSub product and possibly even their CI/deployment offering.
Again - even doing that very possibly does not make business sense and you shouldn't approach that analysis with a specific conclusion in mind. Does your product need something like their AI offerings? If so it's probably a lot faster and more effective to lean into the products they're offering than trying to run an alternative yourself on their compute.
For most B2B businesses offering a SaaS product, I don't think on-demand variable "web scalability" is the decision framework for AWS/Azure/GCP.
Instead, it's really about faster product development iteration. If new SaaS product features can be delivered to market faster because it can leverage many of the higher-level managed services in the AWS/Azure tech stack portfolio like DynamoDB or Cosmos DB -- without your 5 dev team building the equivalent tech stack from scratch, that's when paying the profit margins to AWS/Azure are worth it.
In other words, your CTO and the devs have to look at your future product roadmap and see which features would require extra developer costs in re-inventing aspects of AWS/Azure that may negate the cost savings of a "dumb" IaaS like Hetzner.
your aws infra: 600-1000 month? hetzner: 200 per month?
Is 800/month gonna make a difference to your organization?
Imagine you launch new feature with performance regressions and suddenly your database is at 100% CPU usage
Managed database service will allow you to scale up the instance in a few clicks, so you can redo the feature with a better plan, instead of rushing to roll back or deploy a hot fix.
To me looks like your are trying to solve a non-problem.
If you don't have sysadmin that could tackle dedicated hosting but you have knowledge to keep things running in cloud - stick with the cloud for now.
If it was my money, yes, dedicated FTW. Egress charges... nien danken.
If it's you boss' money, what's the point arguing it. Just let him have his cloud bills.
Also, you can use vms from other vendors. You don't have to go full dedicated.
You have two small prod servers. Yes, other cloud services would be cheaper but it's probably not worth the migration time. Stepping on dollars to pick up pennies.
In this thread, we have a lot of technical people who would want this to be a technical decision. But it is not. It's a business decision and has long term consquences often lasting longer than the technical staff who made them. The reason cloud providers give credits is for vendor lock-in. It's an entirely apporirate decision to want to avoid vendor lock-in and when people are selling technical products to companies they sell the business value first and the technical value second.
The business decision here is do you want to spend time and money just now on moving or do you want to spend time and money later on. If you want to avoid vendor lock-in go for a cheaper hardware solution then you need to spend time and salary money on setting it up. If you want to stay with Azure then from what I've read in this thread from what you've said you're going to have to spend money later on for resources and you're going to have to spend time and money later on managing the costs. This seems literally like technical debt.
While it may be frustrating to technical people that a business person is making a decision that effects how they work this happens in every other industry. Business people decide the vendors, they decide the materials, etc all based on business reasons.
Another thing to consider, as a developer, is how to make your app as independent from vendor requirements. For example, if you build an app in such a way that it only works using SQS you’ll run into issues down the road.
Also, why y’all using dedicated dev VMs? IMHO it sounds like there are other issues to solve before you migrate the app somewhere else. If I was in your position I would work on making the app easier to setup and deploy. Your CTO has a lot to consider and being a CTO at a startup isn’t an easy thing. How long does it take to deliver a feature to customers? How can you help speed that up? What are the engineering goals of the CTO and how can you help with those goals? Make his or her life easier and work with them. Continuing to push an issue like this will cause you frustration and make your CTO dismissive of your ideas and suggestions in the future.
For the sake of security and performance you're better off using azure (especially the hosted database).
Otherwise you'll need to configure iptables, backups, software updates, high availability (assuming it's a requirement) all by yourself. If you don't have experience in these things you don't want to be learning how to do them with the thing that generates your revenue.
To be quite frank: Both hosted and cloud have a learning curve. The difficulty is about the same.
Furthermore, many businesses shoot themselves in the foot worrying about scalability too early. Are you really going to jump up to Google scale overnight? (No one does. Chances are your software has bugs that will prevent this, even if you are running "in the cloud.") When your business is well-run, your scalability needs will be predictable enough that you can provision hardware in advance.
That being said: I really like Azure. If you want to "save money for the company," focus on making sure that your software is reasonably efficient. This way you can pay for 10 servers instead of 100 servers.
Ignoring the weird dev VMs. The setup you have at the moment is tiny so cost isn't the issue here. In terms of dedicated you'll likely have to buy that for a least a year and so I don't think fully dedicated is the correct move here.
There are smaller cloud providers that'll give you those 2 prod VMs at a fraction of the Azure price but again price really isn't much of a factor here so I think staying on Azure is the right move.
I think personally you should look at moving those 5 dev VM's locally i.e. not in the cloud. It's really strange to have that many dev VM's.
For instance, both my postgres master and replica went down one day the other weekend, because another machine in the same rack popped the breaker. In my specific application, I can cope with a once-in-a-blue-moon blip like that -- Hetzner saves me literally thousands of dollars per month in compute and bandwidth, I like doing [a limited amount of] sysadmin, and it's worth the trade-off. But, how do you think your CTO is going to feel about this move when all of your dev VMs go down just before a big customer deployment?
I use Hetzner dedicated servers for some non critical project, I find it fun, but it’s time consuming and requires some knowledge.
Just one example, having an encrypted storage on a dedicated Linux server is not trivial.
Everyone seems to be missing this part. This seems nuts. Why is each dev getting a dedicated VM?
Keep everything in docker + some cloud agnostic orchestration layer so it's easy to migrate to other clouds (or even to dedicated).
I have experience with all of the clouds and I'm never going to use one. They are ridiculously overpriced and they get way more expensive the bigger you are. What other arguments do you need? I've seen mediocre startups doing the same and ending up with a 1M/y AWS Contract while pulling very little money - and eventually scrambling to move everything to physical servers.
If you're going to be a unicorn, it may be worth to pay lots of money to the cloud because it will be a tiny fraction or your future profits - if you're unlikely to hit that, just roll something up on Hetzner. It's not that hard, and literally what people did for a decade before AWS. I've never been purely a devops guy but I've been rolling my own servers for 15+ years; I understand not wanting to deal with hardware failures (they happen and you'll waste lot of time) but a dedicated / VPS is as hard as running things on AWS EC2.
That said, it's a lost battle.
Most technical leaders are dead on the cloud and some even think they're saving money. It's the same with scrum, microservices, performance not mattering or other nowadays myths.
That's the beauty of containers right there.
Eventually you'll end up with a Kubernetes cluster and a bunch of Helm charts either way.
Having a self-hosted server eliminates a lot of these questions. For hosting in production a managed cloud VM might be sensible as these services often provide additional functions like doing health checks, automatic renewal for tls certs, protection against attacks, etc..... Sure, you can get that on your personal server too, but it does need maintenance.
For development/testing/staging I prefer to have a non-managed server. That said, for most applications it is enough to keep the system of the host up to date and that is easier today than ever before and you might even be faster here than large cloud providers who have a bit of a momentum until problems get addressed.
Also cloud providers do change their infrastructure and might force you to adapt. That can mean extra work out of the blue. They tend to leave your systems as they are, but if you need to update certain components you have to read up on their docs again.
I'd challenge the idea of dedicated infrastructure per developer in the first place. There's little reason why this could not be done locally on a developer machine using Vagrant, local Kubernetes, Docker Compose, whatever.
It makes total sense to use the Cloud then, especially if you already have some experience there.
> Our use case does not really need instant scaling all that much since we're a B2B business and know ahead of time how much compute we will need.
If you had the expertise in the team, going with VMs + dedicated on Hetzner would have been a good idea, though.
I started with a dev team of three: One knew Linux admin stuff pretty well, I knew enough to keep the lights on, neither of us had proper cloud or container experience. We've done pretty fine with our Ansible managed Hetzner setup so far.
Especially if there are few moving parts (E.g. just HaProxy, a Postgres instance + your API) and your dev laptop has the same OS as your servers, I'd say it can be a lot easier to work with than some cloud provider.
A lot of it depends on what you're actually doing, but for a B2B SaaS app you could potentially make an argument around vertical scaling. Nowadays you can spin up a 128 vCPU VM with 2TB memory on EC2 in minutes, so why bother having lots of separate smaller servers? I've seen people making this argument quite well recently, especially in relation to using things like SQLite instead of a networked RDBMS. But.. I can't see you making such an argument very convincingly unless you're heavily into ops or play with this sort of stuff on the regular, so I'd stick with your CTO's approach for now and keep alternative ideas in your pocket for hackathons, spare time, or prototyping sessions.
However, something I didn't learn/internalize until later on in my career was that not everything is your call. In this case, you absolutely should defer to the CTO's judgement because, you guessed it, it _is_ his call. He is the one ultimately responsible for these decisions both in authority and blame. If hes "wrong" and you are spending more than you should then thats on him. But being in his CTO position has earned him that right, you should respect it.
Hetzner looks good to me never tried it but have heard it is good. In case you want to look for VPS hosting, you can check, cheap VPS list created by me https://qawithexperts.com/article/asp-net/best-and-cheap-win... Thanks
Your CTO is smart.
But if you use the cloud carefully, selectively and avoid vendor lock-in as much as possible, you will be OK and it will be not hard to move to Hetzner (dedicated) incrementally later.
If you have more significant compute needs a dedicated server can start to make sense. Linux admin is becoming a bit of a lost art in this container-everything world but actually it's not that hard. I run a hardware server for a lab and once it's running there's very little work, super reliable, and very cost effective.
If you aren't paying the bills use Azure. I do pay the bills for my product and use Hetzner.
Regarding production VMs: How much do you expect to save by moving to self-managing everything? I’d expect a few dollars saved for the hardware, but a huge amount of time spent setting everything up and documenting it properly. Net loss.
Let me tell you, I host a B2B app on Hetzner because I cannot use typical cloud providers due to GDPR, and I hate it. So much fiddling around to get everything working and I still don’t have high availability because I don’t have the expertise in that area. It’s a huge time sink and takes significant time away from actual development.
Conclusion: Use your time and money wisely. Do what speeds up adding value to the business. Saving a few pennies but missing out on growth and recurring revenue is a bad deal.
This debate is long since settled. Use the cloud, and devote that mental capacity to building things customers will pay for.
If you are selling to enterprise / businesses AWS or Microsoft are much better places to be reputation wise I think.
https://aws.amazon.com/compliance/programs/ To get a feel for the paper pushing exercise they’ve spent time on.
BTW - I put a large memory instance on ECS anywhere, worked great - if you have very high memory needs something to consider - a basic dell gets you 3TB these days - very nice for development