Particularly interested in:
1. How much work it took to move apps (be honest)
2. How much experience you had at the time of the migration - e.g. on one extreme your entire devops experience may consist of just Heroku (that's me) or on the other extreme you may be a k8s guru (this helps others gauge how they'll go)
3. How valuable were your learnings? E.g. replacing Heroku with an IAAS instead of another PAAS might take longer but give more fundamental learnings, and hence be worth it for some
4. Cost comparison
5. Summary/description of your apps (e.g. 20 tiny apps with a few hits per month, 5 medium with ~20k hits per month, 2 large with 1-2m hits per month type thing). Please give language/framework.
6. Anything else you want to add
Lots of features are added day-by-day and I'm open for new ones. Visit our Github (https://github.com/coollabsio/coolify) or our feedback page (https://feedback.coolify.io/) to get more info.
A few months ago, I quit my job and started to work on it full-time (best decision I've made).
It is not VC funded (said no to 30+ investors - time will let me know if it was a good decision or not ), but funded by the awesome community! <3
Why I'm mentioning this? Because I would like to focus on the functionalities, the community & the users and not the revenue and other boring metrics. I just would like to make a good software, enjoy the process and make people's life easier. I do not want to make millions of dollars from it. If me and my family could live happily, that is totally fine.
Let me know if you have an questions. I'm happy to answer them.
(Also I'm working on a cloud/managed hosting version of Coolify (https://beta.coolify.io/))
In contrast, Coolify has a great GUI that abstracts away the most common things about PaaS hosting, like connecting to GitHub automatically for git push deploys, SSL certificates, reverse proxying and custom domain support, and best of all, having support for Heroku style buildpacks as well as Dockerfiles. I've been quite happy with it, the creator has a Discord and responds to issues very quickly.
With regards to non-self-hosted options, I did try out Render, Fly.io and Railway but I found that their free servers were too anemic. I was compiling a Rust backend and it simply could not compile on their free servers. On Hetzner, for 5 bucks I could get a 2 AMD vCPU and 2 GB RAM machine that was sufficient to compile my Rust apps in a way that the non-self-hosted ones were not. I have a JS frontend app that works fine though but I wanted to keep everything under the same VPS, plus I can run other types of self-hosted services on it too, like Plausible analytics and a Ghost blog. I'm not sure if those are allowed on non-self-hosted options.
All in all, it costs me 5 bucks a month, and I never have to worry about sudden upcharges for traffic à la AWS as in the very worst, my VPS goes down for a while. I'm now running about 20 different services on this 5 dollar box including databases and applications as well as other services, works just fine.
2 are elixir apps that have been created on fly, 1 was an elixir app I had running on heroku, and 1 was rails app on heroku I recently migrated. I used their heroku migrator and it worked pretty much automatically. I had a few steps to follow to move the DB, but their docs covered it - https://fly.io/docs/rails/getting-started/migrate-from-herok...
I didn't want a self-hosted flavor of Heroku, and I value an opinionated and active community around the platform. I'll mirror another comment here:
> Everyone wants to be the new Heroku, but that should be table stakes. Fly is innovative in the same way Heroku was when it launched.
Vs
Heroku with cloudflare in front Postgres from heroku and memcached with an add on.
1. Very little. I put the app in docker a long time ago for local development but still used heroku for prod. I dumped my db and imported it into a docker run db on Hetzner. One thing to note: I did not expose the db to the internet. The app connects to the db over a docker network.
2. I do DevOps for a living.
3. It did not add to my learning.
4. $30/month for a large Hetzner dedicated instance and $12 for S3 (backups) vs $25 for the app (I added 2 more for 8 hours a day during busy season) and $25 for a job runner and whatever they were charging for DB. My total cost now is $42/month and on Heroku it was around $120 during busy season and $100 during non busy season.
5. Rails Postgres memcached and delayed job. I had to expand my app tier during busy season by up to 3 instances (winter). With my load testing the Hetzner instance handles much more than the 3 instances ever did in my load testing.
6. I added db dump cron jobs along with restic backups to S3.
Heroku should’ve closed that free tier years ago! Now they added some very very cheap plans as well which is fantastic for them = gets rid of some of the horrendous cost from platform abuse they dealt with every quarter.
Good for you Heroku, you were always under appreciated by the enterprise, with their clingy sysadmins and ops people fighting you like a plague and at the same time you were taken advantage of by an army of penny pinchers!
Heroku
1. Almost no work. The app was Node.js + Postgres, it stayed that way and only needed a few environment variable changes plus a database migration.
2. Almost none, I use PaaS because I concentrate on making games, and want to spend as little time touching infrastructure as possible.
3. Learned mostly that switching PaaS is not a big deal if you stick to common offerings (Postgres instead of a fancy exact-purpose-fit DB).
4. ~$100 to $27/mo
5. A Node.js app that gets rather low traffic but needs to be stable, in the 1000s of req/day. It only handles highscores/ranked mode seeds, so low requirements.
6. Would heartily recommend Render if you want to stay with a Heroku-like PaaS approach and they already support what you need.
I was looking for somewhere I could run web services and cron jobs both in the same place.
They have a Heroku importer, however I think you need to ask to have it turned on. I found that it was easier for me to build docker images for my apps and use a 'build service' instead however. YMMV
I found their support to be very responsive and enjoy using their UI. The UI, builds and so on all feel very fast.
Northflank can run databases, however for my databases I've been running them on ElephantSQL for some time (https://www.elephantsql.com) - even when I was on Heroku.
Free for Dev's PaaS list is worth a review: https://free-for.dev/#/?id=paas
I think PaaS is awesome. I actually consider moving to one of the managed offerings from time to time... then I look at the pricing.
As a systems engineer specialising in automation, it's hard for me to justify the $75-100/month fee from a PaaS provider (not including network bandwidth), when I currently pay that now for the same thing in AWS including network bandwidth (and I haven't even optimised it with CloudFront yet and static asset deployments to S3.)
I'm not saying PaaS should be a foregone conclusion based purely on my microscopic example and edge case, but if you can configure a CI/CD pipeline (I recommend GitLab CI or GitHub Actions), manage/configure an OS (I recommend Ansible), write Terraform to build out your infrastructure (loads and loads and loads of TF code libraries with example code you can just run with), and a few extra steps, then I'd recommend going IaaS.
You can keep the entire thing KISS, you'll know it inside out, you can grow it (or not) to any scale whilst keeping costs considerably lower than any managed PaaS available, you can hire someone else down the line to look after it when you've reached that level of scale/success/financial freedom, and more.
That being said, I can certainly see a huge value add in doing both: IaaS with a self-managed PaaS stack on top of it. That is, I think, the sweet spot.
Note: yep, maintaining your own infra' is undesirable to a lot of people. If you're one of those people, then stick to managed PaaS and pay the price (it's worth it in your case.)
Note: yep, your time does (technically) have a cost associated with it, but if you're building solutions based on IaaS everyday of your life, you've likely done a lot of the work already.
The reality is that each of the major cloud providers have gotten so good that it basically defeats the purpose of having your own PaaS on top of them. For 95% of cases, the default products from each cloud provider is good enough now. It's all very easy now.
So the remaining bucket from Heroku that still needs to be filled is for those who have small or side projects and aren't willing to pay (a lot) for hosting. If you're trying to build a business around these "customers", you will be in a world of hurt and just racing to the bottom dollar.
Heroku's erratic performance over the last year or so combined with what we see as a steady decline in the responsiveness (and sometimes quality) of Heroku's support made it too painful to ignore. Their security breach was obviously also a big motivation, but by then we had already made the decision.
We are moving as much as we can into GCP.
- Web app: Cloud Run
- Background jobs: Compute Engine
- Heroku Postgres: Google Cloud SQL
- Heroku Redis: Redis Cloud (https://redis.com/redis-enterprise-cloud)
- Deploys: Custom scripts
- Provisioning: Terraform
It has taken about 6 months for me to do most of the work, with some help from the team on some of the heavy lifting. It's not full-time work though, so perhaps 3-4 months if you count person-hours.
I have decent linux sysadmin experience, but that's about it. Had to brush up on Terraform. Testing GCP services took a long time. Their docs are often good, but important details are sometimes missing or not easy to find when you're doing initial research. I don't trust the GCP console (web UI) much these days; the `gcloud` command shows your actual resources and not some half-baked abstraction. Sometimes resources aren't even visible in the UI.
The learnings were valuable for me personally. I've been interested in doing more ops-related things. For the company, the extra control we get over our own service levels is the major benefit. It has a cost in terms of developer experience and resources we need to allocate to ops as well.
Costs are about the same. Cloud SQL seems cheaper. Redis Cloud is almost exactly the same price (which feels like a lot tbh).
If you're interested in working with us (remotely) with ops, send me an email at other.water6357@fastmail.com
The move worked out incredibly smoothly and has saved us money and allowed us to "modernise" our infrastructure to take advantage of some of the newer trends in Infrastructure and Security.
To address your direct questions:
1. Not very long. We were running a NodeJS app with a web layer and several background workers. We were able to get this running on a Google Compute Engine VM in about 1 day using Packer. The whole migration process took about two weeks start to finish.
2. Our team is relatively experienced and had experience with all three major platforms and Kubernetes (although we chose not to use Kube in this case). We are definitely a team of developers, not sysadmins though. This means we had to learn some new things particularly about tuning NodeJS apps on raw linux.
3. I don't think we learnt too much (other than the undocumented rough edges of both platforms) but it was definitely worth it for financial and quality reasons.
4. It's a relatively hard metric to calculate when the company is growing user base and features quickly - but I would estimate it at around 50%.
5. 1 app with around 5000 requests per second. NodeJS / Typescript / Rust
6. If you have only ever used Heroku I think it would be worth getting comfortable with Containers (Docker basically) and making your app run in a container. From there you have tools like Railway (https://railway.app) or Cloud66 (https://www.cloud66.com) that can do most of the rest for you.
1. It took about 1 week due to lack of render documentation for Docker and buildpacks. I posted on the render help forum and contacted their support which did respond but were not the most helpful. I ended up finding a single example buildpack file in a render blog post which shed light on how to structure their build system. If I had not found it I dont think I would have been able to get it to work.
2. I don’t have experience with 3rd party deployment frameworks.
3. Learned how to use Docker in order to utilize Renders blueprint feature which would automatically link a redis instance to my hosted app.
4. $0 to $0
Render has a similar interface to Heroku but some missing features and bugs a few of which I’ve found are: No newline/special character support in environment variables. Applications cannot get the public hostname from calling the standard call in the programming language’s library like hostname(). Unlike Heroku where a publicaly reachable host is return, render returns an internal hostname is returned instead.
To answer your questions:
1) 2 hours for 8 apps — most work was setup of new DNS and testing
2) I have very litte experience in DevOps
3) I think its totally worth it for the additional flexibility and the super cheap cost
4) Just $5 - can’t beat that
5) 8 Ruby Sinatra sites w/ a total of 20k hits/month
Sailor[1] is a tiny PaaS to install on your servers/VPS that uses git push to deploy micro-apps, micro-services, sites with SSL, on your own servers or VPS
API & Platform Results:
Rails: @Railway wins. Render wakeup boot extremely slow (~1m). Flyio cannot run Rails for free.
Crystal: @Render wins. 5-10s wakeup boot time. Railway & Fly keep growing memory infinitely.
Node: @flydotio + Railway both win. Very slow boot on Render.
I really didn't need much it turns out. Most things were old experiments, testing, fun that I let die.
I really just salvaged an old personal page of mine. Setup was super fast, tied to github, very nice features. It's not heroku by any means, but it works for what it is and I really enjoy the simplicity of the setup.
fly.io (as others have mentioned below) lets you build a Docker image and run that. I've found having them serve static files lets me not include caddy in my app (simple go-based website).
encore.dev is, in effect, a URL service. You write go code with comments that say which path goes to that function, and it takes care of routing for you. They have a lot of ancillary services (cron jobs, database, queue, etc.) that integrate very nicely. Different way to write code, but it seems to work very well (it watches a git repository and rebuilds when you push).
It's not as easy as Heroku as there's no managed Postgres, but it's well-documented with a few paths.
However, the main reason Fly stands out is their focus on multi-region deployments. Ever wondered why facebook.com is fast no matter where you are? Fly provides app-level tooling to make this superpower available to everyone.
It's especially important if you're making heavy use of Web Sockets.
Everyone wants to be the new Heroku, but that should be table stakes. Fly is innovative in the same way Heroku was when it launched.
Heroku recently removed their free tier. Render has a free tier that is pretty generous and very usable. For paid plans, Render pricing is roughly equivalent to Heroku. However, I find Render’s pricing significantly easier to understand and thus more friendly. Heroku has pricing tiers with varying features and sometimes you have to upgrade to get a feature even if you don’t need more resources. Additionally, Heroku can be vague about some of their specs, particularly when it comes to how much CPU you are promised. Render, on the other hand, gives the same features to all paid plans and then the resources just scale up and out. I much prefer Render’s pricing model, even though for me the cost is about the same.
My one major gripe about Render is that they don’t have a CLI. I find a CLI to be a very important part of any PaaS. Heroku has a great CLI, Render doesn’t, and the feature request to create one has been open for three years. Clearly it was not a priority for them. That said, the ticket did just get marked as In Progress a few days ago, so I am optimistic, but we will have to see where that leads.
I use zfs for backup of the whole thing.
It is not mission critical so I can accept downtime. Even then, in the last year I had zero downtime.
I think it can be a viable (and fun) option for many small applications.
We had some AWS experience from running our backing services (RDS, Elasticsearch, Memcached, Redis) on AWS despite doing compute on Heroku for a long time, but we'd never done EC2 or EKS for deployments on AWS. Despite having been on Porter now for months, I'd say we still don't know or really need to know a ton about k8s, but we are familiar with some basics around pod sizing, healthchecks, deployments, etc.
I think Porter does a lot to put themselves out as as destination for people leaving Heroku. I'm not sure if this would have been more work to do something like Render, but I was very pleased with the timeframe, hours spent, and the ease of cutting over.
We have a monolith that handles ~50k/min traffic at peak use as well as a couple very tiny services that do some accessory stuff. All the apps are Rails apps.
One of the reasons that we chose Porter over other options is that we really liked their setup for Preview Environments, which are an important part of our workflow here. The experience of running preview apps on Porter is notably better than on Heroku, where we started to see a lot of unreliable behavior on app launch that resulted in the app being unusable and the only fix was to close and open a new PR .
On the whole this was a really positive experience for us. We're seeing better performance, we're paying less (in Porter + AWS compute combined) than we did for Heroku Enterprise, and our ability to deploy mid-day when we're under load is better than it was before. I've spent most of my 12 years or so of Rails development working on Heroku (and we had been on Heroku for almost six years), so we had some fear around moving to unfamiliar tooling, but this has been a big win for us.
1) Took a few hours to have everything setup. Transition was easy 2) Not that much. I chose heroku because i hate doing sys admin stuff 3) Can't say for this one, scalingo is a PAAS 4) Scalingo will probably be more expensive for starter plans but as you grow it won't be like heroku's horrible pricing 5) Mostly app with no traffic to be honest
I program in Go so I get a self-contained executable file at the end of the day that I just upload to the server. So whatever "Heroku" provides is completely irrelevant to me.
Found them fairly easy to use, really excellent support.
- No salt/puppet/ansible, no docker, kubernete, etc. Only standart linux stuff.
- All done with makefiles/ssh/git - Send email with Mailgun
I try to keep the dependencies as small as possible: external services, programs, libraries.
One server can serve an incredible amount of requests if you only do normal web stuff.
Login with ssh to get stats with top. Huge performance, snappy, no concern with cost.
You could do the same with a vpn
It took me quite a bit of time, and it’s not perfect. I need to set up a vpn image as a fall over if the hardware break (low probability)
2. I'm a pretty knowledgeable DevOps guy.
3. Not really valuable but gives me a bit of Linux experience.
4. Flat instance pricing, was able to get all of my stuff into a $5 instance... however I have been increasing it as I get projects from friends and myself in these.
5. Supports almost everything, it just runs your stuff inside a docker container.
6. I liked Heroku, it's where I started with Rails but it gets ridiculously expensive quickly... there's no reason adding a DB should cost $40.
With 20Eur a month and a VPS on Digital Ocean you can get quite far.
I also made a small project to spin up a PaaS like environment with docker swarm, Portainer and Traefik if you're interested: https://github.com/sergioisidoro/honey-swarm
I heard keyob is decent, otherwise fly.io. Render and railways.app have inactive or day limited services.
The tool requires minimum dependencies inside the kubernetes cluster and it tries to follow all best practices on kubernetes. Unfortunately the tool is not ready yet.
Would something like this be interesting for someone else other than me?
I vastly preferred the railway.app migration to the fly.io migration but fly.io feels much more production ready than railway. This isn’t a big deal as none of the projects I moved have paying users so downtime would be acceptable if it happens. I was able to deploy all of my applications from the UI without ever having to install any company specific CLI which I prefer. Railway being such a small company has lacking docs and is very difficult to google for specific question about the provider —- most “railwayapp” google responses are for actual railways. Pretty much everything was intuitive and didn’t require the docs though which was much more pleasant than my experiences with fly.io.
To get the custom domains working I had to move my DNS to cloudflare for all sites which was something I had planned on doing anyways but felt annoying in the moment. I was previously using my domain registrar’s (namecheap) DNS but they don’t seem to support CNAME flattening which allows for CNAME apex domain entries which were required for getting custom domains on railway.
I moved all of the services in about a day and everything has been running pretty well. I do think railway is slower than fly but everything is still plenty fast so far. All projects have automatic github deployments which have been working well so far. Overall for a new project made by a small team I have really really enjoyed using railway for hosting. Railway is organized in a way that matches how I think about project hosting and most things seem well thought out with lots of good stuff still coming.
The websites moved were https://trentprynn.com https://habitapper.com https://weightwatchapp.com https://samueldebartolo.com
these are all open source and can be viewed from my github https://github.com/trentprynn
What can you recommend for an Heroku alternative down under?
1. The migration - from "I just learned about Dokku" to "deployed my app for testing" - took about two hours.
2. I'm an urban planner who dabbles in front-end development. DevOps is not my thing.
3. Not too valuable of learnings. Overall, the dokku commands are almost identical to heroku commands (by design). So the transition was smooth and comfortable.
4. Costs: I chose droplet and postgres services with DO that equaled our heroku costs. But we get about 4x the ram now. The site is much faster.
5. Rails/postgres. We don't track hits, but it's pretty popular in terms of, uh, maps for finding pinball machines.
6. I haven't figured out how to set up logging. I would like some kind of "last 5 days of logs, being piped to somewhere." It seems simple, but I don't quite get it yet. I also miss the "usage" metrics from heroku that showed ram and stuff throughout the day. But I guess I pretty much get that with DO and Scout APM.
2. I'm a dev with a bit of DevOps experience but mostly as a user. I still have to run to my DevOps friends for help occasionally :)
3. I learned tons. Most usefully setting up the entire CI/CD myself and Dockerizing my setup.
4. Cost is one of the main reasons why I moved. Digital Ocean is much more cost efficient for CPU intensive instances (my backend runs audio/video and physics). For the web app, Digital Ocean Apps are a bit more pricey than just running a droplet but save me loads of time.
5. I run a spatial video meeting app (https://flat.social). Tech stack is Node, Next.js, Mediasoup and Typescript. The traffic is rather spiky - anything from a 50-60 users on a chill Sunday to over 25k website hits from Reddit during xmas.
all free tiers, even with a free custom domain.
1. A few months. The actual changes to the apps were not fundamental, but we discovered a lot of things we didn't know about our own codebase and it had to be fixed. The majority of changes were devops plumbing not Ruby/Rails. The move itself, including all the necessary admin changes and supplemental work took about 1.5 years from decision to cut off date.
2. Me - about 30 years of total experience, 10+ years of devops/SW mixed experience. But total Ruby noob (and still is, you don't need to know Ruby to move apps)
3. The move made me a multimillionaire and a couple people billionaires and another 100 or so millionaires or multimillionaires. I mean this move opened the path to the IPO and we became one of meme-stocks of 2021. So even if I never touch AWS, Openshift, Heroku or Ruby it was totally worth it. I didn't learn any fundamental new things (after 30 years in the industry it rarely happens), but solidified my skills in many departments. I never run such a massive containerized workload before, for example.
4. Nobody cared actually. We grew like bamboo at that moment, so very difficult to compare apples to apples.
5. A single monolithic app about 0.5M LOC and a few smaller ones. Hundreds thousands of customers, tens of millions of revenue/month
Self-host redis, do not use their free beta app.
My app ruby on rails, sidekiq, hasura, redis, postgres, 2m rows, 2-5qps.
All apps use 1000 free dyno hours in ~25 days, so at the end of the month, Heroku switched them off.
I didn't find any free alternative for my users and me. I'm thinking about building my own service where users can upload notebooks and publish them as web applications.
I don’t have any apps there myself but I’ve worked on projects hosted on Heroku and I remember being happy with that provider.
Did they raise prices, change their services, or something similar?
No affiliation, I’m actually preferring the experience. Heroku is showing its age now.
Since then the project has seen good traction and we run a few thousand apps in two regions, EU and US.
Anyone still looking for an Heroku alternative, do check us out! There is some free welcome credit to get started quickly.
Overall the process was smooth, the React + Vite frontend app was up and running as soon as I connected GitHub and added the envars. But the dotnet backend took a little bit of trial and error to get running because it didn't like my initial Dockerfile.
Overall the process has been pretty smooth though I wish the docs were a little more robust and had more examples.
I have a linode vps with a wordpress blog up but I want to learn docker and host my apps on there for 5 dollars a month.
Coherence configures managed services in your own cloud such as Cloud Run in GCP or Fargate/ECS in AWS, while adding in all the missing pieces these providers don't include out of the box, such as load balancer/SSL/DNS configuration, database/redis orchestration, CI/CD integration, Cloud IDEs for development, VPC/networking best practices, long-running workers and cron tasks - all with one simple configuration.
Coherence itself has a free tier that fits most projects, but you will need to use credits or pay for you usage of your cloud resources. We'd love feedback and are happy to answer any questions, feel free to ping hn@withcoherence.com!
1. Maybe a day 2. Not a lot 3. You learn event-driven architecture but it wasn't new to me 4. It's free for me as long as you don't attach a company github with Netlfiy or Vercel (then you move out of free tier) and then AWS has enough free credits, Lolo is free for your first two apps (although a bit of a hassle to have to re-deploy every 7 days if you don't want to pay 9€) 5. Web platform primarily with maybe 10-30k hits per month, but also a few web apps. Maybe move the Serverless api from AWS when it increases to not incur too much of a cost.
The real benefit is not having to worry about upgrading stacks at EOF and that stuff.
We don't have big marketing budgets and are a less known alternative, but we see influx of new customers moving off of Heroku every day. It's mostly Node.js and Python apps since we focus on this stack.
The migration is very simple - git push evennode main. This works for most of the apps without any extra steps required. It's really this simple.
Cost is cheaper than Heroku - 4.50 EUR / month. Each app also gets its own MongoDB database at no additional cost.
The biggest advantage I see, compared to VPS, Lightsail, etc., is you get a fully hosted and monitored platform. A server goes down? No problem, your app is migrated to a new one automatically.
I would says a pretty good alternative to self-hosting and Heroku.
Their PaaS is rock-solid. The UI sometimes looks less polished but that shouldn't detract from the fact that their service quality is really, really good. No problems so far. When I had questions, I've been in touch with their support - you have real engineers responding to your message in a few hours. Really cool.
I've recently come to increasingly appreciate profitable, bootstrapped startups, because they have (much) less incentive of getting acquired, subsequently shifting their focus to "scale" and "enterprise sales" (wait, which company comes to mind here.. ah, Heroku). Scalingo is bootstrapped and profitable.
Why not render.com or fly.io? Both are based in the US, so GDPR compliance becomes more tricky. They're VC-funded, so I have no clue what will happen to them in the future. Maybe they'll be profitable and their investors will say "oh okay, it's fine to have a boring, profitable startup in our portfolio". Maybe not. Likely not.
Now I can deploy multiple apps to a single server, not have to worry about provisioning. I pay for the single server instance.
It's been great for my hobby projects.
It took a whole week to set up a simple EKS cluster for one basic Rails app.
Everything is ridiculously complicated, and the docs are terrible. When the Log4j exploit occurred, I had to go and figure out how to search the logs. I had to request them to make an update manually, and it took them 3 days to do it!
Cloudwatch is terrible. EKS UX is terrible.
Unless you want to have a full-time job of wasting time with AWS, stay away!
Cost is horrible as well. Heroku was $7/month and AWS around $80.
Fortunately there are several good options. After seeing HN posts recently, fly.io seems popular.
> 5. Summary/description of your apps
Simple discord.js bot interface to fire Lambda functions, about roughly 5000 times a month
> 1. How much work it took to move apps (be honest)
Roughly 2 hours and it's running! Pretty good replacement.
> 2. How much experience you had at the time of the migration
Experienced with k8s, practically a newbie with heroku-like environment.
> 3. How valuable were your learnings?
Barely any learning, it was quite straightforward.
> 4. Cost comparison
My application is pretty low resource so it remains free with the Railway pricing model.
> 6. Anything else you want to add
Related to point 4. Cost: Heroku did have more extras built-in (logging system, alerting), these are missing in Railway.
Basically these three
- caprover
- dokku
- k8s
I can run it like `m3o app run --name=helloworld --repo=https://github.com/m3o/m3o-app` and then go to https://helloworld.m3o.app
Seems like a similar idea, you give them a Dockerfile and they deploy it somewhere at `$PORT`, and then you listen on that port.
Back-end: - fly.io - nginx load balanced python/node apps running on digital ocean
I use it and they should be paid for it. Why not?
Or... at least as close as you can get. I actually preferred it before the heroku thing.
But to answer your questions: 1. It took around 1-2 months of experienced DevOps works (1-2 devops), this included testing and the actual switchover to new AWS infra.
2. We actually used outside contractors (actual k8s gurus) to do the work. But the decision was made in-house by myself. I'm no k8s guru, hell, I'm not even devops, but I have enough experience with the "big guys" so the choice was calculated from technical perspective, considering our growth needs and technical needs for now and for the future.
3. I haven't had any reasons to regret the decision, but I knew that before. I knew AWS offered the things we need and how to use them and what it's roughly going to cost, so this wasn't really a surprise. I did my due diligence.
4. Heroku bills were around 5$k/mon... AWS ones are in 5 digit, but it was expected because we started to use so much more services and resources than we did (or could) in Heroku. We also expanded into two new countries (one of the main reasons the switchover was done) so the costs are sadly not directly comparable. I can say that the new costs are in expected ranges and I'm happy with what we get for the money.
5. We run a eCommerce site with monthly GMV of 1m€+ and around 700k monthly unique visitors. Our stack is (per country) 3 nodejs services, PostgreSQL, redis and some frontend services. Main cost factors are the database(s) and CloudFront.
6. These decision have to made per use-case, per project and taking into consideration your actual infrastructure needs and budget limits. I don't think there isn't anything AWS can't do, but is the cheapest? Of course not. Hell, there have been plenty of topics on this very page about moving your infrastructure to your own bare metal and saving hundreds of thousands per month.
Think through what are the issues you are currently having (ie, for us it was the Herokus stack limitations, couldn't get http2, couldn't do custom monitoring / alerting, no access to LBs, no scalable DB hosting, the need to easily roll out new countries without having to do too much of manual work, weirdly high cost of some services, like redis for an example). If your problems are in the wallet, you need to consider this as your first priority and find a provider which will meet your technical needs with best price. All the big guys also have cost calculators available which will allow you to get an estimation on what you would be paying. Take time, this isn't an easy decision and it will affect you a lot in the future as well, so you don't want to get it wrong to be stuck with another set of issues.
I was going to use Heroku, but the security requirements for my app made that a non-option. Instead, I went with simple Terraform that spins up CoreOS nodes, using cloud-init to spin up Docker containers. CI process built the Docker images, and deploying was just a matter of making an ECR tag point at the new image then cycling instances. Not quite as simple as Heroku, but pretty close.
I moved onto AWS via this: https://registry.terraform.io/namespaces/GoCarrot
No k8s. No Docker. Beautifully clean system. Blue/green deploys with automatic rollback. Continuous deployment (there's a CircleCI orb as well). Tightly buttoned-down system configuration. Debian built from scratch with an eye towards supply chain security. (There's a root image factory and base image factory to handle the layering of image build processes involved.) Log aggregation and configuration management baked in. Security that can only be described as "insanely paranoid, yet oddly pragmatic." Cascaded image builds, so I can update the entirety of my infrastructure by pushing one button to kick things off then clicking a few buttons to approve deployment of the various services.
1. About a week, although I had the assistance of the author. I'm now rebuilding my personal infra on Omat and it's going much more quickly. Probably 3 days total, with no assistance.
2. More experience with DevOps stuff than most, but not a DevOps person by any stretch.
3. Very, very, very instructive.
4. Given what I was coming from, cost-neutral. Compared to Heroku? Notably cheaper.
5. At present, we're in alpha so traffic is negligible and back-end workload is fairly minimal (tens of thousands of jobs per day). The author of the tools, however, is CTO of a mid-tier SaaS that handles a quite significant (millions of transactions/day, IIRC) amount of traffic, and he is super aggressive about not being hobbled on performance needs -- but also being cost-efficient.
6. Avoiding the k8s iceberg, while having all the modern amenities in a system I actually have a hope of understanding top-to-bottom (modulo my hesitation around reading the systemd source) is nice. This system is an object lesson in "loosely coupled, highly cohesive" design. I haven't felt at any point that I may be stuck in a You Can't Get There From Here situation. Avoiding layering a second layer of software-defined networking (Docker/k8s) on top of the software-defined networking of AWS means I neatly avoid the single biggest source of chronic issues (and, via the workarounds needed, system complexity) that I've experienced with "modern" (Docker/k8s) DevOps approaches.