How would you refactor a big project in 2023?

Speaking as someone who loves refactoring and has done many in my career. For the first step, I'd get VERY, VERY, VERY crisp on the problems I'm trying to solve and why I'm trying to solve them, and start from there. Then I can balance the time/cost of doing the refactor against the benefits and/or other stuff I could be doing. The answer to what/how I'd refactor is also likely to be very different than "my favorite stack for a new project" in 2023.

If first attempt to refactor was over-engineered. They same will happen with the the new stack. New tools won't resolve problems. Start deleting the code.

"We know, for example, that iterating on existing solutions is more likely to improve software than a full rewrite. The dangers of full rewrites have been documented. Joel Spolsky and Stack Overflow described them as “the single worst strategic mistake that any software company can make.” - Kill It with Fire - Manage Aging Computer Systems (and Future Proof Modern Ones) by Marianne Bellotti

Do it gradually using the "strangler pattern" [1]. Find the part in your codebase that is (a) likely to undergo the most change, and (b) that is perhaps causing the most problems due to tech debt and figure out how to just refactor that.

Try using "branch by abstraction" [2] to slowly replace this part of the code by a new component (a microservice, perhaps?). Then release that (via a canary if you need to).

See how it goes in prod, and then rinse and repeat.

Trying to make a massive 8 microservice change in one hit is a recipe for disaster.

[1] https://martinfowler.com/bliki/StranglerFigApplication.html

[2] https://trunkbaseddevelopment.com/branch-by-abstraction/ and https://www.branchbyabstraction.com/

"it is also hard to find good developers for our current stack." Maybe I can help you out. I love refactoring, simplifying software, deleting the code. My personal notes about topic https://github.com/slk500/phprefactor/blob/master/phprefacto... PM me at https://www.linkedin.com/in/slawomirgrochowski/

1. Combine all code to one repo, one main branch. No other branches.

2. Update dependencies.

3. Fix bugs.

4. Delete extra code. Remove not needed dependencies. Look are there better smaller dependencies.

I think the intuition that, in a large enough system, simple new features (from business pov) should be easy to implement is justified. So I always try my hardest to agree. While the code is messy, there must be elements of what I need to implement for a new feature already in there somewhere. Somebody solved a similar problem, or the feature would be too novel and quite divorced from the rest of the system to call it simple. I search for those elements first. Write some tests around them, extract functions/methods, classes, modules, services , whatever really, with those elements. That is my 'refactoring'. Now I reassembly those elements for the new feature. This is usually done with just a few more lines of code. I am very suspicious of refactoring that does now have a new feature implementation at the end of it. I am very suspicious of developers that ask for refactoring iterations. They end up rewriting at least parts of the system. And that, some of us agree, is the worst thing you could do.

My favourite stack, without a doubt, is AWS Serverless. So this is an opinionated suggestion:

Forget the overhead of k8s, forget servers and patching, and ansible and managing VPCs and networks and all that.

Just do the following: Cloudfront, S3-hosted front-end codebase (react, vue, angular, whatever), API Gateway backed by a lambda infra which provides your API, DynamoDB for your databases, and, where needed SQS/SNS between lambdas to fan out workloads. For inter-account messaging, try EventBridge. And then, for auth, go for Cognito*

* Cognito is, in my view, the worst piece of all this -- so, maybe try something else here if you need to (Auth0, or keycloak if you must).

Write the lambdas in Node or Python.

Use SSM Parameter Store as a config management key-value store to configure environments, lambdas etc.

Deploy the lambdas with serverless framework (or terraform).

Build all the rest of the infra with terraform.

Stick it all in a monorepo.

A tool like https://thin.dev/ might be good for you. You can port and serve functionality a little bit at a time into thin and use their managed postgres db and the schema editor to handle the new fields in the backend.

React or Angular for the frontend, and Node.js for the backend. These technologies are popular and widely used, making it easier to find developers with experience in them.

For the database, I would recommend using a flexible, scalable NoSQL database such as MongoDB or Cassandra.

For the CMS, I would recommend using a headless CMS such as Strapi, which allows for more independence in terms of adding fields to the database and displaying them on the frontend

By definition, microservices are independent of stack. Every service in theory can have its own stack. My recommendation would be to stick with microservices but start migrating each service to modern stack. With regards to the stack, you can look at Django if you want to move to a python based ecosystem, Spring boot for java and node if you want to look at server side js backed by a classic RDBMS like MySql or Postgres

I recently read a story where engineers faced similar situation. They decided to migrate their PHP backend to PostGreSQL database https://habr.com/ru/company/lingualeo/blog/515530/

> What would your favorite stack be to start a new project in 2023 where you know that you will have hundreds of thousands of visitors guaranteed per day?

OTP (i.e. Erlang, Elixir, etc.) would be my default for that sort of load.

Start with writing down what the problem actually is, without redactoring as the obvious solution. Write your requirements down, then see what you actually want to do.

don’t refactor, rebuild.

you have multiple reference implementations. learn from them.

have better engineers make the attempt.

do it in parallel and only switch over if it’s not worse.

bonus points for multiple competing rebuilds.

I would just let it be in 2023 ;)