How to get good at designing big and complex applications?

"A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system."

This comes with experience. There is no single answer to get good at the designing complex systems.

The more you work in different applications, the more you discuss with your peers in your organization and other meetup groups, explore different design patterns (Martin Fowler), Follow well known Architects or Company engineering blog [0] posts, Try doing proof of concepts etc. Eventually as you progress with experience, I am sure you will get that knowledge.

I used to follow the below link [1] to understand how the systems work at different companies globally (e.g Facebook, Google, Netflix, PlentyofFish etc). This has very interesting content.

[0] https://github.com/kilimchoi/engineering-blogs

[1] http://highscalability.com/all-time-favorites/

Do people find Martin Fowler and his peers' work to be good in real world? I'm a system/application architect who spent years coding. I used to read books by Martin Fowler and others religiously. But now, I realize most of the books and talks are really just fluff pieces because the examples worked in perfect condition, where as my experience is from the battle field.

As far as getting good at designing big and complex application, I'll give what I know.

1. Everything complex came from a small system. So make sure your small application works first. Twitter/Google/Facebook didn't get to what they have now on day 1. They failed a lot to get here.

2. Designing monolithic system is a lot easier than micro services.

3. Read a lot of codes and how different component interact with each other. For Python, projects such as Scrapy, Flask and Django are good example to start.

Hi,

There's no single way. In fact, beeing a good software architect is all about having many ways in solving a problem, knowing wich solution is the most adapted depending on the context. All is about culture and cleverness To improve, you can :

- Follow/read the well know architects. For example, I read this blog https://martinfowler.com/ and watch most of the conferencies he gives.

- Learn the design patterns and the architectural ones.

- Read a lot of code, like github repos. It does not care if you're reading repos on a subject close to the one you're interested to. You need to see a lot of architectures to see some clever solutions some peoples have used to solve some specific problems. I don't think there's any shortcut.

- get interested in meta development. Like "why does the languages work the way they work ?", read about how code is produced in real conditions (what are the processes involved), etc.

Congrats on your will and enthusiasm, I believe this is already a great first step!

Besides Martin Fowler which was already recommended, I would also recommend the books by Robert C. Martin (Uncle Bob), and his "Clean" series - there are books "Clean Code", "Clean Architecture" - all of them are really good and useful and will give you a solid foundation to build on.

Other than that, the best thing you can do is to get practice & ideally surround yourself with more experienced developers. Either get an internship in a company building something more complex, or participate in e.g. Google Summer of Code.

Doing fun side projects is also a great way - you learn a lot and also have something to show & write about - win-win!

Check the books of Architecture of Open Source Applications - http://aosabook.org/en/index.html

This book (Designing Data Intensive Applications) was an interesting read: https://dataintensive.net/

Study & Experience, that's it.

Also understand the difference between these

     programmer
     application developer
     software engineer
     computer scientists

there are very different roles, it's okay to be one and really good at one.

BTW, you mentioned "design" so I'm going to hinge on that, design not the actual implementation. In that case study existing systems. 90% reading can teach you design, on actual implementation, more like 90% doing.

The early decision you will have to make is whether you want to be a theoretical specialist who will mostly make money consulting working at a consulting firm or university or whether you will be solving actual business problems down to the metal.

Each has a completely different trajectory at this point in your life.

A great example is to read the great debate between Andrew Tanenbaum and Linus Torvalds over kernel design as its closely related to what you're currently studying.

Linus would have clearly failed Tanenbaum's OS class.

Yet I use Linux every day, as does millions of others, while Tanenbaum's MINIX is on a CDROM that came with the book that I bought for my OS class.

This is not to say Linux is still a monolith, but that would be getting in way too much details.

> How to get good at designing big and complex applications?

Write small and simple programs.