Where can I see many examples of real companies' software architecture?
I want to broaden my horizon regarding how things are solved in the real world. Other than some very high-profile companies (like Netflix, github) and companies that I've worked at, it's hard for me to find easily digestible (20-60 mins) examples of actual working architecture of differently sized companies from different business verticals.
> Other than some very high-profile companies (like Netflix, github) ... it's hard for me to find easily digestible (20-60 mins) examples
So I think this doesn't meet your requirements, but I like Tech Dummies Narendra L's YouTube videos [0].
He introduces big tech companies' systems in 30-60min videos and it's not difficult to understand.
[0] https://www.youtube.com/playlist?list=PLkQkbY7JNJuBoTemzQfjy...
There was an article recently (https://news.ycombinator.com/item?id=30936189) that was basically about how far you can get with a simple architecture.
One thing I don't remember explicitly called out, is that most all architectures are grown. There are scarily few situations where starting with the complicated idea is a good idea.
I came across this a while back
It is a court document from one of the UK post office horizon IT system scandal.
It has a very detailed review of the system and its history dumbed down to the level a lawyer could (maybe) understand.
It stands out because it is quite hard to find examples of this level of detail about such a large scale distributed system which aren't internet / web tech companies.
https://www.judiciary.uk/wp-content/uploads/2019/12/bates-v-...
Go to google.com, search "pile of spaghetti", click "images"
I'd really want to have that, but for small companies/services that benefited from avoiding the trends to get some competitive advantage.
Something like "IT Architecture for the Forbes 500-thousand"
A certain amount of information can be gleaned from job descriptions from a company’s careers page.
It's good to note that it depends largely on the company you're looking at.
I work for a very large organization (~£6bil in revenue, £700mil in profit last year) and we suffer from the "mud" problem - nothing about our technology stack is particularly special, it's just a hodgepodge of many different technologies that struggle to work together. That's not entirely fair - I work within a very unique solution inside of this firm, but I'm in a very unique position and I'm sad to say that it took a silly amount of hard work just to be able to not work on legacy applications.
That being said, the companies you mention (Netflix, Github) work completely differently - they were designed with tech in mind! They probably are much more lean in a technological sense, and don't suffer from enterprise architectural issues that large legacy firms do.
I suspect that this inability to move has singlehandedly killed more than one company, though I haven't studied the market to the point that I could really name any. The real kudos has to be given to large companies that existed before the internet and were able to move away from their slow-to-adapt, horribly inefficient legacy systems.
look at public org charts, they'll define the architecture.
only half joking
Work at more companies! Lots of great resources here, however, from experience I would say take all public presentations about how things work inside a company with a big grain of salt. They always have a vested interest in advertising successes, and public presentations always focus on some filter of interestingness. You won’t see the important “real world” parts of what’s left out unless you’re part of the organization.
Slideshare can give you some insights from various companies, most tech presentations discuss something around their architecture!
I was developing some architecture training recently and had this very same question. It’s not easy to find realistic architectures.
The best I found was the German contact tracing app — Corona Warn App. It was done by a group of consultancies in collaboration with the German govt, and went from inception to launch in around fifty days — largely if not totally open source.
Here’s the repo that has all the architecture in: https://github.com/corona-warn-app/cwa-documentation
It’s got full git history so you can see it evolve over time, along with the implementations (also on Github).
There’s a pretty fascinating short talk by one of the people who led the project on youtube too — more about the process side though: https://youtu.be/5y1sHSkPWRg?t=1770
Based on my experience, write down the names of some services and languages on slips of paper then draw a few from a hat.
In the real world [of software], things are solved by choosing the tech with the lowest barrier to entry, not reading any documentation, getting a minimum-not-quite-viable-proof-of-concept working in a development environment, then making that production, over-working a select few to keep it running, and a lot of crossed fingers and heads in sand. The only thing you'll learn from different verticals and sizes is how size and scope have no correlation to how things are built or whether they work well.
The interesting part is how larger scale makes things fail more often, and the response to increased failure can either be running around with your hair on fire for years, or a solid firefighting team, or actually teaching teams not to build products that catch on fire. The only way to get the last one is by focusing on people, not technology.
Twitch recently open-sourced all their software.
Thoughtworks keeps a "technology radar" that I have found very interesting. I won't post the "whys" but it's worth looking for upcoming components and tech that they are seeing used more in consulting. https://www.thoughtworks.com/radar/techniques
I'm looking for something similar for design interview practice purposes in my job hunt.
All the systems design resources I can find are aimed at L4/L5, where the focus is e.g. on how to implement a rate limiter on a single machine, or at best saying you can distribute it by putting the counters on a cache server.
I'm trying for L6 and can identify many of the issues with a L5 design (redundancy, sharding, global latency, hot spots, local batching), but it's hard not to miss the obvious, and to offer practical/realworld solutions, when my day job is embedded compilers and not large scale systems.
This is mostly a rant but I appreciate suggestions.
It's a wonderful question because Github as a zillion projects, and yet there's nary a way to consistently make sense of the system as a whole.
Blobs of code. It's hard to see the systems level.
I think there's a startup idea in there.
I guess searching YouTube is best for these things.
https://martinfowler.com/ but I'm not sure if he touches the real world sometimes, it all feels very academic rather than pragmatic.
I wonder if you'll find "good" outcomes though, it seems to most startups or companies bumble their way to an architecture that works for them. It might not be correct but it might be best way to build a company without architecting everything too much up front.
Read the engineering blogs of big companies like Google, Netflix, Dropbox, etc. and especially read papers they publish. Google has a book out now about its software engineering practices too--although it's not specifically on architecture you can glean a ton of info about how Google services work internally from its software processes: https://abseil.io/resources/swe-book
Try and search for SDKs of some large software. Usually those would be programs for creating content - audio, 3d modelling, 2d drawing, etc. Every major vendor has a plugin architecture that quite obviously leaks implementation details. So stuff like Adobe Photoshop, Autodesk 3dsMax, FL Studio. All these have public SDKs that you can download, explore and write plugins for. You can probably think of some more programs that support third party plugins.
I'm very interesting in the architecture of systems similar to Amazon SQS. Interestingly I couldn't find much discussion on such systems. I guess it's because SQS is such a typical iceberg system that has sophisticated designs to provide dead simple APIs: having a queue that supports competing consumers and simply scales infinitely (in the eyes of users) with users provisioning capacity is no joke.
Looks like it hasn’t been updated for a while though.
This doesn’t really answer your question, but gleaning it from job descriptions is one way I do it.
If I’m curious how a company did something, searching for their job descriptions can turn up interesting stuff like what languages and frameworks they use, and often from there you can infer what their architecture might look like.
Check meetups.com and see if there are local DevOps or other technical meetup groups where people are demoing.
Etsy has a "Code as Craft" blog with lots of interesting reads. It's not been as active the last 2 years but has been re-launched with more posts the last few months.
https://www.etsy.com/codeascraft
https://techengineering.io/ aggregates the technical blog posts from various tech companies and they can be sorted and filtered based on reading time and the architecture you are interested in.
The doom source code was released. If you would like a guided tour, maybe look at https://fabiensanglard.net/gebbdoom/
Unreal is source available too, if game engines are of interest to you
The book "System Design Interview" by Alex Su and Sahn Lam is a good place to get digestible examples. It walks you through step-by-step how you might solve various systems problems, introducing the pieces you need. Each problem fits your 20-60 min request perfectly.
It has link to many other articles and tech blog, besides having a lot of great info on system design and arch
AWS Summit is approaching. I usually find other teams, even nominal competitors, or hulking behemoths of industry, to be quite proud of what they've built, and generous with their battle tested knowledge. All you have to do is reach out and ask ;)
You can use stackshare.io to get an idea of different tech stacks companies use. This might shed some light on their architectures
I'm a big fan of Sean Parent from Adobe, he has a lot of good material on YT; somewhat C++ specific though.
I know it's not a 20-60 min example, but I find reading open source repos very informing:. I'm the cofounder of Budibase, and I like to jump on a call with new contributors and take them through the high-level arch and repo:
https://github.com/Budibase/budibase