Or is it some kinds of fallacy to believe that simplicity has a rather obvious form?
It's more like "simple" isn't "simple enough", isn't it? The latter of which one can get to faster and from his own POV alone while true "simplicity" takes time and consideration.
Consider how much more efficient we should be with allegedly-better processes and allegedly much better tools.
Look at some of those and estimate the team size and time to complete it now. Consider whether you could even deliver the same thing (say, native applications on two or three platforms) with double the team size and thrice the time, or whether you’d have to compromise and deliver something worse to make that happen.
There’s your answer.
[edit] though actually I think part of this is also because we’re on-average worse at managing companies overall, too.
Then: Support a single platform, Windows or Mac
Now: Support Windows, Mac, and Linux, ARM or x86, as well as mobile.
Then: Support a single market
Now: Support a global market with region-specific localization
Then: Write software for wealthy and highly technical early adopters
Now: Write software that anyone in the world could possibly consume
Then: No accessibility
Now: Accessibility requirements for software being codified into law
Then: Zero or low consumer expectations, win by creating something new (blue ocean, lotus123 versus literal paper)
Now: Nearly everything has been done, win by differentiating on comparative quality in a field of options (red ocean, figma vs keynote)
1. A need for standards
2. The industry being too immature for standards to exist and those that are created rarely stand up to the test of time
Look at standards from the late 80s / early 90s, most of it was not prepared for what software has NEEDED to change to.
Look at most programming languages, designed to only run on one machine, when a vast amount of software is designed with high-availability in mind.
Look at most databases, designed to have only one writer which is a huge bottle neck.
Now we need software to coordinate all this software that was designed to run by itself...
Finally, in the pursuit of perfection, we test new ideas in the open, the latest UI framework, a new database, a new paradigm... We are doing the scientific method on NPM and your organization is a part of it :)
Some of it is incompetence.
Occasionally there are claims that it's deliberate attempts to make the author look more valuable though I haven't seen that in practice. Hopefully it's a myth.
Some of it is indifference. You're sufficiently beaten down by the environment that you no longer care about the product.
Also the underlying problems are sometimes very complicated in which case there may be no way to make the solution simple.
That gut feeling only comes through pain, aka experience, botching a refactoring you thought was brilliant to simplify code, and realise there were reasons for it to be overly complex. Experience working with overengineered code that takes a while to realise it's actually much simple than what the code is telling you, and can be thoroughly simplified.
Over many cycles of those experiences I believe one can start developing a sense of form for what's good simplicity.
Another point is it comes from the experience of reading vast amounts of code, from many different skill levels/sets.
Simplicity takes time and consideration from experience, I don't think you can jump the step of building experience to be proficient at it.
In the '10s, everyone was talking about "webscale". VCs noticed that we could write software for hundreds of thousands of occasional users instead of a core group of a few thousand users.
We thought we were making UIs to help people do stuff, but we were really making little feeler apps for vertical integration for corporate interests.
Stuff like big-data and kubernetes looks overcomplicated if you believe the sales pitch. The "tech set" is negotiating for control over business.
- we're not shipping a final product; this needs to change in the future very quickly
- we don't want to give you two years to build a feature. Build the start of a full product now and iterate with no break
- we would like a pixel-perfect custom UI instead of CLI or whatever Windows drag and drop UI builder would give us
- we want to run on multiple OSes
- we want to run on tablets and phones as well as desktops
- we want to have translation into every language and alphabet, rather than staying in ASCII
- we want cloud backups, and configured to the user-privacy compatible regions of the globe
Don't you think?
Software engineering on the other hand exists in a parallel space that is perfect and idealized. It's engineering that never has to leave the textbook ideal models. Mother nature has no claim in this space. This doesn't mean it is necessarily easier however. It means that all the effort typically put into just making something not burn up can now be put into complexity. So you end up with systems that are insanely complex. Engineering minds unchained by the constraints of reality, you just get complexity built on complexity serviced by complexity.
You get what you reward. Our industry rewards more keywords in your CV. Therefore, people try to get as many keywords as they can.
if someone isn't knowledgeable about the problem they're not going to be able to select the most simple solution.
if someone isn't knowledgeable about the available solutions there's a higher chance they'll implement an unnecessarily complex solution.
now multiply this over time and you get a code base that is really good for everyone's job security.
multiply this by millions of code bases and you get a really good tech economy.
That: the simplest solutions look obvious once you see them, but they’re rarely, if ever, obvious from the outset. Simply put, the simplest solutions are rarely the most obvious. You have too actively look for them, and that takes time and effort.
One corollary is that people who take the time to design the simplest solutions that end up making life easier for everyone else, tend to look much less productive than the tactical tornados that masquerade as a heroes. Worse, when the boss looks at their code it’s much simpler and much easier than everyone else. Clearly they took the easy route and chose the easiest things to work on.
And what do you think happens to low performers who chose the easiest work? They can leave, one way or another. But they can also give in and expedite more, more complex code.
Simplicity isn’t just hard. It’s a hard sell.
It takes more effort and time to produce simpler solutions, and generally speaking as an industry, we're way balacned toward speed over quality. We basically only care about "quality" as far as it has direct customer impact. It's relentless optimization for the short-term over the long-term, and it leads to complexity.
So you really can't master anything anymore, at least not like in the old days, because there's just so much more of it and you only have so many hours per week.
But, this is probably inescapable. We have the internet now. You generally just can't write a monolithic app. It's going to be two apps (client and server) or one app that has to think about both client and server.
And also Moore's Law went and died on us. So no more single-threaded... anything. Everything is threaded or parallel or etc etc now.
But today's "get it done now" outlook encourages complexity. That was always around, but ....
About 20 years ago or so, it was OK to get things done right and be a little late. Now seems this "get it in now, do worry about consequences", then "Well we have issues, create a request and will will get to it later". But later hardly ever comes, so the users are stuck.
So here we are, everything is complex, no time to think about what you are doing.
I guess the difference that (in Germany) math exercises cares more about reasoning and explaining while in CS / informatics solution is only the tested result. So yes, coder make the world more complicated ;)
People don't want simple programs today. They want bells and whistle and stuff that doesn't make them think. They want to be, in effect, 'mindless morons'.
In the old days, you had a very simple text program that did every single thing you wanted but you were obliged to remember the command-line options for it.
Today they don't want to have to remember anything, they want a button nicely labelled right in front of them to do the exact thing they want.
If you try to select body text, it starts scroling.
If you try to select bullet point text, it reloads the page.
You have to work pretty hard to break something simple that much.
Nearly 10 years ago, we had the left-pad incident[2]. As I post this, people are in a panic over a similar issue. This time it's called the polyfill incident. The details are different, but they don't really matter.
The root cause is the same:
1. People saw nice code
2. They didn't make backups of the nice code
3. Instead, they trusted someone else to keep serving it
This time, the trusted party served malware instead of deleting everything. Innovation!
A more serious (but still oversimplified) take: static linking and/or vendoring can mitigate this for a price:
* Vendoring is copying other people's source into your repo
* Static linking is the compiler doing the same thing with binary code
It's a trade-off:
* Pros: durability and longevity since everything's in one .exe / folder / zip
* Cons: lose flexibility and gain software license complications with the LGPL/ etc
It also isn't as trendy as "what you need to know" articles.
For educational yet actually funny takes, I suggest reading these:
* The Grug-Brained Developer[1], a light-hearted take about complexity being bad
* The famous ML / AI XKCD[3] about stirring big piles of numbers. What's changed since then?
* The size of the number pile
* How fast we (our GPUs) can stir it
* The amount of investor money having the tallest pile can earn
* NVidia's stock price
[2]: https://arstechnica.com/information-technology/2016/03/rage-...
“Simplicity” can also be defined any number of ways, but is most often defined as “Maximally pleasing simplicity” rather than “Fewest moving parts simplicity”, in part due to misestimating how total volume of moving parts is an exponential complexity modifier when one is not already familiar with the mechanisms. https://xkcd.com/2501/
“Hanlon’s Razor”
There definitely aren't enough people deliberately obfuscating to make it a generally visible cause of complexity; your quote here rings far more true to me.
For example, "refactoring" as a task very frequently just trades one set of things, often aesthetic things, for another -- it is rare to find any obvious best expression.