Masters in my organization have an extremely high ratio of value shipped to hours worked.
This means two things. They are able to discern and avoid work that does not provide value (this is often their greatest skill), and the best ones can direct a whole team away from large swaths of work that is not valuable. And they are able to build systems such that the cost of adding more value to the system remains constant as the system grows in complexity.
Novice engineers will struggle to get something done. Intermediate engineers can get things done but fail to see the tradeoffs. Expert engineers can identify possible solutions with different approaches.
Just like a master woodworker would consistently and reliably deliver good furniture.
Or a master author would consistently and reliably deliver good books.
The key points are:
* Consistent - It wasn't just a fluke or luck, you know what you're doing and can repeat it.
* Reliable - You know how much effort it will take and can deliver it.
* Good - It's not crap, people want it.
Mastery would imply that you can do it all, and do it right, on any project, and there is just too much change and evolution for that to ever be true.
My goal is simply that I continue to improve.
To be exact: "The developers came back and said it would take two months and exceed their budget.
Rather than having a big confrontation over the issue, I told them to just send the project to me and I would do it myself. Cass Everitt had been doing some personal work on the iPhone, so he helped me get everything set up for local iPhone development here, which is a lot more tortuous than you would expect from an Apple product. As usual, my off the cuff estimate of "Two days!" was optimistic, but I did get it done in four, and the game is definitely more pleasant at 8x the frame rate.
And I had fun doing it."
Software engineering, gives you some degrees of freedom, therefore it's kind of an art form. Therefore mastery is about intent and the ability to realize it.
In this canvas, you can ask yourself, is the bearer wielding the tool or is it the other way around.
For me a proficient tool user is no master until he makes his tools his own. He can become one either by creating them, or by understanding them deeply enough that he could recreate them.
There are then various levels of mastery and mastery over various sizes of domain. Levels are more about the capacity of imagining new tools, and having new visions in order to advance your field in some interesting way. Growing domain size is about trying to have bigger and bigger tools while still retaining control over them.
A long time ago I saw a good talk by Jonathan Blow wherein he made a point that really resonated with me. The point was that to get good at programming you need the experience of shipping a lot of programs and thus you should optimize your career around shipping as many good programs as you can. Contrast this with the common advice of being an expert in databases, or distributed systems, or OOP, or testing, or something like that...
Edit: found the talk: https://www.youtube.com/watch?v=JjDsP5n2kSM
Clean, well-documented projects showcasing demos that end up having huge impact. Some of these projects single-handedly spawned entire tech ecosystems. (see FFMPEG, QEMU, JSLinux)
* Consistent delivery of quality work
* Instead of complaining about bad requirements that threaten system integrity, happily works with stakeholders to compromise on them accordingly
* Can consistently describe every detail of every bit of work they have done, and answer any curveball questions about it too
* Succinct and to the point, doesn’t bog down meetings with tangential arguments and what-aboutisms
Could probably keep going but that’s good for now haha. I know I don’t even satisfy all of these points myself at all times, but I certainly try to!
Mastering the language but understanding why the language was developed what problems it looked to address.
Reducing the cognitive load of the design using the language.
Understanding the fundamentals of the types of applications you will be building. SCRUD Architecture Apps Streaming Architecture Apps and Games which are usually a combination of both.
Software Systems from first principles are read and write systems. How do you represent that as a program and reduce the complexity?
How you view software engineering ? and explain it to others what is the method of representation you use to encode software as plumbing? is it text based? time flow graph node based? stack based ? binary based ?
Software engineering as a team sport, how do you get a bunch of developers to reduce the cognitive load manage the relationships and data in the code and how do you measure that like a science.
I am also sure there is a lot of valid advice here as well. This is how. I see it from a mathematical / science perspective that pushes pragmatic narrative.
Many of these traits are fundamental, universal, and even philosophical, than is general perceived, and transcend technologies and stacks, and to a large extent, time.
And yet, I think, the above traits can be described as good, and not really mastery in its truest sense, like in other disciplines like music or painting. Software engineering is way too diverse, fragmented, chaotic, and ever changing.
* I think an intermediate level of mastery is when you realize that a computer is a logic machine built to serve humans and it will always listen to you if you're willing to put in the effort. You can always go a level deeper. Great engineers know all of the tools at their disposal and will bend the computer to their will if they need to. At the same time, they know the difference between a great low-level solution and a shoddy hack.
* Our industry is one of the few where we spend the majority of our time fixing our own mistakes. We call these mistakes "bugs" but they don't just appear, we produce them. The best engineers produce fewer bugs than they fix and rarely end up in a "whack a mole" situation where fixing one thing breaks another thing.You can't master Software Engineering without mastering interfacing with the rest of the organisation.
I don't care how intellectually pure your implementation is or how fast you deliver things. If it can't easily connect to, use and get used by the rest of the business, it's a folly.
A master of software engineering delivers what the business needs in the minimum time possible.
Some businesses care about maintenance. Some don't.
Some care about UI/UX, others with a trapped audience don't.
Some care about efficiency and optimisation, others don't.
You can't be a "master" until you have a sense of what your business is interested in. Not what it CLAIMS to be interested in, but really cares about.
It's been different in every company I've ever worked for.
Good luck.
"Master" really is a great word for it, you have complete faith in the individuals word.
I think of Linus, Lars, Anders, TJ, Rich, etc
Mastery in software engineering needs to be defined very specifically.
An example, I used to build Flash microsites, campaigns and games for an ad agency. I had a very good grasp of the tech, built projects for large clients etc. If you looked at me at that point in time you could say I had mastery, but now Flash is gone, and I didn’t switch over to React and other web based tech soon enough so my “skill” is lesser than someone who purely started in that domain.
I don't see them engaged in "heroic" coding marathons to desperately get something fixed or delivered; they already fixed it weeks or months ago.
Knowing that the correct answer to almost every question is "it depends".
Knowing how to somewhat-less-inefficiently produce larger-scale software that adequately meets the need. (Note that I did not say "efficiently", and I did not say "bug-free".) This takes at least some skill in dealing with people, not just computers.
A master will be able to understand the relationship between systems so he can reduce combinatorial possibilities and reduce complexity to a minimum.
In other words, a master will not shoot herself in the foot. She will also not let others shoot themselves in the foot.
Then, depending on what you do, you will need technical ability: -Understanding basic physics or economics or whatever you interface with your software. -Understanding of low level systems (assembler and C) -Understanding of different paradigms(objects, functional) -Understanding of very High level systems (Lisp, Haskell) -Understanding of serialization(parsing and binary) -Understanding communications. -Understanding data handling. -Understanding interfaces(like command line, desktop or Web). -Understanding standards.
But this depends on the particular work you need or want to do. You will be stronger or weaker in different areas.
But complexity handling is common to any area in engineering.
Competence is a better term. What it looks like is being able to make things happen quickly with minimal errors or problems.
Unless you’re talking about the most extreme cases ..... Fabrice Bellard and John Carmack .... genius developers who are true masters. There are few of these.
Here's a incomplete list of topics that I is worthwhile:
Creating a data schema (sql, no-sql, api based, whatever). This seems simple and obvious, but it becomes very hard very fast, especially when you need to model a complex subject that you know nothing about. Consider being hired by a hospital to model their records, or an insurance company. You need to talk to many people and understand how everything they say and do relates to everything else.
Project management: You may not want to be a project manager, but you should familiar with scrum, agile, waterfall, lean, and kanban. At least you can have an informed opinion when you argue for or against how things are done.
QA: You should know how to write a test plan for a large feature or application. This includes unit tests, manual testing, deciding what areas need more effort because getting it wrong is very bad and what is less important. This may include security and performance testing.
Architecture and design: How would you design a new application? How would you add a new feature to an existing application? How do you communicate this design to a team or many teams?
How to evaluate new things like languages, tools, libraries, technologies, etc. This isn't just reading a couple blogs, this is doing small scale test, proof of concepts, looking at the code to an extent, getting opinions for others you trust.
Finally, communicating. Software engineering is a team sport. You need to know how to write design documents, test plans, maybe architecture plans, progress reports, etc. If you can't effectively communicate your designs, plans, or even why someone else's thing is bad or good, you won't be able to convince others that you are right. And remember, you need to not just convince your peers, but also non-technical people, such as managers, VPs, and many other.
Through-out my career, which is now over four decades if you include the software I wrote as a teenager (before I got paid to do it), I've continuously found a better, more elegant (which can almost always be read simpler or clearer) way to tell the computer what to do.
So my non-snarky view-point is that there's no such thing as mastery but rather that it's the proverbial journey. There is however a baseline at the other end ... if it's not functional and maintainable, there's a definite lack of mastery.
As an art teacher, I have seen many young artists go on to have great careers shortly after leaving school... but most of them burn out or fail in one way or another. Conversely... their aged teachers have acquired character, depth and stability but have in the process lost their wildfire.
- Should know many tools to the level where they know their strengths and weaknesses. This allows them to select a useful collection of them for any new piece of work.
- Should produce incremental project changes which all either improve the value or reduce the cost.
- Should balance all the improvements to produce useful, working software with a reasonable cost. For example, producing super-secure software with no useful features, featureful software with a bajillion bugs, or bug-free software with a gigantic cost are all bad.
Though in software engineering we have a lot of prior examples of successful and unsuccessful projects, we also have lots of open frontiers. And in some ways, it's much harder to "know" that an engineering approach or paradigm is "right" than it is to know that a theorem is true.
How can we make types track the "important" invariances of a system? How can we convince ourselves that a distributed system can guarantee certain properties, or that a modification to that system doesn't break those guarantees? If I build a homomorphic encryption system as a service, how would I build debugging tools for it? Acknowledging the halting problem and its cousins, when can static analysis tools make useful, meaningful predictions about programs?
In the bronze age, you could build an impressive stone tower. Sometimes you could even make it stay standing. But I don't think there were any master civil engineers.
Therefore software engineering, is solving problems with logic, within a set of constraints.
Where constraints can be; business requirements, hardware limitations, development time limitations, performance limits, solution simplicity, release frequency... Etc
Mastery of software engineering is efficiently solving problems, and optimising for the constraints that matter...
You wouldn't ask what it looks like to master the practice of medicine today because there are dozens or hundreds of specialities and roles all of which contribute to medicine.
Firmware engineering and 'agile' website development are very different practices with different assumptions, requirements, etc. I wouldn't expect someone to master both web development and video encoder development just like I wouldn't expect a doctor to be a master pancreatic surgeon and develop vaccines.
On the other hand there is a 'journeyman' level of software development and engineering skill that carries over between different development areas. It's definitely not algorithmic interviews though.
Figuring out what the irreducible complexities of a problem are and at what tradeoffs these complexities could be addressed in different components of an application (including handing them off to the user).
A few corollaries of this are:
- Masters will write less code, as they will eliminate most of the complexity when there is time for this.
- Masters can more easily put the complexity where they want it, having more control over trade-offs between performance, usability, flexibility, etc.
- Masters will have an easier time estimating the required work needed to implement a project, as experience gives them a good intuition for what the irreducible complexity of a problem is.
- Masters will rarely tell you a "right" solution except for simple problems. Their answer will depend as much on your specific requirements as the problem you're trying to solve[1].
- Masters will often fix problems by simplifying code. Refactoring will be a natural part to their coding workflow, not a separate "housekeeping" activity.
- Those who hide complexity behind more complexity are not masters. Those who always fix problems by adding more code are not masters.
[1] It could be argued that "add to numbers and do it fast" is a single problem and that's not wrong. Requirements like "speed" and "usability" are really just another aspect of the desired outcome of a program, but it's rarely thought of as part of the problem to be solved, thus I am considering these constrains as separate from the "core" of the problem.
1) Knowledge of the core theories of computers: core ideas of computer architecture, good understanding of computational complexity (how long a program will take to run), networking...
2) Knowledge of software technologies: Python, Javascript, Java, C, Docker, Functional Programming, Object-Oriented Programming...
3) Knowledge of software engineering concepts: Software architecture, software process (agile, requirements gathering, testing), software complexity (and how to minimize it through good architecture and processes), general engineering principles (redundancy, fault-tolerance, solving problems early in the dev process...).
To me at least, the one that is neglected the most is the third one, "software-engineering concepts." Being able to think about systems at a high-level and well at the detail-level is one the most useful skills for an engineer.
If you can break your problem up into smaller, independent pieces, you can pretty easily manage complexity throughout.
By having a good domain model (which to me means context-agnostic and represented approximately in 3NF/BCNF), you automatically clear up most of the nasty interfacing between components. Also, having a domain model in a normalized form enables much easier querying for any required dimension of fact (i.e. SQL/LINQ/et. al.). The methods will almost automatically follow from a good domain model.
Being able to develop good models of the domain and manage incremental complexity are the hallmarks of a master software engineer. You should note that neither of these fundamental skills requires the use of a computer.
Ward Christensen did it once as well, to the annoyance of my late friend Lloyd Smith, may he rest in peace.
I will try to avoid doing that myself, specially I will avoid implying I am one, but I think it's possible to come up with a few criteria to help you "judge" the level of mastery one has in software engineering:
* can create a project from scratch without pondering forever about each and every choice, while picking reasonable choices that, together, approach optimality given the circumstances (notice that this depends mostly on the team/company involved so there's not one right universal answer).
* can break up the work in manageable pieces so that progress can be made (regardless of how many people actually work on the project) and measured easily. This is really hard and can only be done properly by a practitioner (meaning, no, your PM can't do that for you).
* can establish a good base design that facilitates achievement of the project's objectives (e.g. if performance-focused, then architecture might differ significantly from a more maintainability-focused one).
* knows to apply the right amount of testing/automation (too much or too little can be damaging).
* can focus on the task at hand, avoiding unnecessary work (many great technical developers fail to do this due to their preference for harder challenges that may not be the right ones to address).
* can communicate well (because knowing it all but not being able to pass the knowledge on or convince others will cause most of your knowledge to become inaccessible).
Notice that to be able to effectively rate a person or team against each criterion is itself something subjective sometimes, so you won't get a clear-cut answer no matter how you look at it... but I am sure that there are ways to use metrics for each one, though we all know the problem with metrics (people will game it), so yeah, it's difficult to figure this out. Just make sure you don't fall into the trap of judging people solely on subjective grounds though as our natural biases are quite crippling (see psychological studies about human bias) and will cause you to misjudge others, sometimes disastrously.
2. Think in the abstract. Implementation details are just that, details.
3. Ideas and concepts can be reused, code reuse (in business logic areas, not technical support) is rare.
4. "Design is about constraints" - Charles Eames
5. Understand the foundations, finite state machines, what REST as a style actually means, why Nouns are more important than Verbs, don't reinvent things...
6. Learning never ends.
- High performance (in terms of efficiency in using computational resources)
- Reliable i.e does exactly what we want and doesn't crash. This also includes clear documentation.
I think to reach there, some of the things one may want do are;
a) Learn TLA+
b) Study hardware (processors, memory)
c) Study software, protocols like tcp/ip
d) Build experiments & ship
So, maybe you'll find that to be a helpful practice as well. It's worth a try.
But I'd say a master is simply someone you have a lot to learn from.
I'd say you have mastered the discipline when even though you know the limits of your abilities and you are eager to learn, it becomes difficult to find someone to teach you something new.
The ability to judge risk in terms of time and money.
The ability to judge the amount of security that is needed (a productive level that is still safe).
To think long-term, in terms of avoiding complexity buildup over time, and in terms of avoiding vendor lock-in.
To help others get up to speed.
To stay productive over time.
To stay open to learn something new.
knowing what to Google.
So the master to me is the one who knows which tools for which job !
I'm just a devops guy but my current project is like that, where our lead architect has really impressed me by drawing up the design of a very scalable system. My task is merely to implement and manage kubernetes for it and we have other devs who are using java, .net and golang to write the components.
This impresses me a lot how our lead can come up with such a design and make it work from paper to practice.
- Symfony framework
To answer this question, one must first operantly define what mastery is. So what is mastery? To start, mastery is something on which one can be on the path towards, yet mastery is also a thing which has been achieved. So we need to talk about flow along a path and achievement of the objective.
PATH
The path to mastery is characterized by humility and child-like exploration combined with an intrinsic drive to integrate learning a and take paths that make individual sense to one on the apprentice’s path towards mastery. It is concerned with thoughtful action and adaptation. The ultimate focus is on the seeking of truth (an intrinsic reward) rather than and sometimes at the detriment to receiving external rewards (accolades, financial renumeration, and other materialistic artifacts). Such sacrifices and focus is made early in the mastery process and pay off once one inevitably achieves it. This is a pattern common in most individuals we would consider to be masters and the difficulty lies in staying on such a path despite internalized intuitions and the listening to an internal voice/drive pulling us away from society, the majority of who are not on the path at any given time. Mastery is high risk because it a deviant process often focusing on long term rewards rather than instant gratification or even short-term survival.
ACHIEVEMENT OF
What can a master do? Why is that beneficial?
Mastery is the seamless ability to fuse the intuitive with the rational while performing and understanding at a high level of competency in a given discipline or domain. It manifests as as such a deep internalization of a domain that one is able to spend cognitive effort on seamless cross-pollination between other disciplines, sculpting domains at will. Mastery is only achieved through embracing the humane, which means internalizing the fact that what makes us human is the blending of the rational (a step-by-step-step process) with the intuitive (a distributed and probabilistic process).
MASTERY IN SOFTWARE ENGINEERING
What is engineering? What is software? What commonalities do they share?
Given a definition of mastery, we see that the question of what engineering and software are next to be defined. These three questions should serve to form a foundation for discussion.
Engineering is the process of designing reliable processes and systems given constraints on resources, tools, and available thought patterns. It is also fundamentally concerned with risk management on the path to achieving an objective.
Software is a system which interfaces with either humans, machines, or both to achieve an objective. It serves as a medium for sending conceptual communicative signals across, within, and between various levels of abstraction to include physical computing done at the level of hardware.
As an aside, it must be stated that the delineation we place between software and hardware is an arbitrary though sometimes useful fiction to allow us to get tasks done. With this being stated, such a simplification is often over-used to our collective detriment.
The master software engineer understands hardware and software deeply at both the intuitiveness rational levels in order to design systems and process to specifications and with clarity to both machines and humans alike. This ability to provide and communicate with clarity is especially important when we consider that our colloquial models of machines themselves are over-simplified, a fact that one will quickly become aware of when trying to design applications across platforms that perform “low level” file system operations. In this case, it becomes apparent that even operations such as opening and closing files, tasks which one would think are stable and solved problems, have such variability that cross-platform development is more challenging than it should seem.
Not only does a master software engineer have the ability to communicate at different levels of abstraction of hardware and software—such a master also recognizes that communication is the operant word. Communication is signal propagation, filtering, and more, occurring at the personal, interpersonal, and organizational level as well. The master software engineer is seamlessly able to apply all principles of software development to psychological phenomena as well. To such an individual, there is no difference conceptually from a root pattern perspective.
Those are creations of exceptional masters.
For smaller projects take a look at nginx, Tokyo cabinet (a DB), redis, and, perhaps, python's requests and Clojure (one man's effort to unify obvious things).
Again, these are masterpieces, not just merely good things.
There are a variety of positions one could take once reaching enlightenment: 1. Domination. One could choose to be controlling of the entire field and use it for selfish benefit. 2. Dismissive. Once you reach the pinnacle it becomes apparent that both humanity and the society it has created are horrid and aren't worth interacting with. 3. Appreciation. One could choose to be thankful for being able to reach such a position and to respond by generously using the mastery to use it for the good of humanity. 4. Constructive Legacy. One could choose to form ones own group around that mastery to preserve it beyond themselves.
I believe the dismissive position is the most reasonable and hence most likely to be chosen by those who reach true mastery.
All four positions regularly occur and you can see it readily in masters of respective fields.
The way a master appears depends on their reaction to having gained mastery.
I think the dismissive position is also most likely because I believe there are far more masters than we ever notice. They are not noticed because the majority of them don't bother to use their mastery in an obvious way.
Why should a master bother to correct those beneath them?
If a master interacts with society they have an uphill battle to being treated with the respect they deserve. For the true master there is great value in going their own way and rejecting society.