Good ways to capture institutional knowledge?

Store readme markdown files in the sourcerepo along with the code itself. Make sure during review that changes to code are reflected in the markdown.

Doesn't need to be exhaustive docs - usually just a high- to medium-level explanation of what why and how goes a long way.

Controversial/surprising/confusing choices should be documented in several places - e.g. in the readme, in a bug/ticket, in the check-in comments and also a comment in the code referencing the readme/bug/ticket for more info.

Over-communicating the confusing/surprising stuff helps a lot and helps to prevent the "what the hell is this crap? Let's rewrite it" issues since there is a long paper-trail explaining why things were done that way. Putting code comments referencing bugs/tickets etc (ideally with clickable links direct to Jira or whatever - e.g. "// This does - see the discussion in http://bug tracker/12345678" ) means that the trail starts right there in the code, and people have not had to trawl through some nonsense wiki to find the hidden nugget of info (let's face it - we'll read code but hardly ever go out of our way to find and read wikis etc first)

Write write write and keep writing. Then expect to do 10x as much reading. It's exhausting and definitely relies heavily on employees' writing and reading comprehension skills.

HashiCorp produces a mind boggling amount of prose (non-code text). Every employee can read every RFC going back to the first sketch of terraform which was completely rewritten in a second revision. Mailing lists are alive and well. PR descriptions and discussions are often longer than the code being changed.

No other company I've worked at has had this dedication to recording decisions, and all of them have struggled heavily with losing institutional knowledge. HashiCorp isn't perfect, but I want all future employers to at least pretend they're remote-first as that seems to be the forcing function for writing everything down.

Update: while writing skills are helpful they're definitely secondary to just ensuring knowledge is splatted down somewhere in some form. Perfect is absolutely the enemy of good enough, and I'd rather struggle to gleen knowledge from an unformatted readme in a deep dark corner than have nothing at all.

Automate everything that can be automated. Avoid setting up things using GUIs.

Starting a set of services should be as simple as "docker-compose up", building should be as simple as "make", checking out the code should be as simple as "git clone", etc. You shouldn't need a shitload of wiki checklists that describe how to install dependencies and how to check out all the git-directories with correct versions relative to each other. Save your wiki for high level documentation

A must for this to work is to avoid configuration state in your servers that is not captured in your source tree. This is the most critical because it's easy to forget and it can quickly become a blocker, not just for RnD but also ops. You have this super important service that everybody depends on and it always works, suddenly the server burns while go-to-guy is on vacation and nobody knows how to bring it up again from a clean slate because it involves several hours of clicking around in some proprietary GUI and ticking all the correct boxes. If you can't reproduce this service today, make a VM snapshot already, then start work on fully declarative service configuration.

1. Working in pairs or teams. Avoid solo people working on projects.

2. Common, easily searchable place to put all documentation at. Good search capability is critical. Wiki is ok.

3. A good code & commit search engine. Ability to search code reliably obviates the need for a lot of documentation.

4. Weekly knowledge sharing sessions with the whole team. Both presenters and question askers need to be rewarded to keep engagement.

It is like replication in distributed systems. There are varying levels of redundancy you can get, and each higher levels involves higher overhead than the previous, so there is no golden rule - it needs to evolve as the organization evolves. A startup might have many people who are the only people who know certain things, but a 10000-person company surely should not have any institutional knowledge bound to one person.

The pushback will come from two places: people who can't write (the overwhelming majority of people) and people who usually act impulsively and without a sound basis (also the great majority of people). The presence of these classes of people in a company will lead to the punishment of people who can and do write, for two reasons. The non-writers won't be able to tell that the writers have done something useful and therefore won't be able to highlight it as an accomplishment in peer or manager evaluations. The irrational actors will get all the accolades for "having impact" even when their random, unjustifiable activities have clearly harmed the company. Meanwhile the people who were writing the design docs will be judged for having less impact.

The way I like to measure this is there should be more writing than programming going on within the company. Some investigations, research efforts, or designs will lead to nothing, however every implementation should come with research, design, and retrospective documentation. In that case there will always be at least as many written artifacts as programs.

The way to prevent the nightmare of an illiterate workforce with only oral history is to hire people who can write and practice documentation-driven development, instead of hiring people who can memorize leetcode trivia.

Mandatory vacation. This is something that the finance industry has used for a long time to guard against fraud -- it's hard to cover up something if someone else has to do your job for two weeks straight at some point -- but it also serves as a mechanism for requiring you to cross-train people.

Two weeks of paid vacation where the company isn't allowed to email them or call them for help: I guarantee that documentation practices will increase substantially.

I'm not sure about the "successful companies institutionalize the knowledge of their employees" part.

In such context a process becomes a (written) 'procedure/specification', and some folks stop innovating, they just do it "by the book".

HR quickly grasps this and hires people with less and less skill, cheap personnel 'just able to apply to procedures'. Other ones feel like cogs in the machine (especially the best ones, hunted by competitors) and quit.

Someone departing with the 'procedures' may let a competitor obtain a rather complete grasp of it and adopt the best bits.

Letting each team decide about this and establish cross training seems preferable to me, and has many other benefits.

We record screencast videos demonstrating how to do a process. New people can watch those videos to learn how to do it.

If we find a better way to do something then we make a new video.

There's a team member who transcribes videos into google docs for people who like to read and search in google drive.

It's pretty simple and it works wonders for an international team.

I wish there was one.

In companies I've worked at large and small, the most important information just comes from people's memories ("oh yeah because we decided X in that one meeting six months ago, remember?").

And then once a critical number of people leave the team/company, so much time is wasted reinventing and rediscovering things.

The only solution would be for everything (every meeting and decision) to be rigorously documented with outcomes and justifications, and for every new team member to go back and read the entire history to get up to speed.

But 90% of people seem to detest writing and/or are terrible at it, and it takes up a lot of time. And new hires are never given the time it would take to read the whole history anyways -- they're excited to start working and the team needs to start showing extra productivity ASAP. So for both reasons, it basically never happens.

So you just hope that the rate of gradual knowledge osmosis from old members to new members is faster than the rate of turnover. And when it isn't, you just accept that your team's ability to deliver profitable features will slow down drastically. And at least for the critical business logic in your product, you mostly rely on tests to make sure at least things don't break when a new team member starts changing things.

It sucks.

People who have been in the job for a while aren’t always the best people to explain something.

What I’ve often done is asked new hires to document what they discover. New people are easier to mould to a new behaviour and often have the questions you need to know. When documenting becomes the habit, more people do it. Current 500-person company is very good at documenting many aspects, because we’ve done it since year 1.

A trick I've used effectively is to have a "doc of docs" - a document that tells you where all the other documents for a project or team live.

You can do this as a wiki page or a Google Doc. The important thing is that the answer to the question "where's the documentation for X" should ALWAYS be "it's in the doc-of-docs".

Then you can take it a step further: you can say "it's in the doc-of-docs... and if it isn't, when you find it, add it to the doc-of-docs!"

It's a universal truth that documentation for projects and teams ends up scattered in many different places. A doc-of-docs is a lightweight technique that can really help here.

I started an internal blog - using Confluence, because it was already a tool used by the company so I didn't have to convince anyone to install anything new.

My goal was to introduce a culture of internal blogging at the company. I didn't really succeed on that front - I used my blog a bunch and a few people made a post or two - but I like to hope that if I'd kept at it for longer (I left the company) it would have started to catch on.

The reason I like internal blogs is that they release you from a lot of the pressure of writing more formal documentation. If there's a technique that I think is a good idea but that has not been established as an agreed best practice, writing it up in official documentation doesn't feel like the right thing to do. Writing it up on a personal internal blog as "as-of date X my opinion is that we should do Y" is always OK.

Likewise: writing documentation that later goes out of date can cause harm. Writing a blog entry that says "as of February 2020 this particular system worked in this particular way" seems much safer to me.

Some years ago (before wikis and the www) I worked on a software product for recording Design Rationale - the decision making process that went into designing long lived artfects, such as nuclear reactors and process plants. The idea being that engineers could find out WHY something had been done the way it had decades after the original engineers had retired or been run over by buses. The product failed, despite some intial interest from big companies.

In retrospect one of the main reasons for the failure was social rather than technical. The engineers designing things saw recording the rationale as just an extra job that had to do, with no immediate benefit to them. If anything, they saw it as opening them up to more scrutiny and increasing the chance of their job being outsourced in future. So keep that in mind.

The value of institutional knowledge is that it's already in someone's head. Reading documentation, searching a wiki, email archives, may be better than reinventing the wheel but the real way preserve it is retention.

I would invest in that if your are in a complicated or specialized domain where it takes months or years for someone to really get their mind around it. One thing I have noticed is that private work environments tend to be present at the low turnover environments I've seen. At least a cubicle.

Another option is extending low hour contracts to departing employees as sort of an off-ramp to their role. They are on a few hours a week or as-needed to handle the dwindling number of cases where they are most needed.

I'll focus here on the knowledge of the key people you're about to lose. Ideas:

* If they're good at documenting, and willing to, task them to do as much of that as possible. The documenting might be in adding code comments and API docs, writing separate text files, etc. The person might not be able to document off-the-cuff, but have to work through a topic slowly, such as going through and re-understanding some old code themselves, going through a manual process that they do automatically and reflecting on the whys, etc.

* If some of the information is amenable to giving a talk to other employees, with a Q&A section, that might work, too.

* Another option is to have another employee interview the person leaving about one or more topics, and either type notes as they go, or record it and get a transcription. The interviewing person should be able to understand the topics.

* For tasks the person leaving does, you could have other people do the tasks while the person leaving is available for questions, and one of them documents as it goes. Depending on the task, it might make sense to have the knowledge-holder right there, both answering and observing, rather than only available for questions on-demand.

Side suggestion about accessibility/discoverability/maintainability of all this new documentation: consider keeping the medium simple, and avoiding a proliferation of locations, formats, a dozen bullpoop communication SaaSes, etc. For most software work, for example, inline embedded source code comments and API docs can be an easy way to try to keep a lot of information accessible in context and maintained. Some other information that doesn't fit well in source code, such as ops architecture and procedures, might be Markdown files in that same code repo, or another one. The occasional video file you just can't check into git might be a rare indispensible one, but can still be linked from a Markdown file that's in your repo, but even then, maybe you also have a text transcript in the repo, or someone turns a talk into edited docs in the repo.

Incidentally, much earlier in organizational knowledge sharing, I vaguely recall a study by a consulting firm (sorry, no cite handy, and I'm not 100% sure I remember which big-name firm), in which they found that people were resistant to having their knowledge captured in a system, because that knowledge was an asset of the individual. Your key people leaving might be more altruistic than that, want to help out their colleagues, want to have a good word-of-mouth reputation, have a sense of professionalism about it, have equity in the company, etc. You might like them to do knowledge transfer to a degree that's really above&beyond the call, so consider how you might acknowledge and thank them for that. It might also be a good example for others, and promote more proactive good practices for organizational knowledge.

Cross training. It's easy to lose institutional knowledge when someone leaves if they are one of the only people that has worked on their projects. Much harder to lose that if you do even minimal cross training. Even if it's only once a quarter, have each person sit with someone else, have that someone else explain their job, their major projects, their details, pitfalls, strengths, etc. Lots of notes should be taken. A knowledge base should be a given, but that will only take you so far. A side benefit is such a wider picture of what's going on will better inform the work of everyone, keeping people not only on the same page, but understanding all the details of how those pages interact.

Edit: Yes, they'll hate it, but this should include a tour of duty Sales. Poor relationships & acrimony between sales & dev are based on a failure of understanding each other's jobs. And sales are the closest to the needs of customers, short of sitting with customers themselves.

Get your key knowledge holders to dedicate 20 percent of their time to documenting what they know. However, most of them are so used to knowing stuff that they aren't sure where to start or what the need is, specifically. Therefore, make them take requests from people who want to know what they know. Then what you end up with is really knowledgeable people spending 20 percent of their time asking everyone around them what they can document and documenting it. Check in every now and then and ask to see what they've come up with.

I've struggled with this personally at our company. A sibling comment mentioned a README for each project/process. That's definitely a solid start for building this up from nothing. Copy open source project README files:

1) what is it? (A web project, an automation script, an ansible deployment repo?)

2) what dependencies do I need to run it? (Make, NPM, Java 1.8?)

3) how do I run it? (docker-compose up? make && ./a.out?)

We started with this. Then for the bigger projects/monorepos, we started adding README files in relevant subfolders.

Recently I've been converting these README files in the larger projects into mkdocs subfolders that get hosted in our repository tooling (GitHub/GitLab pages).

Start small. Go slow (if it's institutionally difficult). Build up to more complexity as you get more written material to work with.

I've started creating an "index" project, that links to all the projects that have documentation.

And finally, focus on the pain points first. One of our monorepos was fiendishly difficult to deploy correctly, either locally or in a test / production environment. The very first tutorial I wrote was setting up that environment in a step-by-step, repeatable manner, and it's by far the most oft-used documentation we have. With that out of the way, I can focus more on the esoteric details (and, yes, unfortunately, it's a bit of a thankless, "skunk works" project, but it's worth it)

A per-project or per-process README would be a good first step. Ideally keep the notes/instructions as close to the place where the work will get done. It's important for someone to test the instructions in the README on a new machine --- 9 out of 10 times you'll find a step about authentication or some system dependency was not documented. Credentials are extra tricky, so you'll have to think extra hard how to make that work (e.g. some sort of central key store, shared password manager, or ENV vars that need to be defined so you avoid putting any sensitive info in the README).

For something even better than a README, you could document the steps of a technical procedure in a Makefile (or Fabfile) that your colleagues can run. It's important to keep the scripts readable and stupid (as opposed to abstract and powerful like ansible), so that people can read the steps. Some people refer to this as "runnable documentation."

The easiest way is just to make sure product/business discussions happen over email rather than Slack. This way those discussions can be searchable and discoverable by anyone within the company at any point in the future. We make software for this (FWD:Everyone), but there are dozens of other similar solutions as well. That's the great thing about email, because the standards are open you'll be able to extract more value from that same data with each passing year using products that haven't even been created yet.

Conceptually this is similar to having a wiki, except for that unless your company has people whose full time job it is to maintain the wiki then it will always be out of date and inaccurate; just deploying some wiki software tends to be pretty useless, and even in the rare cases where they are maintained the medium inherently doesn't preserve the tacit knowledge contained within the decision-making process itself.

At each of my last 3 jobs I've been a strong advocate for tracking knowledge in a Wiki. We've used MediaWiki and Confluence, and both worked well. It helps if everyone does their part to contribute, which doesn't always mean everyone has to write. Some people can dictate what they know. Some can provide screenshots. Sometimes just getting someone to leave a comment with a correction is enough. I like to start with stub articles and allow them to grow organically, without harping on folks about it.

Often there are many ways that changes are communicated (for example, an email to the NOC team to advise of systems being decommissioned, an email to the helpdesk to advise of new support procedures, etc). To some people, that email is the documentation. I know that if I want it captured in the knowledge base, that email is my cue to transpose the notification email into documentation that can be referenced later.

My current job is doing sysadmin support for a consultancy, so there are lots of clients and lots of other people doing this work. We have a regular pager rotation for emergencies, so if you get an overnight call to work on something and can't find the information about it, you start to realize how important the docs are. That has been a big motivator in getting people to update docs about the things they know. If you don't want a call overnight, make sure there's nothing that isn't in the wiki!

I use plugins to help point out when content might be out of date, and guide people to archiving out-of-date docs, or to update them. I regularly refer to Stewart Mader's great book Wikipatterns and its list of patterns/anti-patterns to help with wiki adoption and behavior. http://stewartmader.com/wikipatterns/ The book is a great read and covers a lot of your questions, especially skeptics and opponents, and how to address them.

I really don't have any good recommendations, because I've never seen this done well. Never. In small or big companies.

The two biggest challenges I've seen over and over again are:

1. People don't document changes well. You end up with pages and pages of outdated documentation describing things as they were years ago. This is especially hard on companies that have multiple teams using same underlying platforms/frameworks/libraries. Each team is going to create documentation on how they're using it when they do the work. Once the underlying technology does a version change / incompatible change, all of that documentation becomes obsolete but remains in the knowledge bank polluting the search results. I've become convinced this is an unsolvable problem.

2. Search. I have never seen a documentation system that gets me the information I need easily. It's closely tied to the problems described on the previous issue, but also almost all search systems I've seen are really bad at identifying the authoritative sources of documentation and pointing you to them as the first result.

Also think of your audience. The best documentation I've seen clearly separates documentation intended for maintainers of the code and users of the code.

One of the things I practice personally is more-or-less documentation driven development: https://gist.github.com/zsup/9434452 Although I'm using writing developer-level code, so I'm not literally writing end-user documentation. But I follow the principle; I generally start any significant function by first writing the documentation for it at the appropriate level of detail.

I usually also try to budget a day or two at the end to look at the documentation and generally clean it up. I can't always get "fresh eyes" on it per se but at least I can make sure it seems to basically flow.

The side effect is your major project also comes with basic documentation for very cheap. Honestly, it may even be "cheaper than free"; I do this because I think it helps make better code, faster. The act of writing the documentation doubles as a self-directed design review, and I couldn't even tell you the number of times I've documented some particular thing only to realize how stupid it is before I even wrote a single line of code [1], and started rearranging things at the cheapest development stage there is. But I concede in advance that proving it's "cheaper than free" is pretty hard.

I don't present this as a full organizational solution, of course, but unlike those full organizational solutions, this is something anyone reading this can pick up and start trying out tomorrow, whereas "redo how our organization conceives of how we store information" is a bit less immediately actionable, shall we say.

[1]: "Wait, I'm requiring what precondition of the callers? Wait, I'm going to return six values? (Better make a new struct/class/object.) I seem to have an awful lot of functions asking for the same 4 parameters (again, new struct/class/object). I'm asking for how many incoming parameters? These are awfully complicated instructions I'm giving about what they can and can't do to the return values. These instructions on the transactionality of this call are stupid complicated." etc.

At my first job we were in a similar position I think with lots of churn (volunteers, young people).

I argued with my boss and finally got a vm with a MediaWiki instance. I created a simple frontend using Javascript and html to simplify correct addition of common page types.

Obstacles: my boss and certain other people wanted something more businessy. Also they wanted to run the DB on MSSQL, while MediaWiki is supposed to use MySql I think.

At a later place I was tasked with moving/updating documentation from a two page Word document to a Confluence document. It was written by a long time developer so it grew quite a lot.

Obstacles: Confluence :-] Also getting people to admit when their previous docs are wrong (or at least deviates from official vendor docs, and it creates problems.) Also trying to keep it up to date while others are constantly changing things.

Later on I've

- used OneNote,

- got others to use OneNote

- failed to get people to use OneNote,

- updated Confluence plug-ins,

- struggled with people who said they used README-driven development, but were really just their personal notes on what they'd need to remember.

Finally there are some common theme:

- refusing to acknowledge the need for any system beyond mail and Word documents.

- insisting on buying one lf the "commercially supported" but otherwise inferior and close to unusable system

- insisting on using SharePoint (a variation of the point above)

- putting everything into Confluence which means hopeless search + access restrictions so you never know if 1) the document exists but you don't have access, 2) it exists but you cannot find it because it is in the wrong place and search is broken 3) the document doesn't exist.

A couple thoughts:

1.) communicate asynchronously as much as possible. If you have a synchronous conversation like a meeting, make sure there is a written representation of what was discussed. If it doesn't persist, it doesn't exist

2.) Understand that implementing a wiki by itself will not work. There's a gravitational pull to throw a bunch of junk in it (like a file cabinet). Things become out of date and each person will structure things in a different way.

3. Create some loose structure around sharing regular, written updates about what each person/team is working on.

At my company (https://www.friday.app), we've created a tool that is somewhere in-between Slack and a wiki. It's kindof like a work journal. As a distributed team who only has 1 meeting every week, it's a place where all our updates are captured in a single place.

Every commit should combine:

- The code change itself

- Tests that demonstrate that the change works as expected

- Updated documentation relevant to that change (documentation should live in the same repo as the code to support this)

- A link to the ticket/issue that discusses the change

If you use a code review system such as GitHub pull requests or Phabricator you can enforce this kind of commit culturally - in your review point out that the test is missing or the documentation hasn't been updated or there's no link to an issue.

I like building pull requests up from several commits and then using the "Squash and merge" option to merge them into a single commit to master that includes all of the above.

Doing this is great for institutional knowledge, because "git blame" can always lead you to a comprehensive explanation of the change, including a link to the underlying ticket where the change was originally requested and discussed.

Having a single wiki instance for your company is an awesome way to collect knowledge. You can write up operation run books, design documents, references all in an informal way.

Software. Software. Software.

Encode your knowledge into software, into data structures, into tests and active documentation.

Human organisations have faced these problems for thousands of years. And never solved them well. Maybe we need the new kid on the block to help

Have the key personnel fully participate in knowledge capture. As far as the tool to be used for capturing said knowledge, don't waste time...Just use the fastest-to-set-up and the easiest-to-set-up wiki (or something similarly fast and easy). After the key knowledge has been captured, and likely after the key people are gone, you can look to see if you even need to migrate to a different platform than the original wiki. Knowledge transfer can get tricky in general...but since you're on a time constraint (key people leaving), don't waste time with UI or prettiness, etc. Just capture all the things!

I would guess a lot of what you want to preserve is wisdom. You keep that by retaining the older folks that have it, and have them work alongside the younger people that dont. I've learned so much from other people's war stories. More than from any lessons learned database. Documentation is nice too, but writing it and reading it take time and it doesnt always cover the things that didnt work.

Optimizing things (and organizations) tends to make them efficient but brittle. This is another area where that's true.

It is crucial that the organizational culture reward employees for making this happen.

When you think about it, a lot of corporate cultures reward the opposite -- employees hoarding key knowledge so they can use it to their advantage.

To do this, avoid creating a culture that fosters rivalries. I know it's controversial to say this, as many business people seem to think that brutal rivalries are what leads to over the top effort.

Instead, think about the knowledge as processes, and figure out how to best codify and execute those processes.

If you arrive at a step in the process that appears to be something that can only be done by one person, or by a person who is leaving, that is a clue that someone is withholding information or trying to use it to his/her advantage. Nothing is that complicated, just break it down into simpler steps.

Then, when it comes to onboarding, introduce whichever processes make the most sense for the new hire, and encourage him/her to view them both as an example of excellence but also as something that can be improved. Doing this helps the new hire understand the design perspective behind the processes, which is preferable to following them by rote (which is surprisingly common even among otherwise bright people).

It sounds from the question that you have a bit of drama going on, so you may actually want to sketch out the processes as you think they are and then incentivize the team to make them all precise.

I noticed a lot of suggestions in this thread assume you want to capture the institutional knowledge related to coding and software:

- comments in the source code

- commit messages

- readme

- code review

- pair programming

- variable naming and refactoring

Which is all great, but at the same time narrow. What if I want to capture institutional knowledge when it comes to accounting, HR, PR, management, negotiations, suppliers, local laws, seasonal patterns etc. Perhaps there's a more abstract, generalized advice that could be applied outside of the realm of coding.

The best is the enemy of good here.

When I manage engineers, I insist on people having the old-fashion pen and paper notebooks: not scratch paper; not pads; not post-it notes. Simple 5 by 7 grids. During my weekly one-on-ones, I encourage people to make one or two positive changes:

* jot down any odd term they run across so they can google 'strangeNewTerm slides' later and batch through them all * write a quick note on any error taking over twenty minutes * go back and read their notebook here and there

People do use the notebooks, do record the information, do get sick of writing and do start reaching out for other ways to record information. I allow people to use Wikis, markup files, documents, or a mishmash: people are more likely to work on their own idea. I do insist that documentation be in source control and have 'section last updated' dates. Eventually, I encourage someone to maintain a master index of where non-code documentation is found.

The hard part is getting people not to document too much. Every document has a maintenance load, much like every line of code. It is far better to have an up-to-date hint like "TPS reports need cover pages per YoyoDyne contract, 3/2004" than volumes of out-of-date procedures, one of which mentions the YoyoDyne contract on page 37.

One totally different tact: new developers get assigned to the source code control review. That is, they look at the new commits; run the new code, update the project/company specific dictionary with any new terms; give the tests a careful reading and add to them; and write down any question. New developers quickly become familiar with the code base that is most likely to change.

You’re not going to preserve their knowledge via a wiki. At best a wiki would be a snapshot in time. They’re irreplaceable knowledge is likely an understanding of how change happens within the company.

The answer is by requiring close team effort. Pair programming for example.

I’d one person leaves the organization, there should be “adjacent” employees who understand what they contributed.

Wouldnt the answer be: commit these dedicated, talented, key individuals to the knowledge tranfer effort, exclusively? You're trying to prepare for losing them. So, try it out!

Pull them off, and assign them to a cushy knowledge transfer process. You'll more safely learn about your true dependencies & get some of the knowledge you want at the same time.

1. Wiki w/ comments

2. Corporate university

3. Culture of anti-knowledge hoarding, pro automation, well-documented procedures and no "we need Joe, only he knows how to do vital task X."

4. Succession planning

Simple rule: Require that new hires should be able to become productive without having to talk to anyone (physically or electronically). The rest will fall in place.

There is only one way - 1) higher management must acknowledge that this is needed, 2) higher management must approve time allocation for these activities (as opposed to stuffing all new iterations 100% with new feature development). Then they push it lower and lower, down to the engineers. No way it will happen in reverse, when engineers decide they want this it usually end with a several disconnected resources, often using different tools and each maintained by 1-3 persons effectively for themselves, because nobody reads what they wrote. Also everyone need to accept that this activity does not have "end", and that it may need multiple reworks along the way. Basically this must be a thing that everyone just does, all the time, however they can.

Using and integrating a twiki as part of whatever you do ensures there is always a write-up. Encourage everyone to write up and make looking through docs as part of your review/discussion process. For this to work, you'll need to sort of make it integral to everything you do. Info in a twiki becomes stale over time, but better to have some stale info that can be updated than to not have any info at all.

Something I wished companies do is to ensure that discussions happen in internal mailing lists (rather than in individual email accounts) and have all mailing lists searchable and accessible to anyone in the group. This would simplify understanding how certain decisions were arrived at.

Use documentation unit tests - tests that introspect code and then scan the documentation to make sure that specific things are at least mentioned in the documentation.

Applied carefully this can really help encourage a culture of documentation that stays up-to-date. You can't land new code if the tests are failing, which means you at least get reminded that documentation is something that you should be thinking about.

I wrote about this technique here: https://simonwillison.net/2018/Jul/28/documentation-unit-tes...

Force people to stop asking the most experienced person to answer questions or solve problems.

Have people document how they diagnosed problems after fixing it, including queries for searching logs.

Thank people for documenting things.

I think the solution is more cultural than technical. When a company develops a culture that fosters the creation of technical documentation, and encourages employees to document absolutely everything (both the how and the why), then institutional knowledge is simply a byproduct. When a company focuses much more on shipping products and de-values everything from architecture documentation to API documentation, then institutional knowledge suffers.

However, to make the institutional knowledge useful, it must be easy to find. Thus, I think the second most important thing for capturing institutional knowledge is to have a small number of easily searchable places where documentation lives. Markdown files in the source are convenient place for documenting particular projects or code, but more general-purpose knowledge should be in a wiki or any other document store that is centrally searchable and update-able. An example of such general purpose knowledge is "how-to knowledge": How do I request the appropriate privileges to integrate my service with Service X? How do I make and deploy a staging build? How do I set up a new service?

Another sort of general-purpose knowledge that should have a single home is knowledge around context for past decisions that were made for good, but not obvious reasons. My team maintains a document called a Decision Log, where we record the context around and reasons for every decision that required more than roughly 10 minutes of thought. Longer decisions have their own docs, but they are linked from the central Decision Log.

Have a wiki that anyone in the company can write to without knowledge nor approval from anyone else.

Has to be easy to use, easy to write to in particular, and quickly searchable.

Resist all attempts to impose process or standards to wiki entries. If that fails, create a second, "draft" wiki immediately.

A few people are quite disinclined to write things down. A few others are quite inclined. Most of us are in there middle. It's important to remove barriers and make it easy. Think of it as a product that users have to want to use.

This is one of those "It depends..." questions, where it really does - on so many variables that it would be a major chore just to list them all. So an easier way to go about just establishing an idea of where to put the first bite would be to get an idea of the lateral limits. You are already familiar with the startup side of the spectrum, so spend a little time thinking about the extreme opposite: multi-century continuity, the US military's system of knowledge transfer. To be clear - you really don't want to emulate it, even to the degree that IBM did with their manual that helpfully informed employees of the correct way to sit at a desk, but know what the extreme looks like. Maybe pick a few high level concepts out of it:

* Graduated levels of mastery: few people need the details for standing up a production database, but all department members need to know that a company process already exists for it.

* Training dependencies: the mapping doesn't have to be perfect, a little goes a long way in giving shape to a training program.

* Training records: log employee training sessions, from self declarations of 'Yes I read this brief' to consultant led seminars.

Basically 80% of the military's comprehensive training program can be implemented with a network drive of docs and a spreadsheet. A couple of places I contracted for was pretty heavily reliant on Microsoft Sharepoint. I'd be surprised if there wasn't a fairly decent open source suite of components that could give you a solid starting point for a training program what would be somewhere in the middle of the spectrum.

Spending hours updating your Notion or Confluence is busy-work and will be incomplete and eventually become stale anyway.

Try speaking to people in real life. It's not that hard.

Most companies start out with people that are flexible and capable to do a lot of different things. As you grow, it becomes more and more important to have specialists that focus on only a small fraction of the overall work you have to do as a company. Here is what I would do:

* let people work in small teams/duos

* share knowledge by using a Wiki [1], especially the leaders (the flexible people) should write down as much as possible, but collaborate on the knowledge base (answer questions, add stuff that's missing)

* take meetings as needed (don't overdo this, nobody likes to sit in unnecessary meetings)

* automate as much as possible, the common developer does not need to know exactly how the build pipeline works (but a few people should)

* let people try out new things, this keeps them motivated and improves themselves as well as spreading their own knowledge to co-workers

[1] I found this to be very efficient if you have a team that has the right mindset. If you start out with people that are not used to document stuff, it'll be hard to get your knowledge base going. You can take a look at the product we build Emvi (https://emvi.com/), which aims to solve some of these issues.

The issue with written instructions is that people hide it in nested folders, deep outlines, and walls of text. This creates a death spiral, why write when no-one reads. I've been experimenting with different solutions to visible internal documentation and are about to release the culmination of my efforts in a product called Triqla. http://triqla.com

Learn how large-scale open source projects are managed and governed. Set up a team culture that could support remote working, even though teams are at this moment co-located. Can your team keep generating the same output even when they don‘t see each other everyday holding meetings? Such teams tend to leverage more async and written communication, write more complete, comprehensive documentation.

A good code review culture also improves knowledge sharing: We sometimes have two paragraphs of explanation in a pull request that changes one line of code. Those pull requests are forming a part of our institutional knowledge base and they are oftentimes referred during future conversations about past decisions.

The team should understand that spending a bit time today to write things down will save much more time in the future when the same thing has to be explained again and again. Also the team should be encouraged to spend this time on writing documents and not punished because of not starting with the next game-changing product feature.

I think one thing you have to do first is decide what’s really institutional knowledge and what’s really a make-work project designed to appease the people who still get a kick out of playing solitaire. Documenting how to make a Saturn V rocket engine is institutional knowledge, documenting endless workflows that will be stale before you’re finished are not.

If a system is not documented in terms of what it does, how it does it, and common remediations for when things go wrong, then it's a fragile system depending on one person to keep it going.

You need to get (particularly, senior) engineers to buy into the mindset of documenting everything and making the documentation the first place to look (not after all else fails). Leads should hold be held accountable for their docs meeting some standard of "this is a useful doc".

If I were in your shoes I'd get senior engineers in a room and be honest with them about the situation. You're going to need them to do work that they might not be naturally inclined to do, or work that might not seem like a "productive" use of time. You might want to get them to agree on what a good doc looks like (the standard), and what things need to be documented. Maybe take a whole day to do this, with snacks and coffee. Good luck!

I can only say what I know doesn't work.

Powerpoints lunch-n-learns and confluence pages.

Whatever it is it has to be there front and center with the day to day work. Enforcing good commit descriptions, a code review process where a project manager is part of the merge to capture things usually lost in the technical minutia would be my best guess.

To provide a non-coding team perspective. Having an internal database of documents with search capability is critical for explicit knowledge , whether custom built or simply something like google drive. Having a formal process around adding to and updating this store of knowledge is also critical.

For tacit knowledge, the traditional form of transfer is simply interaction in the course of work. having a searchable archive of conversations is valuable. In most organisations, people are only able to search their own emails but not others. Chat is helpful for finding institutional knowledge around simple problems but not optimal for deeper and more thoughtful discussions that tackle larger issues. This is why I’m currently working on TribePulse, an internal discussion platform, designed to surface and capture institutional knowledge.

Don't assume that the architecture is even there for the institution to have control over everything. Small, young organizations are often relying on things they aren't fully aware of, like so-and-so has an uncle or a trust fund or a fast car or something.

When you talk to the outgoing employees, ask yourself if the processes they are describing are even processes that are replicable or controllable by the organization.

So, for example, are people putting things on their own credit card and getting reimbursed? Is this potentially a problem? Do you need to arrange a company credit card so the organization has real control here?

Then you need to document things. But don't just assume that the architecture is even there for the institution to be in control. You may need to create that piece as people leave.

I work at a largish company (~2K employees, many of whom have been here for 20+ years). We have lots of information silos, and vast amounts of tribal knowledge exit the door with retirees every year. Recently we learned that StackOverflow offers their engine for private companies. "StackOverflow for Teams" allows you to set up your own internal Stack Exchange with all the benefits and conveniences that come with that tool. We are just now getting buy-in from IT and Senior Management, and we plan to start rolling it out across the company within the month. It's not cheap, but we calculate that it will pay for itself if it saves every employee an hour or two per year. I'm confident it will do much better than that.

It's a hard problem, but I've had some success with a few approaches. One is pulling the team into a conference room and giving a deep-dive presentation on some aspect of our system. I made a point of doing it via a web conference with recording enabled so someone could always go back and watch/listen to parts in a pinch.

Another is calling an impromptu huddle when I'm about to work on something that nobody else feels prepared to handle on their own. When some crazy bug crops up that I'm about to troubleshoot, I'll often ask a few other team members to shoulder surf while I talk through the approach I'm using to run the bug down. It's a great time to ask the group what they think we should do to test their knowledge.

We work on https://usecodeflow.com, which is a way to capture the way code logically flows (especially used for new-hires to learn a new codebase). Feel free to email me in my profile if you want to chat!

> I am involved with an organization that is slowly growing, is about to lose key personnel, and is looking to prepare.

Let's be honest here. There's nothing you can do to prepare for the loss of these people. You are coming at the problem too late. It's a cultural problem, not a procedural problem.

To fix the problem going forward, you need to establish new cultural norms. Force new employees to document what they find as they learn the codebase and the "processbase". Establish a new norm that a thing must be documented appropriately at each stage before it can progress to the next. There has to be real pain incurred (schedules missed) before this will really start to work. Management needs to have the will.

Strong change control processes: if you want to know why something was implemented the way it was, the ticket authorising the implementation should have all the details including test results and names for who built, who tested, and who signed off.

Firstly, I work at Slab.com — a knowledge hub for teams. So, obviously I'm slanted in my perspective. But I'm not going to pitch the specifics of Slab here for you. Rather, I'm going to share a post we recently published designed to help teams document/write overcome knowledge hoarding (regardless of what tool/system they use):

https://slab.com/blog/knowledge-hoarding/

It's not a direct pitch to use Slab. In it we talk about the three reasons we discovered most folks hoard knowledge:

1. Leverage: If an employee hoards their knowledge, they may feel like they are irreplaceable. 2. Fear: Putting yourself out there can be intimidating. What if colleagues or supervisors respond with negative feedback? 3. Competition: If your workplace rewards personal triumphs over shared victories, employees are less likely to want to share their "secrets."

Also, no matter what documentation tool you use, it's worth reading this first: https://slab.com/blog/documentation-tools/

Many teams initially gravitate toward more generic document editors (Google Docs) as their first team wiki, for logical reasons:

- These editors are familiar to most people, meaning they can be easier to adopt across an organization - Most companies already use (and pay for) at least one of these tools - Teams realize the need for documentation, but aren't thinking of how their documentation tool will scale alongside their business

But there are some issues with these document editors that teams discover as they grow. This article dives deep into those shortcomings. Here are both article links again, and good luck!

1. https://slab.com/blog/knowledge-hoarding/ 2. https://slab.com/blog/documentation-tools/

How do you make people replaceable? By sticking to org structure with roles that most other companies in your sector use, so that they can be replaced by other people who have worked in other companies.

Knowledge is 'institutionalized' via experience first, documentation/formalization second, schools based on that documentation third, mass media fourth, gossip and other forms of informal communication fifth.

Stick with boring tools, boring tasks, then they can be replaced with boring people who won't quit to work on the next cool thing as long as the paycheque is right.

For process documentation (eg, not code docs), we use a company shared Google Drive folder with subfolders broken down by department (sales, marketing, support, finance, general management, hr, etc) and within each we have documents and checklists that serve as both documentation and quality controls for our most critical business processes.

We have videos, slides, spreadsheets, docs - we always try to use the best format for the job.

It's not perfect but you're right in that it has a huge impact on the efficiency of certain processes and therefore, scalability.

Use tools such as QuickQ (https://quickq.app) to capture institutional knowledge when it's shared via Slack!

Disclaimer, I made this app.

I wrote about related subjects here:

https://zwischenzugs.com/2017/04/04/things-i-learned-managin...

Specifically here, the importance of:

- allocating budget to the maintenance of knowledge

- rotating the responsibility for maintenance around the group

- co-locating the knowledge store with the day-to-day tooling, even if that doesn't conform to document management ideals

Confluence, or a similar private wiki is a good idea. As you work, write down steps to do things that were essential to doing part of your work, or things that you will need to repeat often.

E.g. Create tutorials on setting up a development environment, installing dependencies, compiling modules, running tests, creating new components. Write descriptions at a high level of the system, make diagrams of complex message exchanges. Write down best practices, or paste boiler plate code for doing specific useful things.

1. Incorporate documentation updates into your definition of done - at the task, sprint, and project level.

2. Always have an agenda. Always have someone taking notes. Notes must be reference-able (public slack channels count, wiki is better, email doesn't).

3. The first item on all new hire's onboarding: every time you learn something that's not in the documentation, or incorrect, update it. You will be explaining this to the next new hire.

4. Hire a journalism major intern to conduct interviews and cultivate archives.

Here’s a bit of a different answer: implement a company-wide wiki (if your company is medium-to-small sized).

I am in the process of implementing Wiki.js for my team, to a lot of excitement.

Institutional knowledge comes not just from managers, executives, and team leads, but also from those “in the trenches.” The finer details of operation can be lost the higher up on the org chart you go, and that’s where a well-organized and liberally maintained wiki (or other team knowledge base software) becomes invaluable.

Many good responses, but I should also point out: Retention. A bunch of screencasts isn't going to replace someone who got frustrated or just needed a change of scenery.

There are far better tools for managing knowledge than wikis. Check out Bloomfire hhtps://www.bloomfire.com They offer one such solution. You just upload documentation or type into the platform directly and every word in the document is searchable. From the site it looks like they even transcribe video and audio files and make them searchable as well. I know someone who has used it and loves it. Just to say, there are tools built for this....

I've recently made a small things called github-agent. Essentially it lets you keep your github issues as local markdown files. This helps keeping issue descriptions extensive and full of details as you work on them, since you can use your favorite editor (Emacs) to edit them and then sync in a simple terminal command.

https://github.com/k-bx/github-agent

I ended up having to accept that documentation would always end up spread across a dozen different systems: Sphinx/RST + GitHub Markdown + Confluence wiki + Google Docs + Continu + email mailing lists + the customer support website + I found one team using Google Sites + ...

So I built a custom internal search engine that indexed documents from all of those systems and made them searchable in one place. It worked pretty well!

When it comes to documentation, be realistic about how much time - in terms of product features delayed or not delivered - you have to spend on creating documentation and on keeping it fresh.

Also be realistic about how the documentation will be used. I found this paper and its references useful:

https://news.ycombinator.com/item?id=20471577

I have found Architecture Decision Records to be great for this.

They capture why a decision was made and the context/options at the time. They also provide an immutable linear log of decisions which is nice for on boarding.

https://github.com/joelparkerhenderson/architecture_decision...

Just a thought. Companies that are bad at transferring institutional are companies that have employees stay for longer periods compared to companies that do not. If you are looking for job security becoming the internal repo of undocumented knowledge is a good way to keep/grow your position within also be included in more key projects and stay in the loop.

The hardest part of onboarding, for me, is figuring out what to ignore. Every team has so much legacy and baggage.

The culling of deprecated stuff is too hard. The incentives are all wrong. And it's drudge work.

My proposal:

We need to time box and auto expire everything. Like TTL leases. Renew leases on active work. Everything else enters the grooming process.

Have stages, like Trash Bin vs hard delete, to minimize impact.

If your org doesn't already have some sort of wiki or equivalent (we use Confluence here at my day job), that'd be the top priority. Make it as easy as possible for those high-bus-factor colleagues to get their thoughts in writing. Even if it's messy, as long as you're able to search page contents, it's better than nothing.

Have someone learn how to do the things and write it down as they learn. The goal is not so much knowledge (but that is good) as it is being able to function.

Unfortunately, people who already know how to do something tend to skip over crucial bits in their explanations and documentation: "break the eggs before putting them in the pan".

We use a software solution: http://bloomfire.com It's kind of a hybrid between a knowledge repo, search engine and internal social platform. Their customer service is fantastic and they're continuously improving on the product.

One way is to define a project dictionary/glossary.

Shameless plug: I'm working on a tool to help teams define/share the internal jargon that always seems to develop whenever humans work together: https://jargonaut.net

Many people here will talk about practices etc. But tbh I think before that you need to make sure that whoever is in charge is bought into whatever you are going to be doing.

If that isn't in place. Any such endeavours will fail as other people in the business just won't bother.

If large company, making everything - docs, discussions, code etc. searchable through a single box.

In terms of operational knowledge and processes, we work with large clients in France that have huge yearly turnover. We developed operandy.com as a tool for them to document processes. It turned into a way for them to value and automate low-value work later on.

We find that Confluence + JIRA is a very effective way to pass knowledge around.

For my personal use I create detailed "How To" documents that detail the exact steps required to do something non-trivial. Others have found them useful when I posted them on Confluence.

Company tech-talks inter and intra department.

Provide lunch. Have the talks be about something very specific in your infra, a company value, a goal whatever.

But just short of compulsory, make people want to give and attend...

Could someone please post tool names as well? Everyone is talking processes. But I would love to know more convenient tools as well.

This is a great question and the same pain the led us to start ShiftX. It’s a process based knowledge sharing tool where we have focused heavily on simplicity so that both creating and consuming content is as frictionless as possible. We are just starting to grow the company but are getting great feedback so far from our customers in the Nordics. Let me know if you would like to learn more! https://shiftx.com

This is essentially the app my company built:

https://insideropinion.com/

Essentially, we monitor network communications and rank participants in the network by expertise, skills, workstyle, etc. Then we rank content they discuss / share based on expertise (for search). The system will also rank the participants (employees), and monitor the influence.

Hire young junior employees and give them broad responsibilities.

Add e2e tests and design docs for code repos.

Keep your employees around.

tell me about more?