I’ve had this feeling particularly working for established teams or businesses, but I imagine this may also apply to newer organisations too.
Have you felt this too? How do you deal with it or address it? What tools or methods do you use to solve this problem?
At my company, we have monthly organizational meetings where a few senior devs go to represent their teams and share what we're doing that could be dogfooded on other teams.
The more shared documentation that exists at an organization, the easier the archeology portion will be. Even old Slack threads and half-finished README docs can point in a direction, and help with hidden requirements that would have taken more time or painful failures to find on your own. The steps I usually see are:
1) Identify a problem
2) Create a useful documentation infrastructure, like a GitBook or Sphinx, and write down the problem in a well-thought out matter
3) Explore old sources of documentation, and rewrite them into the new structure you created; include the original material and perspective on what's happened between then and now
3a) Find references to other sources of documentation (from when everyone moved from Hipchat to Slack three years ago), get copies of the logs, and search further.
4) Repeat step(s) 3 until you feel mostly fine, and/or other people on your team start to ask you questions about the problem because you've written the docs
5) Architect a solution to the problem, and add your reasoning to the documentation
To do this well, your documentation system from Step 2 should be one that the rest of the team actually uses, and will hold up for a while. At the very least, when your Step 2 is someone else's Step 3, try to be more useful than terrible.
Remote work makes this harder, because it means that you can only use digital archives; a lot of people use paper, and if there's no paper, you don't get random lucky bits of paper that turn out to be exactly what you needed. Instead, you'll have to ask people, hope their archiving system of choice is working, and they are willing to share.
Prevention of these problems involves consistently using the same tools over time, indexing all documentation in a central location which is widely accessible, and meaningfully valuing project document creation as actual work within the organization.