Despite a couple of serious attempts, committing to the process for a few months each time, I've rarely found myself referring back to the stored information. More importantly, the process of adding to and maintaining the knowledge base felt more burdensome than beneficial. I've tried adjusting the content and format of what I'm storing in an attempt to make the experience more manageable, but to no avail.
My main motivation for exploring this further is my growing concern about my recollection ability as I age, and I'd like to fully exploit the available tools to counteract this.
So, for those of you who do have a functioning system that you use regularly - regardless of its simplicity or complexity - how do you employ it to retrieve information or enhance your understanding of topics? Insights into how your system operates would be useful, but are secondary for me at this point. I feel like I've read about all the possible configurations a system could have, yet I'm skeptical about the tangible benefits they can provide. I would be delighted to be proven wrong.
- Idea list: Ideas for art or other projects that I'd like to try some day. I'm occasionally adding new ones and I've started work on old ones I've added that finally piqued my interest.
- Gift idea list: Things friends and family may hint at wanting or outright say they want in casual conversation. I add stuff to this list then buy it for birthdays, Christmas, etc. Can get some real nice surprises for people when they only mention things in passing and you happen to catch it.
- Book list: Just a wish list of books. Occasionally I'll buy a few to add to my personal library and cross them off the list.
Just understand that most of the personal knowledge management stuff is a waste of time. It's a bunch of feel-good busywork that's typically sold to you first as a cheap idea, then sold to you as products like Roam Research, Obsidian, etc. Can't really beat simple lists in text files or on paper. I even like physical notebooks that you go through and keep a hand-written index in the back of. Forces you to think and revisit stuff.
You can do it automatigally, but that can get expensive rather quickly.
I use it as the means to understand things. I write as if I am explaining things to somebody else, that else is the future me, but in the process you can quickly understand what is not clear in your head, break it down to first principles, and build it back up again.
N=1, but writing has had an enormous impact on my growth.
Over the decades, I think I've tried just about every method of maintaining my knowledge base, project notes, to-do lists, etc. And the only one that works even halfway decently for me is a wiki.
It's searchable, the simplicity of markup means I won't get side-tracked into wasting time on layout and eye candy, it's easy to search, if I have internet access then I can access it from anywhere, and the wiki I use allows simple embedded spreadsheets and other such functionality for more complex data.
Plus, it's very easy to allow others secure access to it for when I want to share notes, or keep notes that are updated by multiple people (such as when collaborating on a project), or to distribute very large files.
I switched from vscode (back) to vim, and it has worked as well or better than it did before. I follow my own rules. I like the Zettelkasten idea of one idea per card, but if I put more related things in the same .md file, that's OK. I didn't like the flat directory structure, and so I have dirs organized by category. My /bar directory is inside my /cooking directory, and for whatever reason, that makes sense to me. Ripgrep doesn't care, and I always find what I'm looking for.
This markdown hierarchy, that still lives in a repo called "foam", has become indispensable to me.
My personal notes are a .org file that is backed up on a private GitHub repo.
I have sprawled about 5k lines at this point since around 2020.
Some things get a org-bullet (*) and a TODO.
Then I just do org-agenda and it pulls up a nice view of all my TODO items for pruning and prioritising.
Having an automatic agenda and the fact the bullets are tab-collapsible and can be used as an outliner is the only way my notes are not out of control.
My other writing is in apple notes (short lived) or exists in tickets. I’ve tried various things like zettlekasten and obsidian but nothing sticks for me.
My other daily task setting and personal planning is a bullet journal with a future log and monthly logs. Bullet journaling is how to get it all out and process and plan personal things. Actual Knowledge is in the org-mode.
For email I follow getting things done and have an inbox an @action and a @waiting folder. I keep inbox empty and the others get pruned every day.
https://www.lesbonscomptes.com/recoll/pages/index-recoll.htm...
It allows me to index and find words and phrases in files of many types AND view their contents independently of external readers, or using system readers, including text, PDF, or word processing files of common sorts.
> More importantly, the process of adding to and maintaining the knowledge base felt more burdensome than beneficial
I wonder if being able to recollect things later in an organized way may not be worth the effort for you.
Something I've found is that I really don't need a robust personal knowledge base. My goal is to do my work well (and taking notes to support this is definitely valuable). Building a personal knowledge base doesn't support this goal and has a more unclear value-prop, so it's a drag to keep up with.
Wrote an opinion piece on this https://www.stashpad.com/blog/you-probably-dont-need-a-knowl...
(1) It takes time to reach critical mass. Until then, it's overhead. But even then some content was useful from the very start.(2) After that it became very reliable - it's there and I rely on it. (3) For me, it's clearly not just knowledge base, it's PIM - personal information management - KB is just mixed with the rest. (4) After decades, retrieval is still a mix of ideas and tools - not great. (5) I still wish for more efficient data entry / collection.
In balance, for sure a majority of the content is never accessed.That's fine. The system allows un-foreseen uses to happen commmonly and I like that. Do the positives outweigh the costs over the years? I have no idea. I add stuff constantly, some things I rely on again and again, some query ideas are one offs... and usually they work which is great. I know I have a very solid base for recollection. It's not clear how I would do a cost/benefit analysis.
I feel perfectly fine with the ad-hoc collection and storage strategy. That has worked well enough.
The approach is basically "I write what I've used if I still have around what caused me to need it. I associate what I use with where I access the context of the cause". Because issues are context dependant you organize by the context that it came from. If you name a file as the app name you can have the document come up when you search for the app name. Because the content is based on situations and issues, returning to it is predictably probable. I find this is a pretty low mental burden to writing, maintaining, discovering, and navigating. You don't really have to make decisions about where to place something, how to name it, or how to find it because it follows from naturally arising circumstances and contexts.
I don't think I technically need to use a PKMS as I use the special features lightly but it is nice to have them there when you really want them and to have an interface dedicated and designed to deal with servicing the whole workflow. I also like to write down my stream of thoughts too but I keep them separate and use a separate system for interfacing and managing them.
So, how do I employ it? One is to search on a subject I'm interested in or working on, ie to find notes, previous readings and writings, then add to these or go from there. Another is to keep various kinds of lists, (eg lists of ideas, things to read, etc). Otherwise the way your able to link things together with tags and other forms of linking is also pretty great - these are all features I will insist on when I move to something new, since I use tiddlywiki, which is a mostly dead project.
Anything with these features works like an external brain, allowing for recall, with just enough inefficient that you have to engage with the material, and possible mix some notes and facts together to hopefully inspire some new ideas.
There are a number of feature for organizing content (the main ones being Categories and Namespaces) and it has APIs (so i can write simple python scripts to perform actions -- like updating some index pages with a custom format).
The VisualEditor is very nice and it's fairly easy to embed images, videos or even pdf documents.
It's not exposed publicly but I can access it (both for viewing and editing) from anywhere using my own vpn connection.
I update the main software once in like 6-8 months so it's fairly low maintenance (this boils down to essentially changing the version tag in docker compose file, restarting and checking everything's working alright -- with the occasional fixes).
Since it's running at home (on a small computer that i would keep on anyway and that it's also doing other stuff) it's essentially free and i don't have to worry about space or any other metric.
Once the articles get favorited though there is the problem of navigating them, I just added some simple text search capability but there is a huge amount of room for improvement there. If it gets the ability to ingest articles via bookmarklet than it becomes a bookmark manager, add some ability to enter notes locally and it becomes a more general knowledge base manager.
It could really use "tags" but when it does these will be tags for the ML age, particularly it will be possible to have positive, negative and indeterminate tags. If the ML system thinks a tag applies it goes into indeterminate status and could be set to negative or positive to make it a training example.
I had a WikidPad Wiki (python desktop based wiki software for Windows and Linux) that has 1000 entries. There was a number of pages that were long. But wikilinks in that wiki were often definitional (rather than explain inline, they had their own page) and multiple pages linked to them. The homepage of the wiki is a launching pad for links to other pages.
That wiki is a design of my ideal computer system, it's on GitHub as idea-wiki.
I think I prefer the all in one file thing because I don't need lots of very small files with very little information in. I also read over my notes regularly.
If you want value from your notes I feel you can read over them regularly.
Fast forward another year, I saw a mention of a book called Building a Second Brain by Tiago Forte. I bought it and read it in 3 sittings.
The concept was that you are only capturing what resonates with you. Stuff that is actionable for your projects.
The structure of your folders helps with this. So current projects are easy to locate. You can keep working on them.
He suggests in a bonus chapter on tagging to just use the search mechanism of your app. He gives some other useful ways tags can be used.
Fast forward to today, I am actively reviewing my notes and improving them each time I touch them to make them more useful.
When I was doing my masters in anthropology at the same time as my SWE job, I ran into this issue of having too many notes and not enough organization. Random other tools never quite did it, generic search didn't work, notion et all kind of sucked, so I ended up rolling my own thing on top of Org roam. It's all plaintext org files, easily accessible.
When I do scholarly work, it's great because all of stuff is in one place. It's also useful for tracking the breadcrumbs I do for my real tech job.
With the release of ChatGPT my need for a knowledgebase has decreased by 10x. And I assume that the need for knowledge base/bookmarking solutitions will disaapear completely in the next few years as LLMs improve.
I keep a WikidPad wiki locally. In the past I used word2000 outline mode for logs at work.
I've found ideas from Andy Matuschak extremely useful - https://notes.andymatuschak.org/About_these_notes?stackedNot...
On a practical level the crucial features of what's been working for me for a few years now (inspired by Obsidian community and Andy):
* Notes have to be very clearly "bounded" - break things into smallest logical "statement" units, and interconnect notes instead of having behemoth notes with everything in one place
* Note titles have to be an explicit "statement or sentence summary of the contents, with relevant context" - which really helps with search / discoverability. E.g. "Browsers are the most common and visually advanced applications that make use of the internet, but there are also many others" instead of "Web browsers" or "Applications that use the internet". Or "The day I embarassed myself at football". An alternative of this is to have explicitly clear headings.
* The idea of putting notes into a "reverse spaced repetition" algorithm where instead of spacing things you've learnt further and further into the future, you are spacing things that are "fruitful" nearer and nearer in the future (and when you have less to add, further into the future, when eventually a note becomes "evergreen"). You then have a "queue" (mine is always packed) of notes that are due, and you pick them up as you like. This can be achieved with https://github.com/st3v3nmw/obsidian-spaced-repetition and is described better by Andy: https://notes.andymatuschak.org/zJrfPCbY7GcpV9asEc8NTVzXTAV4...
With the above things combined, I usually sit down just to write and unload things from my mind and enter notes into the (basic) "attention algorithm". That way I never stress about polishing notes, and there's always a time in the future when I come back to a note written a long time ago, make some edits, maybe split it into a couple of notes, clarify the title, maybe merge a duplicate (if something has come to mind twice, it's clearly important to me!!). With explicit titles I almost immediately know what the note is about, trigger new connections (between the time I've written the note and now, some other relevant thing may have surfaced which changed what I think or know, or I've come up with a clearer way to describe the same thing).