Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Good ways to capture institutional knowledge?
547 points by alhirzel on March 1, 2020 | hide | past | favorite | 213 comments
Successful companies institutionalize the knowledge of their employees; this leads to better continuity and faster on-boarding. Things like huge monorepos of useful code, internal tools, process manuals, etc. are example products of this. Young companies tend to depend on the dedication and talent of key individuals, and in maturation, must somehow make the jump to institutionalized knowledge (so that "if someone got hit by a bus" things are ok). What are some successful methods you have used or seen used to accomplish this transition? What are problems you faced (skeptics, opponents, etc.)? I am involved with an organization that is slowly growing, is about to lose key personnel, and is looking to prepare.

Store readme markdown files in the sourcerepo along with the code itself. Make sure during review that changes to code are reflected in the markdown.

Doesn't need to be exhaustive docs - usually just a high- to medium-level explanation of what why and how goes a long way.

Controversial/surprising/confusing choices should be documented in several places - e.g. in the readme, in a bug/ticket, in the check-in comments and also a comment in the code referencing the readme/bug/ticket for more info.

Over-communicating the confusing/surprising stuff helps a lot and helps to prevent the "what the hell is this crap? Let's rewrite it" issues since there is a long paper-trail explaining why things were done that way. Putting code comments referencing bugs/tickets etc (ideally with clickable links direct to Jira or whatever - e.g. "// This does <surprising/confusing thing> - see the discussion in http://bug tracker/12345678" ) means that the trail starts right there in the code, and people have not had to trawl through some nonsense wiki to find the hidden nugget of info (let's face it - we'll read code but hardly ever go out of our way to find and read wikis etc first)

We use the Markdown file method extensively at our company. see: https://github.com/dwyl?&q=learn But just asking/reminding people to capture learning/knowledge in Markdown is not enough to ensure that it actually happens. If the organisation does not have a learning and sharing culture at all levels having markdown files falls apart quite fast! see: https://en.wikipedia.org/wiki/Learning_organization People first need to unlearn what they were taught in formal education. In school we are told not to share our homework with others and the mindset of individualism persists in most workplaces. We need a way to measure and reward people for their contribution to the collective knowledge.

Doing homeworks together was ok, sharing answers to tests was not. Anyway, the goal there was to score individual students, the goal here is to build something together. Of course some people want to outshine coworkers or try to become not replaceable, so they won't share anything unless forced. This is probably your point.

A way to measure people's contributions could be counting how many times those lines in the wiki were copy pasted to solve problems, but in general it's not easy. We could count commits to markdown but as in code, two lines could be worth more than one hundred.

Yeah, agreed. But sharing the answers is exactly what is needed in companies/organisations. I've worked in companies where people hoard data and solutions to problems because it gives them power/influence/job-security.

As for tracking commits, if the metric becomes a goal it becomes useless. People will game the system to have more commits, or worse, write a bot to break up their contributions into as many commits as they can get away with.

One line can indeed be worth more than 100 if the line fixes helps fix some production bug. The biggest issue I have is people fixing things and not documenting the fix.

Our metric of "success" for our shared knowledge is how many people outside of our company/org find our stuff useful. But this is not always possible in secretive or highly competitive industries.

Kathy Sierra said "out teach your competitors". https://youtu.be/Dsryx3Ra5pU I totally agree with this mindset. https://headrush.typepad.com/creating_passionate_users/2005/...

This is what I do. And if the Markdown file is not near the code in question, it's a 100% guarantee you've wasted time writing it because nobody is going to read it.

I also write comments first when writing complex code and then fill in the code in between. More than once this helped more than any documentation could, because people do not read documentation if they can avoid it, and avoid it they'll try.

> if the Markdown file is not near the code in question, it's a 100% guarantee you've wasted time writing it because nobody is going to read it.

This ignores hyperlinks. Why not use in-code comments for explaining nuts-and-bolts matters (for example, to explain strange-looking platform-sensitive code using #ifdef), and use links to wiki pages for descriptions of high-level design decisions?

This isn't for comments directly on the source code, it's for higher level stuff like "These are the three services that interact to provide such-and-such functionality. Here's their basic architecture, their dependencies and some basic troubleshooting knowledge/tips."

Right, but isn't that what I said?

I meant that I do documentation on the side, not in code. In code, I do human readable comments (using a restricted subset of markdown) where things would otherwise be very difficult to understand.

All of this applies uniformly for the code I write for others, as well as code I write for myself.

I’m also a big markdown-in-source advocate. However it’s major shortcoming is that it’s not accessible enough for non-technical teams to maintain.

> major shortcoming is that it’s not accessible enough for non-technical teams to maintain

At the risk of just mirroring my comment above:

Low-level nuts-and-bolts documentation can't be accessible to non-technical people, by nature. Documentation of high-level designs are another matter. Why not use a wiki for the documentation of high-level decision-decisions? A wiki can be accessible to non-technical staff, and it's trivial to link to a wiki page from a comment in source.

It also keeps the source tighter. It negatively impacts readability if the source is full of non-vital comments.

> Why not use a wiki for the documentation of high-level decision-decisions?

Because if you create different solutions to meet the publishing preferences of different groups inside the organisation, then I don’t think you’re meeting the goal of institutionalizing the knowledge. If a consumer wants to find some documentation, you don’t what their first step to be trying to find the system it’s stored in. Especially in very large organisations, where you could easily end up operating quite a few different publishing platforms.

SharePoint is the closest things I’ve seen to a one-size-fits-all solution. But SharePoint is rather terrible to use.

> Because if you create different solutions to meet the publishing preferences of different groups inside the organisation, then I don’t think you’re meeting the goal of institutionalizing the knowledge.

High-level project descriptions don't belong in code. It's a different beast, and doesn't belong in the repo alongside the implementation. Non-technical users probably shouldn't even have repo access.

In-code comments, on the other hand, can't reside anywhere else.

> If a consumer wants to find some documentation, you don’t what their first step to be trying to find the system it’s stored in.

They don't need to look. That's what hyperlinks are for.

I'm not sure if you mean end-user here, or the consumer of a library.

If an end-user wants documentation, that means it's high-level documentation, not nuts-and-bolts documentation on the workings of code. A wiki is a fine solution for this. A non-technical end-user has no business exploring the repo.

If it's someone looking into how to use your library, the distinction is still there. If I want to know what Qt is, I look it up on Wikipedia. If I want to learn about a specific concept, I look for a documentation page like this [0]. In neither case would source-code comments be a reasonable choice.

> Especially in very large organisations, where you could easily end up operating quite a few different publishing platforms.

Maintaining a stable intranet wiki is no great challenge.

> SharePoint is the closest things I’ve seen to a one-size-fits-all solution. But SharePoint is rather terrible to use.

For UI reasons I'd go with a wiki over SharePoint, but they're similar in principle: documentation lives in the intranet, each document has a URL, and documents are mutable. They can coexist if they need to: use hyperlinks.

I'm not sure a one-size-fits-all solution is a good idea in the first place. A legal document about a project is going to end up as a .pdf, and doesn't belong on a wiki. On the other hand, technical documentation like [0] should be handled in a 'web-first' way, such as with a wiki.

Perhaps if SharePoint's document-editing were more like a plain old wiki, I could be convinced that it's a reasonable one-size-fits-all solution. (Although in a sense it's doing several things.)

[0] https://doc.qt.io/qt-5/layout.html

> It's a different beast, and doesn't belong in the repo alongside the implementation.

That's just, like, your opinion, MaxBarraclough. If they aren't in the repo, they're never going to get updated, and as a result they will be outdated within 6 months on any project that's moving anywhere. I prioritize freshness of documentation against most other attributes. It does nobody any good to have documentation for the system as it was 2 years ago.

I take it you agree that high-level documentation shouldn't take the form of comments in source-code.

> If they aren't in the repo, they're never going to get updated

Not so. The Qt folks use a separate repo for their documentation. [0]

My point earlier was that high-level documentation is a separate project than the implementation. You could keep the high-level documentation documents in the same repo as the implementation if you want, that's just a monorepo.

[0] https://github.com/qt/qtdoc

I don't know what the infatuation with markdown is beyond web publishing. Wysiwyg documents have been a solved problem for decades. Use what works on your platform.

Markdown is fine until you realize you would like to have figures, images and tables.

I don't want to spend my time over trite details of a text based markup when I could spend it actually productively.

At that point it's way over easier to use a text processing document of choice (word or open office). I say this as a enthusiast of text based markups over decades from Latex to Markdown.

Wysiwyg document tools aren't applicable to comments in source.

You can't have images in source, but if you really need it you could add an ascii-art-style table. There are tools to generate these, such as https://ozh.github.io/ascii-tables/

This was not about inline documentation in source, but about separate readmes.

If a hand typed ascii graphic does not suffice for code comment embedded docs, then it would be better to include separate docs altogether.

There is a reason why everyone uses LaTeX (or similar) when it comes to professional content delivery - it actually saves you time not fighting the editor.

There's no reason why internal readmes should be of 'professional quality'. It's nice, but a low barrier of entry to explain something non-trivial is more important than nice layout, IMO.

"Everyone" is not using LaTeX.

Yeah. Except WYSIWYGs take way more time to do anything non-trivial.

This Friday I had spent about 15 minutes trying to put an editor cursor within an empty code block in Confluence. Is that a joke? Do you know how I fixed that? I copied a non-empty block from another paragraph and then edited it.

It would take me or someone else half a second to do that in Markdown/Wiki/LaTeX.

By Wysiwyg I was referring to Word or OpenOffice or Google Docs or whatever the generally used word processing tool in the org is.

It's totally acceptable Confluence at least tries to provide a non-programmer friendly interface for the domain experts who are not necessarily programmers. But it's not acceptable if the gui is broken...

> it actually saves you time not fighting the editor.

And from merge conflicts.

For certain use cases, markdown is arguably the best solution. For a team that uses an SCM like github (or any of the ones that natively render markdown), and that also has simple documentation needs, I can’t imagine a better solution. Markdown is fast to create, can be read rendered or not, can be included in your project repo and in pull requests. Images are easy in markdown, simple tables are easy in markdown (complicated tables very quickly move into don’t bother territory though). It uses the tools and workflow you already have to solve a problem very well.

If you need something more complicated than markdown can provide (depending on your renderer, that could something as trivial as line breaks inside the cells of a table), then it’s obviously not going work. But for any solution that’s great in one use case, there’ll be others where it’s completely impractical.


Markdown is great, but I’ve seen so much time wasted on Rube Goldberg solutions that would be trivially addressed by using less cool solutions like Word or GDocs.

If the corporate ownership changes, the markdown files will stay. GDocs or Atlassian subscriptions might be ended without migrating the data, leading to loss of key info. Yes it happens!

I like processing the markdown through Jekyll to create a web site that is used by both tech and non-tech. So the site is auto generated from the last change by an engineer.

I think it's most important that every line changed in production code can be backtracked to a ticket in the "feature management database" whether it's Jira or whatever. This simple rule helps immensely in keeping codebases in good shape.

This is the only clearcut rule that I can think of that is obvious and non-negotiable.

Other things of course make sense (document something in code, other things somewhere else) but are more matters of taste and culture.

We split it out: non-ticketed changes are recorded by forcing commits with specific commit formatting that puts out a changelog at intervals. That gets you a log of changes that can be either tracked back to a ticket or are self-describing.

(public examples are the Angular Commit Message Conventions or the way saltstack uses that stuff https://github.com/saltstack-formulas/.github/blob/master/CO... - you basically enforce some rules using commitlint and record logs using semver, even if you end up not using the versioning for your software)

> with clickable links direct to Jira or whatever - e.g. "// This does <surprising/confusing thing> - see the discussion in http://bug tracker/12345678" )

That does tend to be a great help if people follow it. One problem though: We're on our 3rd tracker since I started at my job. Lots of knowledge in those old cases disappeared because of it.

Keeping read-only archives of the retired bug tracker seems like the right thing to do - even if that had to be done with an external archiving tool (top hit on google: https://www.petekeen.net/archiving-websites-with-wget)

Write write write and keep writing. Then expect to do 10x as much reading. It's exhausting and definitely relies heavily on employees' writing and reading comprehension skills.

HashiCorp produces a mind boggling amount of prose (non-code text). Every employee can read every RFC going back to the first sketch of terraform which was completely rewritten in a second revision. Mailing lists are alive and well. PR descriptions and discussions are often longer than the code being changed.

No other company I've worked at has had this dedication to recording decisions, and all of them have struggled heavily with losing institutional knowledge. HashiCorp isn't perfect, but I want all future employers to at least pretend they're remote-first as that seems to be the forcing function for writing everything down.

Update: while writing skills are helpful they're definitely secondary to just ensuring knowledge is splatted down somewhere in some form. Perfect is absolutely the enemy of good enough, and I'd rather struggle to gleen knowledge from an unformatted readme in a deep dark corner than have nothing at all.

How do/could you quantify the benefits of this culture? I frequently manage groups of “move fast & break things” folks, and RFCs/design docs/etc are a very hard sell (in particular when teams are fully local)

Quantifying human processes is not one of my strengths, however these are some situations a culture of writing helps avoid:

- Fear to go on vacation or take sick days because you'll miss live decision making

- Paternity/Maternity or other extended leave requiring a second onboarding upon return

- Animosity when left out of a lunch or beer where a design was discussed or decision was made

- Cabals of knowledge holders weaponizing their knowledge for job security or advancement

- Onboarding is a huge drain on existing workers as all knowledge must be shared 1:1 synchronously. Discourages team growth.

- Bias toward risk takers and the loudest voices. Difficult for thorough and thoughtful team members to be effective.

If none of these things apply to you, great! I don't want to presume there's one best way of operating.

Do you write about or teach these things.

Ironically: no. It's something we discuss regularly at HashiCorp and, as you might expect, have a lot of internal resources (docs, videos, training) around.

I haven't looked at it myself, but I know Google just released some training materials for technical writing: https://developers.google.com/tech-writing

Over the years my perspective on this shifted a lot. I now feel it’s not worth it to convince people of the need for RFC / design spec process to ensure alignment prior to implementation.

If you’re working with people who don’t agree with that process, just leave. That engineering culture is bad and you’re not going to get anywhere. People will use the empty excuse that careful design docs slow them down too much to convert it into a political debate, and try to make the burden of proof on the person asking for alignment prior to resource committal, burying _you_ in bureaucratic doc writing to avoid writing self-evidently more appropriate design docs themselves.

The idea of changing this kind of culture is a fantasy and you’ll just burn yourself out. Just leave and don’t work for places like that. Don’t hire people like that.

Typically the move fast and break things culture also means lots of smaller changes. Scale back the documentation to fit those smaller changes so it does not seem so daunting (RFCs may be overkill).

The biggest immediate benefit during onboarding. A new hire can review all the broken paths that have already been tried. Second related benefit is existing employees can go back and look up the details on what was tried and why it didn't work. A prior broken solution may become feasible as assumptions/business/etc... change.

It’s demoralizing to write write write when you know no one is going to read, and if you link them something more than 100 words long they’ll ask for a meeting instead.

Fair. I find writing a useful exercise even if no one ever reads it (although processes should enforce someone reads it; like a PR).

Calling a meeting anyway is great! You have a document to reference to guide the meeting, answer questions, and scribe discussions/decisions! If your worst case scenario is that your technical document becomes a glorified meeting agenda, that's not so bad.

Also remember that documents live ~forever, so even if you get no immediate response: write for your replacement 5 years from now who has to figure out wtf you were thinking. :)

But occasionally you find yourself years later giving your own thoughts a read read read, and it's helpful that you've taken the time to write write write. Future you will thank you. I'm pretty happy with past me for obsessively documenting some things.

Read the document to them aloud, word by word, in the meeting.

Automate everything that can be automated. Avoid setting up things using GUIs.

Starting a set of services should be as simple as "docker-compose up", building should be as simple as "make", checking out the code should be as simple as "git clone", etc. You shouldn't need a shitload of wiki checklists that describe how to install dependencies and how to check out all the git-directories with correct versions relative to each other. Save your wiki for high level documentation

A must for this to work is to avoid configuration state in your servers that is not captured in your source tree. This is the most critical because it's easy to forget and it can quickly become a blocker, not just for RnD but also ops. You have this super important service that everybody depends on and it always works, suddenly the server burns while go-to-guy is on vacation and nobody knows how to bring it up again from a clean slate because it involves several hours of clicking around in some proprietary GUI and ticking all the correct boxes. If you can't reproduce this service today, make a VM snapshot already, then start work on fully declarative service configuration.

That's good technical advice but I don't think it works as far as knowledge capture goes. There are two problems.

Firstly, it fails to solve the problem of actually capturing knowledge. In fact, if anything, you're suggesting that knowledge of the systems and processes shouldn't be necessary in order for the business to function and that building a black box that "just works" is good enough. The problem with that is two-fold. First, using code to capture knowledge (eg "read the makefile to see how it works") fails to capture any reasoning for decisions that have been made, and secondly any history of the changes to the system are lost if you do ever decide to wipe out the git history of the repo (eg a shallow clone, or a squash, etc). Those may or may not be important to you but I've found it useful in the past.

Secondly, there are non-functional requirements for things that can't really be captured in code. For example, "The system relies on an external service that can only be rebooted by calling 555-1234" is knowledge that no amount of single start up command automation can fix if there's a problem. That needs to be in a recovery policy document so everyone knows where to look if the system fails. That way the document can be reviewed by non-technical people as well which is a huge bonus.

I've just had to script an earlier manual process and the last couple of steps couldn't be handled in code. The final log message, written to the console, says, "TODO: Now do this thing ...". So everything is under source control and there's a golden source of truth.

Imagine that a guy who automated a certain part of the process just left, and you need to make some changes. Who's holding the knowledge about how X is automated and the thought process behind it?

Ideally, the automation is available and can be read through to divine the steps it is taking to perform the task. As for the thought process, pray that the author left behind hints whether that be in the form of comments in code, descriptive commit messages, etc. Otherwise it gets a bit more tricky.

Comments and commit messages are documentation and knowledge capture, but done in a way that's really hard to read through and requires technical knowledge and repo permissions to even access. If your team includes people who aren't developers and you need to review a process it's very useful to have that knowledge in a more readable format.

I don't disagree with the overall point, but I'm not sure a non-developer would have a need to understand the specifics of how a task is automated. In theory, the automation is built off a defined process already which should be readily available to non-developers already.

In practice... Well, the initial question was about good practices so perhaps we shouldn't lift the lid on bitter experience.

I don’t like having extra steps to convert things to more readable formats because it’s expensive and boring and tends to get out of sync very quickly.

If non-tech people on a team need to understand, in detail, a technical process, I think it’s easier to teach them to read through a script than to pay a tech writer to document the script, all the time.

I’ve run into this problem with non-technical project managers who want to understand in detail. If this level of understanding is necessary then it’s possible to scan a source file to read comments.

Or generate documentation from source and dump it on a web server somewhere.

Although lately even non-technical people are able to read and edit markdown files on GitHub.

Yeah, I think if you are looking at commit messages to understand a process, you have to take a look at if/where things went wrong.

At worst, the code is there. This is much better than a person leaving who did some stuff manually.

Obviously additional documentation would be great, but code itself is a minimal type of documentation.

Yes, but the lack of reasoning behind the implementation means that the new person has to reverse engineer it (practically speaking). This isn't always bad, but in most cases it is.

My point was that by simply automating the process you don't institutionalize the knowledge as much as you move at least part of the problem somewhere else.

It heavily depends on the application, though. In my domain the implementation is rarely self-documenting, and most of the system's complexity is usually baked into the spec or the human part of the process. In some other domains the proportion is completely different.

This is great until you need to change the automation, or something it depends on breaks. Hiding it from a new employee is a wise idea, however, but at some point they may need to know these things and for that to happen it needs to be, you guessed it, documented.

1. Working in pairs or teams. Avoid solo people working on projects.

2. Common, easily searchable place to put all documentation at. Good search capability is critical. Wiki is ok.

3. A good code & commit search engine. Ability to search code reliably obviates the need for a lot of documentation.

4. Weekly knowledge sharing sessions with the whole team. Both presenters and question askers need to be rewarded to keep engagement.

It is like replication in distributed systems. There are varying levels of redundancy you can get, and each higher levels involves higher overhead than the previous, so there is no golden rule - it needs to evolve as the organization evolves. A startup might have many people who are the only people who know certain things, but a 10000-person company surely should not have any institutional knowledge bound to one person.

> Common, easily searchable place to put all documentation at. Good search capability is critical. Wiki is ok.

I have mixed feelings around documentation because I can often read the code faster than the docs, and docs are often incomplete, inaccurate, and out-of-date. Docs for truly long-lived things are nice, though.

As for good search, that's easier said than done. The heuristics Google used for search don't work in code, and searches are too rare to do useful ML for relevancy.

Edit: seeing the downvotes...

I don't actually mind writing documentation, but more often than not, I've found people like(?) writing it because it makes them feel like they're improving the situation by doing something. I'd rather the effort be spent on better naming and factoring in code. Docs are also very prone to rot and drift.

That said, I've found Java docs, MDN, and most man pages to be very good, in part because of how thought-out the docs are and how static the interface is. I'm also a fan of docs that bootstrap new developers. Someone else said they like docs describing "principles," and I like that idea--guidance so you know what A should do vs. what B should do.

> I have mixed feelings around documentation because I can often read the code faster than the docs, and docs are often incomplete, inaccurate, and out-of-date.

The problem I have everytime time I read or hear this statement, is that code is excellent in telling "what" happens, but often very opaque in "why" it's done like that. If the why isn't clear, code might be changed/refactored and stuff breaks, because the developer didn't understand the reasons behind the apparent code smell. It's comparable to Chesterton's Fence [1].

I much rather have some additional documentation than "self-documenting code" that does (apparently) weird things and tells me nothing about the reasons. Also outdated documentation can be helpful, when it's version controlled. That gives you context how the code evolved and if it doesn't state it directly at least pointers why the code is like it is now. That's also why I mostly prefer in code documentation to Confluence/wiki documentation, because the time/change relationship between code and wiki is much harder to comprehend.

[1] https://en.m.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fen...

The problem with "Read the source, Luke" is that even if the code is well written (and that's a big if), reading the code only tells you _what_ it does, not why, not why it doesn't do it differently, nor what it may or may not do in the future. It's the difference between programming and software engineering.

While reading 'teh codez' can work for a (simple) app it will get you nowhere regarding the whole service architecture/infrastructure.

There can and will be several services/apps working together, external services required for some stuff, different set of configurations for different environments, CI/CD, software packaging, etc...

How we deal with it where I work - service developers/owners are responsible for providing docs for their services. Ops provide infra/CI/deployment/high level 'how all this works together' docs. Everything that can be code, should be code - with it's documentation. Of course there's issue tracker, wiki, commit messages all tied together using issue tracker IDs, etc.

And it still takes time for new people to get their heads around the 'how everything is working together'. Amount of docs we're writing is significant. Some of the docs is auto generated. Time for docs keeping is accounted for at estimating the time required for developing new features. Being a member of a remote team I can't imagine how we could be able to work otherwise.

The way I like to think about a well documented system is:

- unit tests show how well the developer understood the requirements at the time they wrote the code - git commit history shows who and when something changed (probably assuming history not rewritten :) ) - git history can be more informative, if and it is a big if, the developers write enough information and not just “changed code” type comments - in-line comments are for “why” - explaining things that look odd or go against standards or best practices, or “I’m doing this now like this, when x is available use that” - notes to help you and others - wiki is for higher level “why” to help people understand the code, where there is a lot of complexity I really like the idea of a “book of the xx” like the book of the runtime here: https://www.hanselman.com/blog/TheBookOfTheRuntimeTheInterna...

Your position is reasonable. The value of spending time on documentation in a vaguely specified and frantically written web app is not the same as, e.g. writing documentation for the frameworks that drive that same application.

Even with the most frantic half effort of a project, its helpful to give the context of why you are doing it, why you approached the project this way, and why you ended up not going with another approach. Maybe you did try what should have been the best approach at first, and it didn't work right, so you had to do something not as good but good enough. Then in the future someone decides to try to redo what you've done using the best approach, and they hit a wall because you never wrote down that you tried that and it didn't work for x reason. Writing up what you do helps keep your own thought process organized, so it's beneficial for you even if you never read what you wrote.


I didn't mean documentation at the code level - I believe well-written code is always better than docs - it should not be used to hide badly written code. I was talking about documentation like product requirements, meeting notes, decisionf and trade-offs, design etc.

Solo people working on projects aren't too bad so long as they communicate the work they're doing and document it well.

Some people just work better on their own.

IMO, what's worse are "knowledge hoarders". Usually they've been in the company for a long time and they maintain their position by being as stingy as possible with their knowledge. You usually find these people in big orgs, and they can be quite toxic.

They also don't write good documentation.

> Solo people working on projects aren't too bad so long as they communicate the work they're doing and document it well.

Strongly disagree, been there, hurt like hell. Solo people on projects mean that other people can't give meaningful reviews (because they don't know the project that well, and because they have their own work), leading to 1. developer not getting good feedback and improving, 2. higher risk of them going down some rabbit holes, 3. bus factor of one and stressful vacations, 4. less guarantee that the docs and the code are any good, 5. a danger that this solo developer will turn into one of those "knowledge hoarders".

Solo development is never OK (imnsho). Sometimes you can't avoid it, but it should be a calculated risk, and only a temporary one.

In my experience it can work quite well. I work alone on many projects. Given, you need support staff. I have someone to document everything aside from the most technical parts that I need to write myself. Also a project manager that extracts requirements and expectations from customers. Doing that myself would increase project duration tremendously.

It is pretty common in the embedded world in smaller companies. There just aren't enough developers to render enough support.

The problem with one-man-shows is the dependability. A lot of web developers go that route. But if they are unavailable because of private stuff or illness, the project they are working on is basically iced.

I've found your comment compelling. I'm assuming you have some unit economics implied here, I'm very curious to learn from your experience.

What would you say, in a team of 6 devs, if a pm just walked up and said "If we can get 6 different tasks done in a week versus 3, guess which one I'm gonna pick?"

Re: easy search. Slapdash can handle search in all your work wikis/docs at once and is very fast. It can search github/gitlab too. I am part of the team there.

Thank you. I find Slapdash a great way to navigate our wikis by app, incorporating Google Docs, GitHub repos and other resources. By lowering the friction to making and using good docs, our docs have improved by a wide margin. The search is lightning fast and I can do most things via keyboard shortcut. Pricing is cheap too!

The pushback will come from two places: people who can't write (the overwhelming majority of people) and people who usually act impulsively and without a sound basis (also the great majority of people). The presence of these classes of people in a company will lead to the punishment of people who can and do write, for two reasons. The non-writers won't be able to tell that the writers have done something useful and therefore won't be able to highlight it as an accomplishment in peer or manager evaluations. The irrational actors will get all the accolades for "having impact" even when their random, unjustifiable activities have clearly harmed the company. Meanwhile the people who were writing the design docs will be judged for having less impact.

The way I like to measure this is there should be more writing than programming going on within the company. Some investigations, research efforts, or designs will lead to nothing, however every implementation should come with research, design, and retrospective documentation. In that case there will always be at least as many written artifacts as programs.

The way to prevent the nightmare of an illiterate workforce with only oral history is to hire people who can write and practice documentation-driven development, instead of hiring people who can memorize leetcode trivia.

Mandatory vacation. This is something that the finance industry has used for a long time to guard against fraud -- it's hard to cover up something if someone else has to do your job for two weeks straight at some point -- but it also serves as a mechanism for requiring you to cross-train people.

Two weeks of paid vacation where the company isn't allowed to email them or call them for help: I guarantee that documentation practices will increase substantially.

I worked in the finance industry and took the mandatory 2 week vacation and dealt with my coworkers taking the mandatory 2 week vacation.

We didn't write any documentation or have any internal wiki or anything like that, and everything seemed fine.

I also find it pretty questionable that someone couldn't write a computer program that can embezzle unattended for two weeks. You don't use your own credentials, you stick it in some other program and have it use the logged-in user's credentials. Are you auditing your HR system before you log in, and are you sure that the "ls" you're invoking is the same "ls" that actually came from Debian? No? Then it all seems very pointless to me.

Has that ever happened? Seems like the stars would have to align for that kind of white collar crime to happen (financial employee who is also a highly experienced programmer who is also highly unethical)

A famous example is the (in)famous trader Jérôme Kerviel who was deemed responsible for a 4.9 billions euro loss at Société Générale.

Having worked in Middle Office before being promoted to the Front, he had a really good knowledge of how operations and risk management worked in the bank (plus some still working write accesses to specific systems) which allowed him to mask his very large positions with fake opposite trades that he was putting in every day before the nightly risk snapshot and cancelling before they could be confirmed.

The guy basically did not take any holidays in two years - otherwise his large positions would have appeared on Risk radar pretty quickly...

If you are into those stories and want much more details than my poor summary I really recommend reading the SocGen post-mortem investigation.

Disclaimer: I worked at Société Générale during the Kerviel era - also there is a lot of controversy in France about how much the bank knew and let things happen (Kerviel was making a lot of profits - until he wasn’t) and if that was used to cover subprime related loss - this post does not represent an opinion on this case!

I mean, 2/3 of those come together par for the course, right?

I feel like the stars would have to align for that kind of crime to even be found. I think it's a guarantee that it's already happening in a lot of companies. So many companies have the most sloppy book work, and regulatory bodies are stretched too thin to catch everything.

If you are a sociopath who knows programming (more and more people in finance do every year) and is in finance (two fields with more sociopaths than some, perhaps), then why would you leave money on the table? You are already morally bankrupt and know how you would get caught, and therefore how to not be caught, and know that this money is going to you with zero issue.

> We didn't write any documentation or have any internal wiki or anything like that, and everything seemed fine.

Just evidence that documentation != institutional knowledge. It's not actionable knowledge if no one reads it.

that is not how 99.99% of embezzlement works. No one in finance is using 0-days (or whatever) against their own companies. It would be much more "routine" types of practices, which might be noticed given minimal oversight.

The proper way to embezzle is by making $20-$100 addons to thousand dollar purchases. You don’t want hundreds of thousands papertrailing back to you. You don’t buy a boat on the company dime you buy trinkets to put on your boat. The real embezzlers are taking $3-$5k/year. The ones taking more end upon the 5 o’clock news

I work as a software engineer in finance and have to take two weeks of mandatory vacation, but this doesn't deter our team from not writing proper documentation, or writing down domain specific knowledge.

When someone is on leave who has specific knowledge, this is just planned into the sprint. As in "xxx knows most about this feature, so let's wait for him to return".

Even when we do write documentation it just gets lost, or developers don't bother looking anything up.

What is mandatory vacatio n?

Two weeks paid time-off, mandatory once a year.

It just means people leaving are not that critical. If they were, you would have that documentation or you would be SOL pretty often.

I don't think it "just" means that people are not that critical, it's a much deeper issue. All the way from individual engineers to managers to product owners to the company in general.

At least in my team, is see:

- Managers allowing people to only take on specific work, therefore people are becoming highly specialised

- Individual developers don't push others to write documentation, instead they wait until somehow they have to do a task, spend 2 weeks to figure it out, and then write some documentation around it

- Individual developers who force themselves to only work on specific pieces. (I think mostly to fuel their ego so they're needed)

- The company not encouraging knowledge sharing, or simply not providing good tools for it

- Product owners who don't really care about the product

No, you can still be critical and not have it hurt the company with just a two week absence. Say a product like Uber ceases new development because of a critical developer leaving who knows how to fix the home grown CI. The company isn’t going to go under in two weeks but it doesn’t mean the dev isn’t critical.

Some of the most “innovative” times are when someone is on parental leave.

And by innovative I mean “we have no idea how anything works - let’s fix it.”

Totally agree. Especially here in Scandinavia where people disappear for 6-12 months at time, their co-workers need to step up.

Just like dropout in a neural network.

What is the equivalent of weight decay?


throwing off 75% of knowledge increases generalization capabilities

(actual experience detecting real criminal behaviors within financial claims)

I'd extend this to regular vacations too. I've worked a lot in both Europe and the USA. In Europe everyone had to know other people's knowledge as people were regularly out of the office. In the US it was easier to rely on someone as people rarely were out for a week and if they were they usually were local and on call.

That works for ops but not really for R&D. Your code will wait for 2 weeks, only running systems won’t.

I'm not sure about the "successful companies institutionalize the knowledge of their employees" part.

In such context a process becomes a (written) 'procedure/specification', and some folks stop innovating, they just do it "by the book".

HR quickly grasps this and hires people with less and less skill, cheap personnel 'just able to apply to procedures'. Other ones feel like cogs in the machine (especially the best ones, hunted by competitors) and quit.

Someone departing with the 'procedures' may let a competitor obtain a rather complete grasp of it and adopt the best bits.

Letting each team decide about this and establish cross training seems preferable to me, and has many other benefits.

This is why I have largely abandoned documenting procedures and instead write down "principles". As long as the principle is adhered to, the embodiment doesn't really matter.

By the book people, especially managers are one reason companies eventually go down/shrink. It kills creativity, hinders progress, and like you said leads good peoples departure

We record screencast videos demonstrating how to do a process. New people can watch those videos to learn how to do it.

If we find a better way to do something then we make a new video.

There's a team member who transcribes videos into google docs for people who like to read and search in google drive.

It's pretty simple and it works wonders for an international team.

Yes, screencasts are good, and IMHO people who struggle with instructions seem much better at re-watching the difficult part until they are able to do it than they are keen to re-read a section until they get it right.

However it is often essential to have an alternative form (eg transcription or at least summary of steps) simply because of discoverability - even with brief screencasts it can be awkward finding the content otherwise.

I see a lot of responses here automatically assuming that capturing institutional knowledge is about code, but there's so much more of business processes than code that needs to be captured, even in a tech rich environment.

I second videos. In fact, I'm trying to think of a way to make animations showing the interaction between microservices without hiring and animator. I'm leaning towards mashing our integration tests with something like Three.js.

I wish there was one.

In companies I've worked at large and small, the most important information just comes from people's memories ("oh yeah because we decided X in that one meeting six months ago, remember?").

And then once a critical number of people leave the team/company, so much time is wasted reinventing and rediscovering things.

The only solution would be for everything (every meeting and decision) to be rigorously documented with outcomes and justifications, and for every new team member to go back and read the entire history to get up to speed.

But 90% of people seem to detest writing and/or are terrible at it, and it takes up a lot of time. And new hires are never given the time it would take to read the whole history anyways -- they're excited to start working and the team needs to start showing extra productivity ASAP. So for both reasons, it basically never happens.

So you just hope that the rate of gradual knowledge osmosis from old members to new members is faster than the rate of turnover. And when it isn't, you just accept that your team's ability to deliver profitable features will slow down drastically. And at least for the critical business logic in your product, you mostly rely on tests to make sure at least things don't break when a new team member starts changing things.

It sucks.

>The only solution would be for everything (every meeting and decision) to be rigorously documented with outcomes and justifications //

Action focussed minutes?

People who have been in the job for a while aren’t always the best people to explain something.

What I’ve often done is asked new hires to document what they discover. New people are easier to mould to a new behaviour and often have the questions you need to know. When documenting becomes the habit, more people do it. Current 500-person company is very good at documenting many aspects, because we’ve done it since year 1.

I've experienced that as well (both as a junior and as an experienced engineer):

- Follow these steps.

- If anything is unclear or doesn't work, update it.

This is what we do. It has the added benefit of forcing new people to read the documentation that was left for them. It’s been my experience that without this directed task, they won’t even look at existing documents. Tell them it’s a deliverable and suddenly they read everything.

A trick I've used effectively is to have a "doc of docs" - a document that tells you where all the other documents for a project or team live.

You can do this as a wiki page or a Google Doc. The important thing is that the answer to the question "where's the documentation for X" should ALWAYS be "it's in the doc-of-docs".

Then you can take it a step further: you can say "it's in the doc-of-docs... and if it isn't, when you find it, add it to the doc-of-docs!"

It's a universal truth that documentation for projects and teams ends up scattered in many different places. A doc-of-docs is a lightweight technique that can really help here.

I like this “master read me” approach and try to also reference the lord of the rings poem to both try to make documenting fun and to explain the intent.

Three Docs for the Product-kings under the sky, Seven for the Dev-lords in their halls of stone, Nine for Mortal Men doomed to die, One for the Dark Lord on his dark throne In the Land of Mordor where the Shadows lie. One Doc to link them all, One Doc to find them, One Doc to bring them all, and in the darkness bind them, In the Land of Mordor where the Shadows lie.

Notion is wonderful for this. It’s like a wiki with the ease of use of google docs.

My company has basically everything on Notion and any time someone asks a question more than once, it gets added

I started an internal blog - using Confluence, because it was already a tool used by the company so I didn't have to convince anyone to install anything new.

My goal was to introduce a culture of internal blogging at the company. I didn't really succeed on that front - I used my blog a bunch and a few people made a post or two - but I like to hope that if I'd kept at it for longer (I left the company) it would have started to catch on.

The reason I like internal blogs is that they release you from a lot of the pressure of writing more formal documentation. If there's a technique that I think is a good idea but that has not been established as an agreed best practice, writing it up in official documentation doesn't feel like the right thing to do. Writing it up on a personal internal blog as "as-of date X my opinion is that we should do Y" is always OK.

Likewise: writing documentation that later goes out of date can cause harm. Writing a blog entry that says "as of February 2020 this particular system worked in this particular way" seems much safer to me.

I also think internal blogs are a great tool. But I also share your experience of how few people are actually using them. At my current company, it is the same thing: I wrote the majority of posts and a handful of people contributed a handful of other posts.

I guess the 1% rule [1] applies to internal systems as well...

[1]: https://en.wikipedia.org/wiki/1%25_rule_(Internet_culture)

Some years ago (before wikis and the www) I worked on a software product for recording Design Rationale - the decision making process that went into designing long lived artfects, such as nuclear reactors and process plants. The idea being that engineers could find out WHY something had been done the way it had decades after the original engineers had retired or been run over by buses. The product failed, despite some intial interest from big companies.

In retrospect one of the main reasons for the failure was social rather than technical. The engineers designing things saw recording the rationale as just an extra job that had to do, with no immediate benefit to them. If anything, they saw it as opening them up to more scrutiny and increasing the chance of their job being outsourced in future. So keep that in mind.

The value of institutional knowledge is that it's already in someone's head. Reading documentation, searching a wiki, email archives, may be better than reinventing the wheel but the real way preserve it is retention.

I would invest in that if your are in a complicated or specialized domain where it takes months or years for someone to really get their mind around it. One thing I have noticed is that private work environments tend to be present at the low turnover environments I've seen. At least a cubicle.

Another option is extending low hour contracts to departing employees as sort of an off-ramp to their role. They are on a few hours a week or as-needed to handle the dwindling number of cases where they are most needed.

I'll focus here on the knowledge of the key people you're about to lose. Ideas:

* If they're good at documenting, and willing to, task them to do as much of that as possible. The documenting might be in adding code comments and API docs, writing separate text files, etc. The person might not be able to document off-the-cuff, but have to work through a topic slowly, such as going through and re-understanding some old code themselves, going through a manual process that they do automatically and reflecting on the whys, etc.

* If some of the information is amenable to giving a talk to other employees, with a Q&A section, that might work, too.

* Another option is to have another employee interview the person leaving about one or more topics, and either type notes as they go, or record it and get a transcription. The interviewing person should be able to understand the topics.

* For tasks the person leaving does, you could have other people do the tasks while the person leaving is available for questions, and one of them documents as it goes. Depending on the task, it might make sense to have the knowledge-holder right there, both answering and observing, rather than only available for questions on-demand.

Side suggestion about accessibility/discoverability/maintainability of all this new documentation: consider keeping the medium simple, and avoiding a proliferation of locations, formats, a dozen bullpoop communication SaaSes, etc. For most software work, for example, inline embedded source code comments and API docs can be an easy way to try to keep a lot of information accessible in context and maintained. Some other information that doesn't fit well in source code, such as ops architecture and procedures, might be Markdown files in that same code repo, or another one. The occasional video file you just can't check into git might be a rare indispensible one, but can still be linked from a Markdown file that's in your repo, but even then, maybe you also have a text transcript in the repo, or someone turns a talk into edited docs in the repo.

Incidentally, much earlier in organizational knowledge sharing, I vaguely recall a study by a consulting firm (sorry, no cite handy, and I'm not 100% sure I remember which big-name firm), in which they found that people were resistant to having their knowledge captured in a system, because that knowledge was an asset of the individual. Your key people leaving might be more altruistic than that, want to help out their colleagues, want to have a good word-of-mouth reputation, have a sense of professionalism about it, have equity in the company, etc. You might like them to do knowledge transfer to a degree that's really above&beyond the call, so consider how you might acknowledge and thank them for that. It might also be a good example for others, and promote more proactive good practices for organizational knowledge.

Cross training. It's easy to lose institutional knowledge when someone leaves if they are one of the only people that has worked on their projects. Much harder to lose that if you do even minimal cross training. Even if it's only once a quarter, have each person sit with someone else, have that someone else explain their job, their major projects, their details, pitfalls, strengths, etc. Lots of notes should be taken. A knowledge base should be a given, but that will only take you so far. A side benefit is such a wider picture of what's going on will better inform the work of everyone, keeping people not only on the same page, but understanding all the details of how those pages interact.

Edit: Yes, they'll hate it, but this should include a tour of duty Sales. Poor relationships & acrimony between sales & dev are based on a failure of understanding each other's jobs. And sales are the closest to the needs of customers, short of sitting with customers themselves.

Get your key knowledge holders to dedicate 20 percent of their time to documenting what they know. However, most of them are so used to knowing stuff that they aren't sure where to start or what the need is, specifically. Therefore, make them take requests from people who want to know what they know. Then what you end up with is really knowledgeable people spending 20 percent of their time asking everyone around them what they can document and documenting it. Check in every now and then and ask to see what they've come up with.

I've struggled with this personally at our company. A sibling comment mentioned a README for each project/process. That's definitely a solid start for building this up from nothing. Copy open source project README files:

1) what is it? (A web project, an automation script, an ansible deployment repo?)

2) what dependencies do I need to run it? (Make, NPM, Java 1.8?)

3) how do I run it? (docker-compose up? make && ./a.out?)

We started with this. Then for the bigger projects/monorepos, we started adding README files in relevant subfolders.

Recently I've been converting these README files in the larger projects into mkdocs subfolders that get hosted in our repository tooling (GitHub/GitLab pages).

Start small. Go slow (if it's institutionally difficult). Build up to more complexity as you get more written material to work with.

I've started creating an "index" project, that links to all the projects that have documentation.

And finally, focus on the pain points first. One of our monorepos was fiendishly difficult to deploy correctly, either locally or in a test / production environment. The very first tutorial I wrote was setting up that environment in a step-by-step, repeatable manner, and it's by far the most oft-used documentation we have. With that out of the way, I can focus more on the esoteric details (and, yes, unfortunately, it's a bit of a thankless, "skunk works" project, but it's worth it)

A per-project or per-process README would be a good first step. Ideally keep the notes/instructions as close to the place where the work will get done. It's important for someone to test the instructions in the README on a new machine --- 9 out of 10 times you'll find a step about authentication or some system dependency was not documented. Credentials are extra tricky, so you'll have to think extra hard how to make that work (e.g. some sort of central key store, shared password manager, or ENV vars that need to be defined so you avoid putting any sensitive info in the README).

For something even better than a README, you could document the steps of a technical procedure in a Makefile (or Fabfile) that your colleagues can run. It's important to keep the scripts readable and stupid (as opposed to abstract and powerful like ansible), so that people can read the steps. Some people refer to this as "runnable documentation."

The easiest way is just to make sure product/business discussions happen over email rather than Slack. This way those discussions can be searchable and discoverable by anyone within the company at any point in the future. We make software for this (FWD:Everyone), but there are dozens of other similar solutions as well. That's the great thing about email, because the standards are open you'll be able to extract more value from that same data with each passing year using products that haven't even been created yet.

Conceptually this is similar to having a wiki, except for that unless your company has people whose full time job it is to maintain the wiki then it will always be out of date and inaccurate; just deploying some wiki software tends to be pretty useless, and even in the rare cases where they are maintained the medium inherently doesn't preserve the tacit knowledge contained within the decision-making process itself.

I've encouraged teams to switch from email to Slack precisely because it takes non-searchable content and makes it searchable. The amount of institutional knowledge that ends up in personal inboxes of a tiny subset of the team (who then eventually leave the company, causing their emails to be lost entirely) has always terrified me.

If you can set up a culture of everyone subscribing to internal mailing lists with searchable archives and wide distribution then I could see it working - but my experience is that the easiest way to get that culture is to switch everyone over to Slack.

I never quite understood why companies don't just setup an NNTP server with newsgroups to capture conversations, rather than email (which is difficult to search IMHO).

They do, it’s just called Slack and Teams these days

Why is email superior to slack (or other chat solutions)?

Can you export slack logs as plain text, or something easy to process like JSON? (Haven't used it in forever, so I don't know).

Being able to

grep -A3 -i foobox -r /nfs/info | grep -i rpc

is useful. Similarly, you can shove plaintext into a more advanced search engine easily.

> Can you export slack logs as plain text, or something easy to process like JSON?

I think you can, the main issues are that:

- Conversations aren't forced to be threaded, and there is no way to go back and do anything with conversations that weren't threaded; it's just lost data.

- Because Slack doesn't export data into a standardized format, there isn't a big ecosystem of tools to do stuff with Slack data. And it's not clear that there will be in the future either -- Slack's growth has started slowing substantially and it's only around 13M DAU, which isn't really big enough to build a business on top of.

Part of the problem with slack is it sells itself to small conversations, fast responses, the "im" type. It is all good and well, but email lets you think slightly more and keep topics contained and searchable.

13m DAUs at $6.67 per month is $86,710,000/month, or $1,040,520,000/year - seems like a pretty big business to me!

Yeah if you can get 13M people paying you then obviously you're doing amazing, but even Slack only has 6M paid seats.

But I meant more like if you can get 1 / 1,000 email users paying you $10 bucks a month then that's $480,000 per year, whereas 1 / 1,000 Slack users is $1,560 per year.

At each of my last 3 jobs I've been a strong advocate for tracking knowledge in a Wiki. We've used MediaWiki and Confluence, and both worked well. It helps if everyone does their part to contribute, which doesn't always mean everyone has to write. Some people can dictate what they know. Some can provide screenshots. Sometimes just getting someone to leave a comment with a correction is enough. I like to start with stub articles and allow them to grow organically, without harping on folks about it.

Often there are many ways that changes are communicated (for example, an email to the NOC team to advise of systems being decommissioned, an email to the helpdesk to advise of new support procedures, etc). To some people, that email is the documentation. I know that if I want it captured in the knowledge base, that email is my cue to transpose the notification email into documentation that can be referenced later.

My current job is doing sysadmin support for a consultancy, so there are lots of clients and lots of other people doing this work. We have a regular pager rotation for emergencies, so if you get an overnight call to work on something and can't find the information about it, you start to realize how important the docs are. That has been a big motivator in getting people to update docs about the things they know. If you don't want a call overnight, make sure there's nothing that isn't in the wiki!

I use plugins to help point out when content might be out of date, and guide people to archiving out-of-date docs, or to update them. I regularly refer to Stewart Mader's great book Wikipatterns and its list of patterns/anti-patterns to help with wiki adoption and behavior. http://stewartmader.com/wikipatterns/ The book is a great read and covers a lot of your questions, especially skeptics and opponents, and how to address them.

> We have a regular pager rotation for emergencies, so if you get an overnight call to work on something and can't find the information about it, you start to realize how important the docs are. That has been a big motivator in getting people to update docs about the things they know. If you don't want a call overnight, make sure there's nothing that isn't in the wiki!

I had a co-worker write about the importance of documentation to folks who are on call, and why it is hard to get it right: https://www.transposit.com/blog/2020.02.26-rewarding-documen...

I really don't have any good recommendations, because I've never seen this done well. Never. In small or big companies.

The two biggest challenges I've seen over and over again are:

1. People don't document changes well. You end up with pages and pages of outdated documentation describing things as they were years ago. This is especially hard on companies that have multiple teams using same underlying platforms/frameworks/libraries. Each team is going to create documentation on how they're using it when they do the work. Once the underlying technology does a version change / incompatible change, all of that documentation becomes obsolete but remains in the knowledge bank polluting the search results. I've become convinced this is an unsolvable problem.

2. Search. I have never seen a documentation system that gets me the information I need easily. It's closely tied to the problems described on the previous issue, but also almost all search systems I've seen are really bad at identifying the authoritative sources of documentation and pointing you to them as the first result.

Also think of your audience. The best documentation I've seen clearly separates documentation intended for maintainers of the code and users of the code.

One of the things I practice personally is more-or-less documentation driven development: https://gist.github.com/zsup/9434452 Although I'm using writing developer-level code, so I'm not literally writing end-user documentation. But I follow the principle; I generally start any significant function by first writing the documentation for it at the appropriate level of detail.

I usually also try to budget a day or two at the end to look at the documentation and generally clean it up. I can't always get "fresh eyes" on it per se but at least I can make sure it seems to basically flow.

The side effect is your major project also comes with basic documentation for very cheap. Honestly, it may even be "cheaper than free"; I do this because I think it helps make better code, faster. The act of writing the documentation doubles as a self-directed design review, and I couldn't even tell you the number of times I've documented some particular thing only to realize how stupid it is before I even wrote a single line of code [1], and started rearranging things at the cheapest development stage there is. But I concede in advance that proving it's "cheaper than free" is pretty hard.

I don't present this as a full organizational solution, of course, but unlike those full organizational solutions, this is something anyone reading this can pick up and start trying out tomorrow, whereas "redo how our organization conceives of how we store information" is a bit less immediately actionable, shall we say.

[1]: "Wait, I'm requiring what precondition of the callers? Wait, I'm going to return six values? (Better make a new struct/class/object.) I seem to have an awful lot of functions asking for the same 4 parameters (again, new struct/class/object). I'm asking for how many incoming parameters? These are awfully complicated instructions I'm giving about what they can and can't do to the return values. These instructions on the transactionality of this call are stupid complicated." etc.

At my first job we were in a similar position I think with lots of churn (volunteers, young people).

I argued with my boss and finally got a vm with a MediaWiki instance. I created a simple frontend using Javascript and html to simplify correct addition of common page types.

Obstacles: my boss and certain other people wanted something more businessy. Also they wanted to run the DB on MSSQL, while MediaWiki is supposed to use MySql I think.

At a later place I was tasked with moving/updating documentation from a two page Word document to a Confluence document. It was written by a long time developer so it grew quite a lot.

Obstacles: Confluence :-] Also getting people to admit when their previous docs are wrong (or at least deviates from official vendor docs, and it creates problems.) Also trying to keep it up to date while others are constantly changing things.

Later on I've

- used OneNote,

- got others to use OneNote

- failed to get people to use OneNote,

- updated Confluence plug-ins,

- struggled with people who said they used README-driven development, but were really just their personal notes on what they'd need to remember.

Finally there are some common theme:

- refusing to acknowledge the need for any system beyond mail and Word documents.

- insisting on buying one lf the "commercially supported" but otherwise inferior and close to unusable system

- insisting on using SharePoint (a variation of the point above)

- putting everything into Confluence which means hopeless search + access restrictions so you never know if 1) the document exists but you don't have access, 2) it exists but you cannot find it because it is in the wrong place and search is broken 3) the document doesn't exist.

A couple thoughts:

1.) communicate asynchronously as much as possible. If you have a synchronous conversation like a meeting, make sure there is a written representation of what was discussed. If it doesn't persist, it doesn't exist

2.) Understand that implementing a wiki by itself will not work. There's a gravitational pull to throw a bunch of junk in it (like a file cabinet). Things become out of date and each person will structure things in a different way.

3. Create some loose structure around sharing regular, written updates about what each person/team is working on.

At my company (https://www.friday.app), we've created a tool that is somewhere in-between Slack and a wiki. It's kindof like a work journal. As a distributed team who only has 1 meeting every week, it's a place where all our updates are captured in a single place.

Every commit should combine:

- The code change itself

- Tests that demonstrate that the change works as expected

- Updated documentation relevant to that change (documentation should live in the same repo as the code to support this)

- A link to the ticket/issue that discusses the change

If you use a code review system such as GitHub pull requests or Phabricator you can enforce this kind of commit culturally - in your review point out that the test is missing or the documentation hasn't been updated or there's no link to an issue.

I like building pull requests up from several commits and then using the "Squash and merge" option to merge them into a single commit to master that includes all of the above.

Doing this is great for institutional knowledge, because "git blame" can always lead you to a comprehensive explanation of the change, including a link to the underlying ticket where the change was originally requested and discussed.

I agree that this can be a reasonable requirement for a completely new feature or a large change in behaviour. However, I'd say that enforcing this for all commits is a safe way to ensure that no one spends any effort on improving readability and robustness of existing code. It's one thing to quickly fix a typo or add a null check in code and send it for a quick review. A whole other thing to create a ticket, add a clear explanation on why this is needed, add a test for a weird edge case that shouldn't really occur and possibly make an documentation update for the edge case.

I'm more inclined than most of my team to do cleanup commits, but even I sometimes avoid them just for the extra cost of the regular review process.

I think a better approach is to have a culture that encourages making context dependent judgement calls, which is also what I try to do.

If it's something trivial like fixing a typo, I ask that this is made clear in the title, so investigators of an issue know that this CL is unlikely to be relevant.

If it's a minor change in behaviour, I ask for a commit message, and/or a comment in code, with a motivation of why it's needed.

If it's a bigger change in behaviour I want an issue linked to it, to make tracking easier in case there's followups needed or there's some regression due to this.

All this is of course based, not on some platonic ideal of what makes for good documentation, but rather on real experience of tracking down issues and adding features.

I know how annoying it can be to track down the commit of some functionality that looks really odd, only to find an empty message without even a reference to the issue that was being solved. But I also don't have this problem with commits that are clearly labeled "Fixed typo in X", those I can simply ignore.

I agree. Typos fixes and cleanups don't need this. The code + tests + documentation + issue link format should be for actual changes, not minor cleanups.

Having a single wiki instance for your company is an awesome way to collect knowledge. You can write up operation run books, design documents, references all in an informal way.

This gets out of date quickly if this is not curated in some way.

Software. Software. Software.

Encode your knowledge into software, into data structures, into tests and active documentation.

Human organisations have faced these problems for thousands of years. And never solved them well. Maybe we need the new kid on the block to help

Have the key personnel fully participate in knowledge capture. As far as the tool to be used for capturing said knowledge, don't waste time...Just use the fastest-to-set-up and the easiest-to-set-up wiki (or something similarly fast and easy). After the key knowledge has been captured, and likely after the key people are gone, you can look to see if you even need to migrate to a different platform than the original wiki. Knowledge transfer can get tricky in general...but since you're on a time constraint (key people leaving), don't waste time with UI or prettiness, etc. Just capture all the things!

I would guess a lot of what you want to preserve is wisdom. You keep that by retaining the older folks that have it, and have them work alongside the younger people that dont. I've learned so much from other people's war stories. More than from any lessons learned database. Documentation is nice too, but writing it and reading it take time and it doesnt always cover the things that didnt work.

Optimizing things (and organizations) tends to make them efficient but brittle. This is another area where that's true.

It is crucial that the organizational culture reward employees for making this happen.

When you think about it, a lot of corporate cultures reward the opposite -- employees hoarding key knowledge so they can use it to their advantage.

To do this, avoid creating a culture that fosters rivalries. I know it's controversial to say this, as many business people seem to think that brutal rivalries are what leads to over the top effort.

Instead, think about the knowledge as processes, and figure out how to best codify and execute those processes.

If you arrive at a step in the process that appears to be something that can only be done by one person, or by a person who is leaving, that is a clue that someone is withholding information or trying to use it to his/her advantage. Nothing is that complicated, just break it down into simpler steps.

Then, when it comes to onboarding, introduce whichever processes make the most sense for the new hire, and encourage him/her to view them both as an example of excellence but also as something that can be improved. Doing this helps the new hire understand the design perspective behind the processes, which is preferable to following them by rote (which is surprisingly common even among otherwise bright people).

It sounds from the question that you have a bit of drama going on, so you may actually want to sketch out the processes as you think they are and then incentivize the team to make them all precise.

I noticed a lot of suggestions in this thread assume you want to capture the institutional knowledge related to coding and software:

- comments in the source code

- commit messages

- readme

- code review

- pair programming

- variable naming and refactoring

Which is all great, but at the same time narrow. What if I want to capture institutional knowledge when it comes to accounting, HR, PR, management, negotiations, suppliers, local laws, seasonal patterns etc. Perhaps there's a more abstract, generalized advice that could be applied outside of the realm of coding.

    comments in the source code
This is an unpopular opinion, but I fervently agree with it.

It one of the most powerful ways to keep a codebase comprehendible.

Some folks say, "That's what commit messages are for." Bullshit. When people say that, I suspect they've never worked on anything other than prototypes or toy projects. Good commit messages are a must, but they don't scale well when it comes to long-lived projects. Think of a source file that's had many dozens or hundreds of commits and refactorings. Tracking down the original author's intent can be very difficult - especially when the original commit doesn't reference what's happening on line 78 or whatever. Especially when there are dozens of such oddities in a single file.

Some other folks also say, "Good code is self-describing." Again, bullshit. Maybe if you're writing code in a vacuum. But code gets weird and messy where it interacts with other things that are outside your control. I can read your code and see what it's doing. But your code cannot tell me the "why." The kludges you implement to work around bugs in other libraries/browsers/APIs are NOT comprehensible without some contextual information about the bug you're trying to solve. Ditto for all the weird business logic that creeps into applications. Why aren't we charging sales tax on Topeka on Tuesdays? Tax law? Were we running some special promotion? Did our vendor in Topeka already collect sales tax in some other part of the process? A quick comment can save the next maintainer hours of wondering.

So, my rules are....

1. Stick a comment in there if you are implementing a kludge/workaround based on some external thing. Wrote your own CSV parser because the one in the standard library is broken? OK, tell me that so that I know this was a conscious choice and not a matter of you simply not knowing that the standard library existed.

2. Stick a quick comment in there if you are implementing business logic. Even if your comment is simply "Implement Bob's marketing idea for the job fair" that would at least give me something to go on five years from now when I'm wondering if I can delete that bit of code.

3. No need to document basic stuff. Retrieving some records from the DB? Writing to a file? No problem. Assume the person reading your code is a competent programmer. No need to describe that stuff.

> Some folks say, "That's what commit messages are for." Bullshit. When people say that, I suspect they've never worked on anything other than prototypes or toy projects. Good commit messages are a must, but they don't scale well when it comes to long-lived projects.

Yep. I've seen codebases that have migrated source control and issue trackers multiple times - sometimes migrating history, sometimes not. The only ancient knowledge that has survived is the code comments.

The best is the enemy of good here.

When I manage engineers, I insist on people having the old-fashion pen and paper notebooks: not scratch paper; not pads; not post-it notes. Simple 5 by 7 grids. During my weekly one-on-ones, I encourage people to make one or two positive changes:

* jot down any odd term they run across so they can google 'strangeNewTerm slides' later and batch through them all * write a quick note on any error taking over twenty minutes * go back and read their notebook here and there

People do use the notebooks, do record the information, do get sick of writing and do start reaching out for other ways to record information. I allow people to use Wikis, markup files, documents, or a mishmash: people are more likely to work on their own idea. I do insist that documentation be in source control and have 'section last updated' dates. Eventually, I encourage someone to maintain a master index of where non-code documentation is found.

The hard part is getting people not to document too much. Every document has a maintenance load, much like every line of code. It is far better to have an up-to-date hint like "TPS reports need cover pages per YoyoDyne contract, 3/2004" than volumes of out-of-date procedures, one of which mentions the YoyoDyne contract on page 37.

One totally different tact: new developers get assigned to the source code control review. That is, they look at the new commits; run the new code, update the project/company specific dictionary with any new terms; give the tests a careful reading and add to them; and write down any question. New developers quickly become familiar with the code base that is most likely to change.

Somewhat off topic:

Not sure who you are managing, but for me being told how to take notes would be a huge red flag.

Give your people some autonomy.

Everyone gets micromanaged for a moment until they move on. On your first day, someone micromanages showing you where the restrooms are and how the breakroom works. Then they never do it a second time.

In practice, you can revisit even a small point like taking notes as often as the other person finds experimenting with your advice makes her a better engineer.

You’re not going to preserve their knowledge via a wiki. At best a wiki would be a snapshot in time. They’re irreplaceable knowledge is likely an understanding of how change happens within the company.

The answer is by requiring close team effort. Pair programming for example.

I’d one person leaves the organization, there should be “adjacent” employees who understand what they contributed.

That makes sense to me. I'm on a small team with several projects. Some people know more about some things than others, but we swap roles frequently. We explicitly try to share knowledge. We pair program and have daily standups. We also have a daily "tech time" where we quickly present the new things we've been working on.

The end result is that everybody knows enough to pick up where somebody else left off. We can review each other's pull requests effectively and go to each other for questions.

I'm not sure I like the pair programming or the open office environment, but I haven't tried anything else yet. Sometimes I think pairing slows us down. And we sit right next to a tech support team that's on the phones all day...

Pairing is supposed to slow you down, that’s a good thing. As in, a quality solution takes more time than a shitty one.

What’s dumb is to do it Kent beck style, both people on the same keyboard at the same time. Cargo cult. I bet that’s how y’all doing it, judging by the open office comment.

Wouldnt the answer be: commit these dedicated, talented, key individuals to the knowledge tranfer effort, exclusively? You're trying to prepare for losing them. So, try it out!

Pull them off, and assign them to a cushy knowledge transfer process. You'll more safely learn about your true dependencies & get some of the knowledge you want at the same time.

1. Wiki w/ comments

2. Corporate university

3. Culture of anti-knowledge hoarding, pro automation, well-documented procedures and no "we need Joe, only he knows how to do vital task X."

4. Succession planning

Simple rule: Require that new hires should be able to become productive without having to talk to anyone (physically or electronically). The rest will fall in place.

Terrible advice.

Part of the knowledge of a large system is due to having discussions with others.

Discussions are good for critic and improvements. But to get up and running i.e. setting up your environment, building things, understanding existing design, running from source, finding roadmaps, getting list of open issues and setting up for debugging should not require talking to anyone.

There is only one way - 1) higher management must acknowledge that this is needed, 2) higher management must approve time allocation for these activities (as opposed to stuffing all new iterations 100% with new feature development). Then they push it lower and lower, down to the engineers. No way it will happen in reverse, when engineers decide they want this it usually end with a several disconnected resources, often using different tools and each maintained by 1-3 persons effectively for themselves, because nobody reads what they wrote. Also everyone need to accept that this activity does not have "end", and that it may need multiple reworks along the way. Basically this must be a thing that everyone just does, all the time, however they can.

Using and integrating a twiki as part of whatever you do ensures there is always a write-up. Encourage everyone to write up and make looking through docs as part of your review/discussion process. For this to work, you'll need to sort of make it integral to everything you do. Info in a twiki becomes stale over time, but better to have some stale info that can be updated than to not have any info at all.

Something I wished companies do is to ensure that discussions happen in internal mailing lists (rather than in individual email accounts) and have all mailing lists searchable and accessible to anyone in the group. This would simplify understanding how certain decisions were arrived at.

Use documentation unit tests - tests that introspect code and then scan the documentation to make sure that specific things are at least mentioned in the documentation.

Applied carefully this can really help encourage a culture of documentation that stays up-to-date. You can't land new code if the tests are failing, which means you at least get reminded that documentation is something that you should be thinking about.

I wrote about this technique here: https://simonwillison.net/2018/Jul/28/documentation-unit-tes...

Force people to stop asking the most experienced person to answer questions or solve problems.

Have people document how they diagnosed problems after fixing it, including queries for searching logs.

Thank people for documenting things.

I think the solution is more cultural than technical. When a company develops a culture that fosters the creation of technical documentation, and encourages employees to document absolutely everything (both the how and the why), then institutional knowledge is simply a byproduct. When a company focuses much more on shipping products and de-values everything from architecture documentation to API documentation, then institutional knowledge suffers.

However, to make the institutional knowledge useful, it must be easy to find. Thus, I think the second most important thing for capturing institutional knowledge is to have a small number of easily searchable places where documentation lives. Markdown files in the source are convenient place for documenting particular projects or code, but more general-purpose knowledge should be in a wiki or any other document store that is centrally searchable and update-able. An example of such general purpose knowledge is "how-to knowledge": How do I request the appropriate privileges to integrate my service with Service X? How do I make and deploy a staging build? How do I set up a new service?

Another sort of general-purpose knowledge that should have a single home is knowledge around context for past decisions that were made for good, but not obvious reasons. My team maintains a document called a Decision Log, where we record the context around and reasons for every decision that required more than roughly 10 minutes of thought. Longer decisions have their own docs, but they are linked from the central Decision Log.

Have a wiki that anyone in the company can write to without knowledge nor approval from anyone else.

Has to be easy to use, easy to write to in particular, and quickly searchable.

Resist all attempts to impose process or standards to wiki entries. If that fails, create a second, "draft" wiki immediately.

A few people are quite disinclined to write things down. A few others are quite inclined. Most of us are in there middle. It's important to remove barriers and make it easy. Think of it as a product that users have to want to use.

This is one of those "It depends..." questions, where it really does - on so many variables that it would be a major chore just to list them all. So an easier way to go about just establishing an idea of where to put the first bite would be to get an idea of the lateral limits. You are already familiar with the startup side of the spectrum, so spend a little time thinking about the extreme opposite: multi-century continuity, the US military's system of knowledge transfer. To be clear - you really don't want to emulate it, even to the degree that IBM did with their manual that helpfully informed employees of the correct way to sit at a desk, but know what the extreme looks like. Maybe pick a few high level concepts out of it:

* Graduated levels of mastery: few people need the details for standing up a production database, but all department members need to know that a company process already exists for it.

* Training dependencies: the mapping doesn't have to be perfect, a little goes a long way in giving shape to a training program.

* Training records: log employee training sessions, from self declarations of 'Yes I read this brief' to consultant led seminars.

Basically 80% of the military's comprehensive training program can be implemented with a network drive of docs and a spreadsheet. A couple of places I contracted for was pretty heavily reliant on Microsoft Sharepoint. I'd be surprised if there wasn't a fairly decent open source suite of components that could give you a solid starting point for a training program what would be somewhere in the middle of the spectrum.

Spending hours updating your Notion or Confluence is busy-work and will be incomplete and eventually become stale anyway.

Try speaking to people in real life. It's not that hard.

Most companies start out with people that are flexible and capable to do a lot of different things. As you grow, it becomes more and more important to have specialists that focus on only a small fraction of the overall work you have to do as a company. Here is what I would do:

* let people work in small teams/duos

* share knowledge by using a Wiki [1], especially the leaders (the flexible people) should write down as much as possible, but collaborate on the knowledge base (answer questions, add stuff that's missing)

* take meetings as needed (don't overdo this, nobody likes to sit in unnecessary meetings)

* automate as much as possible, the common developer does not need to know exactly how the build pipeline works (but a few people should)

* let people try out new things, this keeps them motivated and improves themselves as well as spreading their own knowledge to co-workers

[1] I found this to be very efficient if you have a team that has the right mindset. If you start out with people that are not used to document stuff, it'll be hard to get your knowledge base going. You can take a look at the product we build Emvi (https://emvi.com/), which aims to solve some of these issues.

The issue with written instructions is that people hide it in nested folders, deep outlines, and walls of text. This creates a death spiral, why write when no-one reads. I've been experimenting with different solutions to visible internal documentation and are about to release the culmination of my efforts in a product called Triqla. http://triqla.com

Learn how large-scale open source projects are managed and governed. Set up a team culture that could support remote working, even though teams are at this moment co-located. Can your team keep generating the same output even when they don‘t see each other everyday holding meetings? Such teams tend to leverage more async and written communication, write more complete, comprehensive documentation.

A good code review culture also improves knowledge sharing: We sometimes have two paragraphs of explanation in a pull request that changes one line of code. Those pull requests are forming a part of our institutional knowledge base and they are oftentimes referred during future conversations about past decisions.

The team should understand that spending a bit time today to write things down will save much more time in the future when the same thing has to be explained again and again. Also the team should be encouraged to spend this time on writing documents and not punished because of not starting with the next game-changing product feature.

I think one thing you have to do first is decide what’s really institutional knowledge and what’s really a make-work project designed to appease the people who still get a kick out of playing solitaire. Documenting how to make a Saturn V rocket engine is institutional knowledge, documenting endless workflows that will be stale before you’re finished are not.

If a system is not documented in terms of what it does, how it does it, and common remediations for when things go wrong, then it's a fragile system depending on one person to keep it going.

You need to get (particularly, senior) engineers to buy into the mindset of documenting everything and making the documentation the first place to look (not after all else fails). Leads should hold be held accountable for their docs meeting some standard of "this is a useful doc".

If I were in your shoes I'd get senior engineers in a room and be honest with them about the situation. You're going to need them to do work that they might not be naturally inclined to do, or work that might not seem like a "productive" use of time. You might want to get them to agree on what a good doc looks like (the standard), and what things need to be documented. Maybe take a whole day to do this, with snacks and coffee. Good luck!

I can only say what I know doesn't work.

Powerpoints lunch-n-learns and confluence pages.

Whatever it is it has to be there front and center with the day to day work. Enforcing good commit descriptions, a code review process where a project manager is part of the merge to capture things usually lost in the technical minutia would be my best guess.

To provide a non-coding team perspective. Having an internal database of documents with search capability is critical for explicit knowledge , whether custom built or simply something like google drive. Having a formal process around adding to and updating this store of knowledge is also critical.

For tacit knowledge, the traditional form of transfer is simply interaction in the course of work. having a searchable archive of conversations is valuable. In most organisations, people are only able to search their own emails but not others. Chat is helpful for finding institutional knowledge around simple problems but not optimal for deeper and more thoughtful discussions that tackle larger issues. This is why I’m currently working on TribePulse, an internal discussion platform, designed to surface and capture institutional knowledge.

Don't assume that the architecture is even there for the institution to have control over everything. Small, young organizations are often relying on things they aren't fully aware of, like so-and-so has an uncle or a trust fund or a fast car or something.

When you talk to the outgoing employees, ask yourself if the processes they are describing are even processes that are replicable or controllable by the organization.

So, for example, are people putting things on their own credit card and getting reimbursed? Is this potentially a problem? Do you need to arrange a company credit card so the organization has real control here?

Then you need to document things. But don't just assume that the architecture is even there for the institution to be in control. You may need to create that piece as people leave.

I work at a largish company (~2K employees, many of whom have been here for 20+ years). We have lots of information silos, and vast amounts of tribal knowledge exit the door with retirees every year. Recently we learned that StackOverflow offers their engine for private companies. "StackOverflow for Teams" allows you to set up your own internal Stack Exchange with all the benefits and conveniences that come with that tool. We are just now getting buy-in from IT and Senior Management, and we plan to start rolling it out across the company within the month. It's not cheap, but we calculate that it will pay for itself if it saves every employee an hour or two per year. I'm confident it will do much better than that.

It's a hard problem, but I've had some success with a few approaches. One is pulling the team into a conference room and giving a deep-dive presentation on some aspect of our system. I made a point of doing it via a web conference with recording enabled so someone could always go back and watch/listen to parts in a pinch.

Another is calling an impromptu huddle when I'm about to work on something that nobody else feels prepared to handle on their own. When some crazy bug crops up that I'm about to troubleshoot, I'll often ask a few other team members to shoulder surf while I talk through the approach I'm using to run the bug down. It's a great time to ask the group what they think we should do to test their knowledge.

We work on https://usecodeflow.com, which is a way to capture the way code logically flows (especially used for new-hires to learn a new codebase). Feel free to email me in my profile if you want to chat!

Your website doesn't mention pricing at all before asking me to provide my github details. I dont know if the service will be free or I will be asked to pay until I provide my github credentials. My suggestion would be to add some info in the FAQs section.

> I am involved with an organization that is slowly growing, is about to lose key personnel, and is looking to prepare.

Let's be honest here. There's nothing you can do to prepare for the loss of these people. You are coming at the problem too late. It's a cultural problem, not a procedural problem.

To fix the problem going forward, you need to establish new cultural norms. Force new employees to document what they find as they learn the codebase and the "processbase". Establish a new norm that a thing must be documented appropriately at each stage before it can progress to the next. There has to be real pain incurred (schedules missed) before this will really start to work. Management needs to have the will.

Strong change control processes: if you want to know why something was implemented the way it was, the ticket authorising the implementation should have all the details including test results and names for who built, who tested, and who signed off.

Firstly, I work at Slab.com — a knowledge hub for teams. So, obviously I'm slanted in my perspective. But I'm not going to pitch the specifics of Slab here for you. Rather, I'm going to share a post we recently published designed to help teams document/write overcome knowledge hoarding (regardless of what tool/system they use):


It's not a direct pitch to use Slab. In it we talk about the three reasons we discovered most folks hoard knowledge:

1. Leverage: If an employee hoards their knowledge, they may feel like they are irreplaceable. 2. Fear: Putting yourself out there can be intimidating. What if colleagues or supervisors respond with negative feedback? 3. Competition: If your workplace rewards personal triumphs over shared victories, employees are less likely to want to share their "secrets."

Also, no matter what documentation tool you use, it's worth reading this first: https://slab.com/blog/documentation-tools/

Many teams initially gravitate toward more generic document editors (Google Docs) as their first team wiki, for logical reasons:

- These editors are familiar to most people, meaning they can be easier to adopt across an organization - Most companies already use (and pay for) at least one of these tools - Teams realize the need for documentation, but aren't thinking of how their documentation tool will scale alongside their business

But there are some issues with these document editors that teams discover as they grow. This article dives deep into those shortcomings. Here are both article links again, and good luck!

1. https://slab.com/blog/knowledge-hoarding/ 2. https://slab.com/blog/documentation-tools/

How do you make people replaceable? By sticking to org structure with roles that most other companies in your sector use, so that they can be replaced by other people who have worked in other companies.

Knowledge is 'institutionalized' via experience first, documentation/formalization second, schools based on that documentation third, mass media fourth, gossip and other forms of informal communication fifth.

Stick with boring tools, boring tasks, then they can be replaced with boring people who won't quit to work on the next cool thing as long as the paycheque is right.

For process documentation (eg, not code docs), we use a company shared Google Drive folder with subfolders broken down by department (sales, marketing, support, finance, general management, hr, etc) and within each we have documents and checklists that serve as both documentation and quality controls for our most critical business processes.

We have videos, slides, spreadsheets, docs - we always try to use the best format for the job.

It's not perfect but you're right in that it has a huge impact on the efficiency of certain processes and therefore, scalability.

Use tools such as QuickQ (https://quickq.app) to capture institutional knowledge when it's shared via Slack!

Disclaimer, I made this app.

I wrote about related subjects here:


Specifically here, the importance of:

- allocating budget to the maintenance of knowledge

- rotating the responsibility for maintenance around the group

- co-locating the knowledge store with the day-to-day tooling, even if that doesn't conform to document management ideals

Confluence, or a similar private wiki is a good idea. As you work, write down steps to do things that were essential to doing part of your work, or things that you will need to repeat often.

E.g. Create tutorials on setting up a development environment, installing dependencies, compiling modules, running tests, creating new components. Write descriptions at a high level of the system, make diagrams of complex message exchanges. Write down best practices, or paste boiler plate code for doing specific useful things.

+1. Atlassian tools catch some flack (for good reason), but I worked at a company where anytime someone had a question that their immediate neighbor couldn’t answer, the response was “try checking Confluence, I usually find stuff there”. It made my onboarding SUPER easy because I could basically Wikipedia every internal process/setup/best practice. Awesome feeling.

I think Confluence - so long as it isn't overloaded with plugins - is the best Atlassian tool. Sure, it's basically a WYSIWYG wiki editor, but the page browsing and shortcuts and everything just work really well.

1. Incorporate documentation updates into your definition of done - at the task, sprint, and project level.

2. Always have an agenda. Always have someone taking notes. Notes must be reference-able (public slack channels count, wiki is better, email doesn't).

3. The first item on all new hire's onboarding: every time you learn something that's not in the documentation, or incorrect, update it. You will be explaining this to the next new hire.

4. Hire a journalism major intern to conduct interviews and cultivate archives.

Here’s a bit of a different answer: implement a company-wide wiki (if your company is medium-to-small sized).

I am in the process of implementing Wiki.js for my team, to a lot of excitement.

Institutional knowledge comes not just from managers, executives, and team leads, but also from those “in the trenches.” The finer details of operation can be lost the higher up on the org chart you go, and that’s where a well-organized and liberally maintained wiki (or other team knowledge base software) becomes invaluable.

Many good responses, but I should also point out: Retention. A bunch of screencasts isn't going to replace someone who got frustrated or just needed a change of scenery.

There are far better tools for managing knowledge than wikis. Check out Bloomfire hhtps://www.bloomfire.com They offer one such solution. You just upload documentation or type into the platform directly and every word in the document is searchable. From the site it looks like they even transcribe video and audio files and make them searchable as well. I know someone who has used it and loves it. Just to say, there are tools built for this....

I've recently made a small things called github-agent. Essentially it lets you keep your github issues as local markdown files. This helps keeping issue descriptions extensive and full of details as you work on them, since you can use your favorite editor (Emacs) to edit them and then sync in a simple terminal command.


I ended up having to accept that documentation would always end up spread across a dozen different systems: Sphinx/RST + GitHub Markdown + Confluence wiki + Google Docs + Continu + email mailing lists + the customer support website + I found one team using Google Sites + ...

So I built a custom internal search engine that indexed documents from all of those systems and made them searchable in one place. It worked pretty well!

When it comes to documentation, be realistic about how much time - in terms of product features delayed or not delivered - you have to spend on creating documentation and on keeping it fresh.

Also be realistic about how the documentation will be used. I found this paper and its references useful:


I have found Architecture Decision Records to be great for this.

They capture why a decision was made and the context/options at the time. They also provide an immutable linear log of decisions which is nice for on boarding.


Just a thought. Companies that are bad at transferring institutional are companies that have employees stay for longer periods compared to companies that do not. If you are looking for job security becoming the internal repo of undocumented knowledge is a good way to keep/grow your position within also be included in more key projects and stay in the loop.

The hardest part of onboarding, for me, is figuring out what to ignore. Every team has so much legacy and baggage.

The culling of deprecated stuff is too hard. The incentives are all wrong. And it's drudge work.

My proposal:

We need to time box and auto expire everything. Like TTL leases. Renew leases on active work. Everything else enters the grooming process.

Have stages, like Trash Bin vs hard delete, to minimize impact.

If your org doesn't already have some sort of wiki or equivalent (we use Confluence here at my day job), that'd be the top priority. Make it as easy as possible for those high-bus-factor colleagues to get their thoughts in writing. Even if it's messy, as long as you're able to search page contents, it's better than nothing.

Have someone learn how to do the things and write it down as they learn. The goal is not so much knowledge (but that is good) as it is being able to function.

Unfortunately, people who already know how to do something tend to skip over crucial bits in their explanations and documentation: "break the eggs before putting them in the pan".

We use a software solution: http://bloomfire.com It's kind of a hybrid between a knowledge repo, search engine and internal social platform. Their customer service is fantastic and they're continuously improving on the product.

One way is to define a project dictionary/glossary.

Shameless plug: I'm working on a tool to help teams define/share the internal jargon that always seems to develop whenever humans work together: https://jargonaut.net

Many people here will talk about practices etc. But tbh I think before that you need to make sure that whoever is in charge is bought into whatever you are going to be doing.

If that isn't in place. Any such endeavours will fail as other people in the business just won't bother.

If large company, making everything - docs, discussions, code etc. searchable through a single box.

In terms of operational knowledge and processes, we work with large clients in France that have huge yearly turnover. We developed operandy.com as a tool for them to document processes. It turned into a way for them to value and automate low-value work later on.

We find that Confluence + JIRA is a very effective way to pass knowledge around.

For my personal use I create detailed "How To" documents that detail the exact steps required to do something non-trivial. Others have found them useful when I posted them on Confluence.

Company tech-talks inter and intra department.

Provide lunch. Have the talks be about something very specific in your infra, a company value, a goal whatever.

But just short of compulsory, make people want to give and attend...

Could someone please post tool names as well? Everyone is talking processes. But I would love to know more convenient tools as well.

My company division uses Method Park Stages. https://www.methodpark.de/stages.html

I'm in a regulated industry, so this is probably overkill for a web shop.

This is a great question and the same pain the led us to start ShiftX. It’s a process based knowledge sharing tool where we have focused heavily on simplicity so that both creating and consuming content is as frictionless as possible. We are just starting to grow the company but are getting great feedback so far from our customers in the Nordics. Let me know if you would like to learn more! https://shiftx.com

This is essentially the app my company built:


Essentially, we monitor network communications and rank participants in the network by expertise, skills, workstyle, etc. Then we rank content they discuss / share based on expertise (for search). The system will also rank the participants (employees), and monitor the influence.

Hire young junior employees and give them broad responsibilities.

Add e2e tests and design docs for code repos.

Keep your employees around.

tell me about more?


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact