
Ask HN: How to organize personal knowledge? - _1tan
In the last few years I have collected a large amount of ebooks, papers, articles, movies, pictures, etc. (all digital).<p>I am at a point where I would like to organize these items into a cohesive, searchable and maintainable system. I have looked at various methods (e.g. Niklas Luhmanns &quot;Zettelkasten&quot;, Zotero, Org-Mode) and I am not satisfied yet. Any tips?
======
wenc
This is what I learned from designing data architectures:

If your data is variegated in format and form, don't look for a single
tool/method solution to organize them. There is no single all-purpose
organization method.

Don't start with figuring out a method to organize your information. You will
end up overengineering your org method.

Instead, organize your information per "use-case". Start with a specific use-
case/project (e.g. writing a blog or paper), and work backwards to figure out
how to organize your data to meet the requirements of that project. Do a
couple of iterations. After a couple of projects, and you will naturally
discover your own data use patterns.

If you go with a super organization system on day 1, it will likely be too
general and require too much effort (tagging, keywords, hierarchies, version
controlled, branches, etc.) you will end up expending resources on metadata
management on data that you may never ever need to retrieve and very soon you
will abandon the effort.

My PKM system is very simple: a single unorganized Google Docs for quick
thoughts and ideas (just bullet points), separate Google Docs files for
specific projects, etc. and Dropbox for files. It's simple, searchable, and
multi-device.

I also occasionally use some specialized tools like Jabref (BibTeX) for
specific types of data like references, but I hardly ever write papers
anymore, so these have fallen by the wayside.

I've tried wikis but due to their multipage nature, they segment knowledge too
finely (often there are wiki pages hidden in deep in the link hierarchy that I
forgot existed). Wikis don't fit the PKM use case that well, so these too have
fallen by the wayside for me.

For me, PKMs need to in some way feel like a single broadsheet where I can
easily see and touch my information without having to drill-down hierarchies
and follow too many links.

p.s. I've heard good things about Evernote. It's a little too heavy for me,
but many people seem to find it useful.

~~~
klenwell
_Don 't start with figuring out a method to organize your information. You
will end up overengineering your org method._

I think this is an important point. I like to start with a few simple rules:

\- To retrieve information, I should know where to start: a Schelling
point.[0] For me, this is the home page of my wiki. For wenc, it's a Google
Doc.

\- It shouldn't take me more than three clicks to get from my starting point
to the information I'm looking for.

\- Links/URLs will tie everything together. They are the edges in my knowledge
graph. But as wenc notes, keep the graph shallow.

Then I need to be rigorous, reorganizing things when they don't work
intuitively and adding new nodes when something I need has not yet been
recorded. As wenc puts it, "discover your own data use patterns."

Wikis do work for me, provided it's organized around Schelling points. I've
used and refined these principles in setting up wikis at my last 3 companies
and it's worked pretty well for organizing a collective knowledge base as
well.

[0]
[https://en.wikipedia.org/wiki/Focal_point_(game_theory)](https://en.wikipedia.org/wiki/Focal_point_\(game_theory\))

~~~
bostonvaulter2
What software do you use for your wiki?

~~~
johntash
I'm not currently using a personal wiki, but both MediaWiki and Confluence are
pretty easy to set up. Confluence/Jira licenses are cheap for self-hosted
personal use. MediaWiki is oss and probably not going anywhere anytime soon.

~~~
klenwell
I second Mediawiki. I prefer that for professional use:

[https://www.mediawiki.org/wiki/MediaWiki](https://www.mediawiki.org/wiki/MediaWiki)

Personally, I've been using WikkaWiki for years:

[https://github.com/bakoontz/WikkaWiki](https://github.com/bakoontz/WikkaWiki)

I would prefer something that supports markdown. Neither of these really do.

------
jasode
_> collected a large amount of ebooks, papers, articles, movies, pictures,
etc. (all digital)._

Since you mentioned those specific examples, I see a commonality and so I'm
going to assume you're primarily concerned about organizing files that are
_not authored /generated by you_. (Except possibly pictures/photos since you
might be talking about jpgs from your personal iPhone or digital camera.)

This distinction is key because software like Org-Mode, Evernote, MS OneNote,
etc is better for _personally generated note taking_ \-- like classroom
lecture notes, or brainstorming, or personal todo lists. Those tools are not
suitable for saving and managing a terabyte of movie files. However, they have
one exception to the "being bad at managing collections of the world's data
that I didn't write" \-- they do have very good "web clipping" functionality.
Instead of saving url bookmarks that might suffer digital rot, you can use the
web clip tool. (Personally, I prefer to save important webpages as .mhtml
archives. This doesn't work for javascript heavy sites but for the type of
informative webpages I want to save for later reference, this usually isn't an
issue.)

Likewise, the software that's good at cataloging a "library" of collected
files with capabilities for custom tagging (e.g. Adobe Lightroom for photos,
iTunes for music, Zotero for pdfs, etc) are the wrong tools for archiving
personal notes.

For a disparate collection of digital files, the best method I've found is to
leverage the _native filesystem_ in Linux/Mac/Windows. Create a hierarchy of
well-named folders to build a sensible taxonomy and put all your pdf, epub,
mp4, jpg, mp3 in meaningful locations. There is no good universal organizing
software for organizing all the file formats that has the longevity and
transparency of the native file system.

~~~
zubspace
_> However, they have one exception to the "being bad at managing collections
of the world's data that I didn't write" \-- they do have very good "web
clipping" functionality. Instead of saving url bookmarks that might suffer
digital rot, you can use the web clip tool._

Let me warn you: I tried to use OneNote this way. The web clipper is really
nice and I used it heavily to clip everything interesting on the web from my
desktop and my phone until my OneNote file reached 8GB. Syncing just
completely gave up and there is no way to export everything in a usable
format.

Since then I use a separate Firefox 52.5.0 instance with the old ScrapBook
plugin. I can send sites to it from a current Firefox instance and from
mobile. It works ok and, in case this setup should stop working at some point
in the future, I still have all sites sitting on my local drive.

Unfortunately the browser makers don't give a damn about saving and organizing
Websites locally, although I'd believe that this would be far superior to
Bookmarks. I love scraping my local library for useful stuff and I've been
bitten by link rot far too often.

For personal notes and todo lists I prefer simple text files with a few
categories. Lately I'm also giving Boostnote [1] a try, which simply looks a
bit better and allows me to add checkboxes and proper headings/separators.

[1] [https://boostnote.io/](https://boostnote.io/)

~~~
edwintorok
Sometimes the pages can still be found on archive.org, archive.is, etc. but I
usually find out too late that a page I bookmarked wasn't actually archived.
I've just found this Firefox plugin [1] that automatically sends your
bookmarks to archiving sites. (one at a time, so unfortunately this won't
immediately help with existing bookmarks, unless you visit them all again).

[1] [https://addons.mozilla.org/en-
US/firefox/addon/archiveror/](https://addons.mozilla.org/en-
US/firefox/addon/archiveror/)

~~~
alltakendamned
I wrote a little script that I run as a daily cron job which fetches all URLs
I bookmarked/saved in Pocket and submits it to archive.org.

I wish more people would do that as link rot is a very real thing, certainly
for technical information often saved in gists or github repos that are often
harder to find back.

------
megous
I'm not sure why noone in 150 comments recommended RDBMS yet, but it's great
way to store information in general. I put a lot of personal things into RDBMS
either imported from external sources, or added manually: invoices, double
accounting, time tracking info/worklogs, bank account records, phone company
records, passwords, email, irc/chat logs, things to download, things I bought
in aliexpress incl. description and images, links I liked, scraped backups of
my data from SaaS services I have to use...

It's replicated from SSD to HDD on the same computer immediately (I use
PostgreSQL replication and run multiple servers locally), so if one of those
fails I lose nothing.

If I ever want gui or cli tool it's easy to hack up electron app or PHP/node
cli script in an hour or so. DB access is much easier than parsing text files.
It's easy to extend (add columns/tables) without messing with some random txt
format.

You can also much more easily answer questions about your personal data by
just firing up a psql and writing a quick SQL query with a much better chance
of finding an answer to wider range of questions without resorting to actual
programming/scripting.

You have to write some schema, so that naturally brings some organization to
your knowledge as you have to think about it.

~~~
johntash
I'd be interested in hearing more about the system you use.

Have you written anything about it before, I'm curious about more specific
examples around what you're doing.

Do you automate most/some of what you put in?

I've thought about writing some scripts to export things from various online
accounts/services/apis and importing them into a personal database mostly for
backup purposes. It sounds like you're already doing that, and more.

~~~
megous
I don't know where to start, so here are some random thoughts.

\- Some of the data import is for archival purposes and mostly one-shot. I'm
old enough to have seen many services where I put my data in, or where I had
interesting conversations, go and delete everyhing in the process.

\- Some of it is for ease of access and offline use. For example Jira can be a
very slow system, where each action takes 10 seconds to complete. Some
services have too bloated an UI to be useful on slow computers. You can easily
lose access to some data/technical comments if permissions change, etc. If you
have data locally, it's possible to avoid all those problems and present the
data in any way you want.

Yes, I automate imports from external services. In the past I used PHP, now I
mostly use Node or Electron, because it has a transparent support for binary
data and JSON columns in postgresql, and if I want to parallelize HTTP
requests it's easier to do in Node.

Import/sync scripts always have the same structure. First there's some way to
gather the data from the external service, then I import it into DB with
update/upsert/insert helpers, for example:

    
    
        upsert(db, 'table_name', {id: 123, name: 'qwe', ...}, ['id']);
    

I always try to find and use entity IDs from the external data source, which
eases future syncing. Usually there's not much data so I fetch everything and
update the database (keeping what was deleted by the remote data source).
Often times the service supports queries that result in entities that changed
since some date, which is ideal for incremental syncing. If not, the service
usualy has at least a way to order entities by date, so I sync new entries
until I start hitting entries that I already have in the database. It's very
useful to use async generators for this in javascript, so that I can separate
data fetching and data storage logic in a clean way.

One other method I use, with web services that are too messy or complicated,
is to create a userscript for Violentmonkey that gathers data as I browse the
website and posts it to the database via a small localhost http service. This
is useful for JS heavy websites that don't have nice JSON APIs. The script
just runs on the background (or can be triggered by a keyboard shortcut) and
uses the current state of the DOM to get the data and send them to the
database. This method of import is invisible to the service itself.

Most of the scripts are fairly simple (< 100 lines), because most services are
built around ~2 interesting primary entities that are worth storing.

Because I use upsert most of the time, I can and I sometimes do add some extra
columns for annotation purposes. For example I mentined Jira. I added columns
for ordering issues and for marking them with whether I wait for some feedback
or not, and then I have a simple Electron app for ordering and listing my
issues that I need to work on where issues that wait for external feedback are
not shown until the feedback is provided. It's uncluttered, and as fast as you
can imagine, being backed by a localhost database.

In Node I use: [https://github.com/request/request-promise-
native](https://github.com/request/request-promise-native) and
[https://github.com/request/request](https://github.com/request/request) and
[https://github.com/vitaly-t/pg-promise](https://github.com/vitaly-t/pg-
promise)

PostgreSQL replication is well docummented elsewhere. I replicate the entire
cluster, which doesn't require any maintenance when creating databases/tables,
so it's absolutely painless after the initial setup. All you need to do is
check the logs from time to time. All I do is that I run the backup database
server on a different UNIX socket and disable the TCP/IP interface. It is very
flexible, as you can also put the backup server on a different machine, you
can have multiple backup servers, chain them and it just works even if you
have intermittent connectivity between the machines.

Most of my system is not universal or systematic and is very specific to my
use cases. But that will be true for anyone wishing to preserve some of their
data in a world that is fairly hostile to data portability. In fact, lack of
universality/configurability reduces the complexity by _a lot_ and makes it
all easily manageable for a single person with at most a few hours/data source
of time investment.

------
lgessler
Org-mode. Since it's a plaintext format it's easily searchable, and it's the
most feature complete and ergonomic piece of software I've ever used for task
tracking and knowledge management (including MediaWiki, TiddlyWiki, Google
Docs, Evernote, plain txts, and more).

One major pain point: no totally seamless mobile sync that I'm aware of. I
just write things down in Google Keep and transcribe them when I'm on my
laptop next.

~~~
Arkanosis
Not a perfect solution for everyone as you need a computer to be always on
(eg. a server, a NAS, whatever) for sync, but I've been using Orgzly on
Android with Syncthing to handle the sync between all my computers and my
phone, and it's much better that everything I've tried before.

~~~
kqr
That's my setup too and it has worked so well that I sometimes feel like I
have to double check that it's still working because it runs so smoothly.

Occasionally I have to consolidate multiple edits with ediff-files, but that's
normally only after I have accidentally disabled wifi on the phone for longer
periods of time.

------
sebslomski
As this question comes up quite frequently, here are some past discussions:

\-
[https://news.ycombinator.com/item?id=17586375](https://news.ycombinator.com/item?id=17586375)

\-
[https://news.ycombinator.com/item?id=11945882](https://news.ycombinator.com/item?id=11945882)

\-
[https://news.ycombinator.com/item?id=15736102](https://news.ycombinator.com/item?id=15736102)

~~~
seltzered_
also, comments in 'the software engineering notebook':
[https://news.ycombinator.com/item?id=15473702](https://news.ycombinator.com/item?id=15473702)

------
icc97
This is what Vanevar Bush was trying to create with his memex [0], plus
further what Douglas Engelbart was trying to build upon [1].

Gordon Brander's patterns [2] (on HN recently) seems quite close to these and
what you're after.

[0]:
[https://en.m.wikipedia.org/wiki/Memex](https://en.m.wikipedia.org/wiki/Memex)

[1]: [http://www.dougengelbart.org/about/vision-
highlights.html](http://www.dougengelbart.org/about/vision-highlights.html)

[2]: [http://gordonbrander.com/pattern/](http://gordonbrander.com/pattern/)

------
markpapadakis
I use Dropbox to store everything. There are no files that I care fore that
are not managed by Dropbox (or iCloud, for my photos/videos). I use iA Writer
to create lists and documents for everything in Dropbox folders. I also have a
folder for PDFs/papers (also on Dropbox). There are also folders for my study
nodes ( I study codebases ), notes on personal stuff, notes on Programming, on
pretty much everything. I use Spotlight (on macOS) to instantly locate what I
need among the 100s of such files.

I ‘ve tried all kinds of ideas before settling for this setup, and they all
felt forced or just too much trouble for what they were to me. Text files,
Dropbox, and Spotlight have been a perfect combination for my needs for years
now.

~~~
MapleWalnut
I do the same with plain text files. The only problem I have is searching. I
wish there was a Google for searching local files (i.e. concept search).

~~~
cecja
There was and I shit you not it was from Google.

~~~
jonathansorum
I miss it more than I miss Reader :(

------
kepano
I went down a deep rabbit hole on this about a year ago. Nothing feels totally
right but currently I use a mix of TiddlyWiki and Airtable. I haven't found a
great solution for media outside of Dropbox but those two tools work well for
text and data.

TiddlyWiki is rather inscrutable at first, but extremely customizable and very
thoughtfully architected. What I like most is that it is doesn't rely on an
internet connection, it's very customizable and non-linear. The basic idea of
being able to interlink things and create new pages just by adding brackets
around a word is something I miss anywhere I don't have it.

I use TiddlyWiki in an Electron wrapper made with Nativefier. With a bit of
custom CSS the interface is very pleasant.

~~~
justinnhli
Can you say more about how you use Airtable? To me Airtable has a really
appealing low/zero startup cost, but then I'm always tripped up by not being
able to do arbitrary queries, joins, etc. to create new views.

~~~
kepano
I use Airtable for "quantified self" type things like logging workouts and
health metrics, also tracking reviews of books and movies, planning travel,
and anything that I would otherwise use spreadsheets for.

By the way, you can do simple joins in Airtable by using the "lookup" and
"rollup" field types, but I agree that it is relatively limited in that
respect.

~~~
bfirsh
I use Airtable for this stuff too. Unfortunately it doesn’t work offline (not
even reads!) so I’m very close to reverting back to spreadsheets.

------
IvyMike
I kinda went the non-organized method of organization: dump it all into a
google drive and just count on search to find things for you.

A lot of people think of GDrive as pure storage but unsurprisingly Google
search within drive is pretty good at finding stuff. Although when you know
it's there and search fails, it's frustrating as hell.

~~~
a3n
I wonder if, when you put something new on, you should immediately do a search
and then bookmark the search. Then when you can't find your target in Drive,
you can search your search bookmarks. And when you can't find it there, you
can search your search bookmarks search bookmarks.

Or you could write the locations of your targets on the backs of turtles.

------
duncanawoods
For "ebooks, papers, articles" I use calibre as the database - it manages all
the relevant meta-data and converts between formats. I then sync the calibre
folder in google drive for search and cross device access.

[https://calibre-ebook.com/](https://calibre-ebook.com/)

For notes, I found the majority of my note-taking is research and reasoning to
support decision making for either software or business projects. For that I
built my own tool to capture rationale make it useful:

[https://thorny.io](https://thorny.io)

------
patates
I'd recommend perkeep[0] (previously known as camlistore). It is built for
exactly what you are asking for. It's written in go and it is very easy to
contribute.

[0] [https://github.com/perkeep/perkeep](https://github.com/perkeep/perkeep)

~~~
Quanttek
How has your experience using it been like and how does it facilitate note-
taking?

------
se7entime
Personally i can recommend [http://zim-wiki.org/](http://zim-wiki.org/)

you can organize the "media" think in "attachment" and tag them or create page
about them, and you can still find the attachment in your file manager

"Zim is a graphical text editor used to maintain a collection of wiki pages.
Each page can contain links to other pages, simple formatting and images.
Pages are stored in a folder structure, like in an outliner, and can have
attachments. Creating a new page is as easy as linking to a nonexistent page.
All data is stored in plain text files with wiki formatting. Various plugins
provide additional functionality, like a task list manager, an equation
editor, a tray icon, and support for version control. "

~~~
I_complete_me
zim-wiki for me, absolutely. Text-based, offline, decent plugins, cross-
platform, easy. What I love is that if zim is not available I can use Sublime
Text to view and navigate as a near replacement as long as the
SidebarEnhancements plugin is installed. I stand by the supremacy of zim-wiki
although it does lack "pretty".

~~~
inawarminister
Do any of you sync your Zim/open it from mobile, specifically Android? I've
been using Zim for a few days now and it works well, but would be much better
if I can access the database while on the go...

~~~
dade_
I used zim for a while, then switched to Joplin. Huge fan, cli, modern and
mobile interfaces. Choose your own storage. open sourcen

~~~
johntash
Thanks for the mention of Joplin. I just started looking at it and it looks
pretty decent for an evernote replacement.

------
edwintorok
Some of these have been mentioned at various points:

[https://zettelkasten.de/posts/three-layers-structure-
zettelk...](https://zettelkasten.de/posts/three-layers-structure-
zettelkasten/)
[https://github.com/renerocksai/sublimeless_zk](https://github.com/renerocksai/sublimeless_zk)
[https://tiddlywiki.com/](https://tiddlywiki.com/)
[http://journal.mcmorgan.org/view/incremental-
paragraphs/ward...](http://journal.mcmorgan.org/view/incremental-
paragraphs/ward.fed.wiki.org/incremental-pages)
[http://fedwiki.org/hack.platform.earth/fedwiki-for-
notetakin...](http://fedwiki.org/hack.platform.earth/fedwiki-for-
notetaking/journal.mcmorgan.org/fedwiki-as-memex-journal)
[https://github.com/burtonator/polar-
bookshelf](https://github.com/burtonator/polar-bookshelf)

Perhaps a single system is not the answer, but a combination, e.g. for taking
notes I'd pick fedwiki/sublimeless_zk/Zim/org-mode.

For organizing PDFs with full-text search Zotero is quite good, and you can
link to your PDFs from your notes, but I wouldn't store my notes about PDFs
inside Zotero (because then it'd be separated from my notes about non-PDF
articles).

For keeping track of articles Firefox bookmarks (with Firefox Sync on Android
acting as a 'read me later' bag).

------
ronaldvalente
DEVONthink - great for all things bookmarks, data, searchable PDFs etc - can
also use it for org-mode files and open in your fav editor. Many sync options.
Mobile too.

~~~
kstrauser
Seconded. I have databases for household scans of bills, etc.; one for
computer science stuff; one for my work; and so on.

------
sunjain
I used to dump every thing in GoogleDrive and search. I was enamored with
keeping my notes in plaint text, so as to not tie these with any specific
tools/app/formats as these go out of fashion every few years. However I
realized I also have links/clippings from Web, which needs to be part of my
searchable knowledge-base. And I want to be able to clip/bookmark those from
my phone as well as Mac and Windows machines. Evernote works great for such
use, and has tags(it does not keep notes in plain text format though, these
are supposed to be exportable though). I would not mix video/movies with text
based content though(links are fine).

------
samuraixp
I use [http://papersapp.com/](http://papersapp.com/) for handling PDF papers
from arxiv.org etc.

It seems they have merged with readcube who want a yearly subscription however
I was still able to buy a life time license to papers app a few months ago. I
guess support will slowly disappear but for now its working great, with PDF
sync via dropbox onto my iPad or iPhone, the ability to draw and take notes
and search as well as download new papers. For your own PDFs I guess this
would still work pretty well since you can search and annotate, create smart
groups etc.

------
breerly
Notion for everything: my work, personal, and wife+I joint workspace.

~~~
jeremejevs
Moved to Notion from Marxico+Evernote a couple of months ago, haven't looked
back.

~~~
johntash
Are you using the free plan, or the personal paid plan?

~~~
jeremejevs
Free for now, but will start paying when I run out of blocks (the usage isn't
as high as I expected, only at ~500 out of 1000 right now, though I have
imported 4-5 years worth of notes).

------
sebslomski
Also it's not a product or a service, I really like Gordon Brander's
"Patterns" [1] on his website.

[1] [http://gordonbrander.com/pattern/](http://gordonbrander.com/pattern/)

~~~
based2
[https://news.ycombinator.com/item?id=17880963](https://news.ycombinator.com/item?id=17880963)

------
skunkwerk
Dynalist.io is the best note taking software I've seen in 12 years of using
note-taking software. Features endlessly hierarchical lists, with tagging,
links to other nodes, images, zooming to a single node, full-text search,
Chrome web clipper, Dropbox backup.

\- It's like Workflowy, but better... \- I've tried Evernote, OneNote, Apple
Notes, Google Keep... they all just end up becoming a massive mess of
unorganized notes.

~~~
yigitdemirag
And don't forget inline LaTeX support. With this feature, listening technical
conferences, lectures or noting mathematical ideas are really comfortable.

------
xixixao
I’m super interested in this space! Agree with sibling that focus here is on
knowledge management while question mentions media, which is probably even
more interesting challenge. I recently dreamed up something that kinda
resembled Airtable (which you should check out), albeit I was thinking more of
datomic like system as opposed to tables (more flexible, but tables are nice
and simple). I like the mention of wiki like system, I still find wikipedia to
be my favorite knowledge source. I guess for me the ideal system has bunch of
requirement you haven’t mentioned:

\- mobile and desktop friendly (especially input) \- full local backup,
including any metadata (so that I can always use another software, not cloud-
first-with-export) \- can be purely single-person (plenty existing sharing
platforms) \- it’s gotta support arbitrary nesting (I hate evernote/one note
three levels to a wisiwig page rigidness) \- any others?

I also wonder whether there’s enough market for selling this software
(Airtable is going after business use cases, the single-player use case makes
it quite limited).

~~~
xixixao
Forgot one: full tree history. No way to lose data, even if I do undo, delete
a page, etc. Super important.

------
jen729w
Another shameless plug: you might find
[https://johnnydecimal.com](https://johnnydecimal.com) useful.

Forgive the design, I’m right this very weekend updating the whole thing. I’m
busy writing a little web app which will replace the spreadsheet you’ll
currently need to keep track of your numbers.

~~~
seltzered_
I had come across johnnydecimal at one point but had a hard time resonating
with the idea of having numerical enumerations since I already use tools to
help navigate folder hierarchies (e.g. alfred, quicksilver, etc.).

Do you still find value in the enumeration aspect? Is this something where
there's more value in it within collaborative environments?

------
kasperset
This can be difficult. I was looking for a system to organize all the personal
knowledge beginning of the year. I tried Evernote and several of the
alternatives such as OneNote, but none of them stuck with me well. My main
issue was storing code along with the notes. Evernote does provide code
formatting but not very friendly with the syntax highlighting. I also tried
Quiver for storing code, but it is specialized towards code storage.

Currently, I am trying StackOverflow Team but only for myself. I am so used to
question and answer format that it makes it easy for me to dump the
information. Most of my searches on Google ends up on StackExchange families
of the website that this solution has that familiarity. Plus, it has some
versioning built into it. It lacks on some fronts such as limits on the size
of images. I am still looking for a better solution.

~~~
nickelbob
> I also tried Quiver for storing code, but it is specialized towards code
> storage.

Can you expand on this? Was quiver not good enough at basic text?

~~~
kasperset
Yes, it was but I wanted to access information with a web browser as well. I
might give it a try again.

------
timwis
Personally I use markdown files synced with Dropbox and search with the silver
searcher. But I think I better answer to your question is perkeep (formerly
camlistore):
[https://github.com/perkeep/perkeep](https://github.com/perkeep/perkeep)

------
ahane
Tiago Forte has a lot of thoughts on this topic and a partialy free/paid blog:
[https://www.fortelabs.co/](https://www.fortelabs.co/)

I'm not affiliated in any ways but his ideas a really well thought out and
apparently thoroughly tested with clients.

~~~
alexpetralia
I would highly recommend the idea of "personal knowledge management" (PKM). I
started being much more organized on this front starting last year and it is a
gamechanger.

Personally I store raw thoughts and ideas in Keep, then transfer them daily or
weekly to OneNote. Every so often I review the notes.

I have notes from anything important I've ever learned, ranging from
technology stacks to economics to capital markets to psychology or personal
well-being. I now never worry about losing thoughts or not seeing "the big
picture" \- it's a picture (note really) that I've saved over years now. And
ever so gradually, I just build on that knowledge base, which I see as an
extension of my own brain.

Edit: and if you don't use the cloud, _make sure you back up every few weeks_.
It takes 5 minutes and you will really regret losing part of your brain if you
break your laptop.

~~~
zoomablemind
Very practical approach. Reviewing the saved info is what makes this real
knowledge.

I find that any kind of knowledge search system eventually gets saturated and
requires increasingly more time to scroll through partial hits to find what
was asked. Our brains work the fastest in associative mode, it'd be nice to
have some kind of associative storage (other than tags) that is personally
biased/customized.

Reviewing the kept info is one way to refersh these associations in our
organic minds.

But it'd be cool to ask your 'MindAssistant' to retrieve you the whole context
of your recent or old idea, complete with your thought pathways: 'Hey, Mind,
please remind me what was I thinking about the shapes of the clouds in the
sky', '-Here's your Mindmap, thought-journey, and most important insights,
Sir. At the conclusion of these thoughts, you read the following articles via
HN...'

------
bcheung
Would help us to know more about the specific use case and goal you had in
mind.

For now, generically speaking, I use OneNote for keeping track of my own
generated ideas and pasting in things from other sources. For larger media I
recommend having a network drive or using Dropbox.

I'm happy with OneNote.

For my more technical ideas where I'm brainstorming I have a nice leather
journal (no lines) that I use for brainstorming. I tried using a Surface
Studio as well with the pen and that works good too because you have an
infinite canvas and can move things around. With paper / whiteboard
brainstorming I frequently run out of room. Digital means I can just drag a
bunch of stuff to make more room. Plus having the pen and choosing different
colors makes it really freeform.

~~~
ajmarsh
I loved OneNote until I tried to put some code snippets into it. Somehow over
time they become unusable for cut and paste into *nix systems. Did you find a
way to overcome this? I have no time to dos2unix everything.

~~~
dade_
OneNote even has a code format, but it still messes with capitalization. I
gave up when MS announced mobile only and the desktop app is done.

------
haney
I use Evernote for ebooks, papers, articles and personal notes. Probably
wouldn't work well for movies or large photo albums (I iCloud for stuff like
that). I use their chrome extension, desktop app and mobile app and it works
really well. It's really nice to be able to search everything I've ever
written in notes or clipped on the web. It also has some support for OCR for
paper notes, and does a decent job of indexing text in PDFs. I searched for an
open source or self hosted solution for a long time, but ultimately broke down
and chose a well supported commercial solution.

~~~
aasasd
Unfortunately Evernote gradually becomes worse and worse, at least on Mac. The
very basic functionality of editing notes is buggy and slow. Issues keep
creeping into both simple text notes and web clippings. Users are complaining
about this for years, the devs do nothing but change the looks instead and
bolt on corporate features (apparently corporate users don't need basic
functionality working well?).

They have the weird community forum where employees don't post anything except
for announcements. And now they have an extra tier before actual support,
where your question is 'answered' by some community member or someone like
that, who apparently can't be bothered to understand the question.

The Android app is a slow mess eating all available memory right away and/or
refusing to edit notes. Inability to use SD cards is just what I want when the
notes take 6 GB. And the website now “doesn't support” mobile browsers.

Pretty much the only thing where Evernote doesn't have better competition is
the web clipper.

~~~
johntash
The web clipper is definitely the main reason I still use evernote. I've tried
a few alternatives, but haven't found anything nearly as good.

Other than the speed issues you mentioned, I'm also worried that evernote is
eventually going to start selling my information (if it doesn't already) and I
don't feel safe putting personal notes on it.

------
BrianOD
I've just put this complete thread as a link into my Netsso, to study later.
If I want to avoid future link rot, I tell Netsso to "save content" to my
Dropbox/ GoogleDrive/OneDrive. In doing that, I could also click to encrypt
each file separately, with a 40 character unique password,that only I know,
and only decrypts on my local machine. Netsso remembers the passwords but only
I know them !.It stores any type of document, any url. I load files from my
computer to my Dropbox, and immediately link them to my Netsso, so I can
access them on any device without formally opening the Dropbox. Online Note-
making, with rich text, images included, even videos. Save any new urls with
quick click/copy/paste. Add 400 character description to a link, to include in
later search. Very sophisticated categorisation, sharing (even encrypted
notes), drag/drop, inbuilt credentials for apps, etc (i.e password management
included)...8000 links so far. Preview or Open on line, download, edit. Also I
have a sharing version for Teams, with same functionality (plus encrypted
Chat). Built for private,personal use - all my web knowledge available
anywhere- but very easy to share links, or collections of links, if required.
I can also Back Up complete collection to any computer in seconds, encrypted
by my master password. Netsso seems to be a unique combination of bookmarker,
online file creator and manager, internet file security enhancer and team
collaborator.

------
xbryanx
I use WorkFlowy[1] as my starting place for all notes, info, and tech tips.
It's quickly searchable and the mobile client is easy to use. I really like
the structure of bulleted lists for just about all types of information.

It doesn't really cover your file storage needs, but I think that's a
different beast than notes and information. When I need to store stuff like
that I dump it on Google Drive which makes search/sync/browsing dead
simple/cheap.

1 - [https://workflowy.com/](https://workflowy.com/)

~~~
aasasd
Judging by Workflowy's development pace, the entire team is spending the
profit on opiates. No features were introduced in the past couple years, aside
from the desktop apps—despite users begging for an API at least. Now Workflowy
went the way of Evernote: changing the looks and adding bugs.

The outline paradigm is great, and Workflowy is a nice influx of fresh blood
in that field, but I think I might actually kludge Org-mode into proper form
with good mobile synchronization (and bonus privacy and customizability)
before Workflowy makes any progress.

~~~
delaaxe
[https://dynalist.io/compare](https://dynalist.io/compare) shared above seems
to have more features than workflowy

------
shbhrsaha
For personal notes, check out Nvalt [0]. The search is blazing fast. I linked
it with SimpleNote [1] so changes sync instantly between my laptop and phone.
It also plays well with the Typora [2] markdown editor

[0]
[http://brettterpstra.com/projects/nvalt/](http://brettterpstra.com/projects/nvalt/)

[1] [https://simplenote.com/](https://simplenote.com/)

[2] [https://typora.io/](https://typora.io/)

------
madmarsu
Agree to wenc. If you try a hierarchical organization (you could use the same
classification as in official libraries for example) then you won't find what
you want easily. The best solution is NOT to sort too much, and use a (free)
global tool to find what you search. If you're under windows, then
"everything" of voidtools.com should help. (This program is realtime, it's
amazing.)

if you're using any Unix flavor, activate "locate" and use it. Both do an
index of the filenames. So naming correctly your files is important. The 8.3
schema is over now ;)

On a WinPC I also use TotalCommander (of ghisler.com) to make an advanced
search (filename and/or content). Of course, find&grep under Unixes do that
well!

I got more than 40Gb of ebooks, pdfs, videos, podcasts, etc. in 4 languages
(French, English, German, Spanish) and the sole way to find out quickly is
global searching.

And constant reorganizing. (With no tool to update, it's a breeze ;))

If you prefer a hierarchical way, then using aliases (symbolic links, on
unixes, but since Win7 you can also create links under windoze) can be an
alternative to have multiple entries of the same files in your "database" of
filenames, making possible to have "NoSQL for dummies" in "RDBMS" as well as
"BigData" and "books for beginners" categories.

P.S. I work on documentation systems since 1984. Full text indexing is
probably the best solution.

Madmarsu

------
kkirsche
Devonthink is a great digital archive

------
nathcd
I really think what we need for this is a new sort of file manager, and it
should look more like iTunes or Kodi than Thunar or Dolphin.

I always enjoy reading this post about tag-based file organization, which
feels like it would be a great foundation for this:
[https://www.nayuki.io/page/designing-better-file-
organizatio...](https://www.nayuki.io/page/designing-better-file-organization-
around-tags-not-hierarchies)

------
nickjj
I keep things simple. The following process has served me well for ~20 years:

For brain dumping text I put things into text files, named by YYYY-MM such as
2018-08.txt and they all end up in a notes/ folder.

Then I have a bunch of folders for things that are non-text, organized by
whatever it is. I try to keep folder nesting at a minimum.

Searching text is super simple with grep and searching for specific files
works well enough with file searching tools (and are also easy to browse
manually).

------
KozmoNau7
I did try to find a good cohesive and cataloged system, but in the end I just
ended up going with folders and descriptive filenames.

For instance, my music resides at "Music/Artist/Year - Album/Track# -
Tracktitle.Filetype", with appropriate additions to the Tracktitle to signify
bonus tracks, remixes, live tracks and featured artists.

I've adapted pretty much the same system for all of my other documents. It's a
bit old-school, but it works.

------
toomuchtodo
I try whenever possible to push as much personal knowledge into Wikipedia and
Stackoverflow, and then use bookmarks to keep track of that info in my
browser. This ensures that others can take advantage of that info, forever.

EDIT: I'm still trying to find a good machine readable way to represent the
metadata for all that information. Alas, there are only so many hours in a
day. Will probably end up with tags and Pinboard.

------
stevenleeg
Personally I use vimwiki. It stores everything in plaintext files that I can
search, keep track of in git, and also comes with quite a few useful features
for building articles.

The main downside, as someone else has mentioned here with org-mode, is the
lack of a good mobile editor. This is something I’ve wanted to find time to
fix for years but still haven’t had a chance.

~~~
spicytunacone
How are you currently viewing/editing your wiki on mobile?

I sync to a server with vim and vimwiki installed and on my phone I just
mosh[0] in. I don't generally do extensive editing on the go, though. Did the
solution you have in mind include a specialized editor for touchscreens?

[0]: [https://mosh.org/](https://mosh.org/)

~~~
timokau
I recently started using Markor for that purpose. While it wasn't built as a
vimwiki client, it is good enough for my purposes. Some info at
[https://github.com/gsantner/markor/issues/317](https://github.com/gsantner/markor/issues/317).

------
Jaepa
I generally control the organization of stuff by the way it comes in, and then
have a secondary means to have media organized that comes in from a non-
standard ingress.

Most information is stored in of 3 ways

    
    
      1. Synced to a NextCloud instance running on a VPS.
      2. Synced to a home NAS running Syncthing (though I may change this to IPFS at somepoint).
      3. Added to Wallabag (A read it later like service)
    

_Notes, List & Project based Todos_

These are put into Orgmode which syncs with Owncloud. I can then use the
WebDav endpoint to pull the files to add or edit on my phone.

 _Articles /Papers_

These get added to wallabag then are accessed via an RSS reader.

 _Personal Documentation_

Added with Camscanner, OCRed, and added to NextCloud. At some point I will
probably also store these on my NAS, so I can extracted the OCRed text as a
text file so I can search them, but currently the collection is to small to
justify this.

 _Media_

My NAS handles most of this, and organizes it using Emby. Syncthing is my
secondary means for this, and Emby watches the directory to organize this.

 _eBooks_

Syncthing to add to the server. A headless calibre instance then organizes it.

 _Pictures_

Most pictures come my from my phone, nextcloud will automatically sync these.
If the pictures are part of notes, then instead they go into Orgmode, which
works with images as well.

There are a couple of other organizational thing that use Syncthing +
Organize, but nothing of major note.

This is not A single cohesive system but several scoped systems with a
standardize way to add content to organize (e.g. folders). Searching the
content is generally done via plain text searches, or inside of the
application that manages that context (see emby & calibri).

Links

    
    
      1. https://github.com/nextcloud/server
      2. https://github.com/syncthing/syncthing
      3. https://github.com/tfeldmann/organize
      4. https://github.com/wallabag/wallabag
      5. https://www.camscanner.com/
      6. https://beorgapp.com/
      7. https://manual.calibre-ebook.com/server.html
      8. https://emby.media/

------
dragonquest
I use Personal Knowbase, a small Shareware era program. It has minor
annoyances but does what it says on the tin. It's fast, minimal and allows
narrowing down to related notes in a jiffy. The makers update it slowly
though.

[https://www.bitsmithsoft.com/](https://www.bitsmithsoft.com/)

------
lcall
After trying different things over the years, I wrote this:

[http://onemodel.org](http://onemodel.org) (yes I plan to move to https
sometime)

You can think of it something like a personal wiki + emacs org-mode, using
postgres, but with a much larger vision than today's features, including
sharing (linking/copying) between instances, and computability of the info.
The most current code is in github (AGPL). Comments/questions very welcome,
preferably via the mailing list; be patient if my answers are slow.
Announcements list is low-volume.

Edit: it can store files, but isn't especially smooth about it (yet). For
personal notes of all kinds, it is the most efficient, effective, flexible
thing I know of.

Edit: the FAQs link to a discussion comparing it with emacs org-mode and
others.

Edit: it has built-in text search, and some finicky but very functional
import/export.

------
rasengan0
Really enjoying this one: [https://itunes.apple.com/us/app/mindscope-thought-
organizer/...](https://itunes.apple.com/us/app/mindscope-thought-
organizer/id901513028?mt=8)

Based on this concept [http://www.mandal-art.com/](http://www.mandal-art.com/)

but original mandalart author had/has less effective implementation

Truisms from time: \- Content > tool config \- Simple is better \- Have good
search \- ISO 8601 date format, line based entries

Old and current flames over 20 yrs: \- Google Desktop \- TheBrain \-
MindManager \- Freemind \- NotationalVelocity \- TiddlyWiki \- VimWiki on vim
as Deft folder on emacs in evil-mode \- Org-Mode in native and evil mode \-
log bash script \- sqlite3 \- Traveller notebook, pencil and paper

------
stockkid
My tip is to break down your knowledge into very small chunks and store them
somewhere easily accessible and searchable. I have been looking for a solution
for this, and that led me to build a system for myself and open source it [0].

In short, I find that writing them down every day in bite-sized chunks is most
helpful. This method probably will not help you store non-texts such as
movies, pictures, but for other mediums, it is great.

After about two years, the main benefits I find are:

* Bite-sized notes are easier to index and search for later

* We can easily share those notes with others when in a technical discussion etc, saving a lot of time

* We can do spaced-repetition on those notes easily because they are concise. (I wrote a short Go program to do this for me)

[0] - [https://github.com/dnote/cli](https://github.com/dnote/cli)

------
kd5bjo
I started down this road several months ago, and decided to write my own
system that can grow and change as I discover how I want to organize things.
It's roughly modeled off of a traditional categorized card-catalog system,
with the addition of zettelkasten-style numbered links. As a concrete example,
here's a card that I added today:

    
    
      [[379]]  Sjö Milljarður Manneskjur
            |  Tölvuleikur - Forritun
            |  Inventory - Steam library
            |  Computer Science - Concepts - Multiprocessing
            |
            |  ,,Seven Billion Humans''
            |
            |  2018; re: concurrency
            |________
            |  Backreferences:
            |  [[380]]  Tomorrow Corporation
            |        >  2018: [[379]] Seven Billion Humans

------
miles
I still use Together 2.6.8[0] for collecting and organizing just about
everything. Multiple libraries, built-in previewing, editing, and Spotlight
search, no proprietary database (uses Finder under the hood), etc.

There were some changes in version 3 and the subsequent rebranded Keep It[1]
which kept me from upgrading, but it might fit your needs.

[0]
[https://web.archive.org/web/20130315081858/http://reinvented...](https://web.archive.org/web/20130315081858/http://reinventedsoftware.com:80/together/)

[1]
[https://reinventedsoftware.com/keepit/](https://reinventedsoftware.com/keepit/)

------
tmoravec
I use OneNote currently. Dump (almost) everything in there. The UX leaves
something to be desired - Evernote is so much better. Shame that Evernote kept
losing my data.

And Google Drive for the "larger” and better organized stuff like photos,
e-books, or invoices.

~~~
dade_
Try Onetastic, the calendar feature has saved me countless hours. Desktop app
only though.

------
Dowwie
I don't look back at my notes. So, I don't think I should spend a lifetime
organizing them. I write notes while learning and reflecting. Once finished
with that process, 99% isn't used again.

I use meaningful titles and tags. That's enough.

------
csbartus
Shameless plug: [https://morethemes.baby/themes/log-lolla-
pro/](https://morethemes.baby/themes/log-lolla-pro/)

I do organize my knowledge since years
([http://metamn.io/beat](http://metamn.io/beat),
[http://metamn.io/pulse](http://metamn.io/pulse),
[http://metamn.io/gust](http://metamn.io/gust)) and now I’ve collected these
experiences into a Wordpress theme.

Still under development / test phase ... you are welcome to try it out.

------
based2
Trees -> Typed nodes -> Graphs

MacInTouch More (1)

macOz Omnigraffle (2)

(1)
[https://en.wikipedia.org/wiki/MORE_(application)](https://en.wikipedia.org/wiki/MORE_\(application\))

(2)
[https://www.omnigroup.com/omnigraffle](https://www.omnigroup.com/omnigraffle)

Notes:

TreeSheets: Free Form Data Organizer

[https://news.ycombinator.com/item?id=15057392](https://news.ycombinator.com/item?id=15057392)

Quetree – A Q&A site that is tree-based and hierarchical

[https://news.ycombinator.com/item?id=17848846](https://news.ycombinator.com/item?id=17848846)

~~~
based2
Trilium Notes is a hierarchical note taking application (electron js)

[https://github.com/zadam/trilium](https://github.com/zadam/trilium)

------
davchana
Digital: Notes in Google Keep, Text files in Dropbox. Paper: Bullet Journal,
Lists

Downloaded data: Folders PDF, Video, TV. Own Generated data: Photos in a
YEAR/MONTH Labels Textual: In Dropbox, Excel, Text, Word

All data on master Hard Disk, two more copies.

------
geonnave
Nodebook¹ is the tool that I've been using for the last couple months. IMO, it
beats Zotero and Evernote, or any other tool I've previously used.

¹ [http://nodebook.io](http://nodebook.io)

------
RBerenguel
I use NValt [0] and plain text files for all my personal notes/info tracking
system. I use 1Writer [1] to access/add to it from iOS, and on Mac I either
use NValt itself or Deft [2] in emacs. I try to convert any visual information
into a textual tip, otherwise, most images, PDFs and ebooks go to Dropbox
while I'm actively reading or are in "pending", after being read they go to
long term storage.

    
    
      [0] http://www.brettterpstra.com/projects/nvalt
      [1] http://1writerapp.com
      [2] https://github.com/jrblevin/deft

------
jshwlkr
I always though the smallest federated wiki would be prime for this.
[http://fed.wiki.org/view/welcome-visitors](http://fed.wiki.org/view/welcome-
visitors)

------
kevin_indig
I use Evernote (for notes) and Dropbox (for files). What helps me to stay on
top of things is that I write everything in Evernote myself and then add the
references at the bottom.

First, that helps me to rephrase knowledge in my own words (except for quotes,
of course), which is good for retaining knowledge.

Second, I use tags and note links that help me to navigate through pieces of
knowledge.

Third, I only store files in Dropbox, with a very well-maintained folder
structure.

I don't clip stuff in Evernote and don't just copy/paste. I think that bloats
the whole thing up and makes it unnavigable.

------
SomewhatLikely
I'm surprised to not see any YAGNI voices in the comments. I set up Evernote
with lots of collected papers I had read. However, I found I would rarely go
back, and finding things within a collection of two hundred items was already
not easier than just googling it. Perhaps with improved semantic search the
situation may improve. I just think summarizing down to a small set of core
ideas and storing them in a single small text file or document is better than
keeping extensive originals or even per item notes.

~~~
ahel
Yeah. Summarize and Anki. Otherwise you are just collecting.

------
a3n
Shameless plug: miki (Makefile Wiki)
[https://github.com/a3n/miki](https://github.com/a3n/miki)

"sitemap.html" and "catalog.json" are the most relevant parts. The project
itself assumes you already follow some organizing principle based on the
filesystem, and documents it for you, with links.

It's based on Makefile, and existing utilities, so you should be able to use
it as is, customize it, or as a start of something better.

------
ausjke
I now use gitea to host code and markdown/notes and generic filesystem(or NAS)
to host large size blob(movies,pdf,etc).

for a universal knowledge/media/file organizer the way might be a smart tag-
based system, could not find one still.

used drupal, wordpress for CMS in the past before switched to gitea. Can't use
static-site-generator as I don't want to put random personal notes on internet
without even a login.

a simple, login-included, markdown driven cms with search/tagging will be
really nice.

------
nebulous1
I use Zim but I can't say that I've done an exhaustive survey.

[http://zim-wiki.org/index.html](http://zim-wiki.org/index.html)

------
haloux
Why are you unsatisfied with the particular methods you listed?

------
shubhamharnal
The key to organisation is a WORKFLOW, followed by tools that fit your
workflow.

I found and paid for
[https://www.buildingasecondbrain.com/](https://www.buildingasecondbrain.com/)
. It was immensely useful for me. The course provides a workflow specifically
on managing all the information that feeds into our lives and shows you how to
use Evernote to do this. Happy to answer any questions from my experience.

------
mud_dauber
I use Pocket to grab interesting content as it swooshes by. :-). If it's worth
keeping, I send it to Google Keep, then use Keep's color codes to organize
material by rough use cases (code, product management, hobbies, behavior, ...)

I then periodically clump similar content into a single Keep note, usually by
simply cut/pasting the relevant links. This happens once every 3-4 weeks.

Now if only Google's Keep team would publish an API...

------
JesseAldridge
I've been using the offline version of Draw.io for this. You can save diagrams
as individual xml files. Unfortunately the data is base64 encoded by default
so you have to manually convert them to a searchable format. I wrote a script
to do that: [https://github.com/JesseAldridge/draw.io-
convert](https://github.com/JesseAldridge/draw.io-convert)

------
yigitdemirag
For books, PDFs : Zotero (if book/pdf requires to be sync have a copy at
iCloud) \- Because I usually need to cite these things.

For notes: First condense the thought and put it nicely to Dynalist \- Because
it is all easily searchable, linked and think-then-write tree-structure. \-
Mobile app is a must to capture ideas, thoughts on mobile.

For todo: Things/Todoist etc. would work. \- Because you need deadlines,
reminders.

------
mark_l_watson
I wrote my own Evernote style system and used it for a while. Then I changed
to my current system: Emacs and org-mode for notes, todos, tasks, etc. I also
use Notes in Fastmail (and they show up in Apple Notes automatically). For
PDFs of purchased books, Communication of the ACM, good articles converted to
PDFs, etc. I use Google Drive because it is searchable.

~~~
sridca
By searchable you mean Google Drive can search _inside_ of documents like PDFs
and images?

~~~
mark_l_watson
I don't think search works inside images but works well for PDFs. Just this
morning I was trying to find a particular paper on LSTMs and NLP and I found
what I needed very quickly.

------
xtiansimon
I also follow many of the recommendations here and keep text files for my
personal knowledge. I prefer to keep PDFs rather than HTML files. Currently I
print webpages to PDF after rendering the page in (Firefox, Safari) Reader
mode.

An idea on my back burner is finding a solution for auto-generating taxonomies
using Python NLP. But I've been saying that for years. Soon.

------
delbel
I use qownnotes and Nextcloud on a NAS and back it up on external storage, as
well as every desktop/laptop also has a local copy. It uses Mark Down. So far
it seems to be working fine. Nextcloud is very flaky but better than owncloud.

I keep a lab paper notebook for everything I do and I write the date, time,
and whatever I need. Then I type it in.

------
Findeton
I'm working into this right now at
[https://lyfepedia.com](https://lyfepedia.com), but it's not fully ready yet.

The idea is that you can publish/edit wiki pages, "twitter-like" messages,
pics, videos, do whatsapp-like group messaging etc while maintaining the
privacy of what you write.

------
garyfirestorm
You can organize ebooks with caliber software

------
4ad
Links: pinboard.in

Wiki: I host my own mediawiki instance.

Documents: I host my own upspin: upspin.io backed by ZFS.

Images: Combination of upspin (above) and iCloud Photos.

~~~
Ninjaneered
I also host my own MediaWiki instance, complete with VisualEditor and a mobile
skin. Out of curiosity, why are you not loading documents and images into your
wiki? For images, I don't use it as an image repository, just for images of
things specific to personal knowledge.

~~~
johntash
Would you upload pdfs and things like that to your mediawiki? Bills, invoices,
paychecks, for example.

~~~
Ninjaneered
I do upload pdfs including bills and invoices (not paychecks since I don't
have a use case for that). My wiki is private so it's only available to me.
Plus I have extensions that allow me to render the pdfs nicely and index all
the text in pdfs for easier searching.

~~~
4ad
I have both a public and a private wiki, but most of my PDFs are 5000+ page
instruction set architecture monsters, so I much rather prefer to view them
offline in some decent viewer instead of viewing them in the browser.

I also have passports scans and other sensitive stuff. I don't trust that I
can secure mediawiki well enough to put those in clear on the internet (in
upspin they are encrypted with a local key and the storage is my own server,
mediawiki is in the public cloud).

~~~
johntash
I've never actually heard of upspin before this thread. Their homepage doesn't
give a clear view of what it actually does, but I'm looking over
[https://upspin.io/doc/overview.md](https://upspin.io/doc/overview.md) now

~~~
4ad
This video is pretty good:
[https://www.youtube.com/watch?v=ENLWEfi0Tkg](https://www.youtube.com/watch?v=ENLWEfi0Tkg)

------
lumberjack
I have this problem, except in my case most of my documents are LaTeX. Tried
Gitit but didn't like stick with it because only a tiny bit of MathTeX is
supported and categorizing has to be done manually and you can easily end up
losing pages that you forget to link to others.

------
kqr
This is a crossover between bookmarks and content curation. Also looking for a
good solution.

~~~
jeanlucas
I was thinking of building my own hacker news: own links, curation of content,
tags, and even ranking.

------
humptydumpty001
I bought a $10 starter confluence license and using it since last 6 months. I
am happy with what I have got so far. I have documented lots of things from my
job related to personal life. Although I am also looking it to change to a
different system.

------
malthaus
Evernote (Premium): Saved Articles, PDF, Notes, Project Documents, Scans of my
physical mail

I also export final documents there, eg Mindnode maps, Presentations etc. I
write notes using iA Writer.

iCloud: Documents, Photos, Videos, Music (Production), Programming

Omnifocus: Reminders, Project Tasks

------
davidp670
I have to recommend checking out Bookmark OS. It's like an online desktop and
makes it super easy to keep bookmarks and notes organized
[https://bookmarkos.com](https://bookmarkos.com)

------
nurettin
Perhaps you could be more specific with "not satisfied" and what kind of
documents you need stored. I synch all research documents in mendeley, synch
all links in firefox, synch all structured text using zim-wiki+owncloud.

------
grigoryvp
VSCode and personal wiki extension of my own design,
[https://github.com/grigoryvp/vscode-language-
xi](https://github.com/grigoryvp/vscode-language-xi)

------
wrbishop
I build mindmaps using freemind/freeplane to keep track of my research; I
literally have hundreds of mindmaps from 3Dprinting to ZFS. I also link
mindmaps to other mindmaps (crude "topic mapping").

~~~
mrjazz
Take a look on [https://www.thebrain.com/](https://www.thebrain.com/) It's
better alternative to mindmaps

~~~
gardnr
Can you elaborate on "better"?

------
egypturnash
I would say Evernote but I’m not sure you want to cram movies in there. Works
for everything else though, even searches on its attempts to recognize
handwriting. Won’t give you a transcript of that though.

------
tuke
I'm just impressed to see the name Niklas Luhmann in HN. Well played.

------
Kagerjay
My pinboard,onenote, and evernote is just a dumpster ground.

I organize everything important in dynalist or file folders in dropbox

Whats importanf for me is I can take my notes from any solution and not rely
on any one service

------
gnulinux
emacs org-mode. I make my own wikis for everything I want to remember.

------
dba7dba
I use personally hosted Confluence ($10 a year) to keep snips of notes and
personal project related stuff. And also Google docs.

Yes I'd be interested in having a tool to keep videos organized.

------
pmorici
With the exception of photos delete it all and don't worry about it. A
cluttered digital life is no better than a cluttered physical one.

------
thallukrish
The answers here probably have the clues. But someone has to work hard to
figure out the ONE answer that is minimal and satisfies all needs.

------
kennethfriedman
A combination of Pinboard (for links / images with URLS) and Notion (for
personal notes, thoughts, and lists) works well for me!

~~~
cookiecaper
Same here, pinboard and notion are a great combo. I use NewsBlur for feeds,
but wish pinboard would integrate that.

The biggest disappointment with Notion is that there is no on-prem. I would've
rammed it through at work by now if there were. I've searched high and low for
a comparable open-source alternative, and the best I get is outline:
[https://getoutline.com](https://getoutline.com), but it's got a _long_ way
left to go.

------
h0p3
Tiddlywiki is my favorite so far.

[https://philosopher.life/](https://philosopher.life/)

------
qrbLPHiKpiux
I just use common folder and use the file name as the tags, something familiar
I use to remember what I'm saving.

------
Hjugo
Someone knows a good software to search a large amount of text pdfs locally?
Something like mendeley & zotero?

~~~
jorijori
For MacOS, foxtrot pro is fantastic:
[http://www.ctmdev.com/foxtrot/](http://www.ctmdev.com/foxtrot/)

It gives you just enough context preview around your search term to judge if
the PDF is useful or not. Currently using with 10k+ research papers and very
happy with it.

------
polyterative
Countless trello boards.

Trello is absolutely great for this

~~~
wh-uws
This.

I have several more or less organized ones for specific topics and a catch all
called "personal knowledge base"

with cards for a bunch of random things there is no more specific place for
but not enough for it's own board

------
colund
Focus on learning only a few things and get really good a them. That reduces
the need to organise anything.

------
threatofrain
Has anyone found something that works with Markdown, code highlighting, and
Latex?

~~~
riezebos
There was a show HN this week with something called Madoko which sounded like
exactly that.

------
SolarNet
I use pinboard.in mostly for a raw collection of links.

And I use Gingkoapp to organize my thoughts.

------
simonebrunozzi
This screams like a startup idea you should pursue. I would buy the service.

------
Kagerjay
I copy pasted everything from my blog here. If you want the actual links to
things I use, just use the link since I can't dump it in hackernews without it
being really unreadable.

[http://vincentmtang.com/toolbox/](http://vincentmtang.com/toolbox/)

I only put all the ways I organize my things via software, still a WIP _(work
in progress)_ for hardware things I use _(pens, notebooks, etc)_

\------------------------------------------------------------------

## Notetaking / Productivity

Dynalist – Primary notetaking software

Airtable – Google spreadsheet meets microsoft access. A swiss army knife for
business. Very useful for making a minimal viable prototype

Pinboard – Bookmarking. I consider this 3rd party chrome extension a must-
have, I bind it to my CTRL+D key

Anki – Flashcard software. I use it to speed up learning math, programming
languages, and computer science.

\------------------------------------------------------------------

## Software Development

Chrome – My everyday browser

Slimjet – A simple chromium based browser that I use for webdevelopment
occasionally. Because then I can have a seperate desktop icons

VS Code – My primary IDE

Github – Git + cloud storage for programming projects

Codepen – Isolated frontend environment. Used to A/B test components and
explore cool things. Organize codesnippets here like I do in evernote.

Postman – API testing

LucidChart – Making flowchart diagrams and data models. Not free though, for
that I prefer draw.io

\------------------------------------------------------------------

## Design Tools – (macOS and windows)

Flux – Makes screen red at night so I can sleep easier

AffinityDesigner – I use all adobe products as well, but I prefer this over
illustrator

AffinityPhoto – Same reason, I prefer over adobe photoshop

AdobeIndesign – Catalog & Brochure design

Figma – UX design tool, applesketch is macOS only

AutoCAD – Architectural drafting program, but not free

Fusion360 – 3D modelling program for DIY projects

GooglePhotos – Photo & video backup

Dropbox – Cloud storage for files/folders

\------------------------------------------------------------------

## WINDOWS SPECIFIC

Phraseexpress – Keyboard macro automation. Checkout my 14 phrase-express guide
to see use cases

Autohotkey – When I need an advanced macro that phrase express doesn’t work
on. It has a high learning curve and ugly syntax language though

DirectoryOpus – Native windows file explorer sucks, this is best file explorer
in market. Has a learning curve. Checkout use cases here

7+ Taskbar Tweaker – Smaller icons on my windows desktop screen

Blank Spacer Exes – It gets hard to look at so many icons in windows, so this
gives me eye relief by grouping icons together

Greenshot – The best windows quick image editor out there. First thing I
install on friends/family PC normally, its that good. MacOS version is not
good.

ShareX – ShareX is greenshot image editor that supports gif uploads / custom
image-hosting options. Version 11.6 is the best. MacOS has no equivalent.

\------------------------------------------------------------------

## MacOS specific – alot of programming apps

Iterm2 – A better terminal environment. I binded it to ⌃⌥[spacebar]

Oh-my-zsh – My preferred bash alternative for development

Penc – Gesture based window snapping/resizing

Spectacle – Keyboard based window snapping/resizing

Keyboard Maestro – Keyboard automation for macOS. Similar to phraseexpress but
better

Alfred – Better alternative to macOS’s spotlight for quickly opening files /
apps. I binded it to ⌥[spacebar]

Captured – An inferior version to shareX without gif support, but best thing I
could find.

Dropzone 3 – For quickly handling common tasks like moving folders around

Cheatsheet – Because I’m a new mac user I need to quickly memorize commands

Unarchiver – So I can one click install things on macOS

\------------------------------------------------------------------

## Chrome Extensions – Productivity

Silverbird – For quickly tweeting things on twitter

Tabsnooze – Set reminders to myself to pay bills at end of month, and to do
incremental reading

Pagenotes – When I need to save some specific information on a website. E.G.
terminal commands, regular expressions on regex101.com, etc.

TheGreatSuspender – Suspends unused tabs after 30 minutes. Because chrome is a
memory hog

Tampermonkey – Injecting javascript on a page to enhance notetaking tools. Its
like your own chrome extension. Read about how I use it here

Ankitab – Relatively unknown, but its the opposite of Momentum. I use it to
review flashcards thoughout the day

DynalistCompanionClipper – I use this with my notetaking app dynalist, I dump
all my ideas here as they happen.

Stylus – Add custom CSS to webpages. Since stylish was tracking user data and
got banned

VideoSpeedController – So I can play youtube videos up to 16x speed with
captions to speedread. Its how I consume information very quickly

Imagus – Hover over an image to see its full view. Useful for my notetaking
app dynalist, as well

GoogleDictionary – Select a word to see its definition.

PowerThesaurus – Select a word to see synonym and antonyms. Thesaurus are my
favorite business and coding tool, naming is important

ClipboardHistory2 – SelectAll, CopyAll – you now have a saved copy of whatever
your typing on reddit / hackernews / wordpress

Lastpass – Chrome extension for managing passwords. Cross platform

AmazonAssistant – So I can dump all my favorited wishlist items anywhere to
amazon, export it out later

\------------------------------------------------------------------

## Chrome Extensions – WebDevelopment

Wappylzer / Builtwith – To check what websites are running these days,
accurate 50% of time

Page Ruler – Measure things on screen quickly

Webdeveloper – Powerful swiss army knife for webdevelopment, but I never use
it

ViewportResizer – For responsive webdevelopment, I use this daily. Better than
chrome’s native tools. Checkout bookmark version here

Gifscrubber – I need to pause gifs especially if I want to make a codepen of
it from dribbble, etc. PlaytheGif is a close equivalent.

RefinedGithub – Better looking github made by Sindresorhus

Octotree – Left side file explorer for github repos

\------------------------------------------------------------------

## Android / General

ColorNotes – Every other noteapp I’ve used sucks. I write things here,
copy+paste it to dynalist later

GoogleKeep – When I need to quickly sync an image or text from my phone to PC
and nothing else works

Meetup – Use the specific link, login, and add URL to homepage. Meetup android
app sucks. Shows only the 28 techmeetups I’m subbed to

SolidExplorer – File explorer

Should I answer? – Tells me if phonecaller is a scammer. Not that I pick up
phonecalls I don’t recognize anyways, that’s what voicemails for.

Slack – Team communication

OfficeLens – For making PDF’s from documents using your phone camera.

waze – If I know a driving route is slow, I use waze

okgoogle – My favorite commands are: “Play spotify”, “Next Song”, “Navigate
Home”, “Exit Navigation”, “Find Parking”

Spotify – Music

Musixmatch – Lyrics for spotify

GoogleCalendar – Calendar

\------------------------------------------------------------------

## Content / News

Youtube – I only put an alert on people that post high quality content
infrequently. E.G., SmarterEveryDay, MarkRober, TomScott

ProductHunt – Where I like to see all the new fun tools out there

Hackernews – Lots of smart and interesting people here. Also, where I find out
about news in the tech environment

Reddit/subreddits – Copy this multihub, delete ones that aren’t relevant to
you, add cool ones like r/welding, r/cableporn, r/engineeringPorn

------
tyrex2017
evernote has served me best, with one notebook, and using search

it supports attachments and other stuff

eg when i research “how to do password less ssh”, i will create a note with
that title for future use. it has worked so far.

------
bfuller
For the mundane among us (like myself): trello works pretty good

------
golem14
Could OP outline why he is not satisfied? That might be helpful.

------
amelius
ElasticSearch can automatically index your documents.

------
ddebernardy
EverNote is neat for what you're trying to do.

------
steveeq1
evernote if you want a "well supported" platform

devonthink if you want a personal knowledge database with good searching

------
brudgers
Maybe Apache Tika?

[https://tika.apache.org/](https://tika.apache.org/)

------
stuntkite
Everyone else here has covered explicit file naming and data structure. That's
totally key, but I'm trying to do what you're doing with Keybase[00] right
now. Migrating away from Dropbox, Google Drive/Docs, etc.

A caveat is that files are limited to 250Gb currently and private repos are
100Gb personal and 100gb per team. I'm not sure on team creation limits and I
don't think they are offering paid storage yet. Because of this, I'm also
looking at IPFS[0] for cold storage and using other methods to catalog and
index.

They added encrypted private git[1] registries and dropbox like file
management[2]. That plus the mobile app, teams (groups), chat, and that it's
primarily an identity provider I think it's a killer platform for file
management and organization. It's still kind of immature in the chat and
friends list area (not in a way that makes it unusable, just a little
unpolished), but it's open source[3] and all of the functionality that is
there works flawlessly for me so far.

Their recent edition of "bomb" messages[4], where you can send a private
message with an expiration, I've found to be super useful. It has clients for
all major operating systems and mobile platforms. Source is right there on
github if you'd like to modify/enhance. You can even browse your git repos
from your phone! They haven't added the ability to commit/push to git from
mobile yet.

Now I don't have to worry that if I change a credit card and forget to update
dropbox that I'll loose my stuff. Which happened this year. I didn't loose
anything, but I realized I had not checked in on my over a decade worth of
archives in a bit and hadn't paid them.

I took a look at Keybase a couple months ago and decided to make this move
after hearing the Software Engineering Daily podcast with the found Max
Krohn[5]. It's a pretty good walk through on their goals and current
offerings.

Even with it's lack of commercial SaaS flair, I've actually found using it for
chats, file sharing, git, and organization to be a fairly large productivity
boost. Maybe because it's pretty no nonsense. I've moved half of my work dev
chat to it from slack, and I don't miss Slack at all except for video
chat/screen share. I especially don't miss it's the terrible file management /
organization.

The next phase of my organization, I'm working on connecting it to a personal
web portal where I can share and access control the content, but also curate
my links, writing, and media I create in an HN/reddit style feed and use Kb to
federate to other people's (friends) sites for content to and group that
content by relationship and content category. So basically a smarter web ring
(Everything old is new again!) with RBAC/circle of trust access control and
collaborative versioned content editing.

I'm also exploring other similar tech in tandem or instead of Keybase like
IPFS/Mastodon/WebTorrent. So far, for a lot of what I want though, I think Kb
has it nicely tied up with a bow, though I doubt it'll scale to any
interesting level of traffic without some elbow grease on the ops/storage
side.

[00] [https://keybase.io](https://keybase.io)

[0] [https://ipfs.io/](https://ipfs.io/)

[1] [https://keybase.io/blog/encrypted-git-for-
everyone](https://keybase.io/blog/encrypted-git-for-everyone)

[2] [https://keybase.io/docs/kbfs](https://keybase.io/docs/kbfs)

[3] [https://github.com/keybase](https://github.com/keybase)

[4] [https://keybase.io/blog/keybase-exploding-
messages](https://keybase.io/blog/keybase-exploding-messages)

[5] [https://softwareengineeringdaily.com/2017/10/24/keybase-
with...](https://softwareengineeringdaily.com/2017/10/24/keybase-with-max-
krohn/)

~~~
lousken
While keybase has been reliable so far github readme says "It is approaching
release ready, though currently it is still in alpha. There may still be bugs,
so please keep backups of any important data you store in KBFS. Currently our
pre-built packages are available by invitation only."[0] so make sure to have
a proper backup of your most important files.

[0]
[https://github.com/keybase/kbfs#status](https://github.com/keybase/kbfs#status)

Also one thing which bothers me is online only mode, you can't cache the data
offline and in Europe those speeds are far from great (around 20Mbps for
download).

------
alan_n
I have a large collection (it's 1000+ articles, pdfs, photos, notes, etc) of
mostly "research" for several fiction books I'm writing as a hobby and also
random ideas and DIY inspiration, etc. and although nothing has solved it yet
and my use case is slightly different, I keep seeing people mention Evernote
and similar programs, and I doubt it will work for your use case. I have
tried:

\- Evernote - It's been a while, but at the time it had no highlighting.
Formatting text was also a bit of a pain. Attaching large files was a problem
because of free limits (I didn't care about sync but you need it for
clipping). But the biggest problem was organization. The notebooks were not
enough and it was really slow having everything in one account. Tags did not
really help the organization problem either because what I really needed were
custom fields. Search was slow and it was hard to be precise because there's
no custom fields to be precise about. Additionally for any actual writing,
there's no note history on the free plan.

\- Most of the Evernote alternatives - Although I didn't move my collection to
them, I did try every alternative I could find. If there's any in specific
you're considering, I have probably tried it. The problem with most is they
were either too simple, buggy, old/unmaintained, or just plain Evernote
clones, which already wasn't working. Also many good ones that might have
worked were subscription only because they were cloud solutions with sync.
Then some sounded promising on paper (wiki-like alternatives), but they would
just make me waste time linking and weren't really what I needed. Also many
use only tree organization which just does not work for me. Or they don't
support attachments well. TagSpaces looked really promising for an on-disk
solution but didn't cope well with moved files when I tried it.

\- Scrivener - This is what I ended up moving to. It's not the most intuitive
piece of software. Development, especially for the Windows version is
incredibly slow. It used to not have sync, now it sort of does, but it's not
like Evernote (no clipping). I don't really care for sync though, also I like
that it was a buy once sort of thing. Now it's still pretty slow with search
(made worse by the fact it sort of freezes instead of showing some loader),
but I can divide what used to be my main Evernote notebooks into completely
separate projects. You can also have custom "views" (lists of notes) which
helps reduce a lot of searching. You can also easily search just one specific
field although I do kind of miss Evernote's query language. What really sold
me though was you can have a completely custom organization scheme. You can
add different types of labels/tags, you can change the note icon, you can
color code notes, but best of all you can set custom fields (it's like a
really fancy database really). You can link to other notes. You can have a
custom note card summary for the cards view. There's several different view
types (list, same tree-depth together, note, card/corkboard). It was designed
for writers, so it's easy to write in and keep everything in the same format,
etc (all copied articles are nicely formatted about the same). You can take as
many history snapshots as you like. Best of all you can have huge attachments.
I have videos, pdfs, etc (known image formats and pdfs, you can see inside the
program, videos, and unknown files open externally). You can also set a custom
theme and shortcuts which was also really important for me (Evernote still
doesn't have a dark mode!). Also although the on disk project structure is not
exactly easy to navigate, it exists (i.e. attachments just get copied to a
folder in the project, all notes are plain .rtf files) so you can use
Drive/Dropbox to sync to other computers or edit notes externally.

I'm working on my own custom solution now, but Scrivener has worked the best
for me for notes, articles, and "clippings" if you have a lot of that. I would
not used it though for ebooks, or movies. Ebooks I use Calibre because it also
has the tools to edit them, though you could have ebooks in Scrivener if you
want, and Calibre as the reader. Movies I have only a few, and just use a
folder. For quick notes I use Google Keep or Simplenotes. Then every once in a
while will clean them and extract the good ideas to Scrivener.

------
nvr219
I use evernote.

------
gcb0
question is more about personal media collection. but title and most answers
(so far) are about actual personal knowledge

------
shawn
Pinboard:
[https://pinboard.in/u:shawnpresser](https://pinboard.in/u:shawnpresser)

Also, throw everything into your ~/Downloads folder and use ripgrep.

------
malmsteen
Its useless. Just let your brain do it and ull be able to find the information
back when u needit.

Im not kidding. For any task you dont need more than 5 references. Dont keep
useless volumes of stuff and move on.

~~~
edwintorok
Although one has to be careful that collecting these doesn't turn into a form
of hoarding, it is useful to keep a list of useful/interesting stuff to search
later. Especially for things you wouldn't have found just by using Google
alone : e.g. after reading some HN posts there are a few "gems" that you
haven't heard of before, those are worth bookmarking because they'll be hard
to find later on. For things that are easy to find on Google anyway, or not
relevant in a couple of years time, I agree that it might be beneficial to
just delete it. At some point I realized I had 5000 bookmarks, went through
the titles, thrown out most and was back to a 1000 (still a lot, but I intend
to prune it more).

------
childintime
This isn't a task you should be doing. Simply put, an OS should keep record of
where you got things from, and where it was sent/copied to. That info should
stay with you when moving between computers. It should try to tag the files
accordingly.

Disk indexing isn't enough to answer "where is that file", when you moved it
for example to cold-storage.

Note that by this definition Windows is not an OS :) Linux came closest with
the failed Nepomuk.

------
rcdwealth
You need to master ONE method sufficiently and you may easily become
satisfied.

Simplest method there is is the file system and files. That method is offered
on each computer. Make your own hierarchy.

In general, I suggest 2 folders, one for individuals, one for groups.

In the folder of individuals, you are sorting by name or also by their ID
number their files, files that arrived from individuals.

You may symlink the individual folder to a group file, thus knowing which
individual belongs to which group, or individual may be symlinked to multiple
groups.

Within groups, you can have further hierarchy, those can be groups of
knowledge, for example "All about God" put in one folder with the hierarchy
below, there you can place your research on Elohim, Summerians, or Annunaki,
whatever you may want.

Other folder may be for your personal mind development, you place things
underneath.

Some folders may be related to the humanity, groups, countries, and similar.

Sciences can be organized that way.

All you need is a computer, text editor, and file system, and knowing how to
use the file system.

People placing everything on "Desktop" have yet a long learning period to go
to understand why is there a file system.

