Hacker News new | past | comments | ask | show | jobs | submit login
Contextualise: Manage your knowledge (github.com/brettkromkamp)
209 points by brettkromkamp on Feb 9, 2020 | hide | past | favorite | 50 comments

For personal knowledge, most graph-based ideas were pioneered by Niklas Luhmann in his Zettelkasten system. This system is described with lots of detail in [1] and made him extraordinarily productive.

Essentially, Luhmann had one small card per semantic unit. Cards had alphanumeric IDs. Cards backlinked to other cards using said IDs. He also used a card branching mechanism implemented in IDs as e.g. 123 -> 123a -> 123a1 which he called Folgezettel.

Lastly, he also had cards whose role was mostly to connect topics by serving as a link hub.

It's a really simple system that you can implement using plain text, Org, Markdown or some note taking application like Apple Notes or OneNote plus a few conventions.

After trying many things, for personal use I think nothing beats plain text (or a plain text format). I don't need a server, I can easily sync things, and it's really future proof.

I have also scaled this kind of setup to larger organizations, albeit using a more classical wiki-like approach (read longer articles instead of small semantic unit cards). For example, GitLab has excellent continuous integration. You can use an Emacs or Pandoc inside a Docker to export Org or Markdown files into an HTML.

[1] https://takesmartnotes.com/

Check out https://roamresearch.com, great UX for implementing a similar system using outlining

Thanks for the tip. I’ll check it out.

org-mode is immediately where my mind went when I saw this story on HN.

It's an interesting problem domain that I think benefits from having so many people approach it from different directions, like the person you mention, OP, and all the people that hack on GTD apps, or org-mode customizations, etc etc.

I don't really have any valuable insight in this, just a mishmash of documents with TODOs, notes, plans and knowledge. I wonder how disruptive it would be if we had a highly focused AI that really understood how to organize people?

Org is great because it's plain text, but at the same time offers outlining, timestamps, tables, links and lots of other features. Following Emacs tradition, it doesn't impose any workflow on you. It just gives you some primitives to build the system you want.

A great feature of Org are programmatic views of your data, possibly from many files at the same time. Some are already implemented, but still customizable, like Org Agenda. I use Org Agenda to have a Kanban-like view of my projects. I can simply see all WIP tasks, approaching deadlines and events in a little plain text window.

That's, I think, the minimum viable productivity system. You have a list of projects with tasks, some deadlines, some events and some inbox. You select a few tasks to do every day, like Ivy Lee suggested, and you do them. You want to see these plus deadlines and scheduled events which can be easily achieved with Org Agenda.

A GTD-like system would tag tasks with contexts, and offer some context-specific views, but I find that's too much work. Another pitfall of GTD, in my opinion, is that it doesn't encourage WIP limits which eventually makes it overwhelming.

I credit GTD with popularizing the need for an inbox, the task-centric view to time management, and the distinction between tasks, tasks with deadlines and events. But I think GTD leads to too much planning waste, projects broken into tasks that get constantly outdated.

I have been thinking about incorporating initial AI/machine learning support in Contextualise including auto-tagging and relationship (associations) recommendations.

It should be possible to implement a form of PageRank on a ZK system, non?

Absolutely, ZK is essentially hypertext. Text in cards plus regular links and branched links to other cards (Folgezettel).

Thanks for your feedback. Genuinely appreciated. I’ll take a look takesmartnotes.com.

Very interesting. I always thought of knowledge as a graph (as opposed to a document or a tree), and browsing the graph of knowledge will be really useful for learners.

I think the obstacle is more on the authoring side (how to build/edit graphs as easily as plain documents). There is also standard taxonomy challenges, about choosing the "types" of the nodes and the associations (values of the `instance_of` of the `Association`s in TopicDB). If the platform restricts things to a fixed taxonomy, you lose expressiveness, but on the other hand if you let users create any-which category they want things can grow quickly out of hand).

A related project in that space is https://metacademy.org/ whose code is here https://github.com/metacademy/metacademy-application

I was working on an app similar to contextualize a while back, but had to pause because of other projects, but I hope to come back to it soon. I hand't gotten very far on it, so maybe I can contribute to contextualize. My main focus is on building tools for serialization/deserialization of the graph data to a text-based format that can be stored in git repos so that multiple authors can collaborate on building out a knowledge base using pull requests, see data fixtures here https://github.com/minireference/structure-api/tree/master/d... )

Really like the idea of a version-controlled text-based graph format. I’ll definitely take a look at the GitHub repo. Thanks for your feedback.

Any contribution to Contextualise would be greatly appreciated ;)

If you are interested in this kind of problems, you might want to learn about MediaWiki (the software behind wikipedia) and the SemanticMediaWiki extension. That allows for extracting and representing formal knowledge about wiki pages. It's quite powerful and allows for programming quite sophisticated databases ontop of MediaWiki (kind of in-system effect). Also WikiData is a project/MediaWiki extension worth exploring.

Thanks for the tip. I'll take a look. What I am trying to accomplish with Contextualise is a straight-forward UI on top of a powerful, but still conceptually simple data model (topic maps).

Pretty cool! I've long hoped for an open source tool that's competitive to Analyst's Notebook [1]. It's a tool that's a workhorse in all kinds of knowledge mapping domains. In a pinch you can force press yED to kind of sorta be a substitute, but AFAIK, there's no real user facing replacement.

1 - https://www.youtube.com/watch?v=Siq4LVy-tTg

I’ll take a look at the YouTube video you linked to. I hope to make Contextualise publicly available this week. Thanks for your feedback. It’s appreciated :)

Brett, thanks for this!

Literally just last night I was lamenting that there was no open-source software similar to Roam (roamresearch.com) and how I didn't even know where to start if I were to make it myself.

This is a great option, I can't wait to try it out!

You’re welcome :) It’s my intention to make Contextualise publicly available (free of charge) within the next week or two. I’ll announce it on the project’s GitHub page.

Really looking forward to it!

I use OWL ontologies for some projects and am about to build a very ambitious one for an lightweight ERP (basic idea is to let python users import the ontology into their excel environments to normalize their data across the org). Given tools like json-ld and Protege, what did those not do that prompted creating this? Really interested in alternatives.

Well, I wasn't aware of, at least, some of those tools. But, also... Contextualise really fits my very specific needs and way of modeling knowledge/information. So, I was scratching a very specific itch. But, also I hope that with Contextualise I am putting something new on the table even if that is just a somewhat simple UI on top of a straight-forward but understandable data model.

Thanks for your feedback. Genuinely appreciated.

Graphs aren't enough; we need path equivalence and multiple edges. We need categories.

Could you elaborate a bit on this. Sounds like an interesting take.

When I'm mapping out a narrative arc, sometimes I will want to indicate that some action is monotone or idempotent, or that it is isomorphic or reversible from some other state. Those properties end up being path equivalences.

Similarly, sometimes there can be multiple social relations between characters within a narrative setting; somebody might be both a friend and a parent, for example. Properly modelling this requires allowing multiple distinct edges between vertices.

And, finally, technically any hypergraph with path equivalence is a category.

When I'm doing narrative analysis on a whiteboard, the diagrams that I draw are category-theoretic, not graph-theoretic, and I'd like to be able to bring that richness into my narrative-mapping tool without having to flatten everything down into annotated graphs.

I need to think a bit more about what you are saying. This is a very interesting point of view. Can I come back to you on this?

Don't worry about me specifically. https://arxiv.org/abs/1706.00526 is more food for thought, if you need it.

Thanks. Any feedback, pointers to additional resources and so forth... it's all greatly appreciated.

With regards to multiple distinct edges between vertices: TopicDB (https://github.com/brettkromkamp/topic-db) already allows this. However, in my drive to simplify from a user-perspective, there is no corresponding GUI. A possibility would obviously be to provide both the current simplified GUI (by default) and an advanced GUI (allowing for multiple edges between vertices). I'm actively considering this as multiple people have mentioned/requested it.

Knowledge management as a career typically involves a deep focus on the relationship between people, process, and organization as much as the epistemological hierarchy of data/information/knowledge.

I see this tool as conceptually useful, but like most knowledge graphs I still don't see the inherent process behind the transformation and relation of various nodes.

Is there a layer I'm missing from the overview?

No, I don't think you have missed anything from the overview. Yes, you're right... the process to construct and maintain the knowledge graph is not part of the application. In that respect, I have been considering several related features to help the user with that process: 1) topic map templates that allow for very quick creation of a predefined taxonomy and 2) machine-learning for auto-tagging and recommendations for associations.

Although the above features do not make for the complete process you are referring to, at least they assist users as part of that process.

Food for thought.

Interesting ideas, however it still sounds like you're tackling knowledge as if it's a fixed point in time. That's good to do when you're starting to build your knowledge base, but there's a stage of understanding beyond that which I'm trying to define and quantify.

As a knowledge manager, one of my biggest issues is the temporal nature of knowledge and conveying the cyclical nature of inputs and outputs to people. The "clean" process is often the exception in the real world, and short circuiting often occurs throughout the knowledge development process.

I don't think it really becomes apparent until you've been looking at the same knowledge for a while and start to see there's a pattern above the pattern. Changes in one part of the graph seem to be connected to changes in another part of the graph.

I'm searching for a way to visualize this extra dimension of data. I don't know if graph edges are complex enough to show the dimensionality of the change.

Perhaps a simple time-lapse overview of a graph would provide what I'm looking for. Or maybe there's a temporal relational layer that sits above topics and resources. I'm not sure.

edit: Hmm. A hypergraph seems like a relevant analogy. https://en.wikipedia.org/wiki/Hypergraph

This is almost certainly lower-level than what you are looking for, but I think you are describing a bitemporal graph data model: https://opencrux.com

Thank you! The overview definitely sounds relevant. I will look into this deeper.

edit: Yep. The integrated bitemporality definitely lays the foundation that would be necessary to implement knowledge hypergraphs.

In a previous implementation of Contextualise I included the ability to timestamp topics allowing for a straight-forward temporal navigation of topics using a timeline component. It was somewhat naive, but it did allow you to navigate a knowledge graph from a time perspective. This time around I probably wouldn't add the timestamp directly to the topic but instead tackle this by means of temporal relationships using, obviously, associations. The outcome would be (regardless of how it is actually modeled in the topic map) is that a timeline component could be appropriately populated allowing for temporal navigation of a knowledge graph.

Is there any easy way to sync the DB across machines? TopicDB is built on postgres. I have no idea which file to back up there. My current notetaking tool uses sqlite to store notes, that I know how to backup as it's one single file.

That's a good point. I will, at some point, make it possible for a user to download their topic maps. But, yes... you have a point.

Oh wow! I didn't think anyone was using Topic Maps anymore, much less with Python! I'm definitely going to have to give this a closer look. Thank you for sharing!

You’re welcome. Been using topic maps since, at least, 2006. After all these years still my go-to data model for both knowledge and content management.

I fell in love with them when I took a class on them in Library School, back in 2007. Like many others in the Topic Maps space, I now work mostly with RDF and OWL.

Yes, topic maps did originate from the library world, if I remember correctly :)

When installing Contextualise, please install TopicDB (the topic maps engine on top of which Contextualise is built) directly from GitHub. Instructions to install Contextualise and TopicDB are provided, here: https://github.com/brettkromkamp/contextualise#install-the-d...

You should build it into the Dockerfile so it's easy to try out.

Unfortunately my Docker know-how is severely lacking. I’m hoping that someone will help out with Docker support. In the meantime, I will try to get up to speed with Docker to be able to fix the current Docker setup.

I tried my best to use Google Keep for a couple of years & finally gave up. I exported all my notes (maybe a thousand cards, each with multiple links, text notes & image snips) into a Jekyll-based static site on GitHub pages.

It's still a bit clunky but allows me more freedom to organize as I see fit. Plus, being a public page forces me to consider readability.

Google should invest in making keep into a auto flashcard learning system. It would be awesome if they integrate this with annotations from Google Books like Google Docs.

Very cool. I remember seeing a similar tool called Yewno that Stanford had access to that had a semantic web of topics linked by a graph, and wanting to find or make an open source alternative. I can see a lot of possible uses for this in the future.

Excellent. I will make Contextualise available at some point in the next week or two. I'll let you know when that happens, if you want.

Looks like Tiddly+graph plugin but with extra steps.

I’ll obviously Google it, but what is Tiddly? :)

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact