Hacker News new | past | comments | ask | show | jobs | submit login

Hey author here. Happy to answer your questions!

Big fan of yours. I think this is the second time I’ve come across your site on HN. I’ve been getting deep into developing my own Second Brain lately. I glanced briefly at your Exobrain Repo and it seems you’re using a custom system. Have you tried out using https://obsidian.md or the whole Wikilink/Zeitelkasten Personal Knowledge Management methodology? Curious to hear your experiences.

Yep, exobrain is using org-mode export + some tweaks. I've settled on emacs + org-mode a few years ago (perhaps on the fifth attempt it finally clicked). Before it was gitit/Zim/sublime text/random scripts, whatnot. All the shiny apps like Obsidian/Roam/etc have appeared over the last couple of years (and it's awesome!). But I'm too hooked onto emacs now, maybe the only thing I should use more is org-roam [0]. For me the most useful org-mode features perhaps are tags, agenda, org-capture and org-refile. Search is very important, I have a whole post about it [1].

I started a draft describing my 'process' [2], but it's pretty incomplete.. there are some bits scattered across exobrain too.

Also lately I started playing with Logseq [3] to get a more interactive representation of exobrain [4] (warning, it's pretty heavy, needs some optimization and I might have messed up the physics). Logseq could be a great gateway for people who want to get into org-mode but not ready to go 100% Emacs, highly recommend to try it it!

[0] https://github.com/org-roam/org-roam

[1] https://beepb00p.xyz/pkm_processing.html

[2] https://beepb00p.xyz/pkm-search.html

[3] https://github.com/logseq/logseq#logseq

[4] https://beepb00p.xyz/logseq/#/graph

Looking forward to diving into more detail on this. Thank you for sharing your work and appreciate your engagement in here!

Kudos! I find this work fascinating.

There are moments that I'm tempted to go down a similar organizational road because I'm curious about the possible lessons from the data aggregation, but I usually pass because the payoff didn't seem large enough.

So that makes me want to ask: Have you learned much from the data? or is it more about organization/keeping control of life?

Perhaps mostly about organization, yeah.

The "quantified self" bit is what you could call learning, but ironically so far I've mostly failed to find any significant correlations in my personal data (which is also an interesting finding, in a way :) ). However I did learn a lot about nutrition/exercise/sleep while tinkering with data, so it was totally worth it so far.

I write about it a bit more here: https://beepb00p.xyz/tags.html#quantifiedself

Thankyou so much for sharing your work. Finding this post was like finding something I didn't know I was looking for.

Have you considered monetizing this in some way? As a person of limited technical skills, I would definitely pay for a set of tools that would help me aggregate my data.

Monetizing.. not really. Many reasons:

- I don't really see a good way (not that I'm entrepreneurial in this aspect though), maybe except for providing some infrastructure etc

- dealing with others' data is a completely different kind of headache with legal and security responsibilities

- I don't really have a product or anything -- more of a vision which I want to communicate, a bunch of patterns, techniques and repositories. Maybe know some esoteric gotchas.

- It's a very niche thing to think about and 'want' in the first place (although I am trying to change it through my communication :) ). Most 'regular' people don't do backups, let alone this!

But it would be very cool if someone comes up with a business model to provide such service.

Can I ask - how did you find working with graphviz. Was this your first time working with it, how long did it take you to figure out how to place things and draw?

I am working with drawing graphs at the moment and evaluating what library to use, its coincidentally nice to see what you built but it also seems like it was thoughtfully laid out (less edge crossings etc). I am trying to do something more dynamic so it may not be as applicable for me.

I am looking at your code as well, thanks for providing that

Not the first time, but possibly the biggest thing I've drawn in it...

There definitely are some weird things when you try to plot complicated things, fighting with weird placement, clusters etc. But not sure if it's me or Graphviz to blame for this. But I don't really know a better tool. If I knew how the diagram would look in hindsight I might have drawn in manually in inkscape or something, but when I started I didn't know what I would end up with, so needed to be an automatic tool :)

To minimize the manual work, I ended up with a mix of DSL in python and raw graphviz commands: https://github.com/karlicoss/myinfra/blob/fc6345c31c4e49b534...

Depending on the things you want to represent a better fit might be force layout, for example something like https://observablehq.com/@morvasaaty/d3-force-notes

Working with graphs, I realized its hard to generate dynamic ones and you inevitably end up with domain specific layouts. I guess thats the nature of graph drawing algorithms, its usually specific to the problem.

Thanks for the code, taking at look at that.

Probably too messy to understand what's exactly happening -- but my main takeaway is that if you implement your own DSL (for Graphviz at least), implement in such a way that you can freely mix DSL and raw bits. That way it's very easy to experiment or tweak minor bits without rewriting half of the code.

If you need to be able to diff the code for graphs, graphviz (dot language) is great.

If you need auto-layout, it's hard to do much better than graphviz. The creators of graphviz have some good papers on what it takes to do that right. They'll discourage you from wanting to re-invent that particular wheel, if you read them. Tricky bit here is, depending on your exact graph and constraints, it's not exactly fully solvable in the general case, if you want to have no overlap or no crossing lines under any circumstances whatsoever. Still, you'll struggle to do better, and there aren't actually a ton of implementations out there other than Graphviz that are anywhere near as good (at least, as of a couple years ago).

Templating or building programmatically? Great, it's just text. Generating on demand? Decent, it's fairly fast.

If you need pretty, and I mean really pretty, and especially pretty and interactive... well it's less enjoyable for that, but then nothing that is enjoyable for that purpose is much good at the other points above.

Thanks for that, will try out graphviz. I was doing some custom implementation for minimizing edge crossings and it seems to work fine for me but thats mostly for 2-layered graphs. If I am to expand to multi-layer graphs, its going to be fairly more complicated to figure out how to do that and I dont know if it will work well.

Graphviz seems to be great for getting you a mostly-useful auto layout majority of the time.

I have a requirement to be able to mix fixed-positioned nodes along with dynamically changing nodes, I havent used graphviz to know if it could do that. A library like cytoscape works ok or maybe d3 (I am still evaluating these)

Honestly coming into this, I thought graph drawing would be fairly easy and I could look up online for examples - but it seems to be quite opposite, its really complicated to get graphs that are both dynamically generated with the nodes positioned at the same place in case there are similar nodes

I believe my app (launching soon) is close to solving your "what do I want" but currently wrapping up the API. Are there some other communities, links to other blogs or people that are doing what you are doing? (Tracking this much data, etc.)

For data tracking/analysis, you're looking for "quantified self" https://github.com/woop/awesome-quantified-self#awesome-quan...

For exporting and owning the data the keyword is "data liberation"

Otherwise, the keywords are perhaps "personal knowledge management" and "memex". Have some links here, although they all have different components in scope https://beepb00p.xyz/exobrain/memex.html#cmmnts

I usually maintain related links at the end of articles, so perhaps you can find something there, for example https://beepb00p.xyz/sad-infra.html#links

I just noticed that the awesome-quantified-self on github has been abandoned with no updates for 3-5 years now. Looking for alternatives!

Was it? Seems that it had some updates (maybe not as active as it used to https://github.com/woop/awesome-quantified-self/commits/mast... )

Either way don't think I've seen anything better, so the best way would be to suggest them updates or help maintaining :)

Which of these data-streams do you most often find yourself looking at & using? Are there any that you have been surprised to find yourself using more than you expected?

The most useful are perhaps the one's I'm using for search [0] -- chat logs, web annotations, book highlights, reddit/hackernews saves, tweets, etc. It's literally my external memory, I'm not exaggerating.

Everything in Promnesia [1] is super useful, makes my internet lurking much ... efficient? This word has somewhat bad rap, but it is what it is -- doesn't mean I don't have fun, just makes it easier to keep track and discover new things.

Not sure about surprising data sources -- I'm trying to integrate them all as far as possible and make sure they work for me passively, so I don't have to think about specifics :)

[0] https://beepb00p.xyz/pkm-search.html

[1] https://beepb00p.xyz/promnesia.html

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact