I have bad memory and hence try to write down everything I can. But often throughout a single day/week, I do research on a topic and have a bunch of tabs open that I intend to come back to. Or I read an article that several days later that I cannot recall where I read it at (HN, Twitter, etc.) This usually leads to a frantic search until I can find what I’m looking for as well as having a ton of tabs open.
Manually grouping topics together is too hard. What would be great is a tool that knows where I’ve been, discards bad information (google search result, followed by near immediate close) and some sort of an attempt at topic autoclassficiation (SAP, storage, backup etc.) that gives me the confidence to close tabs knowing that I can get back to a particular topic at a later date.
1. make is as easy to 'bookmark' stuff as possible -- with a single hotkey
2. make it as easy to search over bookmarks as possible -- also ideally with a single hotkey or as quick as you could do google search
My way of achieving this is using org-mode files for 'bookmarks'  and using emacs/ripgrep to search over it . Additional benefit of org-mode is that it's very easy to add notes, priorities, refile bookmarks, so the most interesting stuff propagates through my notes, and I don't feel bad about missing out on information that I don't have time to process because I can always quickly find it when I need.
By the way, I think you should also take a look at Archive Box, which is very much in this direction: https://archivebox.io/
For Promnesia, the goals of Roam and Promnesia are pretty different at the moment (although Roam data can be used with Promnesia, as I mentioned). In addition, I can't personally bet on a closed source tool.
For Grasp, simply because Roam wasn't known (or possibly didn't even exist!) when I wrote it. But even now that I tried Roam now, I don't think I can go back from using plaintext files, it's just so much snappier and more hackable.
Thanks, I used archivebox! Still need to set up a proper automatic archiving, and integration Promnesia with personal web archives is also in my plans!
attachment is suffering
Is author aware of my history based fully interactive offline archiver? https://github.com/dosyago/22120
Haven't seen your tool in particular, thanks for the link, I'll check it out. I only used
https://github.com/pirate/ArchiveBox before, but haven't set up an automatic archival pipeline (yet)!
Also, integrating with local web archives is on my Promnesia todolist! I expect them to be very useful for indirect history retrieval, e.g. "I haven't visited that page, but it's within one link". Having local web archives makes it possible to implement such functionality in efficient way.
Perhaps this page https://beepb00p.xyz/blog-graph.html would be a good start if you want to explore
Although you're right with enough work someone could engineer a way to achieve the same, even without DevTools protocol. I picked the protocol because it made it easy to achieve.
I think in future FF plans to support DevTools, or the standardised version which I think is called WebDriver protocol or something.
You should know about Timeliner; Matt Holt's attempt at solving some of the data sourcing / data silo problems. https://github.com/mholt/timeliner
He also points out Perkeep: https://perkeep.org/
Anyway. Data liberation is a huge driver for me. Making it a primary goal of my next app (primarily for bookkeeping/financial data, but I want to allow users to connect to the third party services I integrate with, eg. Uber, Amazon, etc, and be able to download their own data / play with it via an API).
Feel free to email me / tweet me (@Adys) if you ever want to chat.
Oh, nice, I bookmarked Timeliner recently, but haven't tried yet. Look promising, I expect this to integrate well in Promnesia, and vice versa, my helper HPI package to integrate easily in Timeliner.
For Perkeep -- I tried it briefly, but haven't exactly understood the problem they are solving. I was planning to try again and writeup my experience with it to spark a discussion!
The solution outlined here leaves me with a couple of questions, though.
1. Since there's a local app acting as a service, it's not clear to me how this would run on a mobile device.
2. Once it is running on my mobile device (and home computer, and work computer, and chromebook, and various other machines I use), how do I aggregate all of the data? I'd like to be doing work-related research at home in the evening, and be able to see the fruits of it from the office.
I suspect that the answer to this is the same thing: that rather than a locally-running server, I could put something on my home server or on a cloud-based server, and direct all my various devices to communicate with that rather than localhost?
Yep, you could use a VPS or something and host it behind a reverse proxy, that's what I've been doing so far.
Also for mobile specifically, on Android it works under Termux (haven't personally tried yet, but can't see why not, and the person in the issue I linked claims it works).
For data aggregation: it depends on the data source, but the easiest seems to make sure your data ends on a single computer, index it there, and after than you get an sqlite database which you can simply sync with Dropbox/Syncthing or anything else you prefer.
I've recently started trying Shiori to manage my "bookmarks" and preserving offline copies locally without relying on The Internet Archive, however it still doesn't really help with private content (i.e. Pages only accessible as an authenticated and authorized user) so it'd be great if Promnesia caters for that. Plus the whole data silo thing...
I was a little surprised to see no mention of the "tree style tabs" extension which can help with "where did I get to this link from?" style questions
My personal knowledge management project, Gthnk (gthnk.com), would appear to plug in easily as a Source - without any special plugin necessary. I really like what you've made!
Another thing: Have you considered adding annotation capability directly into the extension? This is something I've thought about creating an extension for, since I don't use anything like Instapaper.
If you're looking for something similar to Instapaper, but local only, your best bet is probably Worldbrain Memex. And as I mentioned in the post, I was thinking of potentially integrating with them tighter anyway.
In theory, I could make it defensive too and allow using without the local backend (only with local browser history), but not sure if there is much value in this.
There is update! Maybe I will look into Promnesia and StorexHub integration next weekend. Thank you for your effort with Promnesia!
Thanks also for using the Yak Shaving - for one, I got curious what was first, the term or the Ren & Stimpy episode illustrating the term and second, I found a description of most of my modus operandi.
# TODO fuck. why doesn't that work???
Perkeep is another project that might be interesting to analyze in this context. https://perkeep.org/
Looking forward to trying it out.
And I'm disappointed :( Given the title, I was hoping for a way to fix Google's broken web history. So it goes.
What do you mean by slurping here? Security-wise, a random website shouldn't be able to query a localhost because of CORS policies.
Either way, I hope I've been fairly reasonable about security so far, but I've mostly been concentrating on the 'plugging in the data' bit, so it's possible I've overlooked something (also I'm not a security specialist!). There is an open issue in case people have any specific concerns or spot something, happy to receive feedback! https://github.com/karlicoss/promnesia/issues/14
On the other hand, some of the other stuff may not be fully baked:
An API call can't be considered authorized just because it came from localhost :)
Safari for iOS, too, has a feature where you can temporarily go "Back" from new tabs. But like you said it breaks if you do anything else. Kind of a hack for mobile convenience rather than a true feature.