The "Queue Management for Inbound Content" section is exactly what I really want as well, so much that I've just started building it instead of waiting and hoping.
For me, this is the lynchpin:
> "all content I think my future self would appreciate me consuming." (emphasis mine)
Current content feeds are optimized for engagement (ie. advertisement load) and thus won't conceive of a "future self", only what your current self will look at and click on right now.
I think that content feeds need to incorporate goal-orientation and move away from right-here-right-now orientation. For those wanting to do anything difficult they need to optimize their information diet over a very long time-scale, like years, so content-feed tools should be aware of human-scale timelines (eg. high school, college, career, parenting).
Humans thrive on learning and growth but so many platforms choose to see their users as merely inputs to an ad-delivery optimization system.
> I have little visibility into required time investment and foundational context until I’ve opened it and started thinking about it.
This is another thing that really annoys me about our current media ecosystem, and is really also a symptom of not properly conceiving of a person's personal development over time and a person's changing needs over time.
---
To look at this blog post from 1000 feet up, I'd say that Jonathan is unfortunately deprived of these tools because our media software ecosystem is madly building things for users who want to look at things and not think, as such platforms are heavily consumeristic and thus fantastic for advertising revenue and monetization generally (eg. Instagram, Snapchat, TikTok, Pinterest, Canva).
> Current content feeds are optimized for engagement (ie. advertisement load) and thus won't conceive of a "future self", only what your current self will look at and click on right now.
Spot on analysis. When I open Reddit it's indeed as if I don't have to think, I can just... turn off. Sure, there are times when that's the right experience be it after a long day or something - but once you're on an infinitely scrolling app it's hard to leave. Our brains just haven't caught up yet. As soon as I've left the app I struggle to list the 5 most interesting things I learned or saw.
> For those wanting to do anything difficult they need to optimize their information diet over a very long time-scale, like years, so content-feed tools should be aware of human-scale timelines (eg. high school, college, career, parenting).
One of the things I like about school is in some way it takes care of that for you, for a while anyway: this book follows that one, don't waste your time understanding this until you've poked at that, etc. But I don't think we actually learn how to develop (or maintain) those long-term learning plans; especially as they are rarely linear but much more akin to an RPG skill tree.
I have been emailing links to myself tagged with a one word keyword for the last decade. They focus on things that I or my (unconceived, unborn) son or daughter would like to read, learn or understand “later”. I have about 10k sitting in my inbox (including this one) that I plan to convert into a critical path and consumption order once I take a sabbatical.
Create a rough dependency graph and perform a topological sort, potentially skipping, down weighting, or skimming nodes that are onerous to consume but of dubious value (e.g. a link that’s a review about a long textbook that isn’t fundamental to a space).
I think even if you have a content feed incorporating goal-orientation you'll still struggle with the learning part. I think this article is very valuable in explaining why: https://andymatuschak.org/books/.
Yep, I'm familiar with that post and largely agree. I personally love books as a learning tool but I've personally not learnt nearly as much about programming from reading compared with 'doing'.
The content-feed I'm aiming towards certainly includes books because they're essential to really understanding certain things, but won't be just concerned with reading material.
The key to supporting goal-orientation is some notion of scheduling, requisites, and knowledge-evaluation. This system should not at all be limited to books. My early prototype will include flash-cards, articles, programming koans, and Leetcode problems.
For programming, you might find this video interesting:
https://www.youtube.com/watch?v=Mgrdg1uwDeA
Larry Sanger (co-founder of wikipedia) found SuperMemo/SRS sort of clunky for use with program so he made a program which lets you review more by doing rather than just recalling.
I think rest of what you're after are in large part SuperMemo which I've mentioned elsewhere in this thread. It basically allows the same goal orientation feature through its priority/scheduling system though it lacks managing requisites (aside from just scheduling something to show up later than something else).
I'm not a huge fan of books for 2 reasons:
-they're not information dense (you can get way more information-per-paragraph from a wikipedia article rather than a book)
-they're not very context independent, as in, if I want to understand some specific part of a book I'll likely have to read the entire thing while with internet articles I can read one part and ignore the rest. The main reason this is important is that I can read the parts I find valuable and ignore less valued things. Reading an entire book for one concept that takes 15 chapters to get to isn't great.
With SuperMemo, if there's a book I find interesting I'll generally try to find any alternative wikipedia article first or otherwise some other web article on the same idea. It's just takes so much more time to read and retain a book than to retain a smaller, denser article.
SuperMemo is a large part of it yeah. I’ve thought about how I will integrate that functionality in as the evidence is there that it works.
In certain domains I really disagree that books aren’t information dense. If you want to properly understand say philosophy, politics, literature it is essential to read primary sources which are often books. Reasons and Persons does not waste a page, for example.
Agree that a lot of books are padded as hell, but to say this is a problem generally with books is myopic. You might just be reading Self-Help and Management books which are notoriously padded out.
> The "Queue Management for Inbound Content" section is exactly what I really want as well, so much that I've just started building it instead of waiting and hoping.
Personally I just use mails for that.
More specifically, I have a VPS with a dovecot mail server, and I wrote a set of tools to write almost anything to any of my maildir folder (shameless self advertising: https://github.com/sloonz/ua). Most of it is RSS feeds, but I also receive a daily summary of operations in my bank accounts (with boobank from weboob), new posts from a forum I’m subscribed to (with boobmsg's tapatalk plugin from weboob), and new posts from reddit
> I think that content feeds need to incorporate goal-orientation and move away from right-here-right-now orientation
I think part of the issue with the current inbound content queues (e.g. Instapaper, Pocket) is that they treat all content the same.
There is a difference between content I need immediately from content my future self will find interesting in the future.
there should be a way to rank/aggregate content according to this.
> "all content I think my future self would appreciate me consuming."
I think The Syllabus [2] is kinda like this for a certain type of content (mostly academic). It aims to find stuff that doesn't get attention in the regular surveillance capitalistic hellscape that is the internet. It's run by Evgeny Morozov, someone who's thinking I admire.
Currently, my online reading is mainly based on hn (and multiple sources that often get posted here [0]), The Browser [1], The Syllabus [2] and people I follow on Mastodon and Twitter. Basically in absence of how to find content for my future self, I look for content that is being recommended by people I admire.
[0]: laphamsquarterly.org, slatestarcodex.com, ribbonfarm.com, etc.
What I see missing is an ecosystem of interoperable tools.
It would be great if I could have a location/format in which all my digital knowledge is stored. And then be able to use different tools which support multiple processes/flows:
* a flow for discovering content
* a flow for archiving the highlights I take on the web
* a flow for writing notes
it would be great if I could change any tool while keeping the flow intact.
The only thing that somehow mirrors this is Evernote + Readwise + IFTT. There are tons of integrations that allow you to populate Evernote from tweets, Highly, Kindle highlights, etc. What I would love is if I could easily swap Evernote with any other editor (e.g. Bear) without changing much.
I wouldn't mind paying for these tools I would just like this personal knowledge software to work in a way in which you can natively add blocks/functionality to it. Most of them have huger vendor lock in.
I like your abstraction of the different flows. Definitely agree the key is not a one-size-fits-all approach but instead much improved interoperability between the user's preferred tools.
It can also be a case of wanting the means to that end. Some have certain minimal needs either professionally or simply personally and the more efficient that process, the more time there is for everything else.
I have been thinking about these questions as well. The gist is, it seems we need tools to manage data created from all the applications that we use. Today, there's so much data, that the most important tools are those that separate signal from noise.
Some things that I have considered:
- I like to read long articles on my e-paper reader, there are several services to do this.
- I need all quotings from articles, books and my personal notes to be accessible in one place, indexed and searchable.
- My whole knowledge base should be easily shared with other people.
- I want to find others with similar interests to mine.
Some interesting technologies that have the potential of bringing this to reality is the Beaker browser and the Dat format for sharing. Also, I really like World Brain's memex for web annotation, searching and sharing.
It seems we are getting closer, but we need to connect the different tools into a widely used workflow.
> My whole knowledge base should be easily shared with other people. - I want to find others with similar interests to mine.
Yes! And I think this paves the way for an interesting collaborative knowledgebase as well.
Agreed on a missing workflow to bind everything together. Do you keep your annotations + notes in Memex? If so, do you add an external annotations (say, from ebooks) into it?
It’s interesting that your “tool” revolves around you... in other words, Revolves around a single user. What I see happening in the content ecosystem are two things: 1. Easier to make content than ever before and 2. Variety of content is collected by topic, not by user.
1. Social Media and all web content tools are always making it easier to make stuff. This might not be very good, but it’s good. Yes we get more noise than signal, but in time there should be tools to cull that noise. The numbers are astounding, 500 new podcasts are started, a day. Millions of YT videos uploaded a minute. Medium made it fast to write.
2. Platforms, even medium and social media, allow you to collect around topics. Yes hashtags, but also publishers/pages. I can search IG through #technology or follow @technology. Same name different stream of content.
If you’d like a mix of audio, video and writing, there needs to be a publisher that creates on a single topic. Digital publishers a plenty. PopSugar, Bustle, Jalopnik, The Athletic, etc. These will continuously get better and worse as their revenue models mature. There will be a hybrid of ads and fees. But the “best” ones, the higher signal to noise will be ones you pay for. $50 a year for The Athletic is a steal right now if you want it.
Trade pubs, The Information, Business of Fashion, and the like are multi-medium publishers.
Spending $200 a year between 4 trade pubs that help me be a better person, saves a couple days total of ads for me and supports 20 journalists do the best work they can. Sounds like a good deal to me.
From 2014 to around 2018 I was part of a team that was heavily invested into bringing these concepts to reality. What we accomplished was amazing and I still use some of them tools day-to-day, unfortunately we had to stop.
What I wanted to share is that it was extremely hard (and ultimately impossible for us) to accrue a critical mass of people that is willing to use these things AND pay for them. We reached a few hundred paying customers; but we were 6 guys working on this full-time, so that means salaries and overall we were losing money like nothing else.
That aside, I would do it again if I could, particularly now that the "standard tools" of the internet have become useless with time. Example, Google never gives me what I want anymore and they're removing the URLs now from the results (why?).
If someone reads this and want to kickstart some kind of "internet renaissance" with me, you can reach me here: alex@<HN-username>.com
I feel as if this is the kind of thing Google might still be able to get right; "to organize the world's information and make it universally accessible and useful" is still the company's mission, after all, although it's been diluted by ten years in the wilderness of developing products in what seems like all directions.
If Google had not strayed Keep would be this. Assistant would be this. Honestly Assistant and Keep would be one and the same thing, perhaps: a single Assistant that is about remembering what you need to remember, finding what you need to find, helping you do what you need to do. This parallel universe Assistant has an open API that nerds like you and me can really use to set up complex workflows. We then are able to share our workflows with each other, and eventually perhaps even build products (apps) on this platform for the less technical users. Split the revenue in an Assistant Store. It could smoothly diversify into Mechanical Turk territory, but with more and better automation.
While Google is asleep, startups can revolt.
Disclaimer: I work for Google but the opinions expressed here are personal and not those of my employer.
The issue with Google is, that Google doesn’t care about privacy and randomly changes its behavior, like recently closing thousands of accounts for using the live chat in YouTube. This company isn’t trustworthy.
I’d prefer such applications to run offline with automatic instant synchronization, so no company can tell me what to do or not to do.
I understand why you think that, and you might be right. But I also know there are a lot of good people at Google, and I dream of the company waking up as a group again and focusing on its mission responsibly. Doing the right thing, openly.
Having said that, I personally also think that social networks and other internet technologies should run on a federated protocol: my current (early stage) proposal for the kind of thing we should try to do with this technology is https://flancia.org/agora. I see it as compatible/synergistic with the post we're discussing.
Hode[1] (the Higher Order Data Editor), which is free, solves some of these problems. (I wrote it. It's still evolving but it works already.)
> metadata is hard
Hode lets you encode and search any kind of information -- entities, relationships bewteen entities, relationships between relationships, relationships of any arity (number of members).
Hash, the language in which one adds and searches for data in Hode, is very close to natural language. I'll give examples below, based on the article's author's wishes.
> Let me compare my reading list with another to see overlap
This is the "and" operator. (There are also "or" and "difference" operators.) The query in Hash would look like
/find (/eval /it #is-in (reading list #of mary))
& (/eval /it #is-in (reading list #of peter))
> Allow me to tag books instead of placing them into static lists
Hode lets you do even better than tags -- you can classify the meaning of the tag. For instance, here are two commands that would add two tags to a book:
/add The Wisdom of Crowds #was a best-seller
/add The Wisdom of Crowds #(helps understand) finance
> Be tied to my highlights, annotations, and bookmarks in a non-proprietary, searchable, and shareable format
Hode is free software, and the data format it uses is dirt-simple.
Hode permits encoding and searching just about anything I can think of. Hode cannot yet create the data for you to search, though -- so far you've got to enter the data yourself.
> Let me query this tool like a relational database. For example: show me all books about scaling startups recommended by people I follow on Twitter or by people they follow.
Hode lets you order your search results according to any transitive binary relationship -- "read before" or "is more important than" or "was written before", whatever you want.
This looks interesting. If I may ask a naive question, why did you decide to build this as a text editor rather than tooling on top of an existing text editor (the parallel that occurs to me being Emacs and org-mode)?
Obair (https://chrome.google.com/webstore/detail/obair/jffaiidojfhf...) is a variant of the memex idea mentioned. It's not quite a universal memex for your digital life (which by the way is really inspiring) but rather made specifically for work docs across different work tools.
Disclaimer: I'm working on it and it's really early in the process :) Should I make Obair more broad?
There was recently a post in HN called "There is too much stuff". It revolved around the massive amount of unnecessary physical goods we need to shift through to find what we are looking for. I believe the same thing happens with content.
There is way more content out there that I would like to consume than my time allows. Is not rare to make my way through an article or a 1 hour talk and think 'oh well, that was a waste of time'. Or content that has pieces that I do find interesting, but is so padded that makes the whole investment not worth it. Currently I am reading Domain Drive Design, and I didn't find insights that I consider useful after well over a third into the book (500+ pages total).
I do believe that content curation is one of the most promising fields in the near future, and it will involve a mix of human and technological solutions. This might include not only the act of recommending content, but possibly modifying said content for the user (e.g. summaries, highlights, etc.). In comparison, current recommendation systems are a pure joke.
Off topic, but is Domain Driven Design suddenly coming into vogue? I just heard of it for the first time this week and went into a rabbit hole of research. I see that the books are not new by any means. I've heard and seen it pop up so many times just this week, including from your comment!
It seems to me that a lot of people are mixing DDD and functional programming concepts, since DDD meshes well with the concept of "functional core imperative shell".
I'm aiming for a federated system in which I don't have to read the whole book because I can read the best of the notes that others have taken. Like if Twitter and Wikipedia were one integrated navigable hypergraph.
I haven't read the entire article thoroughly but I think "Queue management for inbound digital content" is something implemented in an SRS app called SuperMemo. The way it's intended to work is that you have content and over time break it down and through the scheduling system you eventually end up with extracts small enough to be made into SRS cards (though it's more complicated than that in actual use). It has a lot of weaknesses, mainly because a lot of it is based on features from IE (thus poor PDF support) but I still find it extremely useful. If I find any article that looks interesting I just throw it there and eventually I'll import it, go through it with incremental reading (https://supermemo.guru/wiki/Incremental_reading) and eventually dismiss it.
Edit: I also ought to mention, it has a very good priority/interval system for scheduling when I see things. Essentially things I give lower priority are less likely to show up and when I'm going through my material/doing my reps I'm more likely to see high priority material. There's a general issue that you find the thing you see soonest most shiny and thus a strong likelihood of overprioritizing new material but SuperMemo helps intersperse everything to prevent that.
It sort of can do the central search thing too to an extent, I can search through my collection of imported articles/cards and find stuff that's generally relevant to my interests. Though how useful it is depends largely on how long you've used the app. I am also reminded of the brain (https://www.thebrain.com/) but I'm not super familiar with it.
Have you heard of Ted Nelson's Xanadu? I've only looked into it a little but I was reminded of it when you talked about better reading.
Thank you for bringing up SuperMemo - I remember reading through Piotr's article on Genius [0] and thought it was a wonderful piece on the balance of creativity and memorization.
I've looked through SuperMemo's features and while I completely subscribe to the idea of spaced repetition (I have used Anki for some periods) I was put off by the user interface; perhaps I should try it again.
> There's a general issue that you find the thing you see soonest most shiny and thus a strong likelihood of overprioritizing new material but SuperMemo helps intersperse everything to prevent that.
I really the idea of this approach to keep everything at its proper priority level. Thanks for linking the brain as well - I haven't come across it.
I am vaguely familiar with Xandu as well but it warrants another deep dive for sure.
I love that article, it changed a lot of my thinking regarding people's potential for success. Woz has a lot of other articles you might also find interesting here: https://supermemo.guru/wiki/SuperMemo_Guru. I've especially read through a lot of his articles on sleep and memory and found them invaluable.
SuperMemo's interface was not designed with the beginner in mind. When I first started trying it even basic operations frustrated me to no end. But now that I use it as a high(er) level user, it feels very very intuitive and you start to understand why it was designed in a way that looks so unappealing at first. If you do decide to try it again, I would highly recommend joining the SM discord server (https://discordapp.com/invite/pB7jcSK). There are a lot of very helpful people there.
What purpose does this collection of information really serve? I like having things organized neatly and easily accessible, but I have a hard time seeing what goal thousands of scattered annotations, text bookmarks, podcast bookmarks, etc. serve.
Yes -- but it's the scattering that's the problem, not the thousands. I believe the author wants what I want -- a system in which you can easily add any kind of data or meta-data, and easily search and sort those data in whatever manner you'd like.
In another comment[1] I describe software I've made that solves some of these challenges.
I think, I have a hard time seeing the big use from such software, because my work doesn't really require such a reference library.
A searchable collection of data is great if you have a use case that actually depends on such a collection, like writing a paper, news article or doing (academic) research on some topic.
Maybe to ask a better question, what data do you want to collect and what are you using the data for exactly?
The details of what I record (in another comment[1]) might be idiosyncratic and uninteresting to you. Those details aren't important. The important point is that so much information belongs in multiple places. A single statement, let alone article or book, might regard all of economics, justice, common sense and beauty. It might be about parsing, Haskell, and keeping data types flexible. I'd like to be able to search based on any of those things.
If there were a clean distinction between entities and tags, existing solutions would be enough. But what if you want to draw a relationship between two relationships? Most sentences longer than three words, after all, are higher-order relationships. ("Birds #avoid beetles ##because beetles #are hard to digest", for instance, is a second-order because relationship joining two first-order relationships.)
The searches Mr. Borichevskiy imagined are, I think, good arguments for why you would want to be able to encode arbitrary higher-order information. I believe Hode is the simplest solution possible that allows it.
How often do you look stuff up that's older than say a week?
Personally, I'm kind of glad to soak up information as I go and being able to just forget everything else. As opposed to constantly feeling the need to tag, sort, archive, reference, cross-reference, keep track of information, that future self will to 99.9% never actually need.
Yet, I wonder whether use cases start to materialize, once you start using it...
Projects and how-to notes, pretty regularly. Abstract concepts like comedy, maybe yearly. It's definitely an art, knowing what to bother ingesting into the system. I have erred on both sides.
Federation could reduce the hurdle -- both reducing the need to ingest, and the difficulty of doing so. Borichevskiy (author of article that generated this thread) nods at the idea of using someone else's index of books. Such labor-sharing could be much more general.
I don't want to suggest, though, that ingesting something is a waste of time if you don't review it. That's true of boilerplate metadata like ISBN numbers. But for original ideas, or for paraphrased insights gleaned from reading online, I find that sticking it into the graph produces value at the moment it's done. The rigor of compressing an idea to a few words, and determining which adjacent concepts it bears on, and how, has helped me understand things.
(Teachers often ask their students to state things in their own words. It's like that, but faster and harder. Like punk rock for essays.)
I haven't really migrated from Semantic Synchrony[1] to Hode yet. But my use case is almost everything that passes through my brain -- to-do lists, tech notes, things to read, people, places, jobs, companies, economics, artists, music theory, comedic theory, information theory, nutrition, love ...
> Content is published in a variety of formats including but not limited to images, sound files, videos, Google Drive docs, diagrams, long-from paywalled articles, PDFs, powerpoint presentations, and base 64 encoded blobs.
One of the great disappointments I’ve had working in tech for 20 years now has been the way we’ve transformed the entire spectrum of human expression, which includes wildly different forms of art and media each of which we used to engage with in its own unique way, into an undifferentiated slurry of “content.”
Every time I hear a child say they want to grow up to be a “content creator,” I die a little inside. Aspire to be a musician, or a painter, or a dancer, I want to tell them. A filmmaker, a poet, a sculptor. Dream of finding your art and mastering it, not just extruding more gruel to fill some corporate pipe.
You also hear people use “Intellectual Property” to refer to their favorite films. And extended conversations ranking revenues of different media franchises.
I’m sad that you seem to be the only commenter who is taken aback by the implicit commoditization here of not just art and entertainment, but also learning.
There are only few types of digital formats I use really. Text, audio, still photo, video and maybe 3d description of a place or an object, but the last one is still quite marginal.
It would be nice make arbitrary collections of these, add metadata such as place, time and author and send them to other users or bots.
If you want to go crazy you could add permissions and encryption which would enable private conversation and message board / social media style structures.
Pick boring standard file formats that are "good enough" for consumer use (one lossy and one lossless for each media)
Maybe I am dreaming of hypertext. The webserver-client structure of internet makes things easily very siloed and the ways to share documents you come a cross in the net are very clunky because everyone needs to reinvent the format and the player for everything more complicated than simple text or image.
It's sad that RSS is fading away. It simplified ingestion a lot. At the end of the day it focuses on content and a feed provides a quick overview of all the new content that has been published. I don't really understand why people are not into it anymore. Blogs and news sites don't even need to provide HTML pages loaded with JS and CSS when they provide RSS. Why would I bother loading so much stuff when I can get the data straight away without the bloat
Agreed with all points, but especially the queuing system. I started my journey towards becoming a programmer to build something like that. Its been a long and winding road, but it might be time to follow up on that, unless somebody can point me to a project that already exists like described in the post.
I would check out the IR feature of SuperMemo (https://supermemo.guru/wiki/Incremental_reading). There is a very steep learning curve and when you are a beginner you'll think it looks antiquated but if you ever up becoming proficient in it it will all just quick.
Kinda fits into this discussion, but I want playlists for netflix, prime, disney+, hulu, what ever, but not just running down a season, but for example a set of Christmas episodes. FXX has done some of this with the Simpsons, but I want to see other viewers lists and the possibility of across shows.
LiquidText on iOS seems to enable pulling out images, snippets, etc. as the author seems to want. Not sure what happens when that app eventually dies thought.
How popular was Google Reader really? I don’t think many people I know irl ever used it. There’s a handful of known or referenced RSS readers now. Combined they just aren’t that popular. I have a hard time buying that Google Reader was super popular.
There’s a few bookmarking services now. Some startups like linked in the article. And Pinboard has been around for a long time. Delicious isn’t needed anymore. I use Larder.io and Pinboard.in myself
That’s my point. It’s a huge talking point amongst geeky crowds. Is there any data to point to that though? It has always felt to me that Google Reader was never that big. Niche crowds made it seem like a huge deal though. I could be wrong. I’d just like something concrete.
Or are you saying it was a signal of the end for the specific niche geeky crowds? If so then I guess that’s my point. Google Reader doesn’t and never did move the needle that much.
For me, this is the lynchpin:
> "all content I think my future self would appreciate me consuming." (emphasis mine)
Current content feeds are optimized for engagement (ie. advertisement load) and thus won't conceive of a "future self", only what your current self will look at and click on right now.
I think that content feeds need to incorporate goal-orientation and move away from right-here-right-now orientation. For those wanting to do anything difficult they need to optimize their information diet over a very long time-scale, like years, so content-feed tools should be aware of human-scale timelines (eg. high school, college, career, parenting).
Humans thrive on learning and growth but so many platforms choose to see their users as merely inputs to an ad-delivery optimization system.
> I have little visibility into required time investment and foundational context until I’ve opened it and started thinking about it.
This is another thing that really annoys me about our current media ecosystem, and is really also a symptom of not properly conceiving of a person's personal development over time and a person's changing needs over time.
---
To look at this blog post from 1000 feet up, I'd say that Jonathan is unfortunately deprived of these tools because our media software ecosystem is madly building things for users who want to look at things and not think, as such platforms are heavily consumeristic and thus fantastic for advertising revenue and monetization generally (eg. Instagram, Snapchat, TikTok, Pinterest, Canva).