> So for fun, I wanted to see how many steps it would take to get the information contained in those alerts into a usable RSS feed.
The author did it for fun, but if you ever find yourself in this situation and need a solution fast, consider Kill The Newsletter[1]. Free, easy to set up, no accounts needed, open source[2].
Kill the Newsletter is great. I'm subscribed to a number of Patreons and last week I got fed up with the lack of RSS feed for non-audio posts, and got the idea to do something sort of like Chris Hardie and bash together a web server which would receive emails and post them to an RSS feed.
Very glad I looked around and found Kill the Newsletter before I wasted my weekend!
I mostly agree. But really RSS and email have basically the same functionality. The have date, time, content. The main difference is that RSS has better markup for URL "attachments" as well as an alternate URL. The "what if the email server is overloaded" can be equivalently replaces with "What if the RSS feed is down".
Spam is an issue with email, the nice thing about RSS is that you know you want the feed, so spam is almost a non-issue. However I have seen it the other way where my feed reader is blocked because they have "bot protection" on the RSS feed. This can be called a misconfiguration, but it is a common misconfiguration, and there will always be some level of protection on a feed because in the case of DoS you want to try to filter out the attackers. I guess for something this urgent you can configure your reader to alert you on failures to check but that is not even supported by a lot of readers.
The other nice thing about RSS is that I am in control of my subscription, whereas email I need to ask you nicely to stop, and hope that you haven't sold my email address elsewhere.
The main advantage of email is that it is natively push, so the updates will almost always be faster (greylisting aside...). This can be resolved with WebSub but support for that is very rare.
At the end of the day they are both open protocols and you can handle them however you want, although the infrastructure for RSS is probably more inline with this use case.
But then again, I read my RSS feeds via email anyways, so I'm probably biased. My proposal? Support both. RSS is trivial to add support for, and if you have an RSS feed you can add a "subscribe by email" form in a couple clicks using existing services.
And overall these public alerts are a seriously unsolved problem. When I lived in Dublin, IE there was a water advisory like this and their only notification mechanism was TV+radio. As with most people under the age of 30 I don't frequently listen to TV or radio so could have been drinking contaminated water for days. I was shocked that they didn't send out a cell-based emergency alert for everyone in the area. But even that wouldn't have caught everything. It would be amazing if my city, province and country all had emergency alerts that I could subscribe to via RSS and email.
Ehhhh... I get what you're saying here, but I don't think RSS is worth it for emergency alerts.
I'm willing to bet good money <0.5% of the population even use or are aware of RSS. Supporting it also has a very non-trivial cost. Even if it takes a few hours for an engineer to implement it, it'll need to be added to their testing plan to make sure it's reliable.
Then if it breaks, you'll neen to find another expensive techie ASAP to fix it, as it's an emergency service. If someone is relying on the alerts through RSS, there's potential for catastrophe in an emergency situation if it fails.
So I'd rather the money go towards tech that gets a message out to as many people as possible, be it through facebook or other proprietary service. Human lives are potentially on the line, so whatever system is used needs to be stream lined the same way existing emergency systems are.
> The other nice thing about RSS is that I am in control of my subscription, whereas email I need to ask you nicely to stop, and hope that you haven't sold my email address elsewhere.
Sadly too many places think that subaddresses— username+label_at_example.com —is an invalid address. Can't use it in usernames in many places. (It's been valid in e-mail since RFC 822.)
This is one of the reasons that motivated me to get an email forwarding service with my domain, everything at prefix(.*)@mydomain.tld is forwarded to my email address, and there's no special character to block. I can then block specific addresses that have been sold to spammers
I've always wondered why emails aren't used as back-bone for more services. In many aspects, facebook is just a mailing list with ads and a fancy front-end. And in some ways, visiting a web page is just emailing the server and receiving the web page in response. And in some ways, the latter would be perfect to keep an archive of important stuff
> I generally do not connect to web sites from my own machine, aside from a few sites I have some special relationship with. I usually fetch web pages from other sites by sending mail to a program (see https://git.savannah.gnu.org/git/womb/hacks.git) that fetches them, much like wget, and then mails them back to me. Then I look at them using a web browser, unless it is easy to see the text in the HTML page directly. I usually try lynx first, then a graphical browser if the page needs it (using konqueror, which won't fetch from other sites in such a situation).
heh. In 2003-2005 I didn't get why RSS mattered, so I built a mobile photo blog based on pictures+text from a Sony Clié email client, posting pictures + captions to a mailbox, and a .forward file pipe-filter that turned those messages directly into an RSS feed. (About 50 lines of sh - "why do people think this is hard or complicated?" :-)
Turns out (1) UI matters (2) Network Effect matters (3) systems that gamify user engagement basically always win regardless of what they're doing...
A lot* of the criticism against RSS/Atom that I've seen on the internet boils down to appeal to novelty; both are old, so assumed to be "bad".
But hey, they work. They fill a purpose - they warn you about updates in a site that you might want to check out. They do one thing, and do it well; and for me that "one thing" can be new manga series to read, new videos to watch, news from my city's site (it has RSS), so goes on. How exactly would be this "bad"?
The problem is, RSS is not compatible with a few popular models:
1. Closed platform - Places like Spotify, Twitter, Facebook, etc, are dis-incentivized to help you consume content unless they can get you trapped in their UI with your logged in account.
2. Web as a program - Tonnes of people are trying to replace websites with 'apps' and replace programs with the web, neither of which lend themselves nicely to being RSS friendly.
3. RSS popularity - Very few people (compared to the majority of the web) are consuming RSS. Most users will be using Chrome/Firefox/Safari/Edge only. They get their notifications by keeping tabs open or via push notifications.
I think the way forwards is to normalise RSS by rebuilding it into the web. I've often thought about what and RSS-based social media may look like. It's a shame Open Graph [1] didn't use more RSS functionality.
For Youtube, reddit is also possible, might be for spotify too. For Meta owned social media sites like Instagram, Facebook, I don't think it can be done.
I didn't realise it was capable of serving RSS! It still puts you at the mercy of Twitter's API support and a random host to not interfere with the original message...
> For Youtube, reddit is also possible, might be for spotify too.
I don't use Reddit (HackerNews is as far down the internet rabbit hole as I want to go), but I know Youtube still allows RSS (just about). They have been slowly but surely removing the feature though. You used to be able to export all of your subscribers as OPML, now you can't. There used to be an easy way to find RSS feeds, now there isn't. I suspect before long they will move to remove them entirely.
> but I know Youtube still allows RSS (just about). They have been slowly but surely removing the feature though.
Can you elaborate? I can still find the YouTube RSS link in the page source (or just paste the normal channel link into Feedbin and it'll automatically get the RSS feed) and it seems to work fine.
We probably would have all saved tons of money if we listened to the people that were saying don't kill RSS long ago, but we keep repeating history because the decision makers always love a flashy new marketing presentation over "good old stable, simple, and reliable". :[
I did something similar for Bandcamp. Back in the day, Bandcamp had decent RSS feeds, but they dropped them several years ago. Following labels on Bandcamp is one of my main ways of music discovery, but using Bandcamp's "feed" or getting emails wasn't cutting it for me.
So, I wrote something simpler using imapfilter to filter Bandcamp emails and convert them to RSS feeds, which I can read with my freshrss self hosted instance.
Begging website operators to implement RSS is a dead end. If it worked, RSS would be ubiquitous and we wouldn't have these kinds of futile submissions that preach to the choir.
Instead of hoping every website operator in the world does something, it seems far more scalable to centralize efforts into a youtube-dl style repository so anyone can host their own website-to-RSS gateway.
Implementing RSS is extra work, thus not everyone is going to implement it. Even if it contributed to the bottom line. It's the same reason every website that publishes information doesn't have an email newsletter.
Since it's a consumption optimization, it makes sense for us, those who may want to consume a given website, to implement it (while we wait around for utopia).
RSS is easy to add to a website. It's a ubiquitous standard — whatever language or framework you're using has a library or plugin that you can integrate in under an hour.
Keep in mind the example the OP is using; a profit motive doesn't need to exist, the motivation is simply informing the public. If you go to the trouble of creating a website for that purpose, there isn't really any excuse for not including an RSS or Atom feed unless you're somehow still stuck in the "edit pages by hand and upload with FTP" era, or perhaps (I'm being extremely charitable here) if you're stuck in the slightly later "I had to write my own CMS from scratch" or "I'm still using Microsoft Frontpage" era.
I dunno. When I was still writing publicly someone reached out for me to add RSS to my website and it took me like 3 hours maybe to add it to my hand written static site compiler.
It's a pretty simple protocol, even if I would have done things differently. For example, relative paths should be supported with some sort of top level setting called "root" or something like that. That way I wouldn't have to convert relative paths.
> It's a pretty simple protocol, even if I would have done things differently. For example, relative paths should be supported with some sort of top level setting called "root" or something like that. That way I wouldn't have to convert relative paths.
They killed RSS because there was no solid means for data providers to ration, monetize and control individual access to their content. The decline of availability for RSS feeds wasn't based on waning popularity from what I observed. Back then RSS feeds simply went offline in droves, as everyone scrambled to figure out how in the heck to implement OAuth, and most people gave up trying to implement feeds after they came with monthly fees and when OAuth was forced on us by Twitter and Facebook etc... I'd even go as far as to say the requirements introduced for OAuth mainly served the horrible purpose of turning connecting to most (previously free and open) data sources into a fee based and tightly controlled commodity economy that made everyone go closed-data after external data became too janky, costly, and difficult to support.
I'd love to use RSS for everything, but the problem has always been that most RSS readers are really really bad.
I want one where I can add all RSS feeds I care about, then easily filter through the posts by assigning tags I care about, get notifications about certain tags, and easily navigate huge lists.
I basically want to filter based on content, tags and other things, then set what kind of notification I'll get based on tags.
One tag might give an actual toast notification, one might just be highlighted, others might be dimmed and marked as "uninteresting" and some might be straight up hidden.
So far I've not found a single RSS reader I like and they all just sort of act like a basic email client, and usually don't do cross-platform either.
I use FreshRSS which is hosted on my home server. What it does is allow you to categorise the feeds and you can also do some further things like hiding them from the main feed if you choose to. Then beyond that it pulls article tags from the original sites which you can search or predefine a filter for and you can also label individual articles you find. What it can't do unfortunately is allow multiple tags for a site to be applied, so its not possible to classify content from a source multiple ways and have it surfaced in multiple places while only having it shown once and read once. You can put a site in multiple categories but reading it in one place wont read it in the other.
It is one of the better systems I have used, its by no means perfect but its fairly quick and easy to use with some decent shortcuts for working through the sites and I manage hundreds with it and it happily works across every platform since its a website.
I don't think it gets quite to the level of customization you want, but I find Feedly to be very good and is definitely cross-platform. Give it a try if you haven't. The for-pay Pro model has some extra features, and I have had good luck contacting their support.
I make this exact same comment about every new RSS reader.
The information density of Lenns.io is way too sparse.
It displays 9 articles per page on my browser, and I don't seen any way to change that.
By comparison the compact view I'm using in Feedly displays 24 or so articles per page.
I'm not sure if my RSS use is typical, but I suspect it is. I subscribe to a lot of feeds, but I don't actually read but a small number of articles surfaced. I'm not quite sure what the percentage of articles I read is, but I'm guessing it's anywhere from under 1% to upto 5%. It obviously varies a lot by source, but overall it's pretty low.
If I skip 95 articles out of every 100 listed, I need an RSS reader that allows me to glance through a lot of content quickly. Showing me 9 articles per page is going to be far too slow.
I do have some feeds I follow that are visual in nature, for example art related blogs. Those I usually browse with a card view in Feedly, which displays a thumbnail of some sorts with the article.
One default view that's as sparse as the one in lenns.io currently is just not going to work.
I would definitely be happy to alpha/beta test a "different" RSS reader.
Disclaimer: I have used RSS consistently yet, but I have installed a few readers and tried them over the years. Just never broke the old habits. But I should know the gist of it.
feel free to be annoying and email me at the address in my profile if you ever want to (for feedback, brainstorming... etc).
I just want the ability to have some sort of intelligent negative filters, i.e. the ability to nuke content I don't care about.
For example, I'm a bit of a sneaker head, and subscribe to a lot of sneaker blogs. But I really don't like Nike shoes, which is the most popular sneaker brand, certainly in the US, and every sneaker blog has several Nike related articles every day. I never read any of them. I'd like to just completely omit them from my feeds.
That's kind of a simple example, and I can actually do that somewhat easily, but there are more complex examples. Let's say for example human rights related blogs, but ignore all propaganda content produced by either Governmental sources or about organizations associated with human rights violators. Or ignore all articles mentioning musicians, if the mention is about celebrity gossip. Or ignore content about Al Franken, if it's about his political activity, but still list content that's about his comedy work. Or vice versa.
I agree, but would add that nailing these features doesn't excuse a reader without sufficient support for customization of the UI (especially content). I might be able to excuse a reader with a polished, clean, and opinionated design (like Apple & Google's eBook readers), but would prefer to have as much freedom as possible, to the point of user-stylesheet support or some such low-level measure. Bonus points for features easing the manipulation of such 'profiles', such as a simple theme selector for easy swapping between an arbitrary number of custom styles and support for per-feed defaults.
Not to mention accessibility features, which readers are well positioned to support.
Not to be too picky, these are my ideals and I wouldn't so blutly demand such feature coverage from a developer. Y'all do what you think is best and I'll find what works best for me, thx to amyone who's contributed to a reader c:
I wrote a simple "RSS to Email" utility, which I use for reading feeds.
I can filter either in my mail-client, or in the utility itself, to include/exclude entries from feeds based upon regular expressions, and similar things.
It is pretty flexible - at least for the case when the feed itself contains the complete text of an entry. Often you'll find feeds contain only the "summary" of a blog-post, or article. Those are a bit harder to handle.
Sounds like you want something that uses RSS data to build a more freeform database.
I wouldn’t describe RSS software as bad but they do often prioritize the reading experience, which is in conflict with data management due to the small market keeping developer resources minimal.
- Sort articles into lists
- Tag articles by category
- Dim articles that may be uninteresting
- Highlight articles you are most likely to enjoy
- Also have various notification preferences
Instead of convincing me of the benefits of RSS, I think you've convinced me that someone should sign up for every government newsletter possible and do something really interesting to present that information to the public. Like maybe I don't just want to know about water alerts in my town, but I'd like to know something about water alerts anywhere in the country. Or when I'm visiting somewhere can I get snowplow alerts without digging through local sites? I imagine there's a lot of interesting stuff in those newsletters that I haven't even considered...
I have a similar problem: I'd love to have all of my Patreon posts in my RSS reader instead of their shitty app or website. That's sadly not possible. So I'm automatically redirecting all of my Patreon emails to my RSS hosted - which does provide an "email to RSS" feature. It's not perfect, but it works fairly well
I've been sending RSS feeds to my email since 2015, around the time that Google Reader was sunset. Advantages of doing this vs. using a dedicated app:
- I have well designed email readers on all platforms I use (web mail on desktop, app on mobile devices) which I'm used to. Feeds get labeled separately and don't pollute the inbox.
- I can forward emails, archive them, search through them, print them with ease.
- They are backed up as part of my existing backup process.
- Email is here to stay. Dedicated apps are a gamble.
I run my self-made feed forwarder on Github Actions cron schedule [1]. I used existing tools (e.g. rss2email) but each had so many quirks, bugs and outright obnoxious behavior that I decided to just roll my own simple one.
> make sure to include a simple RSS feed and a way for your users to discover it.
There are ways around no RSS. I have a simple rssgen [2] relay which uses CSS selectors to build an RSS from an HTML page. You can then point your feed reader to it.
Instead of hand coding a scraper or subscribing to email driven webhooks, and managing that bespoke code - take a look at Pipedream.com
You can chain together low code steps. An email address can be a trigger, or just a simple timer.
Then you can post results to Slack/Discord with pre built actions, it's a lot less work. No hosting to worry about either. You just build in the dashboard and the steps are hosted by Pipedream.
Does anyone want to try and "fix" RSS? I'd be keen to jump on a chat/email thread about it.
I don't think RSS is a bad thing, I think it's just not sexy anymore and the reading experience sucks - it died for me when I stopped using Opera but I do think there's an app/extension plugin play to bring it back. Most newsletters, alerts would be better as content delivered through a push/RSS model.
We don't get news delivered by post, email is the wrong medium for broadcast.
My city does email for alerts and RSS for news releases.
>But it still locks the information up inside my email inbox, and subjects it to the fragility of email delivery these days; will the town’s email servers be allowed to deliver the message?
I think in practice that fragility is mostly Gmail and some of the other big free email providers. Email providers that actually care will normally make it so you can at least get your email reliably.
This is slightly unrelated/related to the author's problem, but I've been working on creating something that can continually scrape websites for updates for me - and tell me when something has changed. Still a WIP, but if anyone's interested: https://fetcher.page
For this I use I cron job and a bash script which curls sites I want to watch and grep them for content that I am interested in. It diffs to the last check and if there is one, mails the diff and the url to me.
Slightly related, but IIRC WordPress has a native feature that allows you to create posts by email. If it’s not WordPress, it might have been a plugin, Blogger, or WP.com.
The author did it for fun, but if you ever find yourself in this situation and need a solution fast, consider Kill The Newsletter[1]. Free, easy to set up, no accounts needed, open source[2].
[1]: https://kill-the-newsletter.com
[2]: https://github.com/leafac/kill-the-newsletter