How do you update this list on a daily basis ? Github actions is an option.

outcoldman · on July 7, 2023

I am not sure if users of HN will be updating the original HN discussion in general, but yeah, GitHub actions can be an option for this.

For now, I just wanted to add those 600+ blogs to my RSS and filter them out slowly if I need to.

defrost · on July 7, 2023

Good stuff, thanks.

Needs a bit of lint work still, eg: current lines 12 & 13 are dupes

    <outline type="rss" title="bookofjoe" text="bookofjoe" type="rss" xmlUrl="https://www.bookofjoe.com/atom.xml" htmlUrl="https://www.bookofjoe.com/atom.xml"/>
    <outline type="rss" title="bookofjoe" text="bookofjoe" type="rss" xmlUrl="https://www.bookofjoe.com/atom.xml" htmlUrl="https://www.bookofjoe.com/atom.xml"/>

but still appreciated as that's a minor quibble easily polished away.

sphars · on July 7, 2023

There seems to be many valid blogs that got skipped due not having the `rel="alternate"` link in the HTML head (mine included). I just added it, as I'm sure others will, so it'd be a good idea to update the list regularly

revskill · on July 7, 2023

If possible, could you also add to description for anyone wants to add to your list, to go to original thread and submit ?

Currently it seems there're many people fetch from original thread, which makes it ne source of truth, less confusing for anyone to submit.

outcoldman · on July 7, 2023

Yeah, the best way is to go back to the original post and submit comment there on top level. Make sure HN recognize that as a correct href and your blog has alternative link to rss/atom. And for now only users with 100+ karma are included.

I left the console.log there as well to show which blogs aren't recognized.

I am also open for PR. It was just a few hours project to get it going, but if people will find it useful, would be nice to get some tags from the blogs, so it would be possible to extract the list only for specific technology/topics.

BeetleB · on July 7, 2023

> Make sure HN recognize that as a correct href and your blog has alternative link to rss/atom.

Any details on how to do this? My blog (https://blog.nawaz.org/) has an RSS feed here (https://blog.nawaz.org/feeds/all.atom.xml) but I don't know how to set it up the way you'd need it.

zczc · on July 7, 2023

Put this line in html:

UPD: type="application/atom+xml" for Atom, type="application/rss+xml" for RSS

distcs · on July 7, 2023

I don't know Go's flavor of regex but is this correct parsing code: https://github.com/outcoldman/hackernews-personal-blogs/blob...

Are you taking care that if the <link> tag spans over multiple lines (linebreaks in between), you're still parsing it right?

You do seem to be resolving relative paths with:

  feedUrl := matches[1]
  if feedUrl[0] == '/' {
      feedUrl = url + feedUrl
  } else if !strings.HasPrefix(feedUrl, "http") {
      feedUrl = url + "/" + feedUrl
  }

So I think this is good. Just trying to see what could be breaking for the other users who are complaining their feed isn't listed.

outcoldman · on July 7, 2023

I have fixed it for more people, should have used the built-in resolution func https://github.com/outcoldman/hackernews-personal-blogs/blob...

midzer · on July 7, 2023

Do sitemaps like `<link rel="sitemap" type="application/xml" title="Sitemap" href="/sitemap.xml">` work, too?

-> https://stackoverflow.com/questions/13327527/how-to-make-a-s...

RamblingCTO · on July 7, 2023

Didn't work for me and andinfinity.eu, but I'm in the other lists. Any ideas?

outcoldman · on July 7, 2023

Added a note https://github.com/outcoldman/hackernews-personal-blogs/tree...

I checked yours, the issue is that you are escaping + in application/rss+xml, when it should be just application/rss+xml

I have updated the code and will re-generate. I am not sure if what you are doing is allowed or not.

generalizations · on July 7, 2023

Ah, shouldn't have used a throwaway to post my blog. Oh well.

outcoldman · on July 7, 2023

You can comment on that original post again with normal account, and I will regenerate.

distcs · on July 7, 2023

When do you plan re-run your crawler? HN users are updating that original HN discussion every minute as we speak! It'll be nice to get an updated list sometime today. Possible?

And yes, thanks for your work and sharing it!

outcoldman · on July 7, 2023

I am fixing some issues, trying to get more blogs from the post, so do it a few times in a day.

And hope at some point to actually build something larger from that. I love RSS and this is the best way to discover new content. I actually already have found a few interested blog posts, links, information.