Hacker News new | past | comments | ask | show | jobs | submit login

I scrape about 60 blogs and news sites that deal with a niche topic and examine all the hyperlinks. If more than one of them links to the same page, I assume that it's a page that's generating some buzz, so I send it out in an email. It's proved to be a generally reliable assumption.

Have you thought about the case in which a blogger might be using a URL shortener? You might be missing some potentially 'buzzy' links if you are just looking at the direct URL string

Would love it if you posted a blog article sometime about the technical details.

There's really not much to it. Scrape each site/feed every X minutes, find all the hyperlinked URLs on the page, add them to a database table ... and if they're already in the table, send out an email with links to the "buzzed about" URL, as well as all of the sites/feeds that mention it. I keep the links in the table for about a month.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact