
Hacker News Daily (announcement) - cperciva
http://www.daemonology.net/blog/2010-07-14-hacker-news-daily.html
======
niyazpk
cperciva, I really appreciate the effort you put into this.

One thing I find a bit depressing is that even though this scheme hugely
reduces the time one spends on HN by filtering the articles according to
votes, the top voted articles are not the ones that I would want to read. I
usually come here for that "hacker" hacker news (hackerhackernews.com used to
be something like this, but now the domain name has expired it seems). Anyway
I am sure there will be people who find your service really useful.

~~~
Kilimanjaro
That's exactly why I created <http://www.hackerblogs.com> if you check HN
frontpage right now, there are zero posts by hackers, all come from PR blogs
like techcrunch, readwriteweb, thenextweb, etc.

I highly recommend you giving it a try, it is still growing and I hope someday
it may become the new source of real hacker news.

* I dare you to find an article like "Flipping arrows in coBurger King" on the news sites right now.

~~~
giantfuzzypanda
Nice site, just added my blog. Only problem I see is that if it does keep
growing and becomes popular, then people could add feeds of irrelevant blogs,
and like most social news sites it would turn into a digg/reddit.

~~~
Kilimanjaro
I am the curator, I check it every ten minutes (I am that addict) and believe
me, I hate fud, payperpost and propaganda to death.

If I see one blog incurring in a fault, they're banned for life.

~~~
xsmasher
I tried to register <http://deadpanic.com/blog/> , but apparently didn't make
the cut. Any suggestions for improvement, or insight into the criteria you
use?

~~~
Kilimanjaro
Sure it did. I checked the logs and it is registered and ready to serve your
posts as soon as you publish them.

------
troels
This doesn't really solve the problem though. There's still to much
information to manage it.

I wonder if any social-links-sites (hacker news, reddit etc.) have tried with
an algorithm similar to that which last.fm uses, where you get suggestions on
stories based on which previous stories you have shown preference for? (I'm
sure this sort of ranking has some fancy name as well)

If not - Why?

~~~
panacea
Reddit certainly attempted this a few years ago. They had a 'recommended' tab
in their main navigation. It worked about as well as their search at the time
(ie. completely useless).

If I recall, it was based on your activity (this was pre-subreddits) so if you
interacted with lots of political stories, for instance, similar political
stories would be suggested.

The problem was if you downmodded a bunch of articles involving Ron Paul, your
recommended links would be Ron Paul articles.

~~~
alexandros
can anyone find references as to why the feature was removed?

~~~
zck
I don't know, but I imagine the resources they needed to commit to it were
pretty hefty.

------
jackowayed
I really like this idea and will give it a try, but FYI
<http://news.ycombinator.com/best> serves a similar purpose.

I'm not sure of the exact criteria for it, but its basically very highly-voted
articles that are fairly recent.

I've been thinking of limiting myself to /best to keep my HN time in check, so
you're definitely onto something.

~~~
cperciva
The problem with /best in my view is that links gradually come and go -- it's
the right format for "what are the recent interesting stories", but it's not
the right format for "what are the most interesting stories since the last
time I was here".

~~~
sliverstorm
If there is an RSS of /best, the answer is like 10 keystrokes away.

------
alexqgb
More evidence that curation, not creation, is the point of sharpest demand in
the media business. The immediate challenge, I suppose, is finding the right
mix of focus and serendipity.

More broadly, the challenge is having my own life modeled well enough so that
information relevant to short, medium, and long terms plans gets reformulated
appropriately.

I could see this leading to a point where 'news' is not something I check in
the morning over coffee. Rather, it's a feature that presents itself whenever
I shift my attention to doing another thing (e.g. working on project A,
planning weekend B, etc.)

The really fascinating thing would be getting updates about apparently
tangentially related items. It's the classic 'local angle', only with regard
to activity, not place.

------
Jun8
Thanks for doing this, interesting tool but I won't be using it! Let me
explain.

I have often found that the articles I enjoyed most at HN were the outliers,
i.e. not the ten most upvoted ones (anecdotal evidence, never tested this
quantitatively). In fact this is what makes HN interesting: the quirky
entries. My guess is that most of the 10 articles you select will be already
covered by other such sites, diminishing the value of coming to HN in the
first place.

The end effect will be similar to Hollywood blockbuster effect. It's not that
I don't like to go a blockbuster movie, but I don't want to watch those _all_
the time.

------
smork
I subscribed to the RSS feed but somehow the content in google reader is
truncated to this:

"The 10 highest-rated articles on Hacker News on July 13, 2010 which have not
appeared on any previous Hacker News Daily are: "

Would be nice to have the full post in there with the 10 links :)

~~~
cperciva
My blog code defaults to only putting one paragraph into the RSS feed -- for
most of my blog posts this works well. I've adjusted my script so that future
dailys will include the list of links in the RSS feed.

~~~
smork
Cheers, thanks for the quick fix :)

------
epi0Bauqu
Another one I made for myself a while ago: <http://hacker.watrcoolr.us/>. It
includes some other feeds as well as the stories that reach #1 on HN. If you
just want the HN stories that reach #1, use
<http://feeds2.feedburner.com/HNWatrcoolr>

------
bsd_junkie
There are a number of article recommendation engines out there that can fill
the need for "outlier" articles fitting even the most peculiar tastes. I
personally use <http://www.euraeka.com> and even though it aggregates news
from less hardcore programming sources I find it an incredibly powerful source
of science and technology news that fit my taste. I tried Digg and Reddit
recommendation engines but they all work on user-to-user based recommendations
and most of the time i get either inaccurate or trivial recommendations.

------
barkmadley
you beat me to it! I should have a prototype working by the end of the week
for something similar (but hopefully better as well). It was a learning
exercise for me so I didn't really lose any time.

------
pclark
Is there a service that aggregates all the hacker news aggregators?

~~~
mkramlich
since everyone's ideal is diff prob the most surefire and arguably easiest way
for you to get the particular aggregated and/or curated view is for you to
write a small script that does exactly what you want. if others might want
same then make the code avail with a link from your HN profile. If a
particular service or view hack becomes popular perhaps PG will add an equiv
feature to HN itself, etc.

------
roadnottaken
This is a pretty cool idea and a perfectly simple implementation. However, I
think you're solving a problem that doesn't exist: If I wanted efficiency, I
wouldn't be reading blogs and news aggregators in the first place. I come to
websites like HN to relax and browse through interesting articles and
discussions -- sort of like leafing through a good magazine. If you distill it
down to 10 articles then suddenly I'm finished reading and I can get on with
my work... Too soon!!!

------
andrewtj
I'd like to PayPal you $5 toward obtaining daily.hn ($62 + whatever the tax is
on gandi.net) — anyone else?

~~~
jrockway
My browser has a working bookmarks implementation, so I don't care what the
domain name is.

~~~
andrewtj
My bad, I get excessively terse when I'm tired. I wasn't suggesting it out of
utility but out of appreciation — nothing says thank you quite like a
superfluous vanity domain.

EDIT: Just noticed my other post has garnered at least one down-vote so
although I'm pretty sure this idea doesn't have legs, on the off-chance you
dear reader are one of ~13 other folks who'd like to see this happen, drop me
an email to express your interest.

------
aptsurdist
You might be interested in checking out the page -
<http://news.ycombinator.com/best> Sounds a lot like what you're doing.. I'm
not sure if others know about it already - I think I stumbled upon it by
accident.

------
cperciva
Discussion about direct link (I'm not sure which people will vote for, the
site or the announcement of the site):
<http://news.ycombinator.com/item?id=1514039>

------
elai
Could you make it that the actual news articles show up in the RSS feed when
displayed on google reader? I don't want to have to click the page and then
click the article to actually view it.

------
revorad
How do you decide the top 10 links of the day if you scrape every 5 minutes?
(the top links keep changing). Couldn't you just scrape once a day?

~~~
cperciva
The /news page ranks links based on score and time since submission. I'm only
ranking links based on score (and whether it has been on a previous daily).

I _could_ scrape the entire site at midnight each day, but I think PG would be
very unhappy with me if I did that. Scaping /news every 5 minutes imposes much
less load, and since highly ranked links get almost all of their votes prior
to falling off the front page, this gives me almost as much information.

~~~
revorad
I'm guessing that for your purpose, you might catch more interesting stories
from the /classic page than /news.

Do you mind sharing the scraping code?

~~~
cperciva
I'd prefer to not post the code publicly, simply because I don't want to
encourage people to put extra load on the HN server -- but if you want a copy,
send me an email and I'll provide it.

~~~
revorad
Thanks. Just emailed you.

------
nuclear_eclipse
Thanks Colin, I just added this to the feed list for <http://planethn.com> :)

------
tel
The RSS feed doesn't actually display the links, which makes this sort of
thing nearly useless to me. Is this just a bug?

------
joubert
Are points of a story only a sum of direct up votes? Or does it already factor
in age, number of comments, etc.?

~~~
giantfuzzypanda
Yes, but according to the FAQ they're ranked like this:

"On the front page, by points divided by a power of the time since they were
submitted. Comments in comment threads are ranked the same way."

------
kloncks
I'd be interested to know what you programmed this in. Do the .HTML pages mean
you used PHP? If them, cURL?

Lemme know.

~~~
cperciva
I use FreeBSD's fetch(1) to download the page, but curl or wget would have
worked just as well. Extracting the data I want (item #, score, and link) is a
few lines of perl. Managing the data over the course of the day and writing
out the final HTML is done using standard BSD text utilities (sort, join,
comm, cut).

~~~
kloncks
Thanks!

------
hackermom
Part of the problem is how Hacker News seemingly have turned into something
more akin of News. The hacker tidbits are far and few between, and the overall
amount of new submissions have skyrocketed.

~~~
duck
I agree. I wonder how it would work if it cost you X karma points for every
submission? Maybe X varies based on keywords in the title too.

