The idea for HNTrends has been bouncing around in my head for about a year as one of those man-I-wish-I-had-more-free-time-type side projects. I recently decided to take up learning Flash and ActionScript and this seemed like the perfect opportunity to build it.
It's fairly straight forward: A Python script scrapes HackerNews every 15 minutes and saves the data in a MySQL database. A PHP script queries the database and outputs the the last five hours worth headlines as neatly-formatted JSON data, which the chart grabs and plots.
The older, less trendy news items are rendered with a slight transparency in order to reduce the chart's business. You can hover over any of the lines to see it in greater detail. Also, if you click any of the points it'll take you to the comments. The links on the right take you to the actual news articles.
Hope you like it. Let me know if you have any suggestions --
Matt
What would be most interesting is to extend the list past the page with the first 30 articles. That's what would really give you a chance to deal with HNs front-page feedback loop.
There is lots of stuff that does not make it to the front page during busy times.
Any chance of increasing the length of the list to 60 or maybe even 100 ?
This is pretty fascinating. The most important thing I notice almost immediately is that there are generally two classes of articles:
1. those which reach the front page by virtue of quickly getting a few votes while they are young and then dwindle
2. those which barely reach the front page and have some kind of resurgence (discernible by a large jump in front page rank).
It'd be interesting to group articles based on how they perform like this and see if there are any other correlated factors. My feeling is that the ones with resurgence are either long and require some amount of karma "activation energy" before people are willing to dive into them or that they're ones with long conversation chains which have a spike for similar reasons.
Probably only me - but to my mind that's a shame. It's only my opinion, but I think Flash is nuts for anything new except video at the moment. I think Flash is:
- slow
- fairly unstable, even on Windows
- a privacy nightmare (cookie system)
- being left behind by browser vendors, where there is competition
Excanvas on IE and the canvas tag are really well placed for visualisations like these. I built a donut chart in Canvas recently for something I'm working on, and I'm glad I did, because I was able to do an impromptu demo with someone just the other day on my phone.
I know it's not a critical piece of engineering, and it doesn't actually matter, but I do personally find Flash dulls my enthusiasm for otherwise very nice things!
Seriously? That sounds odd I'm running the latest version of Flash in Firefox and it is using between 0-6% CPU with 8 tabs open including this Flash chart.
If he's using a laptop, he's now burning significantly more power. 4 hours of battery life at idle can easily become 30 minutes at a constant 25% CPU load.
you're already looking at HN, it's not like you're really gonna need that 25 for anything else right?
Actually, I did need that 25% for something else. Was waiting for code to finish compiling (well, for Clang's Static Analyzer to finish running). It takes more then a few minutes to run on a sizable project.
Plus, as jrockway pointed out, it absolutely kills battery life, and I'm on a laptop.
Blame Apple if you have an iPhone. This sort of presentation also couldn't work on a small phone, though a text list version should be provided for mobile users (is it?).
I hope the author checks out something like Raphael (JS SVG library) and converts it though, because Flash is definitely tacky.
In all seriousness though: I simply wanted to learn Flash. As an iPhone user myself, I realize its frustrating that it doesn't show up. That being said, even if it did show up you wouldn't be able to make much of it because of the screen size and the sheer business of the chart. It's really meant to be viewed in a normal browser.
No, the other one is sort of a horse race, which is neat. Maybe show the top sites and their ranks over the past five half hours/full hours in a narrow table that would work in any browser.
Yeah, I have click to flash installed, so my first reaction was "what, flash?". It would be so nice to see it done with http://www.rgraph.net/ or something similar.
How about a link to the comments after each article title? I usually read those first and then click on the link (otherwise I'd have to come to HN front page anyway to see the comments on it). Works great though - good job.
Agreed. I've been using the site all morning, and it's really growing on me. But I didn't see how to get directly to the comments until you pointed it out.
Incidentally, I thought the lines looked blurry... in fullscreen it's clear that they are.
The pop-up only appears for the datapoints - not if your mouse is over a line segment between data points. In fact, it was a while before the any popup come up, for me. Maybe make it attach to the closest data point? (google analytics does it this way).
That's odd - most of the data is fairly linear, but as of right now, there's a very sharp spike for 5 stories at 10:30pm (with a corresponding drop in 5 other stores), which is reverted back to "normal" at 10:45.
Less information, true. However, you can't downvote submissions, so the graph will be rather boring. Just a ton of upwardly sloping lines, some sloping upwards faster than others
I am actually mostly interested in finding a data point quickly in order to get to the comments. Maybe just break out the comment link feature into a "(comments)" link next to the article titles?
Quick update: There was a problem with the script which caused it to go down for a few hours. Should be fixed now. Graph might look a little off for a few hours until the logging catches up.
It can be useful in a macro level, that is one might find out the best time in a day (or else, the best day(s) in the week) to post a link in social media site or an article in a blog.
Already there - you just click on a data point in the chart. (The titles on the side take you to the article, which is also the comments for Ask HN / Review my App type things).
It's fairly straight forward: A Python script scrapes HackerNews every 15 minutes and saves the data in a MySQL database. A PHP script queries the database and outputs the the last five hours worth headlines as neatly-formatted JSON data, which the chart grabs and plots.
The older, less trendy news items are rendered with a slight transparency in order to reduce the chart's business. You can hover over any of the lines to see it in greater detail. Also, if you click any of the points it'll take you to the comments. The links on the right take you to the actual news articles.
Hope you like it. Let me know if you have any suggestions -- Matt