Hacker News new | comments | show | ask | jobs | submit login
HN now serving a million page views on weekdays (ycombinator.com)
279 points by pg 2419 days ago | hide | past | web | 80 comments | favorite



As we approach HN's fourth birthday, traffic is now around 90 thousand unique visitors and 1 million page views on weekdays.

Funny, before jacquesm retired from hn last week: 90,001 unique visitors and 2 million page views per day.

http://jacquesmattheij.com/Tell+HN%3A+So+Long+and+Thanks+for...

(just kidding - we miss you, jacquesm. ping us sometime)


Let's have a break from the incestuous cross-pollination on HN please.

Frankly if HN had an option to ignore all posts from the top 10 leaderboard I would opt in to that; I am sick and tired of this entirely irrelevant person-specific drama.


Congratulations and much gratitude to you Paul, all of the YC team, and all of the editors for tireless and thankless work to keep things running well.

> As we approach HN's fourth birthday, traffic is now around 90 thousand unique visitors and 1 million page views on weekdays. (Http requests and page views are identical except for votes, of which there are about 25 thousand on a weekday.)

Interesting, interesting... btw, one set of stats I was curious on - got any numbers on how many people who browse are registered and what percent of registered users vote? I'd be fascinated to hear about it if the numbers are handy and not confidential.

Thanks again and congrats on news.yc and all the recent cool developments and successes at incubating.


A million! Imagine how much bandwidth HN could save if each comment didn't use nested font tags and inline styles ;)


Or gzip for that matter. :)

gzipping this thread page results in a 5x size decrease.

Can't blame pg though. He's just a product of his computational adolescence. In another 15 years, we'll still be using CSS and divs while the kids move on to 3D direct brain interface temporal markup languages.


Or tables with spacer images. What is this, 1995?


Wow, I never peeked at the HN HTML source before but it's pretty terrible. Tables nested within tables, tables used for layout and non-tabular data, CSS only used for colors and fonts. It's such a simple layout that I can't believe they didn't go all-CSS for it and make the HTML really semantic and tight.


How much bandwidth would that save?


I'd guess anywhere from 100GB to 1TB every month depending on whether you did the inline style suggestion and/or the gzip suggestion.

This page gzip'd: http://www.google.com/search?q=30+*+1e6+*+(52KB-10KB)

Front page gzip'd: http://www.google.com/search?q=30+*+1e6+*+(25KB-5KB)

So probably not worth it :)


Gzip is always worth it, it lowers the per client transmission time and because browsers render gzipped pages after they have been loaded the page appears instantly in one go.

It actually creates the impression of a faster site even if the real time to load is the same.

Considering the only thing necessary to enable gzip is to enable it in the front facing web server and that it uses barely any cpu at all on level 4 I don't see why anyone wouldn't use it.


the only thing necessary to enable gzip is to enable it in the front facing web server

Somewhat more is necessary in this case.


Given that gzipping is generally a matter of adding a line to the web server configuration file, and it gives so much savings, I'd say it's worth it.

As for the HTML suggestion, I'd skip that one, it's just a few bytes per comment and needs actual code changes.


ZOMG!

I'm actually really surprised by this .. does anyone know the rationale/reason for such poor markup and lack of optimisation?


> Arc embodies a similarly unPC attitude to HTML. The predefined libraries just do everything with tables. Why? Because Arc is tuned for exploratory programming, and the W3C-approved way of doing things represents the opposite spirit.

http://www.paulgraham.com/arc0.html


HN is my favorite site, what a wonderful community. PG: thank you for putting it together.

However, the site is not fast, seriously, a site that's so simple containing just a few text should be blazingly fast. It does not take too much science to reach this goal.

I understand that it can be interesting running it via an Arc program, but you hit millions of page views, and there are people that are using this site every day to get together, to share their knowledge, and so forth, and it's a shame that there is to wait too much at every page view.


Do you mean it's slow now, or that it was slow a week ago? It doesn't seem slow to me now.


It is no longer too slow, but still not as fast as it could be. I must admit, my experience is filtered by the fact that I'm in Italy and I guess there are many hops between me and the HN servers, but still loading pages seems to take something in the order of one second.

Being the layout very simple to render, no graphics, there are all the prerequisites to make the HN experience "google alike" from the point of view of latency.


What is your ping latency? I get about 200ms from Greece, site doesn't load blazing fast but it's usable, nowhere near 1 second.


From Italy ping latency is 200ms, but page loading is veeery slow (1.3 seconds for home page, 3 seconds for comment pages if they are not too big).

So I tried form UK (linode server):

https://gist.github.com/820303

Ping latency from linode is 100 ms.

So my bad experience is clearly the combination of two things: HN is slow generating pages, taking something like one third of second to generate the home page. Add this to the latency and you are half a second for the home, and more for than 600 ms for comment pages.

This in the UK. In Italy there is apparently also some bandwidth issue, and since the markup is pretty verbose compared to the content, this could be dramatically improved too.

For instance Google.co.uk is served in 73 ms from the same place.

My blog takes 300 ms from my crappy ASDL (the same where hacker news takes 1.4 seconds).

There are huge margins for improvements.


Site seems to have slowed for me too now. Especially the front page. Gzip would probably help with the bandwidth issue but I don't think that's at fault.

I wonder if pg would accept a yc news written in python/redis :)


I can't propose a so biased implementation ;) A faster HN is already a big enough gift.


The problem is more Racket than Arc. By the way I feel Reddit more slower and with worse uptime than HN.


Reddit serves an order (or two?) of magnitude more traffic, and the fact another site is slow does not make HN faster ;)

Btw I've no precise idea about the software stack HN use, I'm just proposing a quality change, that is faster, the mean it is achieved is another (very interesting) topic.


A million page views in a week while Reddit does a billion in a month. So about ~250 orders of magnitude.


1,000,000,000,000,000,000,000,000 is 24 orders of magnitude more than 1. Reddit isn't near that traffic yet.


> Judging from the classic view, the stories on the frontpage are not much different from those we'd have had in the first year

I'll repeat (the gist of) my comment in a less crowded context (http://news.ycombinator.com/item?id=2157384):

First-year users will only vote on what they see, and if they don't consistently use the classic view - which I expect very few would - then they will most likely only vote on what everyone else has voted on. Thus, classic wouldn't be expected to be significantly different from the regular home page, and thus it cannot measure any decline.


thus it cannot measure any decline

That's not true unless those users are somehow compelled to upvote a constant number of stories, which they're not. I upvote a smaller proportion of stories on the frontpage than I used to.


I almost never vote on anything that's not already on the front page; mind you, I seldom vote up more than two or three stories per day.

Assuming that the number of first-year users' votes is noise compared to votes from newer users, and that first-year users vote mostly on items on the front page, then looking their votes alone should just give a different sort order to that front page. That's my intuition, without taking into account ranking algorithms using quadratic decay etc., but I'm not sure there's a good reason to think that would change things.


I'm not sure how accurate your assumption that new users are voters on the homepage is.

As a new user, most of my votes are on the "New" section rather than the homepage. I vote up the things that I think other people should (on New), rather than what everyone is going to see anyways because its up on the front page.

I do think that some sort of weight system for upvotes would be an interesting feature.


You're actually helping to make his point. You're a new user; you help determine what hits the front page by voting in the "New" section.

barrkel is extrapolating his own behavior as an older member: he tends to only consider front page items for upvotes.

Thus, new users determine what makes the front page, all users help determine the front page ordering, and "classic" view reflects how old users would order the front page.

The question is whether barrkel's voting patterns are an appropriate model for older members. Personally (as another older member), I visit "New" perhaps once a week, but when I do I'm much more likely to vote up a story that I find to have even a little bit of value. I only vote up front page stories that I find extraordinarily useful or interesting.

There is no reason to assume that the interaction model used by barrkel and me are representative of all older members, so while his argument is interesting, it's not very well supported.


Actually, there is evidence to support my model: the fact that classic front page is very similar to the normal front page, except with a different sorting order.

This evidence can be interpreted in (at least) two ways, (a) that most voting occurs based on the front page, or (b) that the front page topic themes, and by extrapolation user tastes, have remained consistent over time.

I (based on my own voting, and now yours too, but also the number of votes front-page articles get vs stuff that doesn't make it to the front page, evident from resubmission successes etc.) would guess that hypothesis (a) is more likely than hypothesis (b); but in any case, the fact that there's an alternative explanation for the similarity in the two front pages means that PG's hypothesis of (b) is on shaky ground.


pg, please consider including the year in the dates on the news page.


As a stopgap fix, the anchor links for the news items include the year. Obviously not ideal, but useful if you're really curious or confused about one of them.


I know this is a contentious question, but have you ever considered turning HN into a business with YC as the primary investor? The question is: could you advance it as a business without killing the magic and the splash benefit to Y Combinator. I think the answer to both is yes.


In many ways, I think that's already what it is. HN was created to, and still does, serve the purposes of YC. I don't know where the money comes from to keep it up and running, but I would imagine it's part of the YC operating budget.

In short, I'm sure the value it represents now is far greater than what one could garner from trying to directly monetize it. Just look at any of pgs posts - people upvote the bejeezus out of them: he can instantly and effortlessly access thousands of high quality eyeballs.

There are surely people that would pay egregious sums of money to have this kind of community in any vertical, and tech startups are pretty hot - I struggle to even fathom what could possibly constitute a fair price for jeopardizing that.

[edit: Google says it's spelled 'bejeezus']


It's also amazing leadgen for YC. That's partially because of PG's direct ability to get things like "apps are open" to the top spot, but also just because if you use HN regularly, you get a good impression of YC and are constantly reminded that it exists.


I suspect HN is a fantastic ambient resource for actual Y Combinator members. That is to say, I suspect most of them would have a harder time being productive and successful in a world without HN.


I agree HN is super-valuable for YC right now. But it's effectively part-time product of a pretty small team. Could a small full-time team do good things in terms of user joy, loyalty, acquisition, and even monetization? With PG as an advisor/owner, I can't imagine why not.


I haven't thought about it. We don't need or want to. But you probably could if you wanted to.


HN would be in the same space as Reddit at that point. Reddit seems to be doing well with YC's help. The audiences are different and clearly as we have seen there is enough space for the both of them. There is something about HN being what it is today without the motive of creating profits. The content and community is what drives traffic and I hope that's how it will always be.


The only possible way it could be done is if they start charging a subscription for the service. Anything else would pretty much kill it, and adding a subscription probably would too.


That's jumping to conclusions. I bet tons of marketers would pay mad money for their job postings and other things to be seen by all the high quality eyeballs on hacker news.


All the more reason not to try and monetize it.


HN became my favorite site when it launched, it's my favorite site now, and I expect it will be my favorite site four years from now.

Thanks for keeping it good.


Nomen est omen? :-)


Hopefully with the amount of hits and new users the site is getting, quality doesn't decline. HN has (In a matter of 4 days) taken the #1 spot on my "TechNews" bookmark folder, so as of now the article quality is great.

As a new user, I'm quite surprised with how civil and helpful everyone is.

Often times when you put a lot of extremely intelligent and strong-minded people in the same place, things can get heated really quickly. Granted, strong-minded people are usually not offended easily.


Thank you, Paul for running an amazing site. I think I get a lot of value out of it. I hope YComibnator does, too.


Congrats to the yc crew. It's no easy task to cultivate a community while keeping quality high.

As a side note, does anyone know what kind of hardware setup they are using to serve up HN?


I'd congratulate you but I'm not sure if that's the right thing. It is hopeful to see that the quality isn't significantly affected though.


a million pageviews isn't cool. you know what's cool? a billion pageviews

edit: can't help but presume my intention was lost on the downvoters. not all jokes are purposeless.


reddit apparently did a billion pageviews last month. I'd rather HN not turn into reddit.

While there are still some intellectual safe havens on reddit, the majority of the site is now dick jokes and pun threads. Not that reddit doesn't still contribute informative comments, just that the signal-to-noise ratio has definitely shifted over the years.

HN, by contrast, tends to be intellectual by default, with the odd tasteful joke or pun thrown in for good measure where appropriate.

I really like this place, and I'd hate for it to become ruined by an excess of new users.

(and yes, I know you were just making a TSN joke)


i was actually just curious as to why a million pageviews wouldn't be worthy of congratulations :)

though, now that we've shifted, i agree with you- but also have hope that pageviews are not the deciding factor of community downfall!


I don't really see a way that pageviews could significantly increase without a corresponding increase in uniques. The most obvious reason to see an increase in uniques is to have an increase in new users.

Onboarding new users into the HN culture is relatively easy when you're dealing with a steady but slow intake. I'll admit that I can be an ass on reddit and my highest karma posts have mostly been stupid jokes, but I take things serious on HN because HN presents itself as a serious place.

When you get a significant stream of new users, it can be harder to keep them all in line. If you get a bunch of new folks together voting up their own brand of inane crap, it'll start to pollute the front page. A polluted front page sets a bad tone for newer users, because they think that sort of thing is tolerated.

HN has systems in place to stop this of course, like the ability of some users to kill a post outright and the karmic threshold for downvoting, but there will likely come a time when that isn't enough anymore.

I agree that pageviews are generally a good thing, but HN isn't advertising-funded in the traditional sense (by that I only mean that HN does serve as an advertising platform for YC companies to an extent) so having high pageviews probably isn't high on PG's goals list. HN is pretty unique in that respect, and that's a distinct part of its charm.


A million page views isn't cool. You know what's cool? New material ;-)


Movie quotes have replaced puns as the lowest form of wit. Yes, we get it. You're quoting a movie. Congratulations.


Regarding the classic view, isn't there a bit of bias there? since people mostly only look at the front page, then the stories they vote up are chosen from ones already on the front page. So comparing what the front page would look like if only votes from old accounts counted doesn't say much since in all likelihood you'd see the same stories.

Maybe a better metric would be to track only the first few votes that stories get to see if the same ones would make it to the front page?


What I never understand about the "more exposure will dilute our membership quality" is that it's a social voting site. If you're going to share something with a community that has open registration, and gets exposure by sharing links to other networks (such as Digg and Reddit), this sort of thing is inevitable. I'm glad it's received the exposure it has because it's become such a great content generator. I don't really see the harm in this growth.


More voters means closer to average. It has to.

Quirky communities with strong personalities are more interesting.

(I'm still quite happy with HN though :)


PG, I think you should write the year when you write a "news news" post.

This site has been around for long enough that certain older posts on that page have some ambiguity.


" the stories on the frontpage are not much different from those we'd have had in the first year."

This is the real story. I noticed that about 4 months ago the nature of Hacker News seemed to veer out of its original emphasis and now it's back to what it used to be.


> 1 million page views on weekdays

That's impressive. Congrats.

So why is it still PR6 ? I guess the 136 Errors and 6 warning(s) from the HTML Validator give a big penalty.


What are your plans for monetization? Advertising to such a targeted audience could be very lucrative, and I'm sure lots of startups would be interested in advertising here! Currently, all we have is "organic" advertisement through interesting submissions and comments. What kind of revenue could one expect from 90K unique visitors and 1M page views on weekdays with an audience like HN's?


HN is an advertising site in some way, both for YC startups and even for all its users that have something good to propose. For instance Redis started to get some user base thanks to HN.


I have just one thing to say:

Please do not change the site.

Sincerely.


What, not even removing the superfluous font tags?


Our research indicates most users read the site as rendered by a browser, rather than by reading the html source.


Whilst your response is amusing it just strikes me as so opposite the ethos of the site as to be simultaneously depressing.

Like having a car that leaks oil all around the engine compartment and have the manufacturer explain that this is fine as most passengers don't look in there. Then finding that a gasket alteration could fix it but the manufacturer isn't bothered because, well what's a bit of oil [bandwidth] and mess [sloppy markup] when the car still runs.

I don't know the system that the site runs on but assuming that the presentation layer is abstracted from the base logic it should be easy to fix at least some basics (doctype, font tags, gzip, etags, expire headers) even if one didn't go the whole hog and rip out the tables.

I'm not at all suggesting to alter the visual appearance incidentally I like the minimalist design.


A more accurate comparison would be using a compiler rather than writing machine language code by hand.

When computers first appeared, everyone wrote code in machine language. People thought that's what programs were. When the first compilers appeared, I'm sure a lot of people were grossed out by how ugly and inefficient their output was compared to the hand-written machine language code they were used to. But people who thought that were solving the wrong problem. What matters is how the code looks to the programmer writing it, not how it looks to the machine executing it.

People who think HTML should be elegant are making the same mistake. HTML is object code. What matters is the representation the programmer sees, which in this case is the source of HN, which is quite concise.


Your analogy appears to break down, and forgive me if my analysis is poor, when one considers the bandwidth.

Concerning oneself with HTML is different to fussing over optimising assembler in a regular computer because the mark-up is being pushed along many pipelines which are handling other data traffic, limiting the speed of data transfer limits processes involved in the presentation of the page.

I guess it's like optimising machine code to be run on a massively parallel computer in which real-time operation is desired.

In your analogy the compiler isn't [generally] producing the same assembly as if the code were hand-optimised. Clearly there's a play-off but if you could cut your transport across a data bus significantly with a couple of cheap and simple optimisations (either direct to the compiler or to it's output) then it's quite likely that you would, isn't it?

The main thing for me anyway was the surprise of finding this here. To add to my other poor analogy it's like going round to dinner at a famous chef's house and finding that they ordered the food in and just reheated it; just not what you expect. Sure you get a hot meal at the end of it, sure it might be nice, it's just you expected more.


When HN goes to 10 million page views, HN will no longer be HN. It will be another digg.


I don't know why people are down-voting you. Some people might not like it, but you simply cannot maintain the quality this site has enjoyed over the past several years as we continue to take on a mess of new users. Most of the new users simply don't fall into the hacker/entrepreneur category (as past polls have clearly indicated). They're just people coming from Reddit or wherever who want a "better" experience. Most of the articles on the 'newest' page are about crap like WebOS and #superbowl and politics and you name it. HN will in fact go the way of all previous communities as it takes on hoards of people who don't care about the original intent of this site.


I think that PG will be able to take the quality as high as today, for a very simple reason, he does not reason in terms of pageviews for this site, and will be willing to do all the needed changes, even if this will upset the "wrong" part of the user base, to take quality high. So at max the new rules will have the effect of reducing again the user base to the original one. Otherwise if this does not happen, there will be a next HN site somewhere and we'll switch again. slashdot -> programming.reddit.com -> HN -> ?


I don't think there is anything he can do to prevent what some users consider the decline of HN. He's just not that heavy-handed. He could easily go in a kill every shit article day in and day out, but it's not in his interest to do so. As HN is a vehicle for exposure of YC startups the continued increase of users on HN is only to his benefit. That isn't to say that this is why he started the site, but it's certainly a huge benefit to him. You simply don't #1 every piece of YC startup news on Reddit. :)

I agree with you that eventually something else will come out and we will flock there to get away from HN once it gets to the point where it's mostly fluff articles about Apple and politics.


Yes it makes sense that even in the case of a not real 'startup' as HN it still is too valuable to perform drastic actions. Thanks for sharing your point of view.


This would imply that when dig had 1 million page views was like HN, that is not the case. So I think your claim is flawed ;)


Digg used to be a great place for programming-related news. It was the place for a good programming news when it came out. Why? Because its community was populated by programmers. Over the years the programming section on Digg became desolate and ignored as the programmers sought refuge from the lolcats and poor discussion. A similar thing is happening to Reddit as people want to move somewhere that isn't dominated by discussion comprised of witty comments and mostly high-level tech news.


Strange, I spent some time in dig back when it was created and I never saw this level of quality like in HN. Was much more slashdot-alike.


I was also an old-school Digg user, and you're right, it never compared to what HN is. However, the way Digg declined (and how all communities eventually decline) as a result of an influx of new users with disparate interests is the same.


The beginning of the end.


Aren't we supposed to work hard, LOL?




Applications are open for YC Winter 2018

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact

Search: