Hacker News new | past | comments | ask | show | jobs | submit login
IE has 1% of market share ... on HN: "Show HN" aftermath
50 points by zeratul on Nov 22, 2011 | hide | past | web | favorite | 33 comments
Here is what happened when I published "HN pickup ratio" web app on HN:

- 10,000 visitors from 100 countries in 3 days (good sales pitch, isn't it?)

- 95% of traffic comes from robots (for first few days, after that it goes down to 60%) - this is based on disproportion between number HTML and CSS serves (should I also count here the 405 error?)

- 57% visitors are from U.S. (of that 30% were from California)

- 58% visitors used chrome browser (IE is just 1%!!!)

- 52% visitors used apple device (where 11% are mobile Apple devices)

- 30% visitors used laptops (based on screen resolution 1440x900 and 1280x800)

Here is the raw data (big png poster):


Your calculation of claimed percentage of robots is deeply flawed. Not only do web browsers aggressively cache CSS files, but internet service providers also aggressively cache them.

Though I'm a rare exception, my web browser is configured to ignore CSS completely, including both in-page styles and all additional loadable files. A configuration like this is very rare because it's difficult to do and the result often breaks sites. The only place where it is common is for people with visual impairments.

A well behaved bot will (typically) announce itself by its User-Agent string. A securely configured browser (IMO) will never send a User-Agent string or Referrer, and will reject cookies as well as ignore javascript --Needless to say, this breaks a whole lot of sites, but suffice to say, HN does have a higher percentage of people wearing tin-foil hats than the regular populace.

Same goes for JS files. There is only on page that I'm serving. I also know that there is 10% of visits are returning "customers". So there should be around 90 css/js files served per 100 html serves but it's not.

Few minutes after releasing the app on HN I saw a swarm of "...bot" user-agents in my logs and lots of 405 errors also for the "...bot" user-agents. That made me think that the disproportion might be cause by robots.

After looking carefully at the image you linked to, the first 405 seems to show your web server is configured to disallow HEAD requests (and probably others). The failed HEAD requests by "bot" User-Agents (along with what looks like some bots using forged User-Agents) is interesting since a HEAD request, if successful, does not actually load the page data and should not count in your page load stats. In other words, the bug is in the counting of pages loaded.

The tough questions is, why are you getting hammered by bots?

I'm not sure. It seems that the route is:

  HN -> Twitter -> A swarm of bots sitting on Twitter
When I re-posted my app on Sunday to get extra stats I saw the same thing. The Terminator Judgment Day happened but just on Twitter.

"768x1024 - is your laptop sideways" - you could call it that. It's iPad in portrait mode.

Thanks. There are many numbers in those charts that I don't understand. I was hoping that HN community will help:

Why there are so many 405 (method not allowed) errors? I get this from, e.g., Twitterbot.

Am I right? If someone GETs HTML and doesn't CSS then it's a robot? Why so many robots (web crawlers)?

How were your cache control headers set up?

Most browsers cache CSS files very aggressively (not even requesting a HEAD for them) once they've been loaded once - hence the common use of query strings or asset paths for CSS to ensure users get new versions on new deploys.

Could some of those people have cached css files from a previous page view?

1440x900 resolution is used on desktop monitors, too.

Was just about to echo this. The last two monitors I've bought had this as the native resolution. (yeah, it's been a while)

I'm surprised that only 3.5% appear to be using 27" iMacs or 27" Apple displays. I thought they were pretty ubiquitous at startups now (though less popular than MBPs, of course).

I always imagined the Dell Ultrasharps were the defacto startup montior.

I had a similar amount of traffic from 2 HN posts. My IE percentage was 3.12% and chrome was 55.08% - pretty close to your numbers. Obviously, HN's browser stats will be way different than the general population, still very interestng. My analysis: http://onepix.me/blogs/10-HN-Traffic-Analysis-2-Posts-1-Week

So, 41.76% users used Mac but only 14.84% used safari. 29.94% were on windows but only 1.18% used IE. Interesting.

I imagine many HN visitors develop websites. Developing a website using IE or Safari would be a miserable experience.

I don't understand the Safari bit. I develop webapps, and I use Safari. The developer tool is top-notch. I don't see where the "miserable" bit comes from.

Why would it be a bad experience to develop on Safari? It has extension support, a debug menu (which lets you change user agent) and the same web inspector as Chrome.

Agreed, anyone used to firebug will find Safari's Inspector very familiar. Even more so if they're used to Chrome's Inspector, which is not identical but seems like it shares 90% or more.

It must have improved substantially. I remember having trouble finding out how to even activate a console, and then being blown back by how far behind Firefox+Firebug it was.

Interesting stats! Thanks for sharing.

Something tells me that the majority of those using IE are those forced to at Microsoft-only shops, like I was until recently.

"For me the biggest cost was getting time series data out of the DB"

Is there some reason you can not use memcache or even instance memory for the most part ?

I would like to know who those 100 users are.

Corporate GPOs restricting browser usage. My company JUST upgraded to IE7, though as I am in IT, I can use Chrome

WP7 is a fair reason in my biased opinion

I will come out as one of the 100 WP7 visitors.

but windows phone only has 0.11% of the visits, so .net devs maybe?

It would be nice if the viewport size was included in stats like this as that is more important than screen size.

It's not a standard feature for the Google Analytics tool but I found code that adds viewport logging:



Agreed. I have a 2560px horizontal resolution but my browser is only around 1200px.

I ran some viewport stats on my own site a year back and found that for 95%+ the range was 980-1300px. I should run it again and come up with some more concrete numbers..

Exactly. "Your viewport width is 1010" and many sites have problem with it (as in show horizontal scrollbar if I don't use fit to width). I would use more space for browser but as I have only 1680px monitor, the rest would be unusable.

Interesting to note that normalized for population, New York seriously lags behind CA and MA.

the 1% is probably people currently working in corporates as well

Applications are open for YC Summer 2020

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact