
IE has 1% of market share ... on HN: "Show HN" aftermath - zeratul
Here is what happened when I published "HN pickup ratio" web app on HN:<p>- 10,000 visitors from 100 countries in 3 days (good sales pitch, isn't it?)<p>- 95% of traffic comes from robots (for first few days, after that it goes down to 60%) - this is based on disproportion between number HTML and CSS serves (should I also count here the 405 error?)<p>- 57% visitors are from U.S. (of that 30% were from California)<p>- 58% visitors used chrome browser (IE is just 1%!!!)<p>- 52% visitors used apple device (where 11% are mobile Apple devices)<p>- 30% visitors used laptops (based on screen resolution 1440x900 and 1280x800)<p>Here is the raw data (big png poster):<p>http://hnpickup.appspot.com/hnpickup_web_app_statistics_snapshot.png
======
jcr
Your calculation of claimed percentage of robots is deeply flawed. Not only do
web browsers aggressively cache CSS files, but internet service providers also
aggressively cache them.

Though I'm a rare exception, my web browser is configured to ignore CSS
completely, including both in-page styles and all additional loadable files. A
configuration like this is very rare because it's difficult to do and the
result often breaks sites. The only place where it is common is for people
with visual impairments.

A well behaved bot will (typically) announce itself by its User-Agent string.
A securely configured browser (IMO) will never send a User-Agent string or
Referrer, and will reject cookies as well as ignore javascript --Needless to
say, this breaks a whole lot of sites, but suffice to say, HN _does_ have a
higher percentage of people wearing tin-foil hats than the regular populace.

~~~
zeratul
Same goes for JS files. There is only on page that I'm serving. I also know
that there is 10% of visits are returning "customers". So there should be
around 90 css/js files served per 100 html serves but it's not.

Few minutes after releasing the app on HN I saw a swarm of "...bot" user-
agents in my logs and lots of 405 errors also for the "...bot" user-agents.
That made me think that the disproportion might be cause by robots.

~~~
jcr
After looking carefully at the image you linked to, the first 405 seems to
show your web server is configured to disallow HEAD requests (and probably
others). The failed HEAD requests by " _bot_ " User-Agents (along with what
looks like some bots using forged User-Agents) is interesting since a HEAD
request, if successful, does not actually load the page data and should not
count in your page load stats. In other words, the bug is in the counting of
pages loaded.

The tough questions is, why are you getting hammered by bots?

~~~
zeratul
I'm not sure. It seems that the route is:

    
    
      HN -> Twitter -> A swarm of bots sitting on Twitter
    

When I re-posted my app on Sunday to get extra stats I saw the same thing. The
Terminator Judgment Day happened but just on Twitter.

------
epaga
"768x1024 - is your laptop sideways" - you could call it that. It's iPad in
portrait mode.

~~~
zeratul
Thanks. There are many numbers in those charts that I don't understand. I was
hoping that HN community will help:

Why there are so many 405 (method not allowed) errors? I get this from, e.g.,
Twitterbot.

Am I right? If someone GETs HTML and doesn't CSS then it's a robot? Why so
many robots (web crawlers)?

~~~
bri3d
How were your cache control headers set up?

Most browsers cache CSS files very aggressively (not even requesting a HEAD
for them) once they've been loaded once - hence the common use of query
strings or asset paths for CSS to ensure users get new versions on new
deploys.

------
sciurus
1440x900 resolution is used on desktop monitors, too.

~~~
Groxx
Was just about to echo this. The last two monitors I've bought had this as the
native resolution. (yeah, it's been a while)

------
username3
Clickable link:
[http://hnpickup.appspot.com/hnpickup_web_app_statistics_snap...](http://hnpickup.appspot.com/hnpickup_web_app_statistics_snapshot.png)

------
petercooper
I'm surprised that only 3.5% appear to be using 27" iMacs or 27" Apple
displays. I thought they were pretty ubiquitous at startups now (though less
popular than MBPs, of course).

~~~
oldstrangers
I always imagined the Dell Ultrasharps were the defacto startup montior.

------
dholowiski
I had a similar amount of traffic from 2 HN posts. My IE percentage was 3.12%
and chrome was 55.08% - pretty close to your numbers. Obviously, HN's browser
stats will be way different than the general population, still very
interestng. My analysis: <http://onepix.me/blogs/10-HN-Traffic-
Analysis-2-Posts-1-Week>

------
anujkk
So, 41.76% users used Mac but only 14.84% used safari. 29.94% were on windows
but only 1.18% used IE. Interesting.

~~~
wavephorm
I imagine many HN visitors develop websites. Developing a website using IE or
Safari would be a miserable experience.

~~~
kristofferR
Why would it be a bad experience to develop on Safari? It has extension
support, a debug menu (which lets you change user agent) and the same web
inspector as Chrome.

~~~
syncsynchalt
Agreed, anyone used to firebug will find Safari's Inspector very familiar.
Even more so if they're used to Chrome's Inspector, which is not identical but
seems like it shares 90% or more.

------
oz
Interesting stats! Thanks for sharing.

Something tells me that the majority of those using IE are those forced to at
Microsoft-only shops, like I was until recently.

------
ma2rten
"For me the biggest cost was getting time series data out of the DB"

Is there some reason you can not use memcache or even instance memory for the
most part ?

------
rokhayakebe
I would like to know who those 100 users are.

~~~
fauldsh
WP7 is a fair reason in my biased opinion

~~~
freehunter
I will come out as one of the 100 WP7 visitors.

------
icefox
It would be nice if the viewport size was included in stats like this as that
is more important than screen size.

~~~
zeratul
It's not a standard feature for the Google Analytics tool but I found code
that adds viewport logging:

[http://beantin.se/post/354097397/track-viewport-google-
analy...](http://beantin.se/post/354097397/track-viewport-google-analytics)

[http://wishfulcode.com/2011/11/02/measuring-viewport-size-
wi...](http://wishfulcode.com/2011/11/02/measuring-viewport-size-with-google-
analytics/)

------
ghc
Interesting to note that normalized for population, New York seriously lags
behind CA and MA.

------
purchas
the 1% is probably people currently working in corporates as well

