
Soon you won't find the Language Benchmarks Game with any search engine - gus_massa
http://benchmarksgame.alioth.debian.org/soon-you-wont-find-the-benchmarks-game-with-google.php
======
kylemaxwell
If only there were other places to locate web sites on the Internet...

------
samuel1604
why don't you put it somewhere else?

~~~
Supermighty
That, or give the benchmark game a separate config to allow googlebot
greater/faster access.

------
rietta
I would change hosting providers.

------
eli
Block everything in robots.txt except the homepage, instead ?

~~~
thirsteh
The solution they chose seems to be the absolute silliest way to solve the
problem.

------
secure
I hope this throttling includes the VCS web frontends. It used to be that the
hg (mercurial) and some other VCS (I forgot which one it was) web frontend
were crawled mercilessly by someone (unidentified so far AFAIK). That used to
put so much load on alioth that the git frontend would refuse to work, which
was very annoying for me :).

That being said, I hope that either

a) the author of the post is exaggerating or

b) the alioth admins will improve the throttling to better target abusive
users and not legitimate websites such as this one

~~~
igouy
a) I was not exaggerating

b) All the upstream Fusion Forge generated pages, for all 1,000 projects, were
being served without any HTML robots meta tags -- which, of course, is asking
for

<meta name="robots" content="index,follow" />

and duplicate pages were being served from 4 subdomains, and the 1,000
projects also had their own web pages without any HTML robots meta tags...

In other words, googlebot was visiting alioth because alioth was repeatedly
asking googlebot to visit.

Luckily I found a way for the alioth admins to take some control of the Fusion
Forge generated pages, that did not require changes to Fusion Forge.

Header set X-Robots-Tag "noindex, nofollow"

[https://developers.google.com/webmasters/control-crawl-
index...](https://developers.google.com/webmasters/control-crawl-
index/docs/robots_meta_tag)

The alioth admins were willing to ease throttling and see if asking robots not
to index and not to crawl Fusion Forge pages reduced the server load.

Currently server errors for benchmarksgame subdomain are below 70% and
robots.txt below 5% and the benchmarks game pages are indexed.

------
300bps
Perhaps they could invent a benchmarking tool that would allow them to write
more efficient code that would utilize fewer resources on the server.

~~~
VLM
"There's been no suggestion that the benchmarks game website was the primary
cause of that server load. It's just the result of the default configuration
they use to provide dynamic content for 1,000 projects."

If you wrote it, you'd have to get 999 other projects to bother to implement
its results.

What I would like to see rewritten is a (obvious) JVM view. So that's all very
interesting that a n-body simulator in Clojure takes 60 times the memory and 4
times the exec time of a C version or about half a meg of ram per body in the
n-body problem (uh, what?) What would really interest me is at a Debian OS
level what is the delta when I move an identical benchmark from a Debian box
running -i386 to -amd64 or at least for the interpreted JVM family of
languages what happens to speed and memory when I move from vers 6 to vers 7
or how about alternative JVMs or whatever. More of an OS tweaking focus than a
language focus.

There is a startup lesson in this situation that anything shared can be a
focus of pain, also any single point of failure can be a focus of pain. Not
exactly new observations, but, still true. If instead of a benchmark site,
this project was noobs trying to leverage the synergy of tweeting crowdsourced
dog food delivery over the internet, they'd be dead meat once they disappear
from search engines.

~~~
igouy
>>What would really interest me ... vers 6 to vers 7<<

All the measurements are in CVS so you actually could get at measurements for
different versions.

>>What would really interest me ... alternative JVMs<<

Please take the program source code and the measurement scripts and publish
your own measurements --

[http://benchmarksgame.alioth.debian.org/play.php#languagex](http://benchmarksgame.alioth.debian.org/play.php#languagex)

>>what is the delta when I move an identical benchmark from a Debian box
running -i386 to -amd64<<

Just open 2 web browsers and look ;-)

[http://benchmarksgame.alioth.debian.org/u64q/measurements.ph...](http://benchmarksgame.alioth.debian.org/u64q/measurements.php?lang=clojure)

[http://benchmarksgame.alioth.debian.org/u32q/measurements.ph...](http://benchmarksgame.alioth.debian.org/u32q/measurements.php?lang=clojure)

------
judk
So, move to a new host?

------
Jayschwa
It's still the top result for "language shootout", despite the name being
changed year(s) ago.

~~~
igouy
The website was renamed back on 20th April 2007.

[http://benchmarksgame.alioth.debian.org/dont-jump-to-
conclus...](http://benchmarksgame.alioth.debian.org/dont-jump-to-
conclusions.php#history)

Changing the website banners and page titles and ... proved to be too subtle,
so a new project was created just so the website could have a benchmarksgame
sub-domain.

The URLs were changed December 2012 and the switchover took a few days --

    
    
        16th 0 visits vs 1,048
        17th 265 visits vs 1,026
        18th 1,262 visits vs 136
        19th 1,333 visits vs 8
    

The old sub-domain scripts were changed to redirect --

    
    
        header("HTTP/1.1 301 Moved Permanently");
        header("Location:http://benchmarksgame.alioth.debian.org/");

