
Compare & vote for search results of Bing, Google and Yahoo in a blind test  - vaksel
http://blindsearch.fejus.com/
======
carbon8
1\. This was created by a Microsoft employee (a "Microsoft Developer
Evangelist"). Maybe this doesn't matter to you, but considering the $100m
marketing budget and the widespread use of astroturfing, it seems a little
dishonest to me (certainly treading in a grey area) that there isn't full
disclosure. YMMV

2\. As someone who cares deeply about research methods, I'd argue that there
is little to no value in superficial, subjective and contextless comparisons
like this.

 _Edit: He has now added a statement that he works for microsoft. It was not
there before I posted this comment_

~~~
vaksel
Well if you question the methods, you can always open bing/google/yahoo and
check to make sure the column labels are correct

~~~
carbon8
I didn't say, hint or imply that it was listing inaccurate results. I stated
that it's superficial, subjective and devoid of context. Frankly, it means
nothing.

For example, someone searches for "bongo". The forth results are a radio
station, a page about the animal and an email/calendaring system. Given those
results, someone votes for one. Why? We don't know. They might barely now. One
thing we do know: it was a one-off search anyway, not something that was part
of someone's normal workflow.

~~~
vaksel
the context seems pretty obvious, to let people compare the different search
engines without being biased that they are using a search engine other than
Google

~~~
carbon8
You are misusing the word "context" - <http://www.answers.com/context> : _1\.
The part of a text or statement that surrounds a particular word or passage
and determines its meaning._ _2\. The circumstances in which an event occurs;
a setting._

The searches here are without context, the rating of results are without
context. All we know is that on arbitrary, one-off searches, a set of people
who may or may not be representative of anything clicked the button above a
certain set of results for some unknown reason.

~~~
anigbrowl
And your basis for asserting 'arbitrary and one-off searches' is...?

~~~
carbon8
Because it's a obscure micro app for doing one-off comparison searches that's
making the rounds on social sites.

~~~
anigbrowl
Your argument seems circular, to say the least.

------
pg
I tried 5 searches about subjects I knew well. Results: Google 4, Yahoo 1,
Bing 0. But in every case I had to look closely at the results to decide which
I preferred. None of them returned obviously bad results for any of the
queries.

~~~
socratees
This application is weird in cases where spell correction comes in to place.
Just search for mike tieson, birack obam, pal graham hacker news and you can
see the difference between google results & what's shown on the blindsearch
application. There's a whole lot of difference.

------
amichail
The methodology for this is flawed because people will tend to search for
things they have searched for before on Google and then pick Google as the
best because they find that ranking more familiar.

~~~
socratees
I don't think its flawed. How can you say which results belong to Google or
any other search engine?

~~~
bemmu
For example I searched for "japan", and expected to find "japan-guide.com".
Now I realize the only reason I even know about that site is because it was in
my Google results before, so now I grew to consider it the "correct" answer.

------
dangoldin
Looking at the source code you can find which site is used for which column
when "engines" is defined: var engines=new Array("YHO","MSF","GOO");

Voting will just reveal that information.

Although if you wanted to game the site you could already do it in different
ways.

------
jsrfded
The vote counter is going up like crazy now, and Yahoo is now trumping both
Google and Bing. Yahoo was in 3rd place earlier today when I looked. Seems
like someone is hitting it with a robot to goose Yahoo's numbers. So much for
the data.

------
viraptor
I'm actually surprised - my guess would be something like <10% for yahoo and
almost equal results for google and bing.

On the other hand the test can't take one important feature into account -
profiled results. After 2-3 searches for anything programming related on
someone else's browser I feel there's something wrong with the results... yep
- I'm not logged in. Even if it doesn't make a huge difference, it is
noticeable. I'd really like to check the stats while being logged in into my
google account.

------
fizx
My score: Google 3, Yahoo!! 3, Bing 1.

The test is to a large degree unrealistic. Do a search for "sf restaurants".
Google and Yahoo!! (in a recent upgrade!) both have extremely useful
OneBox/Shortcut results that aren't captured in the comparision.

------
dsil
Cool site, but not a real comparison. I know google does a lot of
personalization, based on being logged in or your cookie, but it can't do that
on this. I'm assuming the other engines do something along those lines as
well.

~~~
sachinag
For someone who's looking to SEO their site, this tool is _exceptionally_
useful just for that reason.

------
randomwalker
As others have noted, the site is being gamed. Someone is stuffing votes for
Yahoo. Rather unfortunate, because otherwise it was interesting and useful.

------
froo
I did 10 searches about subjects I was passionate about. Results

    
    
      Google: 5
      Bing: 5
      Yahoo: 0
    

Having read what other people were writing in this thread I thought I would
also try and rank the results in order, which would be my first choice, which
would be my second choice - so for second choice results, here they are.

    
    
      Google: 3
      Bing: 5
      Yahoo: 2
    

So while the Google/Bing split for first place results were interesting, the
second place results were more interesting (to me at least).

I'm going to keep throwing in results later today as I think up more topics to
increase the sample size a bit to see which is a bit better, as I think it
will trend towards a more even split or even a Google lead, but definitely
promising results for Bing.

EDIT - Update, I just checked the results again, seems like it was just hacked
in Yahoo's favour.

Around an hour ago (when I first published this comment) I noticed it was
about 26.5k votes cast with the split being Google 39%, Bing 31%, Yahoo 30%.

Now having a look the split is Google 31%, Bing 25% and Yahoo 44% with 40k
votes cast... I suspect shenanigans.

EDIT #2 - 2 minutes later, split is Google 30%, Bing 24%, Yahoo 46% with 41.5k
votes cast, so looks like the fixing is in process. Wonder which Yahoo
engineer is doing this...

EDIT #3 - 6 minutes after edit 2, Total is 45k, splits are Google 29%, Bing
23%, Yahoo 49%... Any insight that could be gleamed from this rough poll is
now officially tainted, nothing more to see here.

------
IsaacL
Tried three tests: my own name, "Clojure tutorials" (something I've been
looking for recently), and ""ORG directives in assembly code" (something I had
trouble finding recently). Scores - Google 1, Yahoo 1, Bing 1. Not a very
large sample space, to be sure, but still it was very hard to tell the
difference, and by the stats, a lot of other people have had similar reactions
(with a slight lead for Google).

Maybe this means we should just stick with the company we like the best?
Google it is then ;)

------
davidalln
Search itself is simply too hard to judge. Since many of the results in the
top 10 (at least in my test searches) were the same but perhaps jumbled a bit,
I personally think it doesn't matter that much.

Day to day I use search for two things: very general information and looking
up code errors. Code error searches will pull up the same mailing lists and
forums no matter which search engine you use, and a general information search
will either pull up an official website or a Wikipedia page on any search
engine. 9 times out of 10 that's sufficient.

There's no superficial reason to why I use Google over Bing or Yahoo!, just
simply that I've always used it and I'm always satisfied with it. Yes, I also
like its interface the most and non-search related Google applications are
generally amazing, but I wouldn't be any less productive if suddenly I
switched to Bing for all my search.

(Although I might simply just be biased towards Google... they did just give
me a free phone!)

I think the 40/30/30 results that are currently posted on there show that. For
the most part it's relatively even, and honestly Google is probably winning
just because some die hards are bent on making sure their favorite wins.

------
socratees
He also needs to have a way to mark the worst result out of 3.

------
jsz0
In my personal tests the results are almost identical but the order of results
may vary by a few spots. The most relevant searches are appearing in the first
page so the order isn't all that important IMO. If anything this proves to me
that accuracy is no longer the big deciding factor in search for most people.
It's all about the platform -- mail, IM, maps, blogs, photos, etc. This
explains why Google has stayed strong since they've made the biggest
investment in a complete, cohesive, platform.

------
mortenoffline
I've been using the WebMynd plugin since they posted here a while ago. Been
searching for normal stuff + django development. Yet too see a better top 3
result from Bing. The ranking is a lot worse, might be that they need a bit of
time for tuning. To me it seems to be something wrong with tokenization / word
distance weighting.

------
sachinag
This is about as useful as Yuil was - i.e. very. Sure, it turns out to be a
little self-serving for the person's employer, but you have to acknowledge the
brilliance of the effort and the fact that the results themselves appear to be
on the up-and-up.

------
drp
This is great, but I felt like my mental comparison weighted each individual
result more equally than I normally would in a routine search. Another
interesting comparison would be to show only the top result, top 3, etc. from
each search engine.

------
jpwagner
I use google almost exclusively.

I only used Bing as a test on the day of its launch.

That said, when I chose based on the results: Bing 70%, Google 30%, Yahoo 0%.

Wow...

------
screwperman
Seems like I've finally found a way to Bing for "sex" in India.

~~~
froo
_Seems like I've finally found a way to Bing for "sex" in India._

Actually I found out you can bypass the filter by adding in a generic stop
word at the end which the search engine should ignore anyway (I use the word
"the") and it will search for whatever phrase you want just fine.

Discovered this about 8 hours ago -
<http://news.ycombinator.com/item?id=645812>

------
anirudh
Yahoo seems to be rising incredibly fast. Its greater than 50% now.

------
grandalf
which do you choose?

<http://blindsearch.fejus.com/?q=sex+tape>

~~~
grandalf
uh it's interesting b/c only one search engine ranks wikipedia first!

------
TweedHeads
I tried camino, jetpack, svg, xul and never voted for bing.

As long as MS tries to move down relevant links to competing technology their
search engine is useless.

