Hacker Newsnew | comments | show | ask | jobs | submitlogin
Bing vs. Google (bingiton.com)
100 points by pykello 676 days ago | comments


ajays 676 days ago | link

Google 5-0

Google's strength is not in searching for head or torso terms like "facebook" or "2013 calendar". It is in searching out tail terms; those obscure terms which might include a typo or two, and the phrasing may not be correct.

Unfortunately (and I don't have much time to explain this) one simply _can not_ judge a search engine's comparative performance using random users. And the Bing guys are falling into that trap.

EDIT: To expand more on what I wanted to write.

// DISCLAIMER: This is my personal view, and has nothing to do with my employer.

There are 3 large (fuzzy) classes of queries on search engines: head, torso and tail. The head queries are small in number but large in volume: queries like "Facebook", "Google", "Yahoo", etc. (yes, people do 'search' for these on search engines. Just a handful of such queries may make up 10% of the entire search engine traffic, which would be 100M queries per day (or so) on Google. It is depressing to see the numbers). In general, any search engine worth its salt will get the head queries right.

The torso is basically deciles 2-6 or so (there's no hard and fast definition). Here you'd have most musicians, actors, popular restaurants, etc. Bing and Google both would do a mostly adequate job here, for the first result. But: if you want to explore the concept a little, Bing goes astray and Google does not. For example, search for "idle hand tattoo" on http://bingiton.com/ . The first result is the same, to my local tattoo parlor. But the second result on Bing is from Mesa, AZ; while Google sticks with the right result and offers up other, related sites (Yelp, etc.). Google knows that "Hand" in the title is not the same as "Hands", and gives me not only the store, but also their Yelp, FB and Twitter pages; in other words: Google _understands_ your query better.

And finally, the tail. Here we have obscure error codes, weird technical terms, etc.; most of which are seen maybe once or twice a day. These constitute the bottom ~30% of the query stream. This is where Google really shines. It is very rare indeed to look for an obscure term and find that Bing does a better job than Google (I have yet to find an example).

So, as a user, I have a choice of 2 search engines; and one of them I know will do a better job on obscure terms. Guess which one will I pick?

When Bing claims that "most people prefer Bing in our user studies" (or something similar), their conclusion is flawed because the study itself is flawed. You can't sit a person down in front of a computer and ask him to evaluate your performance on tail queries! Where are those tail queries coming from? If they are provided by Bing ("here's an obscure term, search for it and tell us what you think"), then the user has no way to evaluate how relevant the results are, because, by definition, it's an obscure term! On the other hand, if you ask the user to come up with an obscure term, it will most likely be a term they are familiar with and they'll already know the answer, so they won't really hunt for information. So in my "idle hand tattoo" example, they'll see the top result and claim satisfaction; when, in real life, I would like to see the Yelp reviews, maps, etc. for the place.

If they (Bing) really want to compare how well they're doing, here's a suggestion: setup a search engine (like bingiton, but with only 1 pane of results, and neutral branding), and make it the default for a large pool of users (with their permission). Randomly, switch the backend to Bing or Google. Then, monitor the heck out of what the users do (all with their permission): how often do they click on the first result, how often do they re-formulate queries, the time to the first click, how often do they quickly come back to the results page from a bad click, etc. etc. etc.

-----

thedufer 676 days ago | link

And this is why I use Google. I rarely need to search for "facebook"; I do have a brain. When I'm searching, its more often than not for something I can describe but don't know the name of, and Google blows Bing out of the water on these. Try describing a movie to see what I'm talking about.

-----

tallanvor 675 days ago | link

It doesn't surprise me at all that Google tends to handle the long tail questions better. --They're much larger and have had many more years to collect data, build synonym dictionaries, etc.

When I tried to come up with obscure queries, Google does better at the technical queries, and Bing is doing better at the non-technical queries. But the technical queries usually weren't that far off.

-----

pooriaazimi 675 days ago | link

Yes, they're much larger, bot not much older. Microsoft was in search business years before Bing, and I also think they now have Yahoo!'s dataset too.

They're just better in search. I generally dislike Google (because I don't like their business model, which is advertising [I don't care for privacy much, just hate ads]), but nothing is as good as them in search, nor could be (for the foreseeable future).

-----

wendall911 675 days ago | link

Absolutely agreed. I was also 5-0 Google. Examples: "mdc date" "python math" Google is the clear winner here, as well as many other scenarios where more specific searches yield incredibly higher quality results.

-----

taxonomyman 675 days ago | link

Give http://www.millionshort.com a try for long tail. Set filter to exlude top million sites.

-----

blhack 675 days ago | link

I have actually found myself using bing lately only because it doesn't try to confuse me into landing on google+.

Seriously, guys; this is a problem. I'm searching for a coffee shop or restaurant or something, and I want to link my friends to the map.

I --DO NOT-- want to go to google+. In fact, I don't really want to google+ ever. For anything. Ever.

Seriously I feel like I'm navigating a maze of accidental google+ links every time I use google anymore. It's really frustrating. Bing at least seems to be just...a search engine.

In fact, google hiccuped all over my 8 years old gmail account, deleting my inbox (yay! Thanks for that guys, oh, and it's just lost, too bad for me!) two days ago.

The upside to this? It somehow also unsubbed me from google+!

-----

gruseom 675 days ago | link

I find the Google+ spam annoying too, but my pet peeve is the redirects Google put on search results so they can tell which ones you click on. I hate this, first because it feels invasive and second because it noticeably slows things down. Most of the time it's not that bad, but every now and then it adds a second, or 2 seconds, or 5. I got so mad I astonished myself by switching to Bing for a while. And indeed the Bing experience was much faster. I've switched back to Google for now, mostly out of habit. But I've gradually gone from being a big fan of theirs to a grudging captive. Something could jolt me into a different orbit.

-----

AncientPC 675 days ago | link

There are extensions to get rid of the redirect: https://chrome.google.com/webstore/detail/dohbiijnjeiejifbgf...

-----

gruseom 674 days ago | link

Thanks! I'm going to give that a try.

Edit: holy shit it works! You just made my internet faster. How did I, or rather everyone, not know about this?

-----

Evbn 675 days ago | link

The weird thing is they could use JS to fire an analytics event to avoid the redirect.

-----

gruseom 674 days ago | link

That's what they used to do. This being Google, I'm sure they had a good reason for switching, especially to something so corrosive of user goodwill. I've often wondered what that reason is. Anybody know?

-----

wutbrodo 675 days ago | link

Other than the logo, and the bar at the top that says "Join Google+" (which of course, may be a slight annoyance to some people), what difference do you find between the old Maps/Places page for a restaurant and the new "Google+ Local" restaurant page that you're afraid of being confused into landing on? I just checked a local pizza joint while not signed in to Google and those two things are the only things that aren't reviews and information about the place (i.e. the only differences of substance between the new pages and the old Place pages).

-----

blhack 675 days ago | link

This entire page is just...cluttered.

https://plus.google.com/107551444336725915386/about?gl=us...

It reminds me a lot of myspace, actually.

Okay, instead of whining about this, I'm going to look into a maps API client :)

-----

wutbrodo 669 days ago | link

Well if your only complaint is the design of the page (which is a fair complaint, a poor design can def. be annoying), then the whole "stop trying to confuse me into using Google+ when I just want to use Places" of your original comment is sort of a red herring, isn't it? Aside from having the name "Google+ Local", it's essentially equivalent to having the Places page undergo a visual redesign which you dislike.

-----

raldi 675 days ago | link

Google deleted your inbox? Could you elaborate on that?

-----

blhack 675 days ago | link

Sure. I logged into gmail.com yesterday morning, and my inbox was missing. I still had everything that "skips the inbox", like mailing lists, my sent mail was all there, but everything that was in by inbox had vanished.

:(

This actually made me pretty sad.

-----

Matt_Cutts 675 days ago | link

This actually sounds more like your account got hijacked, or the screen was left up and someone deleted the inbox messages. Try this process to recover messages from the trash: http://support.google.com/mail/bin/answer.py?hl=en&answe...

Can you check for unusual forwarding rules or delegation in settings? As a follow-up, do you use two-factor authentication on this Gmail account?

Also, check your

-----

blhack 675 days ago | link

I didn't use two factor before, but am now. The password I was using was unique, though, and also fairly complex, so I find it unlikely somebody brute forced it.

I tried the link you're suggesting, as well as looking through the trash etc. What seemed very odd to me was that the messages which started in my inbox were missing from he trash, but things that were "skip the inbox" weren't.

Meaning anything I had "trashed" before yesterday was intact.

Again this seems to suggest that the account wasn't compromised, but that I fell victim to some sort of bug. It would have to be a very strange/determined hacker to go through and move my inbox to trash, then go to trash and only remove messages that started in the inbox.

-----

raldi 675 days ago | link

Regarding the messages that used to be in your inbox -- are they gone gone, like you can't even search for them, even with the in:anywhere flag that makes GMail check Spam and Trash?

-----

blhack 675 days ago | link

OH! in:anywhere seems to be finding them!!! Raldi! Thank you!

I searched what I /thought/ was EVERYWHERE yesterday. Thank you thank you thank you!

-----

raldi 675 days ago | link

Is it possible that you accidentally typed *a# into a GMail window? Those are the keyboard shortcuts for "select all" and "move to trash".

Fear of doing that sort of thing has led me to disable GMail keyboard shortcuts altogether on my account. (To do that, press ? in a GMail window and click the link on the second line of the overlay window.)

-----

Matt_Cutts 675 days ago | link

Very strange; I've never heard of a Gmail issue like this.

-----

blhack 675 days ago | link

Matt, they seem to have re-appeared in the trash!

Yesterday was some very odd behavior, so I must have missed them in the morning.

Basically:

1) I logged into gmail, and saw only one email that I had gotten that morning, everything else was gone. I looked in trash, and the things in it were from over a week ago [I've been out of town at Burning Man for the last week], so things hadn't just been moved to trash.

2) Did the "recover deleted messages" request via google, and a few hours got a response that some things should be restored to "All Mail", but still didn't see much.

3) Now they're back!

This is great! :)

-----

Matt_Cutts 675 days ago | link

Woohoo! Glad that you were able to recover them!

-----

raldi 675 days ago | link

That is sad. Do you use POP or IMAP with GMail? And do you see anything weird in the "Last Account Activity" link at the bottom of the page?

Also, were the messages merely archived (i.e., moved from Inbox to All Mail), or were they completely deleted (i.e., you can't even find them via the search box)?

-----

blhack 675 days ago | link

I do use an IMAP client (thunderbird) with gmail.

No, no strange activity in that window. I see myself coming out of Black Rock City, and the sparse signal I was getting on Monday as a result of it, but other than that it's just my office.

-----

GraemeL 675 days ago | link

I added plus.google.com to the list of sites blocked from search results shortly after they started spamming them. Too many results were polluted with google+ above ones I was actually looking for.

-----

Evbn 675 days ago | link

Are you talking about the results with little people icons, Search Plus Your World, that match pages in your circles? You can turn off that feature in settings, or at least you once could.

-----

ChuckMcM 676 days ago | link

Ok, so I found it amusing that they don't include Blekko, I mean we did make this into a fun game with our three card monte game with our /monte tag in search results :-) And I suppose it doesn't really serve their interests to 'lose' to a little guy either.

That said, its an interesting experiment. And at blekko.com we've been running for over a year [1].

What we've found is that there is a huge brand bias, which is to say that if you use some search engine as your primary search engine, you tend to think that is the best one regardless of whether its Blekko, Bing, or Google (or even DDG although DDG is more a search utility rather than a search engine as it doesn't have its own web index).

But 'quality' is also a very subjective thing as well. So if you search for highly SEO'd categories you will find the Blekko and Bing do better than Google mostly because there is a curated input (in Blekko's case it was from day one with it's from slashtags and in Bing's they started doing outsourced curation (putatively after seeing how effective it is in Blekko :-) in some topics with their 'editors' program [2]) At some point Google will realize what Bing and Blekko have which are that the 'indexing the web' problem became the 'filtering the web' problem when the signal to noise ratio started decreasing in about 2005, and that the only viable weapon at the moment for human on human spamming (this is where real humans are working for 5 cents a page to write web pages that draw hits) is human judgement.

If the Bing guys are reading (and I know you are) why not open up your challenge to the new kid on the block, we don't just do Blekko vs Google in our monte results :-)

[1] http://techcrunch.com/2011/06/21/be-the-mark-in-blekkos-3-en...

[2] http://www.bing.com/community/site_blogs/b/search/archive/20...

-----

nugget 675 days ago | link

I used to be a fan of Blekko until you started spamming my favorite download site with a crappy toolbar. Why sell out so early?

-----

ChuckMcM 675 days ago | link

This has come up a couple of times and I'd like to address it.

So one of the things that hinders the adoption/growth of search engines is muscle memory. Sometimes people type into their browser's address bar the address of a search engine, and then search, but more often than not they just click into the search box and start typing. If all the results are 'good enough' people don't bother to check who their search provider is, much less try others. You can manually change your search on Internet Explorer away from Bing to something else, you could manually change your search in Chrome from Google to something else, and you could change the default in the drop down box in Firefox to something other than Google. But if you don't know what you are missing, well you don't do that do you?

Now various sites which have bills to pay and no direct revenue stream, like download sites, offer a service to folks like Blekko which says "We can run an offer to our customers, which if they take us up on it you'll pay us for that." Just like Google gets paid by an advertiser if you click through their advertisement to the site and buy something, these services make your offer for you and then charge you if someone takes you up on it.

So that is pretty standard stuff, the down side is that this space has some pretty scammy operators. We have debated quite a bit about this internally. Our goal has always been to introduce people to Blekko who may not have heard about it (see above :-). However, we've also discovered that sometimes that policing these guys is a pain. They get paid if you install our toolbar (which gives you quick access to our search engine). We insist that they make it an option and clearly spell out how to continue on with your download or what not without choosing to install it, and we insist that the vendor of this software make it straightforward and easy to revert the install if folks aren't happy with it. So that leads to situations where lots of people download software and say 'no thanks' on the offer. Sometimes that annoys the site because they aren't getting paid and they "change things" so that it always installs whether you want it or not. When we find out about that we cut those people off. And if the toolbar doesn't uninstall we cut those folks off too. We had one vendor who gave us a 'final' release to QA where it worked easily and the actual release didn't. That really irritated us. Sadly there isn't the equivalent of the better business bureau for these folks, we discover them, add them to our list of bad actors and move on. We've put a number of changes in place to be better at managing the process and it may be that it ends up there are just too many scumbags in this space to make it a credible channel. Time will tell on that, in the meantime if you ever spot a site telling you that you must install a Blekko powered toolbar in order to proceed (which is to say its not completely obvious how to opt out) please report it so support@blekko.com and we'll fix it.

-----

SquareWheel 675 days ago | link

That's noble and all, but you know what, it's still a toolbar. If I see it in an installer it's still going to put a bad taste in my mouth. You know as well as I do that 95% of those installs are not intentional, or are manipulated. The kinds of users that install toolbars are not computer savvy, and when the computer says "you should install this thing", they're the ones to say "accept".

-----

pilgrim689 675 days ago | link

"so I found it amusing that they don't include Blekko"

Just took a look at Blekko. It looks good, but you can't honestly believe the layman cares as much about it as he does about search engines coming out of MS or Google. Does the layman (MS' target audience here) even know about Blekko?

-----

ChuckMcM 675 days ago | link

So the back story, and the source of my amusement, was that we've talked quite a bit to Bing. And in those discussions they got to see the positive benefit of some curation, and got to play the 3-card monte game for themselves (and lose a lot :-). And then they went out and did 'editors picks' and now they've done this 'bingiton' site. So its always nice to be driving the conversation around search, and amusing too.

So do I think the 'layman' cares? Yes and no. I think they care that they get good search results, I don't think they care where they come from. As pg points out building a replacement search engine is a Hard Problem(tm). You can make a competitive search engine with just 15 - 20 million dollars, but displacing the existing player is more nuanced.

This is something Bing knows all too well, they effectively created a clone of Google's search capability and they only pull in about 30% of the market. Blekko has been steadily adding share primarily by not being the same as Google, rather we've added specific features that were valued by different segments of the market (we're still the only search engine that lets you completely turn off ads as a user preference for example) but mostly folks just use what ever the search box provides.

[1] Note that both Yahoo and Bing are effectively the same search engine as Yahoo is 'powered by' Bing results. http://www.comscore.com/Press_Events/Press_Releases/2012/6/c...

-----

prakash 675 days ago | link

(we're still the only search engine that lets you completely turn off ads as a user preference for example)

You can do that with DuckDuckGo: http://duckduckgo.com/settings.html > Layout tab > Advertisements.

[I work at DDG.]

-----

ChuckMcM 675 days ago | link

That is awesome! I'm a firm believer in giving folks choices in this regard.

-----

calinet6 675 days ago | link

Honestly I'm not even a layman; I use DuckDuckGo and Google and read HN (obvs), but I've never even heard of Blekko. Looks nice, but I completely agree: it's a little disingenuous (or perhaps tunnel-visioned) for him to believe that people really know about it.

-----

ChuckMcM 675 days ago | link

Heh, perhaps.

DDG is a great product, we are one of their partners and we are providing some of their search results. They are also quite popular on HN I think in part because of the cool things they have been doing about creating interesting one-box kinds of things. But that said, they aren't a search "engine" they are a search "provider." That isn't a bad thing, in fact its a good thing because you can get Google results through DDG without having to get Google's "value added" stuff like G+ friends feeds or being "bubbled" as Gabe is great at pointing out. But the difference is that if the search engines stopped providing results, DDG would stop being able to return them until they built their own index. As would a number of other places like search.com or dogpile.com etc. The weird thing about search is that until Blekko came along and built a third, there were really only two web scale indexes being used, Bing's and Google's. Amazing is it not?

So I think DDG is pretty cool, love the ideas that Gabe comes up with, and am happy to partner with him to provide access to our index.

To rebut your statement that I was being disingenuous however, I suppose it depends on how measure or define 'well known.' From the perspective of various traffic reporting companies blekko.com is a bit more well known than duckduckgo.com which is not a perfect measure either but it does provide a different perspective.

-----

epi0Bauqu 675 days ago | link

Hmm, we're (DDG) not a search engine? Come on.

Actually we crawl the Web a lot and have a bunch of our own indexes. A more apt description would be a hybrid search engine. More details at http://help.duckduckgo.com/customer/portal/articles/216399

Just to be clear, are you saying you no longer backfill results with Bing?

Also, we don't use any Google results right now.

-----

ChuckMcM 675 days ago | link

No, your not. But that isn't a bad thing. From your link:

"we do not expect to be wholly independent from third-parties."

That is the technical difference, search engines do expect to be, if not wholly, then at least materially independent from third-parties, they are the third parties, search providers are all about the experience.

But lets be really clear, that terminology distinction really only matters when you get behind the search bar as far as the world is concerned we're all search engines, even if, like search.com, they don't index anything, and while DDG provides tailored indexes which support key aspects of your site's experience. The differentiation is the experience in that case not where you get your data. And I fully recognize that this distinction is not important for a large chunk of the Internet [1].

For a general audience, yes, we're all search engines.

For a technical audience, no, we're not. And the distinction is whether or not you have a generalized web index or not.

For organic search they present to the end user in a nearly identical way, caveat the 'experience' benefits of one over the other.

But for other things, like "tell me all the sites on the internet that copied this article verbatim." or "give me a rundown on link authority to all inlinks to this page" those kinds of things you need both a web index, and rights to use it like that. That's a pretty objective difference in capability.

So in technical company (and I consider HN to be technical) I try to be crisp about the terminology, in non-technical company I refer to all of these offerings as search engines because it is less confusing and frankly they don't care what its called, they type in words and get results.

[1] My father in law (part of the 99%) thinks he logs into "Google" to get to the internet because Chrome defaults to Google's home page when it starts up.

-----

jrussbowman 675 days ago | link

Awesome, now I know what to call Unscatter. I've been calling it a search site, because I don't currently have my own index of anything. I like search provider better. Thanks!

-----

jrussbowman 675 days ago | link

I first heard about Blekko through HN. I first learned of their API which I used to build most of unscatter.com on through HN too. The 3 card monte the original poster of this thread referred to, once again heard about it on HN. HN has also had links to articles about Blekko's investments from Yandex.

These were all front page. Over all I think Blekko has done a great job of being relevant and covered on HN.

Actually, here's a simple example. 44 pages when you search for Blekko using HNSearch - http://www.hnsearch.com/search#request/all&q=blekko

-----

ralfn 675 days ago | link

>and that the only viable weapon at the moment for human on human spamming (this is where real humans are working for 5 cents a page to write web pages that draw hits) is human judgement.

You are aware that Google hires thousands of search engine quality raters, all over the world right? But rather than directly curating, i suspect they just use these humans to train their algorithms. There is likely some inductive step going on, in selecting which algorithm (or 'sort' of the web) is selected, and then filtered by keywords.

Secondly, most search engines, actually do also use end-user curation. I have funky enough search queries, that I often get google redirects as search results: they want to know (and record) which link i pick. Also, because of all the google-adds-cookies, google-analytics-cookies, etc, they likely have 'session-bags' full of destination links and search queries that happen to correlate.

For me personally, neither search engine did very good in this test. But that's likely because neither result set is personalized in this test. And i don't know, if Bing does that now, but even if it does, for me to switch to another search engine, means to train another search engine in my contextual world. Hell, i even notice difference in result sets in Google based on my operating system. I get different results in Linux, than i do on my iPad.

People that get a lot of spam in search results, really ought to login, and stop clicking those links.

-----

autophil 675 days ago | link

Blekko is pretty awesome. I'll admit. I've found stuff on Blekko that I doubt I'd ever find on Google.

To Everyone: Try it out. Blekko's results are a breath of fresh air.

-----

epi0Bauqu 675 days ago | link

Actually we (DDG) do have our own indexes (including web links). More details here: http://help.duckduckgo.com/customer/portal/articles/216399

-----

ChuckMcM 675 days ago | link

This is a great summary from that link Gabe:

"we've focused on building a better search engine by concentrating on what we think are long-term value-adds -- having way more instant answers, way less spam, real privacy and a better overall search experience."

Now I'd probably word it 'building a better search experience' and remove the duplicate at the end but that's just word-smithing. One of reason we love you guys is that we share the same values in this regard. The more 'engine-like' you get away from 'experience-like' the more dubious Google and Bing get with sharing their index with you :-)

And, like you guys, we like to partner with folks who can provide the 'long tail' as it were.

-----

natex 674 days ago | link

"or even DDG although DDG is more a search utility rather than a search engine as it doesn't have its own web index."

Looks like you might be wrong about that.

DuckDuckGo gets its results from over 50 sources, including DuckDuckBot (our own crawler), crowd-sourced sites (in our own index), Yahoo! (through BOSS), embed.ly, WolframAlpha, EntireWeb, Bing, and Blekko.

http://help.duckduckgo.com/customer/portal/articles/216399-s...

-----

MarkMc 675 days ago | link

As a Google shareholder, I really want to know whether Bing search results are better than Google. And I really, really want to know what the trend is over time.

However I don't think Microsoft's study is any good. They say, "In the test, participants were shown the main web search results pane of both Bing and Google for 10 search queries of their choice."

So it seems that the study participant had to think of 10 search queries, one after the other. But this is not how search works. In the real world users need to perform one query every so often, and their choice of search terms is based on an immediate need rather than what comes to mind.

A good Bing vs Google study would ask 1000 people to use a split-test search engine for their browser search box, then monitor the results over time. I've asked HN about this before and got nowhere: http://news.ycombinator.com/item?id=3966121

So is there business opportunity here? I imagine that there must be thousands of Google shareholders willing to pay £100 per year for the results of an unbiased, ongoing study into who gives the best search results. In fact, why doesn't Google set up the study and provide such information to shareholders?

-----

Hermel 675 days ago | link

You can keep your shares. I picked Google 5 times out of 5.

-----

mynameishere 675 days ago | link

The only person who would do such a study would be an analyst for a private investment firm.

-----

Sindisil 676 days ago | link

Google, 4 to 1.

Now, all of my searches were technical; I plan to try again later with more general, "normal people" searches. Bing may do better there.

Also, good results from even google searches have become harder to come by in the last year or so. I have to work harder at my search terms. I don't know if that's Sturgeon's Law at work, or if there's room for a search engine to improve enough to become better than google.

Perhaps targeted search engines are the answer (i.e. engines's who have both their search and crawler algorithms tuned for a specific target audience) are the answer?

-----

calinet6 675 days ago | link

This is the thing for me.

Of course it's going to be a draw for most searches (especially the suggested ones: "math games", "coupons", "time zones"...), they're generic and easy to find.

You can use any search engine you want to find those things. Shoot, even AltaVista was good at those kinds of searches, and that was 10 years ago.

The ones I care about—even if I'm not a programmer or technical user—are the difficult searches. What if I need to find a specific car part that only ever existed in the 1972 Ford Pinto? What if I want to find a plumber who specializes in pre-20th-century homes? What if, what if, what if.

I used searches like that and more in this comparison, and it was 100% Google. I could even tell easily why it was Google, and it was pretty clear which one it was. I still chose Google (despite being basically unblinded) because it was fundamentally better at these specific searches.

That's what bing needs to compare. But will they show it? Of course not.

Edit: I did 5 more searches all non-technical but specific and picked only based on result quality. Again 5/5 google. (for those curious, the searches were: "lego mindstorms", "Mars curiosity hi-res", "home developments in myrtle beach", "used bookstores in portland, or", "top baby names in 1982"—the last one was very cool because Google also provided links to the 5 surrounding years right below the 1982 result. It's the little things you know...)

-----

FigBug 676 days ago | link

I went 4 to 1 google as well. My searches weren't technical, my name, my job, my hobbies, etc.

-----

sondh 675 days ago | link

I got 3 for Google. The results are surprisingly similar to pick a winner. Probably because I chose some easy searches (Mr. President, person name, place).

-----

rfergie 675 days ago | link

I did a mixture. Google won technical searches easily. Bing marginally better on the others

-----

ken 675 days ago | link

I've noticed this as well. The closer my searches are to what (I imagine) a typical Google employee does every day (Linux/Python/Java/HTML/HTTP/CSS, English language, American locale, someone in their 20's/30's, etc.), the better it does. Google search can be incredibly good at taking a partial Linux error message, and leading directly to a solution.

The further my search is from this, the worse the results I get. I'd say this is a common theme throughout Google's software. Google Wave, for example, seemed to be aimed squarely at the "I'm a Google engineer" use case. (I assume not many Googlers ride the bus, because that's a weak point of their maps!) The Chrome commercials they're running right now are a great example: they look like something straight out of the old demoscene, that Google engineers might do in their spare time, not something that I could imagine a use for.

-----

pmelendez 675 days ago | link

I got Bing with 2-0 with 4 draws. I used 1 technical keyword and 1 "normal people" keyword in spanish. It was really tied though, most results were different but pretty relevant, very hard for me to pick a winner.

-----

bwanab 675 days ago | link

3-2 Google. 3 technical searches all went to Google. 2 popular culture searches went to Bing. Interesting.

-----

JepZ 675 days ago | link

Google, 3 to 1.

All searches were related to pages I know and I searched for some keywords or parts of the domain.

-----

mindprince 675 days ago | link

Google, 2 to 0

Here's what I searched for -

-my name (Google, I chose Google because it gave more of my results, which may not be fair.)

-extract articles from websites (Google, in this search google's result were much better.)

-python ide for mac (Draw)

-new york time (Draw)

-glusterfs (Draw)

-----

kodisha 675 days ago | link

Google 5:0

-----

ximi 675 days ago | link

Google 5 to 0

-----

kaolinite 676 days ago | link

Google 4, Draw 1, Bing 0.

Why? Because I tested it with fairly specific queries. A while ago I tried to switch to Bing because after testing it with a few general searches, it seemed just as good. However, when I switched and started searching for compiler errors, etc - I soon switched back because the results were just nowhere near as good.

Here's what I searched for:

    bash: !": event not found error
    gtk calendar tutorial
    python import gtk error
    seed funding in uk
Something to remember by the way is that Google has spent a lot of time and effort on their "bubble", that is, search results personalisation (http://dontbubble.us) and it is a big part of what makes their results so great. That influence won't be present in this test.

-----

corin_ 676 days ago | link

Maybe I was imagining it, but four out of the five I did I felt I knew which was Google and which was Bing - not from doing searches that I would recognise, but just from the extra non-search information / lay-out.

I did my best to still pick without bias to my current search engine (Google) but ended up picking them 5/5 and Bing 0 times.

-----

quandrum 676 days ago | link

The site breadcrumbs using ">" always give away Google unfortunately.

-----

addandsubtract 675 days ago | link

So does the lighter blue of the bing links.

-----

Evbn 675 days ago | link

And the collection of similar pages when the result is a forum.

-----

joelrunyon 676 days ago | link

Ditto. The layout of the two's result pages are so different now, it's quite easy to tell them apart.

-----

dspig 676 days ago | link

Same here - ignoring that I could sometimes tell which one was Google and just looking at the returned links, Google won by 4 and a draw.

-----

parfe 676 days ago | link

>Bing services aren't available right now

After 5 searches the screen greyed out and was unresponsive.

Guess Google wins by disqualification?

-----

sgricci 676 days ago | link

Same here.

-----

timothya 676 days ago | link

I remember a while back when Bing was shown pretty conclusively to be taking Google results.

So I wonder if this test is really "Google vs. Bing + Google".

EDIT: Source - http://news.ycombinator.com/item?id=2165469

-----

Matt_Cutts 676 days ago | link

Last time I checked, it looked like Bing was still using clicks on Google search results as a signal in Bing's rankings.

-----

MarkMc 676 days ago | link

Matt - do Google's terms of use allow users to send the results page clickstream to Microsoft? If not, aren't there legal avenues you could pursue?

-----

option_greek 675 days ago | link

I don't think we need any more litigation and definitely not in search. Google search wins by pure accuracy of their results. They should be proud of it and just ignore scavengers.

-----

sc0ttbeardsley 676 days ago | link

How does msft have access to google algo click data?

-----

Matt_Cutts 676 days ago | link

IE and Windows, I believe.

-----

sc0ttbeardsley 676 days ago | link

Interesting. The privacy implications are disturbing if true. I'm neither an IE nor windows user but I suspect most users wouldn't opt-in if given that choice. Any refs to research/evidence about the topic would be appreciated.

-----

Matt_Cutts 675 days ago | link

The IE privacy policy is at http://windows.microsoft.com/en-US/internet-explorer/product... . Look under the section titled "Suggested Sites." Here's the relevant section:

"When Suggested Sites is turned on, the addresses of websites you visit are sent to Microsoft, together with standard computer information. ... Information associated with the web address, such as search terms or data you entered in forms might be included. For example, if you visited the Microsoft.com search website at http://search.microsoft.com and entered "Seattle" as the search term, the full address http://search.microsoft.com/results.aspx?q=Seattle&qsc0=... will be sent."

Most people have little idea that allowing a feature called "Suggested Sites" will result in their Google searches and clicks being sent to Microsoft, or that Microsoft will use clicks on Google search results in Bing's ranking.

MSFT also uses something called the Microsoft CEIP (Customer Experience Improvement Program), and I think that's either opt-out already or they're making it opt-out in Windows 8--it's built into the "Use Express Settings," I believe.

Again, I haven't looked at this very recently, but if you're using a recent version of Windows and IE, you're probably sending your searches and clicks to Microsoft unless you've been very careful about how you configured your computer.

-----

sc0ttbeardsley 675 days ago | link

Agreed about "most people have little idea...". I wish more companies were more honest and open about these things instead of burying it in the fine print. I wonder what would happen if you enable "Suggested Sites" but then install IE 9+ (since it appears to support the DNT flag) and select "Do Not Track". The pessimist in me suspects it will just turn off tracking for all non-msft experiments.

In any case, I think you've convinced me that my decision to use a non-windows OS and a non-IE browser was the correct one.

Thanks for mentioning this...

-----

majormajor 675 days ago | link

For the longest time, I didn't even realize that clicking a Google result sent the click back to Google (I just didn't think about it).

Then I right clicked one of the results one day and did "copy link" since I wanted to send it to a friend, and got something like this (with a bunch of other stuff in it too that I've yanked out cause I have no idea if any of it would be identifying info, but the end result is that this link no longer works):

http://www.google.com/url?&source=web&url=http%3A%2F...

I just hadn't thought about it before. It makes sense that they want and use that info, but it wasn't obvious at all that they were collecting it in that way, since the address shown in the status bar is the destination site, not the actual link target. I had just always assumed that the results were based on links and such around the web, and didn't think clicks would be factored in.

-----

sc0ttbeardsley 675 days ago | link

To answer my own question: http://www.bing.com/community/site_blogs/b/search/archive/20...

Seems it is opt-in users.

-----

CodeCube 676 days ago | link

Ended up picking google 4-1. I'm somewhat of a bing fan in the sense that I find them good enough for most searches. So I was a bit surprised that the results ended up skewing in google's favor ... guess that kind of backfired for them :-P

-----

mtgx 676 days ago | link

5-0 in favor of Google for me. 3 of my queries had 5 words, one had 4, and one had 2. Bing is almost as good at solving the easy short ones, while Google can solve even the long queries.

-----

BitMastro 676 days ago | link

I got a draw and then a 5-0 for Google. A couple of result for Bing were weird, like displaying "Seattle" when I searched for "a" (the letter), NSFW images when I was searching for a novel ("accelerando"). Google knowledge graph it's just too much ahead ("age of mcgyver") and it has better results on maps and evaluating expressions.

-----

esolyt 676 days ago | link

I did a comparison by actually searching the same phrases on google.com/ncr with Incognito. The actual Google results are much better than the Google results on this page.

Search for "Hunger Games" on Google returns the IMDB page as the first result, which is exactly what I want. But neither panes on the Bing's test has the IMDB page as the first result.

-----

ch0wn 676 days ago | link

5-0 for Google. I tried mostly programming related queries as I find the differences most noticeable there. Especially when searching for specific APIs or objects, Bing tends to give you just the project homepage and some unrelated pages, while Google gives you the actual deep link.

-----

Karunamon 676 days ago | link

Bing seems to be a bit better at generic, broad searches (games, file extension $something, etc) and Google seems to be better at exact, specific queries (stop error 7B, RPGs announced at PAX 2012).

Bing ended up winning, which shocked the hell out of me, but after trying a couple more times, I noticed that pattern.

-----

moondowner 676 days ago | link

And at the end of the day, what I need are exact/specific queries.

Whether to get the job done, to get some certain information or something third.

-----

korr 676 days ago | link

DuckDuckGo is now a hundred times better than Google for those exact & specific queries. I was looking for some info on my truck engine, and Google sent me to sales pages for completely unrelated parts (and sometimes, entirely unrelated cars!). DDG sent me to deep in an old forum thread, where someone had posted a diagram of exactly what I was looking for.

-----

thomasjoulin 676 days ago | link

Every time I see a link comparing search engines, I try them with hope, but always go back to Google. My last search confirms this :

* Google https://www.google.com/search?q=insertObject+atIndex+zero+bu... * http://www.bing.com/search?q=insertObject+atIndex+zero+bug&#... * http://duckduckgo.com/?q=insertObject+atIndex+zero+bug

-----

korr 676 days ago | link

So, five results at the top from StackOverflow... Why not just skip the search engine middle man and go straight to StackOverflow?

-----

DanBC 676 days ago | link

Proper search engines are better at finding content than a site's own search feature.

-----

thomasjoulin 675 days ago | link

I usually type in my browser address bar, which searches to Google directly. I tried my query on Stack Overflow... Google still gets a better SO question than SO itself : http://stackoverflow.com/search?q=insertObject+atIndex+zero+...

-----

DannyBee 675 days ago | link

Because the goal is to have something that finds any particular information I want, not go to every site and hope it may show up in a site search?

-----

BitMastro 676 days ago | link

Because you still don't know if StackOverflow will have the result you're looking for

-----

saalweachter 675 days ago | link

I've seen this in shopping results -- I'm probably going to buy from Amazon, but I like to see the top N competitors' results along side the Amazon results, so that I can easily confirm that no one is more than a buck or two cheaper than Amazon and can then feel good about my purchase.

-----

More



Guidelines | FAQ | Lists | Bookmarklet | DMCA | News News | Bugs and Feature Requests | Y Combinator | Apply | Library | Contact

Search: