
We Need Viable Search Engine Competition - nemesisj
http://peebs.org/2014/01/04/we-need-viable-search-engine-competition-now/
======
antics
The comments here about building search engines that viably compete with
Google in terms of quality are hilariously wrong. _Hilariously_ wrong.

The dominating cost is not hiring smart people to work on the problem, it's
hiring _enough_ smart people to work on the problem.

Consider. Here at Microsoft, we employ a search relevance staff of -- what,
20% the size of Google's? And Google's engineers are rock solid. MS might be
able to make up a small margin, but our search relevance staff -- good though
they are -- cannot compete with a brilliant workforce that is 5 times larger
than them. If we are going to actually compete with Google, the problem is not
hiring smart people, it's hiring _5 times the current number of smart people
employed for this task at MS_.

Never mind _buying_ such a team. MS can afford that. How do you even _find_
that many people in 2-3 years?

When you consider the rest of the data, the case for building something like
Google really begins to look grim. For example, how much does it really cost
to build a search engine? We've poured _at least_ tens of millions of dollars
into _just_ search relevance (I'm not even counting infrastructure). That's
good mileage, considering that this way is littered with the corpses of
companies like Cuil, who made investments in this area and failed miserably.
Still, while this has been a great deal for us, we're still not there yet, and
it's not clear we will be in the very near future. And so it's worth
wondering: if MS can't buy something like that, realistically, who can? DDG?
lol no.

In the end, this is the true dominating cost of building a search engine:
people capital. Other bottlenecks, like engineering debt, politics, _etc_.
pale against the sheer, awe-inspiring investment of Google in people capital.

~~~
nemesisj
You know what? I really can't get behind the idea that MS is somehow the
stirling example of engineering prowess and discipline. This is the
organisation who fails their way through every other operating system. One of
the poster children for missing the boat on the internet, tablets, operating
systems, cloud services, and more.

I also reject this idea that Google's got a lock on every smart person ever.
That they don't have any politics or wasted effort. That they're the snowflake
they think they are. This kind of thinking is pathetically misguided, and a
huge part of their marketing. Google is not perfect. They are staggeringly
weak in the face of real competition (witness Facebook vs. G+ and Apple vs.
Android).

Your argument is a replay of the prevailing attitude in the mid nineties
regarding operating systems (from the same company, surprise!). And then some
college student named Torvalds pushed out the most influential operating
system ever, and did it for free, with the help of the world.

Nobody and nothing is invincible, and people who say that a situation is
unassailable are always, categorically wrong, given enough time.

~~~
antics
Hi OP. Your points, as I understand them, are the following. Correct me if I'm
wrong.

* MS is not a good example of a strong engineering org. Further, it is not a good example because Windows sucks.

* Google doesn't employ every smart person ever.

* Google can't actually compete. (ed note: in any field, or just against iOS and Fb?)

* My argument is like the argument that no one would supplant Windows.

* Given enough time, everything dies.

Points 2 and 3 are about Google. Let's start there. You're right that Google
doesn't employ every smart person ever, but then, who said they did? :) But
there are a limited number of search relevance engineers, and finding enough
to keep up with Google is a monumental, and maybe insurmountable feat. If you
want to compete with Google, you will need to have a serious advantage that
supercedes this. That is just a fact.

Point 3 is about competition. I'll grant you that G+ is no Facebook, but
Android is the most widely adopted mobile OS on the planet, and by a _huge_
margin -- iOS is basically not even competitive, except for the top 5% of the
market. Further, a billion smartphones will be bought this year, the most of
which will be Internet enabled _Android_ devices, and most of which will be
bought in developing countries by people coming on the Internet for the first
time. You tell me who's forward thinking there, because when that market comes
online, it will be huge. The fact that you mention this as being non-
competitive indicates to me that you might not know what you're talking about.
:(

Point 1 is about the MS org. What can I say, OP, I work here, so maybe I'm not
the best person to have this discussion with. But FWIW I _chose_ this place
over some much sexier jobs because the team I work with is arguably the best
of its type in the world. There are bad neighborhoods, but the disparity
between a good team in MS and a good team at Google is basically negligible.
Also I think Windows is one of the great engineering feats of CS, so ... :|
(Note that I still use UNIX at home.)

Point 4 is probably the result of confusion. I don't think no one can compete
with Google. Bing has 20% market share! Clearly we can. But I do think it will
be hard to compete with Google on _search quality_. I don't see how you can
argue that.

And point 5 is obviously true but not relevant.

EDIT: Actually I now see your point 1 as saying "MS couldn't pull this off
because they're not a good engineering org, but someone else could". Maybe
someone else can build a better search engine, but I think what MS has pulled
off with Bing is a _monumental_ feat.

For starters, we built the entire Bing stack from scratch. No OSS. No common
platforms like the JVM. Nothing like that. We started from nothing, and
invented the server infrastructure, the data pipeline, the runtime that would
support the site, the ML tools, everything. The fact that the site runs at all
is a small miracle, but the site does not "just" run: the most remarkable
thing by far is that the quality of our tooling is quite incredible, generally
an order of magnitude better than the OSS equivalents. For example, the
largest deployment of an OSS NoSQL datastore seems to be a few thousand nodes.
The small NoSQL cluster backing our MapReduce implementation is stably
deployed on a cluster _an order of magnitude larger than this_. This is
something you only really see at companies like Amazon, Google, or MS.

I understand that the consumer market is not something MS is strong at, but I
am hoping this gives you a taste of the scale and quality of what's happening
behind the scenes. Happy to talk more about this if you drop me a line or
skype me at `mrclemmer` :)

~~~
devcpp
You make some good points but you lost me there:

>Windows is one of the great engineering feats of CS

Do I have to point out all the lame exploits and bugs which take so long to
get fixed, the terrible design, the "things which should have been there years
ago but we still don't have" like a decent file manager, task manager, copy
utility? And AFAIK Windows made few major contribution to the theory of OSs
(semaphores, threads, paging, scheduling and so on) so there's really nothing
to be amazed at.

I'll spare you my opinions on Bing. Reinventing the wheel is not worth
describing, no matter how beautiful that wheel is, although I'm happy for you
to be a part of the team making that wheel. If only Bing had more ambition
than just being a clone of Google Search, some people under 70 would actually
consider migrating. But if you're happy with your default-search-engine-
bundled-with-IE market share at 20%, good for you (eh, it does bring a lot of
ad money). You may not call the shots at MS, but you can at least admit all
the shortcomings.

~~~
tptacek
You could point out all the lame exploits and bugs, but only if you want to
spark a long argument that you will end up losing. There are a variety of
things I don't like about Windows, and I avoid using it. But the idea that
it's somehow less secure than other operating systems is for the most part a
Linux advocacy myth. The fundamental security architecture of WinAPI is just
not that different from that of Linux or OS X (which also has a legacy compat
issue that complicates its security).

~~~
devcpp
Fair enough, you know much more than me in that domain. :)

Are you going to argue about the famous general instability of Windows
compared to its large competitors though? It seems like a good indicator of
bad design in low level implementations.

~~~
wglb
What do you mean by _famous general instability of Windows_? Windows is
perhaps my 3rd choice for OS, but almost all my clients use Windows and among
the problems they deal with, instability is not one of them.

------
chestnut-tree
My biggest concern with Google is the phenomenal amount of data they track and
record about users. They have no self-restraint when it comes to tracking the
online behaviour of users. Now they can track across devices (phone, desktop,
tablet) giving them unprecedented knowledge of your online activity.

Companies are collecting more data about online behaviour than ever before.
And no-one has the same online reach as Google. From analytics, to apps, to
fonts to jquery - there's barely a site that doesn't link in one form or
another back to Google in some way. Google's digital fingerprints reach into
every corner of the web.

I've said this before, but I was hoping that 2014 would be the year we become
more privacy-conscious, but I don't actually think that will be the case.
Google get an incredibly easy ride on the subject of privacy and online
tracking from the tech community. They're probably salivating at the prospect
of capturing even more precise user behaviour through an OS (Chrome) that
potentially captures _everything_ you do online. Google aren't capturing this
data anonymously either. The tech community's response to this seems largely
to be - so what? For anyone who cares about privacy, that's pretty depressing.

~~~
Karunamon
I didn't and still don't particularly care. The arguments to why this is bad
always seem to be of the slippery slope / what if somebody does something
untoward with the data variety.

The first one is a easily-dismissed fallacy, the second is not limited to
Google or any other company. I have yet to see a convincing argument that
Google is misusing this data or doing anything bad with it.

On the other hand, a service that knows you intimately enough can provide some
very cool things _that are otherwise impossible_. The cards on Google Now, for
instance, rely completely on the search history on your account and the
location data from your phone. I get up in the morning and my usual route to
work is plotted out with an ETA. I search for a nearby restaurant on my
computer and the directions appear on the phone complete with ratings. Things
like that.

My philosophy is to deal with any abuses if/when they occur (and mitigate the
forseeable ones), instead of walling yourself off from the ever-more connected
world. I'm starting to think there's a fundamental shift happening in what
"privacy" is, why it's necessary, and what it means nowadays. And as usual,
the choices are hop on, get out of the way, or get run over. For better or
worse.

------
sigil
_Is PageRank really the indomitable tech of our generation? Nobody can do
better algorithmically, or integrate some kind of crowd sourced feedback, or
measure browsing time and habits, or simply hand tune some of the most
competitive key phrases? I’m sure I’m oversimplifying, but I wonder if we
haven’t all been hypnotised by the complexity, much of which is marketing
hype..._

Actually, the tech _is_ extremely complex, and PageRank is only a tiny part of
that. Try building a search engine sometime. To hackers inspired by this
article: here be dragons.

DuckDuckGo took the only sane approach and aggregated results from existing
search engine apis, then gradually mixed in some secret sauce. Even then, is
DDG viable competition for Google? Will it ever be?

All the same, I join you in wishing for some Innovators Dilemma-style
disruption here. A "toy" service comes along one day that's a substitute for
Google search, but only within a tiny niche. Google doesn't take it seriously
enough until it's too late...and we have a real ballgame on our hands again.

Thought experiment: what could that disruptive niche be?

~~~
erikpukinskis
> Thought experiment: what could that disruptive niche be?

Someone needs to write/buy a search engine and then build a really rich API
into the internals of it that lets 3rd parties write customized search
engines. How about an auto parts search engine, or a search engine for
spanish-speaking people living in southern california? What about a Christian
search engine, or a meme search engine?

When someone can empower 3rd party developers to make the same kinds of
decisions as Google does, but with different tradeoffs, and they really put
the full institution behind supporting that, I think:

A) Google will have a hard time competing because they won't be able to give
the personal attention to the needs of what is, effectively, a community of
modmakers.

B) This new company will capture the long tail of search. Only a sliver of
that is covered right now, with a scattering of niche search engines (Google
Scholar, etc).

C) The number of users could be VERY large. It could be the cable to Google's
broadcast television.

D) Getting started wouldn't require any massive technological achievements.
Just find an underserved niche where even a really _really_ stupid search
engine would work better than Google. Write it, figure out how to make money
off it it, grow. Start with something that requires only a small index. Slowly
expand into additional niches according to what will help keep the company and
the tech moving forward.

Google's bet is that any information you can glean from someone coming to a
niche search engine can be reasonably approximated with contextual information
in the query. That's proved true for many queries, but the key is to find the
queries where it's really not.

~~~
Isofarro
Sounds like Yahoo Boss:
[http://developer.yahoo.com/boss/](http://developer.yahoo.com/boss/) And
that's been around for years already.

Essentially, you get the Yahoo results, and decorate, filter, reorganise,
improve, combine as you wish. So you get the organic results, which you can
then innovate upon.

And it has survived the transition from Yahoo-powered search results, to Bing
powered search results: [http://searchengineland.com/yahoo-boss-moves-from-
yahoos-ind...](http://searchengineland.com/yahoo-boss-moves-from-yahoos-index-
to-microsofts-adds-pay-structure-64140)

Which means you have API access to the second most-used search engine in the
industry. So what better way to voice your discontent with Google by
supporting a competitor.

Running a successful search engine is expensive, it needs continuous
investment into R&D. That's why Yahoo took a step back and partnered with Bing
instead. The level of investment needed just to hold status quo with the
existing market runs into billions of dollars a year, something Yahoo baulked
at. Microsoft, however, were still strongly inclined to invest that every
year.

~~~
nashequilibrium
Yahoo screwed up so bad here, read the research papers coming out of
labs.yahoo.com back then and you realize they were onto knowledgraph before
Google and actually had a head start, but they shattered their research
division and lost a lot of those search people to microsoft and google. They
just didn't have faith to create their own path and also lost their identity
by deciding to become an entertainment company.

------
AnthonyMouse
The complaint seems to boil down to how Google deals with spammers. The
question is, how is competition going to make it better? If we instead had
five search engines that each had 20% market share, would people stop trying
to game them? Would false positives in efforts to thwart spammers be
eliminated? How?

The problem isn't even one that responds to market forces -- the "victims"
(sites that should rank highly in organic results) are no one's customers. If
you want to flip the script, they're the product. They have no leverage, and
having more search engines leaves them with still no leverage. Only the users
of the search engine have leverage because they can switch to another search
engine -- but they can do that today. The problem is the alternatives are no
better.

What we need is a cost-effective accurate way to identify spammers and exclude
them from high rankings in search results. Someone who could do that more
effectively could challenge Google, because there is a market for spam-free
search results. But if that was an easy problem to solve then why hasn't
Google solved it? They have the right incentives. It's just not easy, because
spammers adapt. Whenever a search engine does something to thwart spammers,
the spammers do something different.

Having more search engines doesn't make it easier, it makes it harder because
each search engine has less resources to dedicate to it and they have to
duplicate each other's work, and the cost to sites of legitimate optimization
for a larger number of search engines increases which creates an even larger
advantage for major institutions over small timers who can't afford the higher
cost.

It's silly to say "why doesn't Google do X" and list some abstract thing you
think would solve it which they've already considered and declined to do.
There is probably a reason. Maybe manually curating every website is too
expensive. Maybe arbitration proceedings would be overrun with spammers trying
to challenge legitimate removals of their spam. And if they're wrong, don't
speculate about it on a blog, prove it by building a better search engine. So
far no one has been able to do it.

~~~
nsns
You falsely assume a new search engine can only be "as good as Google".
Google's search has become highly commercialized over the years, it used to be
about finding the "best" answer to your search query (the "original"
PageRank), but it has increasingly become the "best consumerist" answer to
your query (where to buy something). This make it more susceptible to
spammers, as their basic incentives are similar. New, different, search
engines might try another approach, perhaps less personalized, less
commercialized, and less mass-media oriented, and be less of a "petri dish"
for spammers.

~~~
scholia
A search engine that didn't bury good quality results under vast amounts of
recent ("fresh") blog spam would be a useful start...

------
jorgecastillo
I don't think anything can compete with Google when it comes to search in
English. I almost feel like Google can read my mind. With the adequate query
you can find anything you've ever read on the internet if it's still online.
Yesterday night I wanted to re-read a blog post about a guy that made a few
bucks with QNX, but I didn't remember much I tried a few queries and I got the
expected blog post with this:

[https://www.google.com/search?q=netherlands+blog+qnx+a+few+b...](https://www.google.com/search?q=netherlands+blog+qnx+a+few+bucks)

[http://jacquesmattheij.com/how-to-make-a-quick-
buck](http://jacquesmattheij.com/how-to-make-a-quick-buck)

I just tried the same query with DDG and Bing and they don't even come close
to giving me the link I want.

~~~
srathi
I feel the same. I wanted to find a video about a zombie prank in a New York
cafe and did a Google search for it. The first result was the actual video
that I needed.[1]

Needless to say, Bing fails here.[2]

[1]
[https://www.google.com/search?q=telekinesis+prank](https://www.google.com/search?q=telekinesis+prank)
[2]
[http://www.bing.com/search?q=telekinesis+prank](http://www.bing.com/search?q=telekinesis+prank)

~~~
throwawaykf03
The 2nd and 3rd results on Bing the same as the 1st one on Google.

------
gradys
I would say that Bing already is a viable alternative. Having used it for
several months now, there aren't too many queries that it doesn't handle well,
and usually when I check Google's results for those queries, they are equally
bad.

~~~
teddyh
So instead of an entrenched monopolist, I’m supposed to choose… Microsoft?

That just feels _wrong_.

~~~
romanovcode
It's 2014, times have changed.

~~~
teddyh
I don’t think _Microsoft_ has changed enough to earn my trust or respect. I
doubt they could.

------
lawl
I wish more people would use and/or improve YaCy[0]. It's a decentralized
search engine. Client is written in java. With the power of decentralisation
hardware shouldn't really be an issue. It's just that the search result
somewhat sucked the last time I tried it.

But I guess now is a good time to install it again, help scraping the
internet, and maybe hack at the code.

[0] [http://yacy.net/en/](http://yacy.net/en/)

~~~
rocky1138
I love the idea of this stuff but I'm so scared things like this install a
backdoor to our PCs. Is there any way to mitigate this? Run YaCy in a VM on
VirtualBox?

~~~
trit
Since it's open source, I doubt something like that would go unnoticed. You
can check out the source here though:
[https://gitorious.org/yacy](https://gitorious.org/yacy)

------
ceph_
>1\. Google is making arbitrary rules on how sites should behave, because they
have a monopoly.

How are rules against paid linking scams or procedure generated content farms
considered arbitrary? It's clearly trying to game the system, and the rules
are explicitly laid out to tell you NOT to do it.

>2\. Google needs these rules, because Google’s rankings are apparently
trivial to game.

If this were true, you can make millions executing your plan for any number of
websites. It's not.

~~~
bushido
_> How are rules against paid linking scams or procedure generated content
farms considered arbitrary? It's clearly trying to game the system, and the
rules are explicitly laid out to tell you NOT to do it._

The whole case with the delisting/penalization and subsequent(and extremely
speedy) re-listing of RapGenius is a great example of Google's current
arbitrary practices. I covered this is some details on another thread[0] in a
couple of replies[1][2][3][4].

[0]
[https://news.ycombinator.com/item?id=7010997](https://news.ycombinator.com/item?id=7010997)

[1]
[https://news.ycombinator.com/item?id=7011063](https://news.ycombinator.com/item?id=7011063)

[2]
[https://news.ycombinator.com/item?id=7011111](https://news.ycombinator.com/item?id=7011111)

[3]
[https://news.ycombinator.com/item?id=7011356](https://news.ycombinator.com/item?id=7011356)

[4]
[https://news.ycombinator.com/item?id=7011597](https://news.ycombinator.com/item?id=7011597)

~~~
ceph_
The rule they broke is clearly laid out [1]. The practice of manual penalties
is as well [2]. They outline a course of action on how to remove the offending
links [3]. How is this arbitrary?

[1][https://support.google.com/webmasters/answer/66356](https://support.google.com/webmasters/answer/66356)

[2][https://support.google.com/webmasters/answer/2604824](https://support.google.com/webmasters/answer/2604824)

[3][https://support.google.com/webmasters/answer/2700611](https://support.google.com/webmasters/answer/2700611)

------
r0h1n
Perfectly valid expectation with near impossible odds of coming true.

Google has gotten so rich, entrenched and popular that IMHO no competitor can
dislodge it. I say this as a thoroughly-disappointed user who's tried nearly
all the alternatives to Google in the various segments that it operates in.
I've managed to stop using nearly all Google services except Android (running
CM) and Search.

As others have pointed out in the thread, DDG is _nowhere_ near as good,
especially if you're not in the US (I'm in India). After having forced myself
to use DDG for a month, I've now resigned myself to Google searches with a
couple of extra steps: \- all searches performed while logged out of all
Google services \- browser plugins to rewrite all Google tracking URLs from
search results

So sure, we need viable search engine competition. But don't wait up for it
either.

~~~
tsurantino
"Google has gotten so rich, entrenched and popular that IMHO no competitor can
dislodge it."

Isn't this the definition of a monopoly? And if so, isn't that reason enough
to consider search as a public good or a publicly regulated means of accessing
information?

~~~
r0h1n
The key word there is "IMHO"

Judging from the various anti-trust cases that have been brought on against
Google around the world it's clear that _proving_ that is nearly impossible
too. More so because Google operates in a sector (Internet software) that is
theoretically open to infinite competition and zero switching costs.

~~~
gress
That theory - infinite competition and zero switching costs - seems to be
completely ungrounded in any kind of reality.

~~~
mgkimsal
There's a whole issue of what the definition of 'using google' is to people.
Even if I never search with Google, I'm still often using google stuff - or,
more precisely, Google services are using/tracking _me_. And there's no
obvious way to turn it off or opt out of it.

~~~
skj
incognito mode.

------
Aqueous
Part of the problem is that Google's PageRank algorithm is patented and the
patent won't expire until 2017.

Google insists that its current search mix is based on a lot more than just
PageRank, but it seems that PageRank probably contributes the foundation of
their business. I don't see a way of competing with Google's results unless we
are allowed to use something like PageRank, which we can't do unless we pay
royalties or wait until it expires.

I'm not saying that PageRank is the be all end all of search algorithms, and
certainly someone somewhere could come up with a different method of ranking
superior to Google's. Ranking pages by how many other pages cite them seems
like a pretty fundamental insight, and where I would start with any new search
engine.

~~~
greglindahl
The PageRank patent is owned by Stanford; anyone can buy a license. And BTW,
it isn't necessary or even that useful these days to compute it.

~~~
nostrademons
I'd thought that Google had an exclusive license, but I just checked and the
exclusive license expired in 2011:

[http://www.seroundtable.com/pagerank-
patent-12731.html](http://www.seroundtable.com/pagerank-patent-12731.html)

So yes, anyone can license PageRank now.

------
drewvolpe
This is one of PG's frighteningly ambitious problems and I believe it's an
important (and really hard) problem. I think the answer is to focus on an
important niche and grow from there. DDG has done a good job by focusing on
privacy; they're currently at nearly 4m queries per day.

I'm starting by building the search engine I want to use while writing code,
which does parallel searches of different parts of the web dynamically based
on the query.

I have an initial (ugly and very limited) prototype here:
[http://gigglebang.com/](http://gigglebang.com/)

I'm looking for co-founders. If this is an interesting problem to you, email
me: gb@dewdrops.net

~~~
greenyoda
Thanks, your prototype looks interesting; I'll try it out. The one-click
redirection to DuckDuckGo, Python, etc. is very convenient.

It would be nice if you could add a privacy policy to your "about" page that
disclosed what kind of data you collect about your users' queries.

------
avalaunch
The internet is now littered with badly written content linking to more badly
written content, all in an attempt to increase inbound links and ultimately
rank higher in Google. And since Google can't tell the difference between
badly written and well written content companies continue to pump this crap
out. A lot of problems might be solved with some healthy competition but I
don't think this particular problem would be. It would require a new way of
ranking that severely devalued inbound links and that seems like a really,
really hard nut to crack.

------
mark_l_watson
I agree with the article. I wrote last week
([http://markwatson.com/blog/2013-12/practical-internet-
securi...](http://markwatson.com/blog/2013-12/practical-internet-
security.html)) about using two browsers. Chrome with default security
settings: used only for gmail, google search, facebook, and twitter; Firefox
with ghostery: all normal web browsing, search using duckduckgo and bing.

My setup encourages me to at least use duckduckgo and/or bing when I am using
firefox.

~~~
andrenotgiant
FYI: I would keep tabs on Ghostery, there's a for-profit company behind it
that works with Ad Networks - They don't do anything bad now but experience
shows that a massive user-base that you make zero revenue off can be a
tempting fruit when profits start falling.

[http://en.wikipedia.org/wiki/Ghostery#Criticism](http://en.wikipedia.org/wiki/Ghostery#Criticism)

~~~
mark_l_watson
Thanks Andre, I will watch out for that.

------
gesman
One of the viable venues to compete with Google is not by developing "new,
better search engine", but rather "better vertical platform".

What's the best search engine where buyers go? Not google.com and never really
been google, it's amazon.com and ebay.com.

What's the best search engine to find answers on technical questions?
Stackoverflow.com is approaching google very fast.

So the answer would be for sharp minds to develop sharper and better vertical
portals/platforms and pick google apart piece by piece.

~~~
lubujackson
That's exactly what my startup is trying to address - how do you help users
navigate 100, 1000 or even a million search engines? Ultimately, you need a
way to browse search engines by topic and bundle them into custom lists for
your specific needs. It's early going, but check Nuggety out if you're
interested: [http://nuggety.com/](http://nuggety.com/)

~~~
namenotrequired
I'm interested, but I've checked the front page and am still confused what /
who it's for, could you clarify?

~~~
lubujackson
Sure. I know the site needs some more clarity and that's what I'm working on
now. The basic idea is a Pinterest-type community where you can build a list
of sites on any topic and share lists with others. But each list lets you
search each website directly from the same search box (results come up in a
new tab, like Kayak).

So what is it good for? Any subject where you want depth or breadth over
surface results. So collectors and researchers can build a portal for
themselves. And there are some interesting possibilities for specific needs,
like a search list for many places to find a cached URL:
[http://nuggety.com/u/nuggety/cached-
webpages](http://nuggety.com/u/nuggety/cached-webpages) or a search list to
search any of Google's 190 international websites:
[http://nuggety.com/u/nuggety/international-
google](http://nuggety.com/u/nuggety/international-google)

------
hitchhiker999
Last year i wrote this: [https://medium.com/surveillance-
state/32ba2b38c219](https://medium.com/surveillance-state/32ba2b38c219) \- and
got a proper dollop of hate from certain segments in the webmaster community.
They claimed it was 'sour grapes' \- I have 140k visitors a day, am doing
great, better than great. The monopoly is bad for all of us, even though I'm
benefiting as it is.

This Google thing is WAY out of control.

------
marc0
[http://www.yandex.com](http://www.yandex.com) is a real competitor

~~~
gregholmberg
On a whim, I asked yandex.com: 'k-d tree'.

Nine results were about the space-partitioning data structure. Also, there is
a tree-trimming service on Long Island called K&D Tree Masters.

Surprisingly, pleasantly -- all ten items are useful to me.

------
digisth
Is there any new information regarding the future plans for Bing? This article
from a couple of months ago had it looking ominous:

[http://www.bloomberg.com/news/2013-11-08/microsoft-ceo-
candi...](http://www.bloomberg.com/news/2013-11-08/microsoft-ceo-candidate-
elop-said-to-mull-windows-shift.html)

"Besides emphasizing Office, Elop would be prepared to sell or shut down major
businesses to sharpen the company’s focus, the people said. He would consider
ending Microsoft’s costly effort to take on Google with its Bing search
engine, and would also consider selling healthy businesses such as the Xbox
game console if he determined they weren’t critical to the company’s strategy,
the people said."

------
arikrak
Yes, search is hard. Look how most websites that build a search for _their own
site_ still do a worse job than Google's site search. And its not like people
haven't tried to take on Google. Google always faces the threat of companies
succeeding in smaller areas of search, but so far it's dealt with those pretty
well too. The other threat it faces is a (big) company developing an
alternative approach, e.g IBM Watson.

------
jpdelatorre
Here's an idea... how about instead of using just one search engine website,
browsers will have a functionality that allows you to search for something and
have it query to 3 different search engine at the same time? Users can
configure which 3 search engine he/she wants to use. That way, website owners
won't be so affected when one search engine screwed him up.

~~~
GoodIntentions
[https://en.wikipedia.org/wiki/Dogpile](https://en.wikipedia.org/wiki/Dogpile)

I haven't used dogpile in forever - actually thought they were defunct, but
your comment brought it to mind. Seems like what you're after?

------
CombAce
or: We need an inclusionist wikipedia - where every possible search term
starts with an empty page/any possible search term would have an entry, even
if it's just for redirecting a spelling mistake. Eventually every search term
would grow its own little community of experts. Unfortunately, as pg said,
"Deletionists rule Wikipedia."

~~~
examsheart
Appreciate the lateral thinking. Several issues related to evil SEO-like
behavior would need to be addressed as well:

* Blow away my competitor's links on the topic I'm related to

* Always put my links at the top on my topic page

* Sneak my link into other pages not directly related to my topic (viagra spam on comp.lang.*)

------
nilved
DuckDuckGo provides better results than Google. We have viable alternatives.

~~~
BTurkE
You know their web results are from Bing, right?

~~~
nilved
Bing is one of many places that they get results from, yes.

------
namenotrequired
Alternatives:

[https://www.blippex.org/](https://www.blippex.org/)

[http://www.yandex.com/](http://www.yandex.com/)

[https://duckduckgo.com/](https://duckduckgo.com/)

And of course:

[http://www.bing.com/](http://www.bing.com/)

[http://www.yahoo.com/](http://www.yahoo.com/)

~~~
namenotrequired
Oh and also:

[https://epicsearch.in/](https://epicsearch.in/)

------
adamseabrook
We (meanpath) released[1] some crawl data which may be of interest to those of
you hacking away on a search engine project.

[1] [http://blog.meanpath.com/meanpath-
january-2014-index/](http://blog.meanpath.com/meanpath-january-2014-index/)

------
yodha77
as lot of people have said.. u cannot beat google at search. but, there is a
way.. push information that users are looking for before they even think about
it (similar to Google NOW). search is still a sophisticated operation for
regular users. by showing the right information at the right time, users don't
even have to search.

------
thedawn
You are so right. I agree with every word. Competition is very important and a
real helper to the us the users.

------
PLenz
duckduckgo.com

~~~
sleepyhead
I use it but for many queries it still isn't good enough.

~~~
Derbasti
Funny. I use DDG as my default. Frankly, I think it's results are way better
than Google's. Yes, they are more simple-minded and less "intelligent", but
then they are much more predictable as well. As such, it took some getting
used to, but now I feel like I search more precisely than I ever would with
Google.

Way too often, Google will try to be "smart", and search for some
"intelligent" interpretation of my search, where no such thing was called for.
Especially for technical, precise searches for "strange" strings, this can get
really annoying.

Also, those bang-searches are just genius. I regularly search Wikipedia,
programming language docs, or maps using those. And yes, sometimes Google as
well, mostly for fuzzy "how do I" searches or searches that I don't know
precise keywords for.

~~~
stephen_g
> "Way too often, Google will try to be "smart", and search for some
> "intelligent" interpretation of my search, where no such thing was called
> for. Especially for technical, precise searches for "strange" strings, this
> can get really annoying."

Wow, so it's not just me then - I was describing my exact same feeling to
someone (perhaps less eloquently than you!) just three days ago!

Even when I go and set it to 'Verbatim' search (which is well hidden), it
still often gives me useless results for these kinds of technical queries...

