
 Russians dent Google’s world domination - nickb
http://business.timesonline.co.uk/tol/business/industry_sectors/technology/article4596416.ece
======
mojuba
Yandex doesn't index the entire Internet though, but only some selected part
of it, mostly pages under .ru and all ex-USSR TLDs, plus domains submitted to
them manually. Not very impressive I must say. news.ycombinator.com, for one,
isn't in their index.

Edit: in addition to ex-USSR TLDs they also index web sites that map to IPs
physically located in ex-USSR, regardless of the domain.

~~~
ii
AFAIK it's their policy to index only Russian language content and they do it
pretty good. They can index domains in any zone as long as their content is in
Russian.

Anyway IMHO Google gives better search results in Russian. I guess they have
more users here for the same reasons Yahoo does better in Japan. It's all
about culture.

~~~
mojuba
The problem with Yandex is that it does index English language content,
however not all users (by far not all!) are aware that it's limited
geographically and by TLDs. If I look up, say, "Succinctness is Power", which
should bring up PG's essay on his web site, Yandex instead retrieves a bunch
of pages that mention the article and also Russian translations rather than
the original. So the user is left thinking that the original is not even on
the web (unless she carefully examines the links on web pages that pop up in
the result set).

They were clever enough to index en.wikipedia.org, although it is usually not
at the top for some very obvious requests (e.g. try "Turing Machine"), which
is an indication of a rather poor ranking system.

My impression is, Yandex does not compete with Google in any way. Despite its
slogan ("Everything can be found" or something like that) it's just a limited
web index that doesn't even state explicitly about its limitations. And that's
strange at least.

Edit: NB: they have a minimalist front-end at ya.ru

~~~
raquo
Yandex is local - limited to CIS (ex-USSR zone) in terms of people the search
results (and the search index) are optimized for.

Yandex is mainstream, it's not a geeky site, and most Russians don't speak
English well enough to prefer originals to translations. So most of its users
just don't arrive at the limits of Yandex's index ever.

Of course it's nice for me to be able to go to google.com and search in
Russian or in English or in any other language (and get results in the
language I search), but it's just not what the majority of Russians needs or
wants.

Their policy on the search index was 'only russian-language content' (their
bot analyzes page text rather than looks at the TLDs or IPs), then a couple of
years ago they started including more and more sites in english, starting from
encyclopedias etc. This is possibly also because their indexing technology was
deep into morphology of Russian language from the very start in late 1990-s.

PS their slogan is a bit more dramatic - 'Everything WILL be found' :)

~~~
mojuba
Thanks for clarifications, but to my understanding they won't index Russian
content if it's not under exUSSR TLD AND it's not physically hosted in the
exUSSR. It took me 3 years (!) to convince Yandex (through submissions, mails
and even phone calls) to include my Russian-language blog, only because it was
under .com and hosted in Texas, despite the fact that my blog had a lot of
references from the .RU space.

And no, they confirmed, it wasn't because my web site was marked as spam. They
explained to me that "some" servers have to be included manually because they
are beyond their "default" space.

What puzzles me is that Yandex is definitely not the entire Internet, not even
the entire Russian-speaking internet, and yet it claims to be finding
"Everything" without mentioning of the limitations in place. My point is, this
is misleading for the users, if not just plain cheating.

Soemthing is definitely wrong with their policies with regard to indexing the
Web.

~~~
ii
It's definitely _everything_ for most "normal" users (like my sister or
Russian-only speaking friends). Mentioning of Russian-only limitations or
providing English-language results would be confusing for such users (most of
them don't even know what Google is).

Not long ago I submitted .com English/Russian/German site to them and they
refused to index anything but Russian part, but Russian part was index
successfully (although the sever is in Russia).

------
rgrieselhuber
The same is here in Japan, where Yahoo has a much stronger lead. It's one
thing to localize your UI, quite another to localize your approach. The second
piece is crucial.

------
siculars
/rant

There can be no doubt that Russia has many, many brilliant mathematicians and
engineers (poets, writers and musicians too). Everyone knows this. And,
individually, I know their people can be quite lovely. The problem is that
these people are in Russia. Russia crushes all. How many times do people have
to get burned by Russia to realize Russia is a trade - not an investment? How
long before the wannabe czar declares "all your
routers/servers/data/money/life are belong to us"? "Russians", by and large,
are not affluent. A relative handful of Moscowvites are.

Mark Franchetti is obviously on the payroll but why he would write this
blatant puff piece about "yet another indexer" is beyond me. These people have
nothing Google, Yahoo and even Microsoft can and have put together. The only
thing they do have is an advantage in the understanding of native Russian
language and society. Ok, so? There is no market for Russian services outside
of Russia. The rest of the world speaks English, French, Spanish, Arabic or
some version of Chinese with a smattering of Portuguese thrown in for good
measure.

Since the Empire struck back in Georgia, international monetary outflows from
Russia have been at their highest levels since the Great Ruble Implosion of
the late 1990's. Lets face it gang, when the big ugly bear flexes its muscles
people run for cover. Nobody wants to be in bed with his highness, the dread
nightmare Putin. Who here thinks Putin has his peoples best interest at heart?
Who here thinks a goal of Russia is to increase the prosperity of its people?
Putin is out for numero uno, Putin. And the main concern for Putin is Gazprom.
Who here thinks that Yandex will continue to fly under the radar of the FSB
indefinitely? Who here thinks an independent, privately owned and operated
company can exist in Russia without connections to their
political/military/underworld/secret police?

(off topic) People in the West love to bitch and moan about the US all day
long. 'Hegemony', 'Imperialism', 'Totalitarianism' and, lest we forget,
torture. Has everyone fallen asleep at the wheel? If you are from the West and
do not have an ancestral connection to mother Russia, you have no idea what
any of those words mean. It will be many generations (hopefully that short)
before Russia is friendly enough as a national entity to do anything even
related to business with.

In closing, if this Yandex knows whats good for them they will pull up stakes,
run dont walk and leave Russia with extreme haste and plop themselves down
anywhere else. London, Paris, Silicon Valley, New York, Madrid. Pick one, any
of a dozen or more cities would be better than the dump that we know today as
Russia.

Oh, in the fairness of full disclosure, if it sounds like I am anti-Russia in
the extreme, I am. Honestly, which Olympic medalist of any color has defected
from the West to Russia? We here in America have Russia to thank for our brand
new Olympic individual gymnastic all-around Champion, Nastia Luikin :)

/end rant

~~~
asdflkj
Your simplistic view of Russia is what happens when you learn some select
facts and plug them into a formula that's designed to accommodate Western
societies. If every fact you know about Russia (assuming even that your facts
are accurate, which is unlikely if you read Anglophone media) suddenly became
true in America, things would certainly become worse. Sort of like if you
added garlic salt to ice cream, it would taste worse, and this doesn't prove
that garlic salt is bad.

For people who want to understand Russia, I suggest learning more about the
90s, and what was actually happening there while The Economist and the rest of
them were falling over themselves with praise. It will put into perspective
what they're saying now. Here's a teaser:

<http://media.hoover.org/images/graph2.GIF>

If nothing else, it will remove the desire to pontificate on "the big ugly
bear" and "the dread nightmare Putin".

~~~
rms
Suggested reading, please?

~~~
asdflkj
I don't know, to be honest. My knowledge of the 90s is first-hand, and of
today--reliable second-hand. I have relatives there.

After my experience with Russia and Western media's version of Russia, I had
been wondering how to get reliable information on nations whose cultures are
isolated from the Western culture (in particular, China). I haven't come up
with anything yet, so now I pretty much ignore what comes from the media, and
try not to form opinions.

~~~
siculars
I second you, asdflkj. It is always best to analyze first hand sources.
Unfortunately, free media does not exist in Russia. Pitty.

------
mynameishere
Wow. I can't believe they don't switch to local languages, as with google.ru.

------
ii
Another nice thing about Yandex is that they use Django in several project and
actively contribute to it. They even held Django sprint this summer in their
Moscow office.

