
A Relevant Tale: How Google Killed Inktomi - nachopg
http://diegobasch.com/a-relevant-tale-how-google-killed-inktomi
======
paulsutter
Great article. One note: Google's dynamic abstracts were not only very useful,
they also improved perceived relevance because they let users see why the
pages were selected.

When I was at Altavista, we were also blocked from doing dynamic abstracts by
cost.

Google's main advantages were:

\- managed by the founders with a total focus on search and measurable results

\- google's hiring process produced a very strong team early on

\- strong focus on controlling costs from the beginning (Altavista's use of
the DEC Alpha was a huge handicap.

~~~
gaius
When Altavista launched, it was an impressive showcase of the DEC Alpha's
power. Intel only became usable for serious servers (with the exception of
exotic stuff like Sequents), as did Linux, years later. Google had the good
fortune to be in the right place at the right time, when Lintel became a
commodity in the datacentre. 5 years earlier, they'd have been on Sun
probably.

~~~
minimax
Initial versions of Google at Stanford ran on a mix of both Linux and Solaris.

<http://infolab.stanford.edu/~backrub/google.html>

~~~
paulsutter
I'll bet you a dollar that Larry and Sergei never actually bought a Sun server
to run the search engine, but rather that they may have used some free
resource available to them at Stanford.

------
wavesplash
Inktomi killed Inktomi long before Google helped put the nail in the coffin.

What the article doesn't say is Inktomi had a dual sided business. One side
was in Caching Proxies the other was licensing a search API.

Inktomi decided to focus on the caching proxy business and de-emphasized their
search product, only to watch the proxy business evaporate as internet
bandwidth became cheaper/better.

The focus on a shrinking market (proxies) and the lack of focus on growing
market (search) killed them. Had search been a priority from the beginning
things may have ended very differently with Inktomi creating their own front
end.

~~~
jusben1369
Thanks for that. I too remember Inktomi much more as the "cache kings" with
licensed search a secondary focus.

------
darrellsilver
Another lesson from this article: If your engineers refer to those who make
decisions as "the execs" instead of "we": Quit.

------
lacker
_It was clear that Yahoo.com was the definitive result for the query "yahoo"
so it would score a 10. Other Yahoo pages would be ok (perhaps a 5 or 6).
Irrelevant pages stuffed with Yahoo-related keywords would be spam._

As someone who worked on search quality at Google for some time, this bit
jumped out at me as a terrible mistake. The correct way to judge results for
the query [yahoo] is:

(a) Where is yahoo.com? At the top?

(b) There is no (b).

It seems like a slight difference, but it leads to the wrong priorities. For
the query [yahoo], it does not matter if spam or non-spam is in spot #5. The
only thing that matters is where you put yahoo.com.

~~~
bitops
Can you elaborate a bit more on this? I think I understand your point but a
few more details would be great. Thanks!

~~~
ricardobeat
Most people don't look farther than the first 3 results, so what comes after
that isn't very high priority. Might be the reason some more specific/obscure
queries on google return terrible results :)

~~~
Zakharov
More specifically, if people find what they're looking for in the first
result, they ignore the rest.

------
avichal
I just realized that Google won on search the way Apple has won on
smartphones. They control the full stack -- frontend, relevance, indexing,
advertising -- and tightly couple these pieces. Inktomi couldn't control the
user interface the way Google can't really control the interface on Android.

~~~
dwc
You are right, but IMO you miss the real point. Both Google's search and
Apple's iPhone were about delivering a wholly satisfying product/service, and
controlling the whole stack was needed to do that _for those cases_. This is
not always true (though it often is).

~~~
avichal
Yeah, it's pretty hard to know when it is and isn't. Vertical integration is
as old as business.

------
zmmz
Relevant: <http://youtu.be/E91oEn1bnXM>

It's a recording of a very very good talk by Inktomi co-founder Eric Brewer
called "Inktomi's Wild Ride - A Personal View of the Internet Bubble"

~~~
diego
Yes, that talk is excellent. Deserves its own submission.

------
lubujackson
Most interesting to me is that Inktomi had all the power to beat, acquire or
replicate Google but didn't have the right mindset. They were operating under
a few bad assumptions:

\- search is a commodity for licensing (making them resistant to launching a
"cleaner" engine that would alienate their clients)

\- what worked for a smaller internet (100 million pages) could scale
appropriately with the growing internet (100 billion pages) without rethinking
everything

\- "Page rank" only helped relevance (it was also about spam)

I think Google is stuck in a rut of their own right now. Here's some faulty
assumptions I think Google is making:

\- users always want faster, more direct answers (rather than controlling the
filtering/categorization of their searches)

\- users want Google to predict what they mean rather than clarify what they
mean

\- algorithms > human decisions

~~~
tytso
\- users always want faster, more direct answers (rather than controlling the
filtering/categorization of their searches)

That's a very power-user centric attitude, don't you think? As a power user I
preferred to type long, complicated Sabre queries to find exactly which
airplane flight I wanted. It was much faster, and I had memorized all of the
complicated mnemonics. But that's not what a casual user would want to use.

Asking users to specify categories for what they want means requiring a
certain orientation in their thinking which is shared by computer scientists
and trained librarians. But to an average user, that's extra work. And think
about how this might work if you're talking to an actual human librarian: if
you start asking about TV shows, and then mention "The Big Bang Theory", do
you think the librarian will ask you, "Did you mean the scientific theory, or
the TV show?" That's only something a stupid computer would do. A smart
librarian would take the context of the previous queries that you've made of
him or her, and provide the right answer quickly and efficiently. Wouldn't you
want the same thing from a search engine?

------
bstar77
After I had switched to Google, I never understood why all of the competition
just disappeared over night. You would think they would have given it a fight,
but that never seemed to happen. At least this article gives a little insight
to that. I still wonder what happened to Altavista.

~~~
lobster_johnson
AltaVista tried to jump on the 'portal' bandwagon. I remember at the time how
stupid I thought they were, trying to beat Excite and Yahoo at something that
was already old hat, a concept that had the Internet had outgrown. Then they
screwed up pretty much the same way Inktomi did.

AltaVista still exists. It's awful, and it's powered by the Yahoo search
engine. Which is pretty much the same thing, I suppose.

~~~
dangrossman
Yahoo! bought Overture (which owned AltaVista) way back in 2003. They only
replaced the search results with Yahoo!'s a year ago after announcing the site
would be shut down. Seemed like they were using it as an occasional test bed
in the meantime.

~~~
simonw
Yup, when I worked at Yahoo back in 2006 I remember altavista being described
as a useful site for running experiments.

------
chrisacky
I had no idea that that Diego worked on Inktomi. Although it makes sense why
IndexTank worked fantastically.

If I had to pick one reason why Google triumphed (and you can only pick one),
I think it would be their Page Rank algorithm. It added that extra bit of
awesome-sauce to and already tasty stew.

~~~
silvestrov
I'd say: The Google CEO's have the point of view from an end user.

The Inktomi managers had the point of view of a generic MBA.

Everything followed from that difference of view.

~~~
Produce
I would modify that to read (since this article even stated that the engineers
there found this to be obvious):

>The Google CEO's have the point of view of an engineer.

And, more generally:

>The Google CEO's have the point of view of the people doing the actual work.

The lesson to take away from this is that one shouldn't try to manage what one
can't do themself. The disconnect between the manager and the problem domain
becomes too great and they end up making ridiculous decisions since they are
acting on the wrong information.

~~~
wpietri
I'd disagree on a couple of points.

First, engineers very often build crappy products when left to their own
devices. The ones that they do the best at are ones where they are also the
users and are more or less representative of the target audience. Google's a
great example, and so is Firefox. So I think Google execs' perspective as a
user was much more important. Consider as contrast Google Plus, which
definitely was not built because was an avid but dissatisfied Facebook and
Twitter user.

I think your second point is almost right. You should never try to _control_
things you don't understand. It's ok to _manage_ things you don't understand,
because good management in that case is not directing the people who
understand, but supporting them in achieving common goals.

As a non-tech example, few hospital administrators can perform brain surgery.
But that's fine as long as they ask the brain surgeons what they need rather
than telling them how to work.

------
FilterJoe
According to a friend of mine who worked on the search team, Inktomi shifted
its (management and CapEx) focus away from search and onto other projects. He
thought at the time that even with the constraints of not competing with their
own customers, there were things they could have done to better compete if
management had chosen to do so.

Edit: fixed typo

~~~
diego
Yes, that's true. They didn't think search would be a huge business. Back then
the model was to sell search to portals charging by query volume, and it was a
race to the bottom. Our Solaris servers were more expensive than Google's
Linux boxes.

------
kloc
After reading the "Tale" it appears that Inktomi killed itself. It is a good
example of what happens if top dog companies fail to innovate in face of
sudden superior competition. RIM is another example, but would be wrong to say
that Apple is killing them. Apple is just making and selling superior
products.

~~~
gee_totes
_Apple is just making and selling superior products._

Agreed. If I had a nickel for every time my Blackberry crashed since I
upgraded the OS a year ago, I'd buy some Apple stock.

------
gaius
Not competing with your customers is a not-invalid reason, tho' there are
real-world examples that go either way.

~~~
diego
True. In this case, I forgot to mention that not having a world-facing UI
deprived us of vital signals that Google used to improved their experience. We
had query logs, but we had no idea of what our customers' users were doing
within a session, no user history, etc.

~~~
nirvana
We had a world facing UI, and all the insight that produced, and we _still_
weren't able to sway management that there might possibly be a profitable
search business.

In the age of napster people were using our service-- and falling in love with
it-- finding music on the internet.

Nope, no money there. Better to sell to ISPs in canada and hope they integrate
our results with Inktomis.

------
drucken
Think I would qualify his key takeaway somewhat:

"Are there any lessons to be learned from this? For one, if you work at a
company where everyone _wants to use_ a competitor's product instead of its
own, be very worried."

This is because companies sometimes (maybe often?) ban the use of competitor
products to their detriment.

------
dm8
That was a good read. I remember I used Yahoo for searching the web. Due to
relevancy factor I moved to Altavista (but it didn't improve anything until
the day I found out about Google and still use Google). I didn't know that
Inktomi was powering search at that time. If Yahoo was so dependent on Inktomi
or Google for its Search, I wonder why didn't they work on Search by
themselves. After all they were information organization tool. Why did they
ignore such a huge market. VC's were going crazy for funding search engines
and number of search engines companies were either getting funded or going
public. Based on these signals and the traffic they had during dot com era,
they could have easily built substantially good search engine; yet they
ignored it. Can anyone shed light on it?

~~~
thrownaway2424
Yahoo owned Overture so their results were pure pay-for-placement. If anyone
wanted to pay $100 per click to have the #1 spot on "beef" go to a site about
chicken, that was a-OK with Yahoo management. When Yahoo realized their
auction system was stupid, they had "project Panama" which was also a joke and
by that time Google had the market to themselves anyway.

------
aidenn0
Inktomi engineers using Google reminds me of TI engineers using HP
calculators. TI banned them, which apparently just moved the calculators under
the desk

~~~
ajays
When I started at Yahoo Search (in 2005; it was Inktomi), I quickly learned
that "just Google it" was frowned upon, and modified my vocabulary to say
"just use web search". To this day, I still use this phrase.

Most engineers that I knew in YST did not use Google at all. We preferred to
eat our own dogfood, and filed query triages against bad results (and only
used Google to compare).

------
ChuckMcM
Ok, so the timing on this is really amazing. Techcrunch, reporting on
Facebook's S1, mentioned that Yahoo! has suggested another 12 patents they may
try to throw against Facebook and their Open Compute project. The Yahoo!
letter is here: [http://www.scribd.com/doc/92280387/TechCrunch-Letter-From-
Ya...](http://www.scribd.com/doc/92280387/TechCrunch-Letter-From-Yahoo-to-
Facebook-regarding-Yahoo-Inc-Server-Patents-and-Facebook-Data-Centers) and the
source of many of these patents? Inktomi!

Is that crazy or what?

------
kenrikm
Why I switched to google.

A) it was fast, it loaded fast. B) it was not filled with ads and pop ups.

Only one of those is still true.

~~~
rkudeshi
I suspect quality results also played an outsized factor in most people's
switching, even if they don't admit it.

Easy test: are you using Bing now? (with their new clean results pages)

~~~
wazoox
Oh yes. I remember back in early 1998 I was using Copernic to aggregate and
filter results from a whole bunch of search engines, because individually
their results where so poor and irrelevant. Then I discovered
<http://google.stanford.edu> and never looked back.

~~~
cglace
Same here except I used dogpile.com

------
brown9-2
_In the next year and a half the stock went down by 99.9%. In the end, Inktomi
was acquired by Yahoo for 250M._

This is unrelated to the main point, but does anyone know if Yahoo re-used a
significant amount of Inktomi tech acquired for that $250M? Or was it spoiled?

~~~
brendano
I've been told (by people from Inktomi who then worked at Yahoo), that yes,
the Inktomi technology became the main Yahoo search engine. (How much it
changed or not in the process, I don't know)

------
The1Mirage
Anyone else notice the really weird link supposedly of the Ipad Screen
Resolution that contained the initial query of Domino Pizza Phone Number?

x's so it won't become a link!

hxxps://wxw.google.com/search?ix=aca&sourceid=chrome&ie=UTF-8&q=domino+pizza+phone+number#hl=en&gs_nf=1&tok=DBTJEp2_3oW1F2ietDMecQ&pq=ipad%20screen%20resolution&cp=1&gs_id=5w&xhr=t&q=new+ipad+screen+resolution&pf=p&safe=off&sclient=psy-
ab&oq=nipad+screen+resolution&aq=0&aqi=g2g-b2&aql=&gs_l=&pbx=1&bav=on.2,or.r_gc.r_pw.r_cp.r_qf.,cf.osb&fp=8316e992ae23057e&ix=aca&biw=1363&bih=647

------
cabalamat
If the stock went from 25 G$ to 250 M$, that's a decrease of 99% not 99.9%.

~~~
diego
It peaked at 241 1/2, bottomed out at 24 cents in 2001 before going up 10x
again.

------
troytoman
Diego,

Thanks for capturing the train of thoughts that seem to run through my head
almost daily. Lots of lessons to learn from that experience. Although it is
easier to connect the dots in the rearview mirror that it was looking forward
at the time, There were some clear lessons about not forgetting the actual end
user (which is not always your customer), using a single metric as a proxy for
user experience, obsessing about a competitor and trying to get big instead of
great.

------
andrewfelix
As an Australian I used to use 'Anzwers' as it seem to give great relevance
for Australian specific information. I think it used Inktomi?

I'm not sure why I switched to Google. Not to discredit the Google UX, but I
think I switched because the name 'Google' was so catchy and eased it's way
into my university's vernacular. "Just Google it" rolls of the tongue nicely.

------
yaix
Wow, not only is there still a search page under hotbot.com, its also still
run by Lycos! It's like time travel...

------
xunzhang
I want to check out at that period of time does stuff in Google use Inktomi to
do some experiment also?

------
dpacmittal
From the article, it's clear that more than Google, inktomi killed itself.

------
gruseom
I remember why I switched to Google: 1. speed, 2. simplicity. Relevance, meh.
Never noticed much difference there. This led me to believe that PageRank was
more about marketing (brilliant marketing) than technical edge.

~~~
mcguire
Are you kidding? I was using Altavista, IIRC, and would routinely have to page
down to the second or third page to find any kind of relevant link. When I
switched to Google, I almost immediately stopped looking at more than the
first page of results.

~~~
gaius
There was nothing wrong with Altavista's algorithm for "finding stuff" - it
was just too vulnerable to SEO pollution. I remember the quality of it
declining almost overnight once spammers (yes that's what SEOs are) figured it
out.

------
buster
Sounds much more like Inktomi killed Inktomi.

------
nirvana
This was a very interesting read. I was working for a vertical search engine
in this very same period. I, and the other engineers, also attempted to get
our management to recognize what google was doing right. Unfortunately, we
were delayed greatly by bad technology choices forced on us by venture
capitalists (e.g. "build your search engine on top of Oracle! they say they
have full text search, it will save you time!"[1]) and management short-
sightedness ("our future is selling audio video search results to ISPs and
portals, not being our own portal." -- this despite google not being a
portal.) They actually got worried when the search box on our homepage started
getting more queries than some of our larger customers.

Now, FWIW, I'm building another search engine. Instead of 20 engineers we have
just me. Instead of 4 years, we're going to do it in one. While I have no
interest in going up against google (different plans entirely) the radical
change in leverage you get with open source and PAAS or IAAS, combined with
Google's having taken their eye off the ball and run off to chase Facebook
down a blind alley, means that something like DuckDuckGo actually could take
real share away... maybe. (%1 of google's volume would be "Real share" right?)

[1] Oracle did have full text search but did not have the performance or per-
machine-efficiency we needed, so it cost us a lot, and it was a constant fight
to get it to do the kind of queries our relevancy algos required. we had a
constant stream of consultants in from oracle HQ, and in the end dumped it and
wrote our own db from scratch in about 4 months.

~~~
goochtek
As you haven't mentioned what exactly your search engine will focus on, I'll
just throw this idea out there. I would love a search engine that returned
results equivalent to Google, yet offered me the privacy level that I desired.
With Google, currently I am the product. I know this. I understand this. Thing
this is, I don't want to be the product any more. I want to pay for the
product on say a yearly fee which guarantees me I won't be sold ads or have my
search habit information sold.

~~~
davidjohnstone
Like <http://duckduckgo.com/>?

~~~
simonh
It seems like a nice idea. But I use other Google services like GMail, Google
Docs, Google Maps, etc. If I switched to duckduckgo but still used the other
Google products I'd feel wrong about it. There aint no such thing as a free
lunch.

Now if their search results got so bad that other alternatives were clearly
superior, that's different. They have an obligation to provide a good service
to us the customers too, but we're not there yet. For me.

~~~
bira
After the recent penguin update, their search results have got REALLY bad.

Don't you guys agree?

That's an opportunity, right there.

------
RedwoodCity
Bitter?

