
Google's Secret Weapon: MapReduce - hellacious
http://discussionleader.hbsp.com/stibel/2008/12/mapreduce-googles-secret-weapo.html
======
henning
Articles like this just encourage pointy-haired bosses who know just enough to
dangerous and annoying.

"We're shifting over to a MapReduce-based architecture, just like Google!
Google's website is only an _interface_ to a larger architecture. It all ties
in to cloud computing and SOA and..."

~~~
olefoo
bingo!

------
enomar
Vague and misleading at best. If the author does actually understand
MapReduce, he's taking the brain analogy a bit too far here.

~~~
frisco
I really doubt he does understand it. That's a terrible explanation of
MapReduce, and then makes it sound like it's secret Google tech that will take
over the world-- parallel computing isn't new, and even MapReduce itself has
an open source clone now. It's really surprising to find an article like that
on HBR.

~~~
incomethax
I think he must have heard the term "Artificial Neural Network" and made some
hasty inference that it means something having to do with the brain.

On a side, is it just me or has HBR putting out more link bait in the past
couple of weeks?

------
jgrahamc
It's interesting how much noise there is around MapReduce. I'm working as CTO
of a start-up with a very large dataset (terabytes) where we need to do quite
complex queries across the dataset and very quickly.

Naturally we are taking a distributed approach to the problem since hardware
is cheap and relatively easy to coordinate via software.

We decided to go our own way using a mixture of some databases and raw file
system access with our own application coordinating both the cluster of
machines and performing the queries themselves after performing extensive
benchmarking of everything.

We looked closely at Hadoop and came to the conclusion that MapReduce was just
the wrong architecture for our problem. Its performance was horrible.

The moral of this story is that MapReduce is interesting, but that doesn't
prevent you from having to actually test algorithms and make an informed
choice. Just because Google's using it doesn't mean it's a panacea.

------
jsteele
I didn't get through the whole article, but "Google is building a new secret
weapon that has more to do with the brain than search."? Google published
their MapReduce paper in 2004...
<http://labs.google.com/papers/mapreduce.html>

------
vlod
errrh why is this on hacker-news? does anyone here not know what mapreduce is?

i got really excited when i was about to find out abt googles secret sauce..
_sigh_

~~~
michaelneale
Yeah its the worst kept secret in the world then !

------
kqr2
For open source, there is hadoop:

[http://hadoop.apache.org/core/docs/current/mapred_tutorial.h...](http://hadoop.apache.org/core/docs/current/mapred_tutorial.html)

~~~
wheels
Latest version of Qt also implements MapReduce patterns (only for multi-core,
not across the network). There's also good old MPI, which isn't terribly sexy,
but gets the job done. I found this post rather interesting when I ran across
it:

<http://jaliyacgl.blogspot.com/2008/07/july-2nd-report.html>

------
sh1mmer
Map Reduce is hardly secret and it's not like other people such as Yahoo!,
Facebook and others aren't using it. Google might have a head start but that's
about it.

I wonder how long that head start will last with a collaborative Open Source
project backed by some other big players (that would be Hadoop).

------
dchest
Harvard Business Publishing? They edited my comment.

I wrote a comment, part of which you read see here:
[http://discussionleader.hbsp.com/stibel/2008/12/mapreduce-
go...](http://discussionleader.hbsp.com/stibel/2008/12/mapreduce-googles-
secret-weapo.html#c038971)

Then a few hours later my comment has been edited -- they removed a few
sentences. (I don't remember the text I wrote exactly, but I said that: 1) the
author of this article should have showed his text to at least some CS
student, 2) I didn't expect that Harvard Business Publishing could publish
articles by such incompetent authors.)

Here we go. They could have deleted my comment, but instead they decided to
edit my speech. Why just they don't rewrite everything I said?

~~~
dchest
Great, now they deleted my comment. Better than editing it.

------
lsb
Their secret weapon is not their assembler language, MapReduce, but their HLLs
atop that, like Sawzall, <http://labs.google.com/papers/sawzall.html>

------
jcapote
I stopped reading right after I read that "MapReduce let's google use the
internet to think"

~~~
ggrot
Oh you should keep reading, it gets even funnier.

------
geuis
I am interested in learning about map reduce but I have yet to find a very
simple tutorial that explains what it does and demonstrate at a very simple
level how it works

~~~
vlod
its not entirely obvious and you think its more complicated than it really is.

you might want to try: [http://stackoverflow.com/questions/28982/simple-
explanation-...](http://stackoverflow.com/questions/28982/simple-explanation-
of-mapreduce)

or a whole bunch of tech talks: <http://vodpod.com/tag/mapreduce>

the first 2 videos should give you a good overview

