

Ask HN: Why did Google release their mapReduce algorithm/cluster setup? - monological

I was just wondering why Google, back in 04, released their map-reduce white paper. They didn't have to. Was it released simply out of good will? Was it done to fend off antitrust litigation? Were other companies already doing the same thing, so it wouldn't matter if it was released? I think they believed that the amount of information that was revealed in the paper wasn't low level or in-depth enough to cause any real damage to Google, even if it were implemented(Yahoo's hadoop). Or am I missing something completely?
======
SwellJoe
To engender the feeling that Google is a factory for big new ideas, making it
the place every top-flight developer wants to work, and the tech company that
everyone talks about when innovation is the subject. I'd say it's paid for
itself, and I'd also suggest that it hasn't hurt them. Google has as much of
the search market as ever, as far as I know, and the only dents have been due
to non-technical factors (Microsoft, and others, modifying the search bar in
browsers, ISPs making deals to push other engine, etc.).

~~~
scorpioxy
I would also add good karma. As in, it doesn't hurt them. I say it was also to
showcase their tech and provide more confidence in their later services.

------
bayareaguy
In this particular case the ideas weren't particularly new - when I first
heard about Map-Reduce I thought people were talking about MPI's AllReduce
function <http://www.mpi-forum.org/docs/mpi-11-html/node82.html> from the 1995
MPI spec and even then it was nothing new. Teradata and others shipped systems
operating on the same principles a decade earlier.

But to answer your question: all large companies need to publish the central
technical ideas in one way or another to ensure their own current and future
technical staff understand how their systems work (e.g. IBM Systems Journal).

------
geocar
Distributed systems are nothing new (AST's Amoeba goes through some of the
mapreduce contortions), but there are very few programmers comfortable
thinking that way- you may recall, even SONY had an awful lot of trouble
recruiting talent.

It's in Google's best interest to increase the market size of "parallel
programmers"- to actually be able to pick from more programmers who can
"think" in mapreduce, improves their hiring options.

I doubt this fact escaped Google: mapreduce stands on shoulders of its own
giants who faced this problem as well.

------
inklesspen
Or maybe, like many scientists, they were truly interested in advancing the
sum of human knowledge and the state of the art.

~~~
monological
I'm just afraid they're being driven by their desire to increase revenue
rather than benefiting humanity.

------
jlouis
Probably done to get feedback in the area. If you publish a paper on a better
map-reduce strategy, Google benefits. The important thing is to seed the
research community with the right idea.

Of course, when some researches comes up with a better idea, you headhunt him.

