

Mistakes Google made in scaling its organization - plinkplonk
http://www.quora.com/What-are-the-main-mistakes-Google-has-made-in-scaling-its-organization?srid=n7

======
shadowmatter
As an ex-Googler, I disagree with this second-hand account. Yeah, political
infighting sometimes happens and things get shook up, but this person paints
the picture of no project ever launching. Addressing some points:

1\. See item 3.

2\. VP changes rarely happen compared to a PM change. And a PM who comes may
change some of the goals, but the focus area of your product doesn't change,
e.g. your optimization tool for display advertising will continue to be an
optimization tool for display advertising. It would be hard to find your
project suddenly disagreeing with some "existing strategy."

3\. New products that launch should have a UX designer contributing or at
least advising. There should be mocks for the frontend engineers to follow,
and the backend engineers are just told to make it work.

4\. Yeah, there's a checklist, and it is long. SREs want to minimize the
number of pages there, but pages happen because something's going bad, and
they want you to have proper monitoring in place to address the pages quickly.
The launch checklist also focuses a lot on addressing potential security
vulnerabilities. Google is tough on security.

5\. Google's all about the software compensating for commodity hardware. I
don't think too many teams have specialized hardware. You can ask for what
resources you need, but would have to fight for what form you get them in.

7\. Can't speak for this. Why reorganize marketing?

8\. While I agree that Google likes to fail fast, in my opinion some Google
products probably didn't get the love or time they needed to take off. I
wasn't really privy to how those decisions are made.

------
birken
I think calling Google processes "mistakes" is a little strong. For example, I
recall at one point at Google where they were trying to improve latency on the
products, so you weren't allowed to launch a new feature if it hurt latency.

I could imagine somebody who just made an awesome feature that made an
application 5% slower complain about how stupid that policy is. However, they
don't realize that however awesome their feature is, speed of applications is
a hidden cost that everybody bears, and sometimes the priority for a company
may differ with a specific developers vision. Google wants their products to
be fast, wants them to be available, and wants them to be secure. I'm not
saying that everything they do is great and their processes and checklists are
optimal, but when they have processes designed to do a certain thing, it isn't
fair to say their processes are a mistake because they don't do something
else.

~~~
lemmsjid
Very much agreed. When an organization becomes large it needs to make tough
choices. In an organization built around product-centric teams, it may be
easier to release products more quickly and the average employee may perceive
a smoother course from concept to execution, but the organization will quickly
discover that horizontal changes become extremely difficult. The frustrated
product developer who can't get products out is replaced by the frustrated
operations guy who can't patch unsecure systems fast enough. Which one is
worse? For Google, I would say the latter. While it always seems like a
tragedy to hear about "great" products getting killed, pre-birth, in the
corporate mill, I much prefer my stable, high uptime, browser-performant
Google to a Google that releases way more product than it already does.

Not to mention that most products (like most startups) fail. As a product-
centric company, do you break teams apart after a product fails? If a product
succeeds, do you take that successful team and move it to the other product?
That may work if you're lucky enough to have a team with the proper skills
balance (not to mention passion) to move across different products. What about
career goals? People in a particular field tend to want to work with the best
people in that field--a huge draw of working at Google is working with the
best people in Field X. In a functional organization, the functions (almost)
never go away--so the teams are stable.

Of course, being on the outside I can't speak for the problems that are
mentioned, such as in-fighting, lack of executive discipline ("You will not
launch your product because I don't like your hair style, FUUUUUUUUU!"), etc.,
but those problems are bad regardless of organization type.

~~~
VladRussian
to me presence of in-fighting is a manifestation that whatever executive is
responsible for both parties and the issue at hands is just not able or
avoiding doing his/her job - taking decisions. Usually it is a combination of
the inability and being afraid to take the decision and responsibility for it.
"Fish starts to rot from the head" as they say in my old country.

------
Hominem
Honestly that sounds typical of a large organization, everyone wants to put
their fingerprint on a project when it's way up. Once everyone gets their
hands on it, every vp packs the team with their buddies, burdens you with no-
show contractors from a firm they have a "relationship" with. Then everyone
starts scuttling away to the next big thing. Of course someone gets bored one
day, and decides to redefine the entire scope of the project, pushes it as the
next big thing, then everyone come scuttling back.

~~~
cageface
Exactly. It's a scramble for symbolic ownership that obscures the fact that
everybody is increasingly reluctant to take real ownership. Accountability
doesn't scale well.

------
adambyrtek
It's always good to have an open discussion, but I don't find it convincing
when an anonymous guy on Quora based on second-hand unverified information
claims that he knows better what is good of bad for Google.

------
SoftwareMaven
I've never seen success when an organization organizes around business
function. Invariably, the function takes precedence over the product.

~~~
CaptainZapp
I'd enter Oracle as a counter example.

Personally, however, this is not something I would want to achieve, but to
each his own I guess.

~~~
jbooth
Maybe a corollary for businesses whose product is based on lock-in. Poor
software design is sort of the point as long as you keep getting your hooks
into people.

------
Steer
Not sure if this is off-topic, but have any company ever grown as fast to be
as big as Google have? Successfully?

------
yason
In the light of this article, Google should have a play area with only limited
network and processing power for publishing betas of new products.

For example, anything that's Google Search or GMail or similar must meet some
performance, stability, and realiability criteria but those applications would
also get to run on the big clusters. Conversely, it wouldn't matter if a beta-
toy is a bit buggy or hogs down the servers on the playground since it's just
playground.

Some users would bear with the unfinished tone and start using the apps there,
and at times it would happen that one application would gain lots of momentum,
and Google could consider fixing and lifting that one to the full service
level.

------
ccarpenterg
That is not a mistake. It's a corporation.

~~~
staunch
That is not a mistake. It's a large corporation. But I repeat myself.

------
ojbyrne
So this basically reads as someone who's taken a couple of Org Design courses
in college arguing that it's actually significant whether the top level of an
organization is "product" or "function" ("geography" is another alternative),
and as a result he basically comes across as someone who could use a little
"experience" so that he understands that scaling an organization isn't just
about some analytical framework that he learned in school, but about people,
dynamics, history, politics, egos, a lot of stuff that is really "hard."

And that almost every fast growing company gets it, in some form or another..
"wrong." But they also find success despite their failure.

Whereas the guy who took the couple of Org Design courses probably ends up
working for a VC running innumerable other companies straight into the ground.

~~~
babar
I would encourage you to talk to some people who you trust who have worked on
a few different projects at google (especially anything not related to search
or ads) and see if they think this answer is theoretical.

If you don't have people there you can ask, take a close look at what has
happened to the products and teams that google has acquired.

~~~
notauser
The problem is that people who say it's a "functional organization" problem
are both correct and incorrect at the same time.

The kind of problems described certainly seem to have been exacerbated by the
way they are structured. But all organization structures have their own
problems, and the issues caused by (say) a project based organization* might
have been even more detrimental to success.

The most professionally managed organizations I have worked for have generally
been matrix orgs (sort of functional, with projects 'borrowing' resource for a
year or two at a time) and that worked pretty well, but only because they
burned a lot of time and money managing the process. In many cases it may be
cheaper just to live with doing it imperfectly.

*Lack of product integration, break up of technical pool of talent, even bigger resource fights, duplication of effort, lack of communication, and zombie projects amongst other things.

------
Joakal
What can Google do to fix those underlying issues?

~~~
danbmil99
Allow autonomy to product groups. They should be run like little startups:
choose whatever tech stack they want, hire their own PM's, do their own hiring
(Google recruiting is a complete wasteland), and live or die by their
scucess/failure.

It'll never happen, because it's diametrically opposed to the founding
ideology, which is that every problem has a rational solution. Without getting
too Ayn Rand on this, it's the classic mistake of central planning vs personal
initiative.

~~~
Hominem
Are you saying they should be able to go off into the wilderness, decide to
run on Heroku or something and come back with an app? Companies invest huge
sums of money on infrastructure and the expertise to manage it. Google has too
big an infrastructure to let each project roll it's own.

Let's say google reengineered to provide something like ec2,each team can run
it's own images. does each group hire security experts? Does each group have
to solve scaling problems google already faced?

I think maybe it could work like this.Google proper hires domain experts,
security, engineering etc. Provides each mini startup with a budget, mini
startups can spend the cash on the Google proper experts, or go outside for
areas where Google is lacking. Mini startup pays hosting fees to run on
Google's EC2 like infrastructure, or use whatever Google has in place now.

~~~
waterlesscloud
What if you let teams select from internal security teams, internal scaling
teams, etc?

You can run a wide range of things on EC2, but I'd be willing to bet certain
sets of things dominate.

If the solution google has internally really is the best solution available,
let it compete. It'll win until it's no longer the best.

~~~
cma
You'll get an internal 'Microsoft' who focuses on things like lock-in instead
of quality, and other zero or negative-sum games like that. You'll need an
internal patent and copyright system to keep one team from freeriding on
another, etc. etc.

~~~
Hominem
Right, I don't have to implement a queuing system because a guy down the hall
did it. But he sunk his budget into developing it and I get to swoop in and
benefit. That is not a bad thing, that is how corporations are supposed to
work, it would be worse if we burned two budgets implementing a queuing
system, but it is a bit unfair that he burns his budget for my benefit. Maybe
if I paid him "licensing" fees it would be more equitable, but then we have
another porcupine to hug, managing all the internal licensing.

