

Hiring: The Lake Wobegon Strategy (2006) - awa
http://googleresearch.blogspot.com/2006/03/hiring-lake-wobegon-strategy.html

======
johngalt
This assumes from the outset that you know the difference between a good
future employee and a bad one. Restating the problem so that the hard issues
are out of scope makes any strategy look good.

How to find the perfect car: Just pick one better than all the previous cars
you've owned.

How to find the perfect date: Only ask women more attractive than all your
previous girlfriends

~~~
metageek
"This assumes from the outset that you know the difference between a good
future employee and a bad one." -- Not really; his simulation assumed the
hiring process had error bars of 15%.

~~~
krschultz
It should probably be a lot higher than that.

~~~
crystalis
And that only further proves the importance of Norvig's point.

------
thorax
While interesting, I wanted to point out that this is very old (2006) and may
have changed somewhat since.

Now that they've grown to a certain size and have offices in different places,
I would imagine they (just logistically) have to hire at the "site" level
rather than at the company level now, which would encourage each site to bring
in employees at a different mean than the entire company itself (i.e. like the
hiring manager issue).

~~~
timcederman
Not really. I was given a choice of offices during my interview process in
2008, when they had more employees then than they do now.

------
WillyF
I don't care what weaknesses you can find in this post or Google's strategy,
the fact that they're thinking about talent in this way is impressive.

Typically HR and statistical analysis don't mix.

------
techiferous
I worked with someone who told me about a hiring strategy he encountered at a
former employer. HR came across some study that showed that people with high
IQs were the most likely to have personnel issues (or didn't work as well on a
team, or something like that). So they decided to avoid these personnel
problems by giving job candidates IQ tests and only hiring those with IQs that
weren't above the company's mean IQ. I kid you not.

~~~
megablast
Was this the Police Force. They regularly reject people who are too smart,
because they would get bored in the job.

~~~
jeffcoat
I've heard this story, but I can't find any evidence for "regularly". (I'm
also not sure what form such evidence would take.)

The highest profile case I can find is of Robert Jordan, of New London, CT,
who sued for discrimination because his job application was rejected for
scoring too high [sic] on the IQ test. (He lost.)
([http://www.nytimes.com/1999/09/09/nyregion/metro-news-
briefs...](http://www.nytimes.com/1999/09/09/nyregion/metro-news-briefs-
connecticut-judge-rules-that-police-can-bar-high-iq-scores.html))

It appeals to my own personal prejudices, but it was also 10+ years ago.

~~~
techiferous
From the article:

"I was eliminated on the basis of my intellectual makeup," he said. "It's the
same as discrimination on the basis of gender or religion or race."

If he got what he was asking for, employers would not be able to reject job
applicants that were not intelligent enough for the job.

------
strlen
Would it be possible for the source for this simulation to be released? To
allow others to play with this. Namely, I'd like to see:

* Effect of Microsoft/Netflix policy of periodically laying off the bottom n% of the company

* Effect of churn (which tends to hit both the bottom and top more so than the average).

* Seeding by initial founding engineers. Founders generally have to be strongly competent engineers (95th-98th percentile), but not very many are actually 99.9th percentile "10x" engineers (e.g., Brad Fitzpatrick of Livejournal, Bill Joy of Sun, Max Levchin of Paypal).

* Speaking of "Nx engineers", how about a logarithmic (with log_2) scale of skill rather than a linear?

Of course, there is also the question: an interview shows tests aptitude (and
an error rate of the interview process implies error rate in determining
aptitude), but more factors play a role in actual measured performance. I
wonder if this is factored in to the metrics:

    
    
      foreach interview_score in range(1,5):
        foreach year in range(1,4):
          foreach performance_score in range(1,5):
            p ( performance_score | interview_score ) at year 
    

A company very likely must (by law) have performance_score data and it should
be easy to reconstruct interview score data (unless interviewers are sloppy
about writing down their feedback and decisions are made by a hiring manager
based on oral input).

------
fizx
Here's a more detailed simulation that I just wrote in javascript:

\- I assume an error margin in our ability to evaluate employees.

\- I assume an applicant pool that isn't necessarily centered on z=0.

\- I evaluate z-scores, not percentile.

\- I calculate the average number of interviews for a successful hire.

<http://fizx.github.com/hiring/>

------
dfranke
I can't take seriously the assumption that skill level in this context is a
scalar.

~~~
apgwoz
Yeah, I'd really like to know how they get the Y-axis. All of the google
interview recaps I've ever read have made it seem as though candidates aren't
likely to get the same interview.

That said, it could be that their interview questions are like the questions
on the SAT, or GRE, with the difficulty being normalized in some way.

------
krupan
Hmm, when I looked up "The Lake Wobegon effect" on Wikipedia, it linked to
"Illusory superiority." I'm guessing that's not quite what Google is going for
here.

~~~
Periodic
I was wondering about that as well.

In Lake Wobegon, "all the women are strong, all the men are good looking, and
all the children are above average." The Lake Wobegon Effect is one where most
people think they are above average at something they are good at.

In this case, I guess they're just hiring so that everyone is above average.
It makes some sense, but it does have a darker double-meaning.

------
awa
What I found interesting was that the no hiring manager strategy doesn't help
much... also I wonder where they got the data from for other hiring strategies
that they don't use.

~~~
btilly
That was just a Monte Carlo simulation.

If you change the parameters of the simulation, you will get different
outcomes.

------
paradox95
Here is how to get hired at Google:

1\. Start and sell at least one company. 2\. Set a world record - the more
obscure the better. 3\. Have a Ph.D - it doesn't matter what it is in. 4\.
Know at least one manager at Google. 5\. Do all of the above before your 25.

~~~
jeffcoat
Sure. Do that, and you can get hired anywhere.

1, 2, 3 are by themselves exceptional-- pretty much by definition, setting a
world record in anything is unusual. Having started and sold a company means
you've built something worth real money. (The Ph. D. is similar, though it has
a different metric.)

[Really, it'd be a lot easier to study algorithms and graph theory and ace the
interview.]

------
tjmaxal
All this really is is an application of micro economics to your hiring policy.
In an ideal world this strategy works but as others have pointed out, without
perfect data, the noise builds up so that it is hard if not impossible to
maintain a rising average.

------
studentrob
1\. Who says growth must come from hiring better or even more employees?

2\. Does Google feel they already make the most of the talent they have?

3\. Do existing Google employees feel they themselves are in the best position
to shine?

~~~
mkramlich
Regarding #2: I think they're not making the most of the talent they have.
Google does some great/cool things. But I expect a lot more of that based on
the number of employees they have, and the average level of talent/IQ they
supposedly have. I suspect the discrepancy is caused by the dampening effect
of having a large infrastructure and bureaucracy, and a possible
clone/conformity effect from their hiring process. I believe their hiring
process mostly filters for the wrong thing. At least with respect to folks who
aren't already famous to begin with.

------
apike
For a lot of employees, the project they'll be working on and what their
manager is like are major factors in whether they want a job. Wouldn't a "no
hiring manager" strategy cause problems because of this?

~~~
blacksmythe
I understand (at least in the past) Google makes it easy to move between
groups, so you are really joining a company, rather than a group. In most
companies if you don't like your manager, your best option is to find another
job.

