
The "well-calibrated interviewer" curse - chuhnk
http://rachelbythebay.com/w/2013/01/04/cal/
======
esperluette
Loved, loved the "I let him dig the deepest hole he could imagine by doing my
best ditzy/derpy voice and just saying things like "they want me to ask about
the load... average...?" bit. I do that too. Honestly, if you're patronizing
me _in the interview_ , you're gonna be insufferable to work with.

The best candidates light up when they have a chance to explain something
cool, no matter who is listening and get even more excited when you ask a
question about a detail. Good candidates are patient and thoughtful and ask
YOU check-in questions ("so now we have X, right?") Mediocre candidates get a
bit lost, sometimes overshoot or undershoot the right level of technical
detail, but muddle through and sigh in relief at the end. Asshole candidates
try to dump a bunch of smoke and mirrors on you, say shit like "well, if you
haven't taken an algorithms class ...," wave off your questions, and pretend
they don't have enough time to answer the question "seriously."

If you think in your engineering career you're never going to have to explain
"how something works" in at least some detail to a semi- or non-technical
person, you are not going to have much of an engineering career.

~~~
nerfhammer
I thought that was needlessly vindictive. If you don't like the guy that much,
just stop the interview.

~~~
tptacek
Vindictive would be halting the interview immediately because the candidate
made the (egregious) mistake. She's playing this part of the story for humor,
but one imagines that a non-brain-damaged candidate could immediately correct
for that boneheaded move. "Wow. I'm so embarrassed. I must sound like a
complete tool. It's been a rough couple weeks dealing with HR teams. I'm very
sorry, can we start over?" And, what she's saying is, predictably enough, most
nerd candidates don't do this.

~~~
xentronium
I don't know. In a world of sucky interviews, wasting a candidate's time on
purpose, because they mistook you with an HR (statistically, that's a much
more probable occurrence) is as unethical as it gets short of plain lying in
the interview result sheet.

I can only hope that her not-that-productive-in-interviewing colleagues don't
do such shit.

~~~
tptacek
If she'd just stopped the interview, some other HN commenter would be writing
an angry post about how _that_ was unfair, too.

~~~
jerrya
_"Oh, this is (candidate), but let me save us some time. I thought this was
going to be a technical interview." That guy bombed the interview right then
and there. I didn't tell him that, though. I let him dig the deepest hole he
could imagine by doing my best ditzy/derpy voice and just saying things like
"they want me to ask about the load... average...?". He'd make up some
garbage, and I'd log all of it. This way I could bury him both for being a
sexist bastard and for being a lying piece of trash at the same time._

He has already sunk the interview in her eyes, but she doesn't want to tell
her employer that this guy bombed due to his interactions in the first 30
seconds with her. Moreover, he is a sexist bastard.

He then tells her something she disagrees technically about load average, and
this makes him a lying piece of trash.

Regardless, he has bombed and sunk the interview, and her only duty now is to
spend another 44:30 seconds to bury him.

Of course, she is playing this for humor, so I am out of line for thinking
Rachel sounds like a jackass who has done a disservice to the candidate as
well as a disservice to her employer.

~~~
vacri
So the sexist guy has an incorrect opinion on women in tech. He encounters
one, who decides to play along with his stereotype rather than just be who she
is. While it's not her responsibility to teach him not to be sexist, it's also
specifically avoiding allowing him to see a positive role model. And all out
of a vindictive power play. If he has 'already lost the interview', then there
is no point in keeping up with the charade - you've already made your
evaluation and now you're just being a cat playing with a mouse.

And at the end of the interview, we are left with a sexist man who has had his
preconceived notions _confirmed_ by encountering a female tech. That's
something of an own-goal.

~~~
chris_wot
Wow, so Rachel encounters a sexist jerk, and then asks him a number of
questions to let him hang himself, and it is _Rachel_ who is the problem?

Given they have very few women interviewers, and the guy has to work in a
team, any woman who has the unfortunate luck to work in the jerk's team is
going to have their life made difficult. Rachel seems to have done a service
to the company by flunking him, IMHO.

~~~
vacri
That's a good strawman fallacy you've got going there. It's pretty clear that
I think the main problem is the sexist man.

Rachel didn't 'ask him a number of questions', she purposefully played the
role of a ditzy, clueless airhead to play to his stereotypes rather than just
ask the questions straight. Sure, it might feel good at the time and we can
all laugh at the guy, ha ha, but what actually happened was the guy got
reinforcement for his shitty behaviour. If the guy was giving weak answers,
then being a straight interviewer isn't going to suddenly make him Einstein.

~~~
chris_wot
He might have gotten reinforced for his bad behaviour, but he still didn't get
the job. To my mind, serves him right.

------
nbm
I'm a "well-calibrated interviewer" at Facebook for my role (Production
Engineer), and some things from the post sound fairly similar.

I consistently do two-ish phone interviews or on-site interviews a week. I'd
say 2 hours is a little on the high side for average time per candidate
(probably around 1.5 hours), but it does happen. I am sometimes called in to
do tiebreaker interviews (before or even after on-site interviews), or
requested for particular candidates. I occasionally help other groups
(capacity engineering, software engineering, operations engineering) with
their interviews on top of my usual load. I've flown overseas for people who
can't easily get US visitor visas. I'm in the top 5 interviewers by quantity
for the role (which isn't too hard, admittedly), and in the top few percent of
the company (I even got some ice cream as thanks!).

In addition to interviews, I also spend an hour a week in a recruiting meeting
trying to improve our recruiting process (including giving feedback to our
interviewers on how to improve their interviewing and feedback). And I've done
non-interview recruiting at universities (manning booths at career fairs,
mentoring at hackathons) and conferences (at Velocity, SCALE, Surge, PyCon
just in the last year).

My feeling about spending 4-6 hours a week on interviewing and recruiting
activities is that it is some of the most valuable work I can be doing.
Avoiding a false positive - ensuring we maintain a consistent high bar - is
going to save people in my group (and others in the company) potentially
hundreds of hours. Making a great hire is going to add thousands of hours of
productivity to my group and the company.

And we're a small enough group (and even a small enough company for the non-PE
hires I've helped with) that I can easily keep track of the people I've helped
hire and it is supremely fulfilling.

~~~
kator
That's very cool. I think what grinds on most people is the people up stream
tend to send lots of low quality leads down the pipe and it burns people out.

> In addition to interviews, I also spend an hour a week in a recruiting
> meeting trying to improve our recruiting process ...

Exactly.. If it's broke "let's all work to improve it". Not "oh no here comes
the recruiter again with some horrible candidate".

I don't know about the team you work with but the recruiting team I'm working
with is full of bright excited and passionate people who want to learn. They
don't code.. SO WHAT.. They know people well, and they're willing to learn.
That's all I ask.. from anyone... I can teach you anything.. I can't teach you
to have a good attitude and I've yet to find a way to increase people's IQ..
:)

I might not be able to teach you as a recruiter to code, or "be smarter" but I
can help you "work smarter" and help avoid burning people out further down the
pipe.

> My feeling about spending 4-6 hours a week on interviewing and recruiting
> activities is that it is some of the most valuable work I can be doing.

4-6 hours and once in a while you get to add 2,000 hours of new capacity to
the team! :-)

If you think about the ROI interviewing and getting the right people on the
team can be amazing. Much better then those hours spent refactoring 20 lines
of code...

~~~
nbm
Yeah, I really enjoy working with our recruiting team - they display the same
interest in being better at what they do and willingness to put in the effort
necessary to improve as the engineers I work with, and that earns them my
respect.

Taking some time to understand what people do, the challenges they face, the
work they head off without you seeing any of it, and that they're as
interested in getting things done, and getting better at what they do, as you
are is well worth the investment.

------
fbuilesv
For those unfamiliar with the company in the article, Rachel's talking about
her experience at Google.

One problem I've seen in big tech. companies (especially Google) is that most
employees hate doing interviews, they think it's worthless and a waste of
their time. They don't seem to realize that this is a necessary part of the
job (especially with a company that went from 20 to 20000 in such a small
amount of time). In the end, would you rather have your co-workers be
interviewed by yourself (or another technical fellow) or by an uninformed HR
department?

Having said that, I would've loved to work with Rachel. I have a deep
fascination and respect for responsible and involved people who can pull the
weight of an entire team when everyone else ain't doing shit.

~~~
tptacek
Another problem is the disconnect between "recruiting is extremely important
to the company" and "the interview process should be as thorough and
regimented as possible", as if the latter followed from the former.

Everyone in software management is familiar with the concept that good
employees are worth Nx more than average developers. But a good chunk of them
have also attached themselves to the idea that employee quality scales
continuously from average to good, and so there's significant incremental
value to sussing out everything you can about a candidate.

I don't believe this is true. I believe there are "average" (meaning, in
reality, "bad") candidates and there are "good" candidates, and you need to
find out which of those two buckets people are in, do some extra checking for
culture/work habits/communications skills, and then stop.

Instead, the formal interview processes I've seen are gauntlets of trivia and
case studies and successive whiteboarding exercises that exhaust everybody.
They don't generate value proportional to their cost. And they're
unsustainable, so that whatever marginal effectiveness they have is
compromised totally within a year when everyone who participates gets sick of
it and starts phoning things in.

Tangential point: some people just can't interview effectively, and you need
to keep them away from your interviewing process. You're probably not going to
be able to perfectly balance the recruiting load across your team.

~~~
btilly
_Another problem is the disconnect between "recruiting is extremely important
to the company" and "the interview process should be as thorough and
regimented as possible", as if the latter followed from the former._

There is a reason for the regimented process that you're talking about.

The challenge for any large company is to delay, for as long as possible, the
inevitable reversion to hiring the mean. Under normal circumstances, different
teams will have different standards, and by the time you realize this you'll
have a lot of teams whose standards have slipped, and you will have a lot of
marginal employees.

Therefore smart large companies try to create a minimal process that aims to
create uniform process across the whole company. The process tries to be
lightweight as possible on individual people, but fights the good fight
against entropy.

Unfortunately for Rachel, Google's process centers around the idea of having
well-calibrated interviewers whose interview characteristics had been so well
tested against others that hiring committees know exactly what to make of
their interview feedback. (What absolute scale you grade on does not matter
much. How your scale compares to everyone else, and how well it predicts
everyone else, does.) Once an employee winds up as a well-calibrated
interviewer, HR would like to use that person as much as possible (and each
use does a better job calibrating the employee, and increases their
desirability as an interviewer).

This is inconvenient for the employee to whom it happens. But, on average
across all developers, the process is quite light-weight.

 _Instead, the formal interview processes I've seen are gauntlets of trivia
and case studies and successive whiteboarding exercises that exhaust
everybody. They don't generate value proportional to their cost. And they're
unsustainable, so that whatever marginal effectiveness they have is
compromised totally within a year when everyone who participates gets sick of
it and starts phoning things in._

Google's process has been sustained company-wide for a number of years now.
There is much that is imperfect about it, but evidence suggests that it is
sustainable.

~~~
tptacek
So first, I should be more careful. Google is operating at a scale that I
don't understand. My experience tops out at ~100 employees. Let's stipulate
that Google has a hiring problem that very few other companies have, and let
me shift the focus back on to startup hiring, which is what I care about.

And then: I want my point to stand: for a given role, there are good hires and
there are bad hires (where "bad" includes "meh"). There is not a lot of value
in grading past that. But lots of hiring processes are premised on doing
exactly that; they take candidates who can demonstrably perform the task
required for the role and then beat them up with trivia questions, with
puzzles, with nerd dominance quizzes, and with subjective stuff.

I point that out because if you can find a way to accept that, you can
streamline a lot of recruiting process and make interviews less painful for
everyone. The way I finally managed to make that leap of faith was to reflect
back on everyone I've had a hand in hiring that didn't work out, and then
consider the process that hired them. Then I think back on everyone I've hired
who has surprised me with their awesomeness (some of them are even people I'd
negged). I am now sold on the fact that most tech interviews are voodoo
rituals.

What I would do/am doing instead:

* Get people out of HR/phone screening as quickly as possible.

* Have an explicit up-front call with me or someone senior in the hiring process to level-set people, try to defuse some of the tension, explain the process, and answer questions. Things go faster and more smoothly when candidates are convinced they _want_ the role. The amount of sales effort it takes to get a candidate into the pipeline is not enough; you need to engage them on what the job is like, let them ask what the day-to-day is, and get them revved up _before they interview_.

* Minimize phone screening, both in length and in depth. I am on a mission to get rid of them altogether but am not convinced I'll succeed.

In the interim, what I'd like to do is take all the effort and expense we'd
sink into hour-long phone screens and redirect it to coming up with a small
rotating battery of really, really good questions that we standardize on.

* Replace phone screens with challenge problems. Calibrate them for the length of time a series of elaborate phone screens would take. Try to make them fun. Here I admit I'm not sure what I'd do back in my pure dev jobs to make this work, but I'm sure I could come up with something.

* Give the on-site interview team a full dossier of objective facts about what our up-front process learned about the candidate, so that nobody is trying to read tea leaves out of resumes. Candidly: I barely glance at resumes anymore, and I wince when I read interview reports that go "I read XXX on the candidate's resume and so asked YYY".

The most important thing I think I've learned about recruiting in the last
year --- apart from getting better at getting people in the door --- is to
generate sets of facts that are easy to compare between candidates. I'm
learning _a lot_ by reviewing the history of how people do on challenges, and
I really very much want to be in the same place by the end of the year with
all person-to-person screening interviews. A nice side effect I'm hoping for
is for prep time on phone screens to approach zero, and for actual phone time
for interviews to approach ~20-30 minutes. We'll see.

~~~
Dn_Ab
I don't know if you have read Thinking Fast and Slow but your observations are
in strong agreement with research.

From what I have read (mostly Kahneman) most interviews as performed are
useless and suffer from the illusion of validity. Instead, structured
interviews and scoring that are the same for all candidates are strongly
recommended. Which is, draw up a short list ~= 6 independent factors which
from experience are reflective of the job and whose answers you can easily
judge objectively. For each factor come up with a small number of questions
with a score, say from 1 - 5 and then simply add up the scores. No fancy
weighting required - research shows that regression does not add much to this
simple model. To avoid the "halo effect" do not jump question categories. This
will vastly improve upon the more common unstructured interview and though
still far from perfect you will be hard pressed to improve on that short of an
IQ or equivalent test.

Interestingly, and reminiscent to bagging in machine learning, unstructured
interviews can be brought nearly up to par by averaging multiple independently
scoring interviewer opinions.

------
kabdib
I love interviewing people. I've done hundreds of interviews in the past few
decades, and learn something new every time.

From the superstar who you really want to start working with right away, to
the bozo who can't code their way out of a paper bag, I can't say that a
single interview was a waste of time.

The bad ones teach you how to ask questions, the good ones keep you on your
toes (and during the "sales job" part of the interview, make you introspect a
little on your current job, which is always good).

[I wouldn't have failed the guy immediately for the "I thought this was going
to be a technical interview." Then again, maybe it was the tone of his voice?
He'd probably been dealing with a bunch of HR / recruiter nonsense]

------
kator
LOL two interviews a week.. I do about five to six a week and had three in a
single day! My goal is to get four in one day...

When you're growing staff by 100's of percent year over year recruiting has to
be one of the most important tasks you can put energy behind. You want the
right people on the right teams and you don't want a random non-tech person
making the determination.

I see engineers over-worked, up to their ears and always complaining about
interviewing. I often wonder, we're in a rational, problem solving industry
but we can't see the simplest problems. You're over worked and up to your ears
because the team doesn't have enough people on it or the "right people" in it
and you're slammed. Does it make more sense to just keep working your backlog
or take time to get quality people on the team so you can double your velocity
on your next sprint? Why is this so hard for people to see?

You have to kiss a lot of frogs to find the right people for your team. It's a
numbers game, perhaps it's hard for programmers to understand that the hiring
process is a lot like debugging, most people don't like doing it but if you
don't things will break and stay broken.

All this said I do think companies have a hard time figuring out how to
balance this. I'm consulting right now with a company that has a very large
recruiting team and crazy hiring goals. Meanwhile they were sending a lot of
"questionable" candidates to the engineering teams. Engineers were bitching
about this to me over lunch in the hallways etc. My response, simple, I swam
up stream. I started meeting with the recruiting team, reviewing as many
candidates as I can and helping them develop a better process. We're setting
goals and measuring our rejection rates and getting feedback on each one even
if it went bad. Recently I told them that if we're cutting someone on site to
let me do it. I want to meet these people before we cut them to understand
what went wrong, did we do something wrong? Did we cut the wrong candidate? I
want to give everyone the benefit of doubt and figure out how to improve the
process. Could we have avoided wasting their time and ours?

In the OP's case perhaps upstream should have been measured by how many she
rejected. They should have been incentivized to improve their own "quality
score" instead of burying people on the "well-calibrated interviewer" list.

You can sit there and bitch about something that is broken.. Or you can choose
to fix it.. Which will it be!?

~~~
Nate75Sanders
2 things:

\- Most people didn't get into programming so that they could interview people
10+ hours per week.

\- Plenty of managers won't really count interviewing time in the project
management tools (or give you credit however time is managed). You're still on
the hook for whatever work you would have been responsible for with 0
interviews.

~~~
sliverstorm
The second point is very important. If you are expected to interview people
10+ hours per week, that needs to be considered in your project goals and
timeline expectations.

~~~
kator
Yes and if your manager doesn't understand then it's time to find a new
partner in your career..

I am always amazed at people who will let a PHB torture them and gladly bitch
about it at every family function and with a round of friends at the bar.

Really.. So you're an amazing talented person and you're letting a PHB ruin
you life.. What does that say about YOU.. ????

------
zomgbbq
I'd like to talk about this from the other side of the table - the
interviewee. I think interviewing is broken but I really don't know the answer
how to fix it so I have to just "play the game". I'm pretty good at "playing
the game" but I am feeling the same level of interview burnout while looking
for work as interviewers do looking for candidates and I can't help but think
it hurts the industry overall as engineers don't look for new jobs just to
avoid the hazing process of interviewing.

Without trying to sound like I'm bragging, I consider myself a senior level,
talented engineer. Recently, however, the startup I have been working at ran
out of funds and I have been pounding the pavement looking for work. The
problem is that I don't feel like any interview I have been doing, or for that
matter, have ever done, is really a measure of how good I am as an engineer.
It's more a stamina test to see how well I can be tortured and hazed and it
makes me reluctant to take on more interviews. Most interviews are about being
quizzed on the minutae of a language or about solving a programming quiz in 60
seconds while the interviewer critiques thought of every line. I just don't
see how these interviews are actually filtering out bad candidates or
measuring good candidates. They are measuring how someone performs in a very
specific, non-realistic situation that does not emulate how every day work
would occur. The worst example of this are the horrible SHL/Brainbench tests
that seem to measure memorization more than skill. I do quite well on those
silly tests, but I'm not sure they really measured anything accurately.

I was the happiest when I got homework assignments as my tech prescreen or was
asked to walk through examples of my own code with their engineering team. I
feel these are less stress on the interviewee because it is code you are
familiar with and are a closer exemplar of what goes every day at work.

Or maybe I'm just a wimp.

~~~
nbm
I've been hired on bravado, reputation, pointless trivia contests, and other
ways of interviewing, and it scares me that companies are willing to take on a
risk without verifying skills in some way.

I've also been on the receiving side of delightful trivia questions like "What
are all the ways the character '*' is used in Python?" and puzzle questions
that test "creativity"/"ingenuity", whose value are totally alien to me. (I've
also done online quizzes/tests, including personality ones(!))

I would like to think there's a broad middle ground in between, but that there
isn't a single process that is going to be effective for all companies or for
all candidates.

In the past, I've used "homework" as a final step in a hiring process, which
worked for some (it certainly washed out a few candidates who got passed less
strenuous interviewers), but which others weren't interested in doing -
including two that we hired anyway and who ended up being great coworkers.

I imagine I would feel a bit put out if a startup wanted me to spend 2-4 hours
on something before they allowed me the opportunity to get to understand the
position and the company (especially since I would hope the startup would look
at my open source contributions in lieu of the homework). New grads might have
a bunch of opportunities, and they might decide on skipping something with
that sort of requirement.

~~~
mikle

      1. Multiplication (2 * 4 = 8)
    
      2. Power of (2**4 = 16)
    
      3. *args and **kargs in functions definitions and function calls.
    
      4. Imports (import * from blah).
    

Did I miss anything?

~~~
nbm
I have no idea offhand if that's the full list - it's the list I come up with
when I think about the question.

But that's the thing - it's a bad question. It should be asked as "what does
this do?" or "how do you do this?", like:

"What does this code do: foo(1, 2, __kw)?"

Or:

"How do you import all exported names from a module into your module's
namespace?"

These show your code reading and code writing skills.

The original question is based on unreliable use of memory. Memory is great at
slightly-fuzzy-key lookups. It is usually terrible for table scans applying a
pattern filter.

Think of the challenge "Provide 5 common Unix command line utilities start
with t"?

Most people who actually know 15 or so of them and regularly use 7 or so of
them would have trouble providing all 5. Some will just blank for a minute
(because there are just no pathways to work from), most will find 2 or 3.
People who studied a Unix text book the night before would also know 2 or 3. A
few might just totally blank and become affected by the fact they can't even
remember something so easy that they will be unable to answer that question,
and may affect their interview performance after that.

So it just is not in your interest to ask those types of questions.

~~~
mikle
Oh, I never said I'll use this question. It is totally useless - like you said
it is a merely a test of memory. I just wanted to see how my memory works :)

------
interviewer
I'm also a "well-calibrated interviewer" at Amazon. I'm an engineer and a 'bar
raiser' with 700+ interviews. My thoughts are:

1\. Interviewing and evaluating a candidate has very, very little to do with
engineering. Yes, you have to be able to evaluate code but this just scratches
the surface of what needs to happen during an interview.

2\. As a corollary to #1 - just because you are a good engineer doesn't mean
that you will be a good interviewer. Never mind whether or not you _want_ to
interview. It is just a different skill set.

3\. Interviewing can take a lot of time. But it also has high value for the
company. If you are passionate about the people that work with and really want
to shape the company, then get involved in interviewing. There are very few
activities that will influence things more.

4\. A lot of the time that I spend in 'interviewing' is actually time that I
spend with co-workers to make them better interviewers. For in-house
interviews the breakdown is 15 minutes preparing, 45 minutes interview, 30
minutes writing debrief feedback, 30 minutes attending the debrief, and then
60 or more minutes working with the other interviewers on the loop or helping
the hiring manager understand what they should be looking for.

5\. In some ways the interviewing process is no different than any other
process employed by a company. In particular there is almost always room for
improvement. As an employee of a company you can spend time complaining about
the process ('it is broken', 'it is unfair', 'it is too heavy weight', etc) or
you can spend time fixing it. If you are working at a company where you are
not empowered to fix things like this then you should consider working for a
new company.

6\. I like lists!

------
rdtsc
One key missing element is to let member of the team interview their future
co-worker as well.

That way it doesn't seem like waste of time and it gets people invested. If
they don't pick the right person they could end up stuck with someone dragging
the team down and they'll eventually learn the value of having a thorough
interview and being interested in it.

The other side of the coin is that they should also be allowed to influence
the type of offer and how aggressive the salary negotiation would be.

I have seen a cases were a good candidate technically hasn't returned back
because of presumably a low offer.

~~~
mjmahone17
Letting a member of the team interview the candidate may be a best practice,
when possible. However, someone else said Rachel worked for Google. Their
hiring practices, at least for university recruiting, is basically find all
the "right" candidates, hire them, then, after they've accepted an offer,
match them with a team. I know other companies, such as Facebook, actually
have you begin working, then have teams court you after about a month of work,
then you choose which team you want to join.

~~~
nbm
I'm not aware of how many other companies have a "bootcamp" program like we
have at Facebook.

Basically all software engineers (including some whose titles aren't Software
Engineer, and including managers) go through this six week process - first
four weeks is mostly getting people up to speed through two or so
presentations a week and doing real tasks (which may mean meeting up with some
other engineers to understand enough context on how to proceed) that go
through the same review and push process as everyone else, and the last two
weeks is mostly learning about specific teams and a bit of a sell exercise
through a set of presentations.

All through this, you share a software engineer mentor with two other
bootcampers who helps you navigate the code repos and also the company.

At the end, the bootcamper decides what team to work on, with some background
information like company priorities and maybe with some teams specifically
reaching out to them.

That's a fairly huge investment, so I'd love to know which other companies
offer this sort of program.

~~~
mjmahone17
You're right, I don't know of any other company tht does it exactly like
Facebook does. Tripadvisor, though, will let a significant portion of their
candidates do a 1-year rotation through four different teams, then you get to
choose the one you liked best (or potentially a different one if you're sure
that's what you want). Not exactly the same, but still a larger than normal
investment prior to setting you on a team.

~~~
nbm
That's pretty awesome opportunity and investment.

I wonder how they deal with understanding/measuring performance with these
short engagements.

(Facebook has a "hackamonth" program which people who have been at the company
for 18 months are encouraged to do - work on a new team for a month, and then
decide to return, stay, or join a third team.)

------
thinkling
Some people are good at interviewing, and in particular at filtering, and some
are not. If you're good, and you do more than your regular share, you're doing
the company a huge service.

I've heard of this function being applied at the other end of the interview
loop. Microsoft has its "as appropriate" addition to the loop at the end.
(Senior person isn't bothered to interview you unless previous people liked
you.) Amazon has their "bar raisers", seasoned interviewers added for quality
control, at least one on each loop.

Doing this up front seems to make more sense, as you waste fewer employees'
time. At the same time, as Rachel indicates, it's less satisfying to the
valued 'well-adjusted' interviewer, as they see few of the people they
interviewed get hired.

Are there other models that solve that problem?

------
blacksmythe
Most companies hire by department, so a good manager is incentivized to do a
good phone screen (or at least make sure a good phone screen is done), because
it is their people who waste time on a plant interview of an unqualified
candidate.

Google's policy of separating the interview process from the job assignment
breaks a lot of the incentives to interview that most companies have in place.

~~~
nostrademons
That's intentional, because it also breaks many of the misaligned incentives
of departmental hiring that result in pathological organizational behavior.

Most managers, if given a choice between hiring a "good enough" candidate and
hiring no one, will choose to hire. Particularly in a growing company, where
everyone is overworked and there's a lot of grunt work that people would love
to have a new recruit do. Over time this results in a degradation of overall
employee quality ("A players hire As, Bs hire Cs") and a bunch of deadweight
that were needed to solve a temporary staffing crunch but later end up holding
the group back.

It also tends toward fiefdoms and empire-building; when you're hired by the
department, it can be very difficult to move between departments. One of
Google's strengths is that transfers are pretty easy; I doubt that would be
true if you were brought in by a department manager instead of assigned to the
company as a whole.

~~~
greenyoda
From the corporate side, not having managers do their own hiring seems to make
sense. But from the employee's side, it means that you don't get a chance to
meet and evaluate the manager and team who you'll end up working with. I'm
sure that Google is similar to most companies in that the quality of managers
varies widely and different departments have very different work and
environments. Not being able to evaluate the manager and team that I'd be
working with if I were to accept a job at some company would be a deal-breaker
for me.

~~~
stephencanon
For me it wouldn't necessarily be a deal-breaker, but I would demand a
significant premium on salary or cash signing bonus for the risk of not
knowing the group and manager I'm going to be working with, or the work that
I'm going to be doing. That seems like a pretty bad incentive for the hiring
company too.

------
frozenport
I wonder if Google has an analytical models to try to understand what makes
candidates successful in their company. 1 of 2 new hires leaving is a problem.

~~~
greenyoda
You can't really draw any valid conclusions from a sample size of two. The
fact that this happened with the two employees who Rachel interviewed doesn't
give us any information about what the attrition rate of new employees at
Google is.

~~~
nerfhammer
You can draw valid conclusions. You just need a very wide confidence interval.
If p=.05, we can assume that the real attrition rate is between about 2.5% and
97.5%.

------
rheide
Company pays you money. You work for them. They ask you to do something you're
good at. Sounds good to me.

~~~
esperluette
It's a ROI question: all that time, week after week, for two hires, one who
lasted a year? That's not a great ROI for a good engineer's time.

~~~
Steer
I think that's the wrong way to look at it. The time she put in also meant
that some people that could potentially have been a bad fit were not hired
which was good.

~~~
michaelt
While Google goes straight from application to a telephone interview, a lot of
companies ask for a work sample test first - either by e-mailing them a
problem and asking them to reply with a solution, or through automated systems
like Codility.

If you're spending a lot of employee hours filtering candidates who could have
been filtered more easily through other means, I can understand feeling a bit
indifferent about it.

------
josh_fyi
You need a graduated system of interviewing.

Start with quick-screening questions online, then a phone interview, then more
in-depth online testing (nothing too burdensome on the candidate, say an
hour's worth), then several hours of interviews (perhaps all on one day) by
various interviewers.

You should also gather candidate's expectations/requirements in compensation,
corporate culture, etc as early in the process as possible, to avoid
disappointment.

At any point in the process you should be willing to stop, politely but
decisively, whether for jerkiness or for technical incompetence.

------
fideloper
I wonder why she ended up with such a large work load of interviews? Are they
allowed to deny interview requests given to them?

~~~
calpaterson
She explains in the text. They were using her as a first hurdle because she
got a good reputation for interviewing. The rationale seems to be that if they
put "well-calibrated" interviewers in first, they might weed out time-wasters
earlier.

I work at a company with a similar policy on technical interviews (though I
don't do any), and here I think it's uncommon to decline a particular
interview unless you have some very pressing engagement.

------
gruseom
I hate job interviews so much that I want (someone) to figure out how to build
a company entirely without them.

~~~
mjmahone17
There's a good reason many job openings are never posted, and having good
connections can lead you to getting a job that doesn't, from the outside,
appear to exist. However, not having interviews at all would seem to imply
building a very insular company, because you'd need an intimate knowledge of
how strong somebody is as an employee. And the only way to get that is either
to look at their past work, similar to published papers in academia, or to
know that person well. And even in academia, to my knowledge, professor hires
still get interviewed, even if they're a nobel prize winner, to make sure
they'd be a good cultural fit, and so that they know whether they'd want to
work for the interviewer.

~~~
martinced
+1

Last consulting gig I told I was willing to help them for a few months. I sent
them a CV and they saw I meant business. Got an email saying: "you're starting
next monday if it's okay for you".

Quite an efficient process ; )

(I don't know which part of my CV they did background check that said)

~~~
meric
What did you think of their programming chops?

------
nanook
Is the website not accessible for anyone else? It always happens to me with
this website.

~~~
sp332
Seems fine, does this link work better?
<http://rachelbythebay.com.nyud.net/w/2013/01/04/cal/>

------
michaelochurch
In a typically managed, closed-allocation company, your job is to suck up to
your boss-- not to do what's best for the company. Rachel's ex-company is one
where your career rises and sets on "calibration scores" set unilaterally by
the manager. (You need good peer reviews to get a promotion, but transfers and
firing are based on the scores, and you need good projects to get promoted.)
If your manager doesn't think interviewing's important (and most don't) then
it hurts your career to spend time on it.

It's also sub-optimal, from a career perspective, to work on long-term
projects. Those are the first projects to be cut, and in the modern asshole
corporation, a project being cut usually means your job is over (because no
one will take a transfer from a failed project, especially because managers
tend to blame project failures on the people below them and have months of
lead time before shit goes down).

Recruiting is the ultimate in long-term projects. It's extremely important for
the company, but very hard for someone who's good at it to get personal
credit. Most of your time is wasted on horrible candidates, and the good ones
have a lot of great options, so it takes a long time to get a successful hire:
6 months to find the first, plus 12+ months for that person to be successful
at the organization (at which point, it reflects on him, not on you for
hiring).

Companies rely on good citizens like Rachel who keep the place going, but
rarely reward them.

~~~
fizx
If you like to travel and you're a good interviewer, you might get some free
flights and a week of additional vacation as you help with foreign interviews.

~~~
michaelochurch
This is different, because if a company is buying in to you enough to send you
to foreign countries, you probably don't have to worry about getting "perfed"
by your manager.

However, most of the traditional closed-allocation companies view (despite
their claims to the contrary) non-executive recruiting as undignified grunt
work (you're associating with people, most of whom the company won't hire) and
don't reward it. It just leaves you with one less hour per day to serve your
boss's political needs.

------
Sherrilbfx
I didn't really get the gist of the article: so she was doing more interviews
than everyone else and not finding candidates?

