

Hiring Engineers, a Process - kldavis4
http://hueniverse.com/2013/02/hiring-engineers-a-process/

======
54mf
The real TL;DR: "I have no patience for jumping through _your_ hiring hoops.
Now, here's a list of _my_ hiring hoops you'll need to jump through before I
consider working for / hiring you."

I would never, _ever_ hire this guy. With his hubris and self-centered
worldview there's no way he'd work well with a team, and with his apparent
aversion to writing code under any circumstances other than inside his own
delicately-crafted bubble, I'd have serious doubts about his ability to hit a
deadline.

Nor would I trust him as a manager. He walks out of an interview (multiple!)
because they asked him to code (how _dare_ they), then interviews the guy who
responds to (an incredibly ridiculous) code test with a jokey response? Sounds
like a great way to make friends, and a terrible way to find job candidates
with strong skills.

Even if he was the greatest programmer/manager in the world, unless you're
looking for a solo developer who doesn't need to produce within a given
timeframe or a manager who values a "cold call with attitude" over a first-
class résumé, you're in for a world of hurt.

~~~
makmanalp
Eh. I think he complains a bit too much, but consider the reverse direction.
Why do employers get to be such dicks?

This doesn't happen very often, but every once in a while I get one of these
"oh we're so interested, you're so great, you did so well in the interviews
... <silence on the line for 3 weeks>".

Or, when I bother to write you a cover letter, you can bother to write me back
2 sentences, even if you don't want to hire me. "I'm too busy" is not an
excuse. I'm busy too! If you can't take the 30 seconds to copy-paste a nicely
written canned message and edit it up to let me know you want / don't want me,
then you suck. It's just common courtesy, and you're not exempt from it.

You also can't pressure me to make a decision once you've made an offer. I'm
going to shop around, just like you shop around. You probably don't have a
hiring emergency for the one junior position you're hiring me for, so shove
it. Or just hire someone else, but quit calling me to pressure me. I
understood the first time. I almost fell for this when I was fresh out of
college.

You also really shouldn't be claiming IP rights on unrelated things I work on
out of office. That's _my_ business and my work, and I shouldn't need
permission from you to own it myself.

Where is the decency? I see this post as a reaction to that frustration.

------
tptacek
I thought I'd write a graf about how much I disliked the "Screening" section
of this, with its instruction for candidates to have "attitude" and to "make
it fun" for him, but then I reread it and at the end he's saying things like
"send him an email about one of his blog posts", and that sounds totally sane.

I recommend reconsidering "culture fit". The "culture fit" meme is poisonous.
Mostly to the profession, but also to you, because it'll cause you to eject
people who would be solid performers, eventually in favor of prima donnas who
will underperform, feel no ownership of their work, and use your team solely
as a stepping stone.

If you're convinced that "culture" matters, find a way to distill it to the
culture attributes that directly implicate job performance. But then make sure
those attributes don't exclude whole classes of life circumstances, like
parenthood.

PS: I interviewed at Bloomberg about 10 years ago and enjoyed the experience
immensely. I got hammered with concurrency and distributed systems questions;
everyone who talked to me was smart. Maybe it depends on the group you're
talking to there.

~~~
donw
I see things the other way when it comes to culture -- it's the end result of
people finding ways to work together, and if a candidate can't fit in with the
culture of the existing team, then they're not going to be a good hire, no
matter how brilliant they are.

I once interviewed an otherwise brilliant developer that was militantly anti-
TDD and anti-pairing, both of which are attributes that both myself and my
team considered to be essential parts of the business of building software;
working as a part of an engineering team is about more than just writing code.

Considering cultural fit helps you _avoid_ hiring prima-donnas that are
unwilling to be a member of the team, and it's also a good way for a candidate
to know whether or not they will be happy working with a given company.

That kid wouldn't have been happy working with us, and the team didn't want to
work with him. He was a competent developer, but not a good fit for the
culture of the team, and not hiring him on that basis was a good decision all
around.

~~~
ardit33
Did you just call all developers that don't like TDD and pairing as prima
donas?

Do you release that's probably 90% of engineers out there, and maybe 95% of
the good ones.

You can work and make great software without using heavy handed processes like
TDD and Pairing, which in my opinion it only helps junior/inexperienced
developers a bit, but hampers the productivity of the good ones.

But yea, it seems that the place wouldn't be a fit for him, but there is no
need to call him a prima dona. He was right to stick with his guns, and you
guys stick with yours, but keep in mind that you are limiting yourself to a
very small subset of engineers that both like TDD/Pairing and are good at what
they do.

~~~
donw
Not at all. He just wasn't a good fit for the team, and nor was the team a
good fit for him.

Prima-donna programmers tend to be vain, demanding, and incapable of receiving
or acting on constructive criticism.

More often than not, they're also not particularly productive, either.

Whether or not you do TDD is unrelated to any of those personality traits, and
I've interacted both with excellent developers that had a myriad of reasons to
not test, as well as horrible prima-donnas that were religious about TDD, but
only if you did it the One True Way That Only They Could Understand.

------
dmbaggett
Re: Google (et al), "I’m purposely focusing on companies that got it all
wrong."

As someone who's hired hundreds of people and worked to build three startups,
I am increasingly of the opinion that it's critical to carefully study the
hiring practices of companies that appear to employ a consistently high level
of talent, like Google. And Google's talent base was one reason I thought the
employees of my last startup would be happy there, and argued (to me) in favor
of an acquisition by them. (Virtually all of them are still there, too.)

That said, entrepreneurs should take with a grain of salt any definitive
pronouncements about hiring practices. In my experience, hiring and
organizational design are not solved problems yet, by any means.

~~~
tptacek
I agree, and note that Google also has radically different hiring challenges
than a startup does.

~~~
dblock
I think Google's interviewing process completely ignores someone's background
and quickly tries to send you into the kind of Googler they would like you to
be and see if you fit. They probably miss on having a diverse engineering
force this way, which is a fine trade-off in an army.

------
troels
I'm curious as to how the "real-world" programming task works. I find that
most task are either new features, in which case domain knowledge is essential
- e.g you need to talk to the users before you can write any code. That
doesn't lend it self well to a homework task.

The alternative is bug fixes and inceemental tweaks and improvements to
existing systems. These are often of a more technical nature, but they are
also 90% about understanding existing code, which means that the candidate
would need copies of the existing code base. I'm not sure I would be
comfortable dishing that out to random strangers, which job candidates
essentially are.

How do you do this in practise?

~~~
tptacek
We have it easier than he does, but I'll tell you what our practicals look
like:

* We test a lot of web applications, so we rigged up a sacrificial web app that we run on random EC2 instances and have candidates take an hour or two seeing what they can find in it.

* We test a lot of crazy custom client/server stuff, so we built a client/server system with a custom protocol that we have candidates reverse and then attack; we tried to calibrate this so it'd take less than 2 hours to beat.

* We write a lot of fuzzers, so we have candidates write a fuzzer for a file format.

Key attributes of good practical tests in our experience:

* They can't take too long to complete.

* They have to relate somehow to the actual work candidates would be doing within the first six months.

* They have to produce results that are _objectively_ comparable in some fashion to those of other candidates, so we can learn from the trends. This is probably the most important thing I've learned about hiring in the past year.

We'll probably swap the fuzzer challenge for a memory corruption challenge
sometime this quarter, because the fuzzer scores high on the first two
attributes and low on the last.

I'd love to do a crypto challenge (mail sean AT matasano dot com if you'd like
to do a bunch of crypto challenges, by the way; just say "Thomas told me to
mail you"), but they score low on the first two attributes.

So if I was doing engineering interviews again (About 10 years of my career
were as a full-time dev), I'd probably look to:

* Taking a time-inefficient algorithm and making it faster

* Taking a space-inefficient system and making it tighter

* Writing an importer for a specific file format

I agree that actually asking candidates to jump in and produce code for issues
in Github is a losing strategy. In particular, because you're making it
unnecessarily hard to learn from your hiring decisions; when you look at
someone who is kicking ass on your team, can you go back to the hiring
decision that selected them and see whether they outperformed or
underperformed a metric and adjust the metric accordingly? If not, your hiring
process is suboptimal.

~~~
troels
Thanks. I guess this is a bit easier for you because you are a consultancy and
have lots of different projects going on. You also work in a very technical
domain (I assume - really, I know little about what you're doing).

Where I'm having a hard time seeing this play out, is for a business that has
an in-house software platform supporting it. Or for shops that develop and
maintain a products.

------
tomx
Is "I don’t read resumes", "I don’t care about your resume", very common? It
seems a bit cold hearted and unfriendly. It says to me "I don't care enough to
read a bit about you, and your achievements". But that's just me.

~~~
tptacek
I usually read resumes, but I don't really care about them. Candidates usually
send "cover letters" (this is all email, so, really we're just talking about
the 2 grafs that precede the resume in the resume email) and a cover letter is
enough to get me on the phone. If I like you on the phone, I will never bother
looking at your resume.

~~~
chiurox
So going from "I don't read resumes" and "I don't care about your resume" to
"I usually read resumes" seems a bit self-contradicting. Regardless, I agree
with your point of view. When we were interviewing, we got a bunch of
impressive resumes but in reality...

~~~
tptacek
I usually read resumes.

I do not care about resumes.

If I get on the phone with you because of your cover letter and haven't read
your resume, I will probably never read your resume.

Regardless of whether I ever look at your resume, it's very unlikely to have
any impact on our hiring process. We use practical tests and, to a much lesser
extent, 1:1 interviews on concepts; the practical tests have been so valuable
that we're moving towards doing more of them and less subjective interviews.

Your resume fits almost nowhere in this process.

------
tjtrapp
i continue to question the "whiteboard coding" experience.

on one hand: a dev cannot put their thoughts about a problem set into code on
a white board.

on the other hand: a dev answers all problems with code on a white board.

i find it hard to believe that taking a dev away from their desk and providing
a whiteboard with marker is so different that it would make one become a "bad
developer" incapable of solving a problem.

i find it easy to believe that the dev has just not practiced coding on a
whiteboard. therefore, when asked to do so, cannot bc of to anxiety from not
practicing.

if one can write english on a whiteboard and not get anxious, then why would a
professional developer get anxious about writing c++ or java on a whiteboard?

like i said, i go back and forth on this when it comes to interviewing people.

~~~
tomp
I've been a developer now for a year and a half, and right now, I don't think
I would have any anxiety about writing code on a whiteboard (even if the final
outcome would be no code - I can't know everything!) or even over the phone.
However, when I interviewed for the present job, I was straight out of
college, and even with 10+ years experience being a hobby programmer, I was
very very nervous. So, I would say coding questions are OK, except maybe for
junior, inexperienced applicants.

------
suresk
This seems like a reasonably good process. This item under "DONT'S" is a
particular annoyance for me:

* Don’t ask those who interviewed the candidate before you what they think. Form your own first impression.

For some reason, some people feel the need to grab you before stepping into an
interview and gush about the candidate "This person is AWESOME! I'm really
excited about them!", which, if you aren't cognizant of it, can heavily skew
your evaluation of them and basically destroys the point of having multiple
people interview a candidate.

The lone exception might be if a candidate is so unbelievably bad that you
want to save everyone's time by ending the whole thing early, but hopefully
your screening process is good enough that it doesn't happen often enough to
worry about.

~~~
RyanGWU82
Why do we conduct multiple interviews in the first place? Because, as a group,
we want to gather all the information we need to make an informed decision.
Some quick discussion between interviews can help make sure we gather _all_
the information we need -- as long as you keep it to narrowly defined topics.
In particular, I always try to find out: "What did you cover in your
interview? What _didn't_ you cover? Is there anything I need to know?"

Let's say we're interviewing someone for a role with HTML5, CSS, and
JavaScript. If the first interviewer ran out of time before getting to
JavaScript, I should probably start with that.

Or, if an interviewer noticed any red flags, we can use the following
interviews to gather more information. If I was told that a candidate
complained loudly about being overworked, then I could spend more time
discussing time management and the way he works. Maybe he legitimately _was_
overworked, or maybe he's a complainer, or not very good at prioritizing his
work.

Finally, it helps if there's at least one recruiter or a manager that you
_can_ give real-time feedback. Recently I was on an interview loop of a
candidate for a very specific role. I quickly realized she was a terrible fit
for this position, but she'd actually be perfect for a different one. I went
to our recruiter and explained the situation. He was able to reorganize the
schedule so that the candidate spent the rest of the day interviewing for the
right position. If we had just continued without any feedback, it would have
wasted a lot of time and we would have missed out on a great candidate for the
other role.

------
pathy
I think most of the things mentioned in the post is a good way to go about
interviewing. However, I don't really agree with the "no resume - look at open
source" part.

I know I am not the target demographic for this company as I am no coder and I
don't really have any interest in contributing to open source development,
even if I were a coder. Surely there are other/better ways of initially
judging potential new recruits?

As far as I can tell the author is an open source advocate so I guess I
wouldn't fit into the "culture" but ignoring those who aren't as into open
source as the author would exclude a number great of people who might
contribute greatly to the company (and a missed opportunity to convert them
into open source advocates/fans/contributors).

------
zenocon
Wow -- this is one of the first write-ups on this dead, beaten horse that
makes a considerable amount of sense. I've often done something like this
"lite", and had reasonable success.

~~~
tptacek
Wow is recruiting and hiring practice ever _not_ a dead horse.

~~~
llimllib
Which reminds me, I'd love to see a post about how you do it, and especially
how your perspective has changed as your company has grown. #justanidea

------
fatalerrorx3
It sounds like they've found a legal way of getting free work completed.

Is there some kind of signing bonus if you make it through the interview
process and get an offer? I think there should be.

~~~
lukeschlather
Yes, they must be making a killing getting people to build them cars for free
in C++.

More seriously, it sounds like they go out of their way to avoid giving their
applicants any work someone might profit from.

------
dayyan
Where is the terse, elegant version of this.

