
We’re Bad at Interviewing Developers – Interview with Kerri Miller - GarethX
http://blog.fogcreek.com/were-bad-at-interviewing-developers-and-how-to-fix-it-interview-with-kerri-miller/
======
rorykoehler
As a non-CS graduate looking for my first full-time job as a Junior Developer
I can only ask one thing of prospective employers. Please give clear feedback
that will help give direction to the candidate if you reject them.

I have done a number of code challenges and while I don't mind doing the
challenges (I look at them as training) and I can always create a working
solution there is nothing worse than receiving a simple 'we won't be
continuing with your application' email with no other feedback. If you are
going to take up a candidates time with a code challenge the least you can do
is help the candidate become a better developer by providing constructive
feedback. You owe it to the applicant and also to the community at large.

I have had about a 50/50 experience in this regard. Of all the companies that
reject me with good feedback I am left feeling content that I have improved as
a candidate throughout the process. The others who don't provide feedback
leave me feeling like I've wasted some time that would be better spent reading
about best practices and techniques or developing my own software.

~~~
rasz_pl
Your mistake was doing them in the first place. Code challenges and
whiteboards are for cattle and cubicle cannon fodder at big companies.

Show them your github, credits in some os software, side projects, dont be
another no name code monkey.

~~~
rorykoehler
I'm thinking of taking this approach in future. I might not be the perfect dev
yet but I have taught myself to build reasonably complex apps in 18 months and
already have a track record of developing tech solutions to business problems.
As a Junior dev I expect to be taught best practices and to be continually
learning on a very steep curve for the foreseeable future. The fact that I
have gotten this far on my own steam should say enough about who I am. The
interview is useful purely from a cultural fit perspective.

------
agentultra
The golden nugget for me was the suggestion to ask the candidate to teach you
something.

I'm terrible at operating under pressure and generally need to take my time
with any problem I'm given. Unless you actually work in an environment where
it's critical find the optimal solution to a large constraint search problem
in 30 minutes or the tower blows up I don't see what the difference is to
taking a couple of hours to find the solution. The only way I'd be able to do
that is if I'd seen the exact problem before a couple of days ago and can
recall the solution from memory. For problems I haven't encountered before I
need a pad of paper and make a lot of mistakes before getting to the right
answer.

I'm just slow, deliberate and not well equipped to win at Jeopardy. It's not
my bag.

But if you asked me to teach you something I could go on for hours on
procedural generation algorithms, pseudo-random number algorithms, neat tricks
from old languages, compilers, macros... yadda yadda.

~~~
vonmoltke
> Unless you actually work in an environment where it's critical find the
> optimal solution to a large constraint search problem in 30 minutes or the
> tower blows up I don't see what the difference is to taking a couple of
> hours to find the solution.

Not to mention, if this scenario is frequent enough that you really need to
screen for it in interviews there is probably something fundamentally wrong
with the way you are running your business.

~~~
agentultra
And yet it's bewilderingly common to use it as a low-pass filter.

The reason I like the suggestion is that it asks the candidate to show you how
something works which seems like it would demonstrate their ability to
understand a topic, how they approach it, and importantly whether they can
communicate what they know in a way that other people can understand.

I'm more interested in whether a candidate can write a decent bug report and
form a coherent patch than how many puzzle solutions they've memorized to slip
through my filters.

------
PaulHoule
It tires me that there is so much talk about the problem of hiring developers
and very little about the problem of "what do you do once you hire them?"

That is, they may treat you like a rockstar when they sign you, but then if
you complain because you got a crappy Dell laptop that was a hand-me-down from
a salesman who couldn't sell, that your build process takes 40 minutes and
that this could cut that to 20 minutes. Of course they have the "sick system"
excuse that there is no money, but you get blamed for complaining about the 40
minute build, and then you get blamed that it takes forever to do anything
because you are always waiting for that 40 minute build to finish and the
system is such a house of cards that if you work on anything else in parallel
whatever you would have learned from that 40 minute build is invalidated.

~~~
morgante
It sounds like your perspective is colored by one particularly bad experience.

Every company I've interviewed at definitely seemed to make developer
happiness a priority (especially the smallest ones). Even the smallest of
startups (ie. one guy with a tiny seed round) understand that buying your
developers great gear is an essential investment.

~~~
vonmoltke
Try working for the defense industry. They tend to treat engineers like they
should be lucky to have a job.

~~~
linkregister
Depends on the contract =)

------
robmcm
Contract to perm is the best way by far.

In the UK the contract rates are typically significantly higher, so even if
the person isn't offered a job at the end they aren't bothered. Everyone wins

~~~
kangman
Isn't it probably also due to the fact that the UK offers unemployment and
healthcare benefits that far surpass what is offered here in the States? So
that there is a safety if at the end of the contract you are left with no
offer.

~~~
robmcm
I expect so, but you can also get double your salary with tax breaks when
contracting. So it would be easy to cover private health care etc with the
additional money.

------
chucky_z
I had written a large chunk but removed it after reading most of the article.
I will go out on a limb and simply comment that I felt the interview where I'm
currently at was done correctly, and addressed most of the issues that this
article raises. As a note, I was hired as a sysadmin/dba/devops for a smallish
startup.

They brought me in for a total of ~20 hours of interview, and in that time I
discussed work-related topics with every single C-level. I went through a
standard interview with questions, and then a longer skill test that included
every current engineer at the company.

For the last few hours they paired me up with a current engineer to work on
some database issues they'd been having.

I've been very happily employed here now for around 7 months, and after adding
this company to my resume I've had quite a lot of offers on LinkedIn. I feel
nothing other than "Oh, this company has offered me an interview, that's
really cool!" towards these offers.

I think it's one of the best things in the world to have that feeling, and I
wish everyone who was hired in an engineer position felt the same way.

------
jasode
_> We don’t do a really good job of hiring with intent. We decide that we need
more people, but we don’t do a really good job of figuring out what we need
those people to actually do, and who we actually need to hire._

I don't understand what she's talking about here. Every time I see a job
opening, the software hiring manager and the team peers know there's a pain
point that needs to be solved. E.g. "We need to an embedded C programmer to
make this firmware talk to our USB protocol". "We need a developer to port the
Java code to Go." "We need to expose our mainframe transactions as a REST
API", etc.

Look at the previous "Who's Hiring" thread. Do those posters look like they
post the job slots without a clear intent?

[https://news.ycombinator.com/item?id=10152809](https://news.ycombinator.com/item?id=10152809)

 _> just automatically do, and we don’t think about, “Well, what questions are
we trying to answer by asking a candidate to solve a problem?” Are we dinging
people for trivia questions, for not remembering, “Oh, I need this third
option flag, or an obscure method from a core library.”_

I'm not sure where her experience with whiteboarding comes from. In my
experience, companies use whiteboarding to sketch out algorithmic thinking.
Whiteboarding is the opposite of reciting trivia such as the " _3rd parameter
to an obscure library function_ " Interviewers don't care about perfect syntax
or missing semicolons.

 _> Instead, I really want to focus on questions that are asking about
decisions that they’ve made, what choices have they made, and what choices
would they make again in the future? Are they reflective about mistakes that
they’ve made? Are candidates looking for opportunities to improve, and how do
they actually go about it? Do they make plans for themselves, like how they
would improve a certain skill set, whether that be a technical skill set or a
more soft skill set, for example, management, or project shepherding for
example. Those are the kinds of questions that I think really get you at the
heart of not necessarily what somebody knows, but what they’re capable of._

Those questions are fine but it's wrong to prioritize them over concrete
programming questions. Companies are trying to evaluate if the candidate _can
actually program_ and those "Emotional Quotient" questions she favors are easy
to bs about. Instead, companies need to assess real programming skills and
they can start with FizzBuzz and then move on to more comprehensive
evaluations -- whether it's the take home programming problem or the onsite
whiteboard algorithms discussion.

~~~
wpietri
I'm hiring right now, and I read a lot of job descriptions before I wrote
ours. [1] I've definitely seen what she's talking about: generic go-hire-
engineers job descriptions. I also think the kind of thing you describe,
hiring for a very specific technology, is a similar failure mode. The
technological pain point of the moment rarely lasts; the more stable needs are
actually organizational. For example, over the next couple of years I want to
be able to hire junior engineers, so right now I'm much less concerned with a
particular technology stack than with someone who really likes mentoring and
working in closely collaborative environments. That way I can bring the junior
people rapidly up to speed.

I also have seen whiteboard coding interviews where interviewers are inclined
to ding people for not remembering small details, so it's a legitimate
concern.

[1]
[http://www.codeforamerica.org/jobs/chime/](http://www.codeforamerica.org/jobs/chime/)

~~~
RogerL
I've been explicitly told that the code should compile, as I write it (Hi
Microsoft!). That's daft, and I told the interviewer that that is not how I
think and did it my way.

~~~
Retra
I would ask for a white-board compiler, then.

------
leetrout
I know this is OT but on the off chance someone from FC sees this- these
videos are painful to watch with the low frame rate.

I'm glad this one has the soundcloud link. Some of the others I've enjoyed
don't and it's much easier to just listen.

------
nerdy
Testing for communication skills is awesome advice and probably rarely ever
consciously focused on.

~~~
adekok
As a related note, the best predictor of university physics scores is high
school English scores.

Why? Because physics is about making a conceptual model. i.e. telling a
_story_ about a problem. And then solving it.

For interviewing, communications skills are a must. People will be working in
a team, and must be able to explain themselves. And get along with others.

Another thing rarely checked for is the flip side... understanding
explanations. If you have to explain something four times to a candidate,
you'll probably have to explain things 1000 times on the job.

~~~
RogerL
I don't think so. The best predictor of university physics scores is...wait
for it.... university physics scores.

Likewise, the best predictor of on job performance is on job performance.

If this isn't a recent grad, interviewing on the resume will tell you more any
proxy for on job performance. Sure, people may be seeking to step up from a
drudgery type job, but I think that usually comes out (citation needed, I
admit). "I wanted to replace the terrible X with the wonderful Y, but I was
over ruled".

"question about why this was rational".

"rational explanation ensues"

"want a job here?"

I admit it ain't perfect, people lie on resumes and take credit for other
people's work, but we all know the whiteboard tests and other things are just
voodoo. Google's hiring data proves it out - only one person in their company
was able to give better than chance ratings based on interviews.

And, who really cares about resume inflation? I'm scrupulously honest in my
resume, but if someone can describe all the trade offs, costs, and benefits of
an architecture from a previous job, and answer speculative questions about it
("what would you change if you needed to scale X"), then heck, they can
probably do the job. If I understand engineering and am an active contributor,
what more do you want? No, "doesn't get nervous during a highly artificial
situation of whiteboarding" is not a rational want, since it will not add 1
dollar to your bottom line.

~~~
vonmoltke
> I admit it ain't perfect, people lie on resumes and take credit for other
> people's work

What frustrates me is that, as an interviewer, you can catch out all but the
pathological liars if you know what you are doing. The problem is, very few
interviewers in this industry know what they are doing. From the transcript
intro: "We typically don’t get trained on interviewing". _That_ is a glaring,
fundamental problem.

Worse are those who treat interviewing like a distraction from their "real"
job. Those people do not belong anywhere near an interview table.

------
biztos
I love the idea of asking the candidate to "teach me something, preferably
something nontechnical."

It sounds like a great way to test for communication skills while also
learning something about the person's interests. Plus it could make the
process a lot more fun for the interviewer.

However I expect it would depend as much on the personality of the interviewer
as on the interviewee. And at least in the US, it could have legal
implications if you decide not to hire someone because they didn't have
anything "interesting" to teach you.

~~~
mc32
I'm one of those people who take a long time to warm up to others. I'm not
someone's buddy just because we happen to be in the same situation, place or
because we agree on something.

So this exercise would be artificial and forced. And, yes, for sure, I can put
on a show, but it's not me and I'd be doing it because that the game or
expectation and further, I think it's rather beside the point.

------
paulus_magnus2
perhaps you should not hire developers but people who will own a whole
"vertical slice" i.e. do what typically is sliced horizontally between
business - product owner - analyst - architect - developer - tester

~~~
wpietri
I like the spirit of this, in that I personally enjoy working some in other
areas. Plus, I think dividing out work like "analyst" and "architect" can be
super dangerous: it shifts individual focus from project success to looking
like an awesome producer of architecture documents or analysis documents.

On the other hand, I think the model you suggest would also be problematic
because on a project of any size you need to glue back together a lot of very
thin slices. It adds the problem that no one person is going to be amazing at
all those things, so a fair bit of important work will get done poorly.

I think the best model is instead cross-functional, closely collaborating
teams where the decision about who does what is extremely fluid. That gets the
benefit of lots of people doing (and understanding) different sorts of work.
But it lets key work be done by people with strong expertise.

Real-world examples of this include IDEO's pursuit of "T-shaped" people:

[http://chiefexecutive.net/ideo-ceo-tim-brown-t-shaped-
stars-...](http://chiefexecutive.net/ideo-ceo-tim-brown-t-shaped-stars-the-
backbone-of-ideoae%E2%84%A2s-collaborative-culture/)

And Atomic Object's use of "poly-skilled, co-located teams of generalists":

[http://spin.atomicobject.com/2011/09/16/poly-skilled-
teams-d...](http://spin.atomicobject.com/2011/09/16/poly-skilled-teams-
deliver-products/)

------
gloves
@patio11's starfighter project could help this...

~~~
brobinson
When it's available. Can't wait for it!

------
warrentr
These fogcreek interviews are pretty good. It is too bad the video quality is
so poor.

------
keehun
Wonderful article! Expected an interview with NPR' Kerri Miller

------
dominotw
meh this tripe again.

