
Ask YC: Does interviewing developers leave you depressed? - Flemlord
I'm hiring a senior developer right now. Our first interview has a 4-question screen, where I ask them to implement a simple function, implement a recursive function, write a simple SQL select (join across two tables), and create a simple object model (how would you represent a car as an object).<p>~70% of the "senior developer" applicants fail<p>We're extremely happy with our process--this is only step 1--and it's been amazing at yielding a great set of candidates for each hire, but two weeks of administering the tech screen to applicants leaves me a bit depressed. (sigh)
======
tocomment
Just out of personal curiosity, you should try to find out if any of these
failed candidates are actually skilled developers but there's something about
the questions or interview that's messing them up?

For example I consider myself a reasonable programmer, I can certainly do any
project put in front of me. But I failed a YouTube phone interview because
they asked me to "say" a function for finding the longest common subsequence
of a set of strings.

I learned the following issues completely stop me from being able to program:

1\. Someone waiting on an immediate answer.

2\. Someone watching/listening to my though process.

3\. Not being able to experiment with the code on a computer.

That didn't cut it for them because apparently at YouTube a lot of your
programming is done over the phone (sarcasm).

I guess my point is, try to think of the exact tasks this hire would be doing,
and ask them to do that. Will they really be writing recursive functions
often? Also if possible give them a take home project, so they can work in a
more natural environment.

~~~
soc
I agree. When I first meet someone over the phone or in person I'm in a social
mood and prefer to talk rather than do a code dance for them.

Once I'm alone at my desk though it's easy to get down to business.

Think it would be cool if companies just said show up to work, and you work
for free for the first week. After that both parties can decide if they want
to continue.

~~~
HeyLaughingBoy
Maybe it's just me, but I tend to immediately discount any advice or proposals
that suggest I work for free.

~~~
pj
I would actually enjoy a one week type engagement to get a good fit for a
company.

It works both ways. You are selling your time. They are selling their
environment.

The problem is that the employer is accepting a lot more risk than the
employee. Do you like paying for something you can't return and you can't get
your money back for if it is crap? This is the situation a lot of companies
are in. They hire someone, they pay them to write the code or build the
project and it turns out it's all a lot of garbage and the company is out the
money.

This problem is really tainting our industry. I think a lot of people who call
themselves programmers are really just some kids off the street who grabbed an
open source package and started hacking away. They can build a website with
some graphics and stuff, but they have no vision, no experience with large
systems that are actually going to be used for any significant period of time.

It is quite depressing actually.

~~~
HeyLaughingBoy
_I would actually enjoy a one week type engagement to get a good fit for a
company_

So would I. But since I'm _WORKING_ for that one week, I expect to be paid for
it. I really don't think anyone should be surprised by that!

~~~
pj
Hmm... I see your point. Perhaps we disagree on the word "work." When I think
of "work" I think of moving a company in a desired direction.

A lot of people think of work as "sitting in a cube."

The question is, "How do we determine if the candidate's definition of work
matches the employer's definition of work?"

Maybe what you think you are doing when you are working is actually causing
more problems and moving the company backwards because they are going to have
to undo all the crappy code you wrote. I'm not meaning to attack you, I'm just
trying to illustrate a point. Know what I mean?

Maybe such an engagement could work out where, "If we hire you, we pay you for
the week. If we fire you, we keep the money and you are out."

This reduces the risk for the employer while also giving the employee a chance
to examine the company and be paid if the work is satisfactory.

~~~
seasoup
What are you going to do in one week at a place? It could easily take that
long and much longer to ramp up and even understand a part of the existing
code base.

~~~
pj
I've gone into a company with software systems that were a complete rat's
nest, learned the enviroment, accessed files and utilities needed to make
changes, and implemented a significant change in 2 days. A lasting and
flexible change that will enable the company to operate more cheaply and be
more agile in the future.

Many information systems can be implemented in a week. You could set up email,
an e-commerce site, a CRM, SCM, or even substantial portions, if not an entire
ERP system depending on the size of the company.

A week is a long time if you know what you are doing.

~~~
seasoup
Ah, the area where we disagree is on the size of the company in question. I
was thinking medium to large, while it looks like you are thinking more start-
up. I agree that a week is much more viable to assess someone in a startup. In
a large corporation, they'll be lucky to have a computer by then!

------
larryfreeman
Senior Developer does not always mean coder. It means, in my view, senior-
level experience and senior-level judgment.

What are the job requirements?

I was a manager for 10 years and didn't code except to fix bugs. When I coded,
I used C/C++. As a senior manager, I oversaw the JavaEE application strategy
at Sun.

At any rate, today, I am a Senior Developer at a start up that uses LAMP. I
would have no problem answering your questions today but it took me a full
month to get back up to speed.

It may be that you are focusing on the wrong questions, it may be that your
job description is attracting the wrong candidates, or it may be that you are
not promoting the job in the right channels.

Good luck.

------
gstar
Not depressing at all, completely representative of the job pool.

The technique you're using isn't common - a lot of traditional companies don't
interview on skill and technique (like Google do).

Backing up your 70% of failures, we recently hired a fashion designer at our
startup (we're in an interesting market!) I decided we should use the same
kind of interview we would for a developer.

Everyone we interviewed was completely baffled, but it really sorted out the
wheat from the chaff. At least 80% of the people we interviewed couldn't do
what they said.

Thankfully though, we've ended up with someone who is really incredible. In
fact, I think she has the 10x higher productivity that us developer types talk
about, and her old employer is trying to hire her back.

tl;dr - looks like you've got a normal pool of applicants!

~~~
clistctrl
what is an objective interview for a fashion designer like?

~~~
gstar
Difficult. IANAFD so this may be slightly technically inaccurate:

We got them to actually draw something (called a flat) from a half-drawn
garment, draw a full flat from scratch, identify trends from a series of
photos, pick textile colours from photos, identify designers from "signature"
garments, have them debug a spec. to see if a garment was reasonable (say the
spec called for a notion like a chunky zip on an unsuitable fabric), and a few
other things.

------
DanielBMarkham
I would be careful with your questions. There's a lot of bias that you can
create without being aware of it.

I once had an interviewer ask me for the five rules of database normalization.
Beat the heck out of me at the time -- I was just drawing a blank. I could do
the work all day long, but I couldn't remember the stupid names he was looking
for.

Similarly, I had a guy ask me to name the seven categories of this course he
was interested in that I had taught years ago. Same deal -- just couldn't
recall it, although I was very familiar with the material and had no problem
teaching it.

The problem with memory questions is that they bias against people who don't
currently have the answer in working memory. Simply because somebody has done
something recently is no indicator of performance. The best interviews warm up
on general questions about the material before diving deep. That gives a
candidate time to "change gears"

I also very interested in "why" questions over "how" questions. Classrooms and
cram-lessons are really good at "how" questions -- it takes some degree of
actual work to understand the "whys" of something.

I've interviewed a lot of developers over the years. 99% of them really aren't
anywhere near as good as they think they are. That's sad, but it's also an
opportunity for developers who care about their work to get ahead. It doesn't
take much ongoing effort to be truly outstanding.

------
Eliezer
Good developers get hired and removed from the pool. Bad developers go to the
next interview.

<http://lesswrong.com/lw/gy/you_are_not_hiring_the_top_1/>

~~~
Leon
That doesn't imply the top 1% isn't there; just that they are more likely to
be under-represented. Also, programmers are prone to move to new areas that
are more interesting, especially the top/best, so that can skew it back in
favor of the upper echelons. It always seems that the mid range to bottom
developers are more likely to find somewhere and stay, as they have so much
trouble finding a new place to go to. They are the ones that have
'stickyness'.

What is the biggest factor, from what I can tell, is that the best don't send
out their resumes but get poached by other companies or use their networking
to land a job.

------
noodle
do you screen them over the phone? seems to me like it would be much easier
and less depressing if you just gave them the work, gave them an hour, and
told them to email the results back to you. stuff that is nontrivially
difficult such that they can't just search and find answers. they have to
know.

edit: i think i'd prefer having to do this myself, as well, because i'm a shy
person in person and i find myself floundering when i'm on the phone and have
to "explain my thought process" to someone as i'm working through it. i'd much
rather write down an explanation.

~~~
felipe
I think "quiz" type of questions are pretty ineffective at filtering out
candidates. You do filter out the bad candidates, but great ones could also be
left behind. Who knows, the person could be an brilliant developer who got
nervous during the interview and had a blank. Or the phone call is breaking
up. Or the person is in a non-ideal location. Etc... I would not filter out a
potential hire just because the person couldn't remember the SQL syntax on the
spot...

For me, a senior developer is someone who is able to work in a TEAM, be
dependable (i.e.: The guy who you can call at the 3am with an urgent matter),
mentor others and be able to contribute and implement new processes and
practices. The technical stuff is important but should not be the main filter
IMO.

It also seems to me that you want to apply a certain logic or algorithm to the
process. Personally I prefer to call up and just have a conversation with the
person (trying to find out the aspects I listed above). Unless I am Google and
I am getting a gazillion applications, that's different...

~~~
noodle
i can agree with this. i don't like quiz types of questions either. if i were
to hire someone, i'd rather hire someone who i know i can rely upon to get a
job done, regardless of their knowledge. someone who knows how to teach
themselves, who is able to learn and learn quickly.

but, i still think that a longer-form, more essay-like quiz is better than
grilling someone over the phone and expecting them to continuously talk to you
while they're working through something.

~~~
enjo
I agree. I used to make the mistake of screening on language trivia and simple
functional ability (like they're doing here). HUGE mistake.

Now I screen on much bigger topics that tend to be more philosophical in
nature. The role of design patterns, benefits of one language over the other
(particularly functional languages), value of unit tests...

They submit their answers via e-mail, and within a minute or two I already
know if there is any value in at least talking to them in a more formal
interview setting. I've found that folks who understand development at this
much higher level _always_ handle the basics well.

------
lutorm
Are you depressed because your filter has proven very effective? It seems you
should be happy. ;-)

The thing to worry about with a very discriminating filter is, like tocomments
post, whether you are filtering for something different from what you really
want...

------
_pius
Yes, it is depressing.

That said, I've had success with doing the pre-screen in a Campfire room and
asking them to Pastie/Gist their code into the room. That way, you eliminate
some of the weirdness associated with coding over the phone, but you maintain
much of the immediacy.

~~~
Flemlord
That's exactly what I do. The first interview is over the phone and they type
out their answers over a WebEx link.

------
kmano8
For my current job (for an entry level position), I took a one hour test
asking to do pretty basic coding prior to interviewing with anyone. It asked
to find errors/poor practices in a code snippet, some quick logic/binary
operations, and a larger question involving linked lists that required a few
minutes of planning before I began writing code out.

This was a comfortable setting for me. Though in previous interviews, I would
completely freeze when someone wanted a technical explanation, simply because
I need a moment to collect my thoughts, and it's difficult to do so when
someone in front of you is looking for an immediate answer.

------
jerf
The only ones that somewhat depress me is a particular class of bad developer:
Fresh out of college, these people have their curriculum _nailed_... and know
_absolutely nothing else_.

One example that really struck me was an interviewee that I asked to write a
simple pre-order traversal on a seven-node tree that we print out for our
interviews. Instead I got a moderately lengthy and correct description of how
to search for a value in a sorted binary tree. Correct, except for A: it
wasn't the question I asked and B: immediate visual examination of the tree we
were looking would reveal it was clearly not sorted. Other things in this
interview led me to the conclusion that I got that answer because this person
had studied that topic, but almost couldn't even hear, couldn't perceive a
question that wasn't essentially right off an exam.

These are the ones who depress me a little, because they have clearly done
everything everyone told them to do (study hard, do well on tests, etc) and
must have some significant raw intelligence to be able to do this well on a
challenging curriculum... yet, for all that, they have walked away with so
very little useful knowledge or skills. I also feel some anger at the system
that they have learned too well to navigate... but by the time they get to me,
it's too late to change anything.

I comfort myself in the near-sure knowledge they will find a job somewhere.
Some percentage of them will even adapt to the real world and perhaps they
will become great programmers. I don't know. I can't know. That's life.

But I can't in good conscience give a thumbs up for them. We are willing to
train solid developers and we have a track record of hiring people that don't
even know the language they will be working in (since they have demonstrated
the ability to learn languages in the past), but you can only take that so
far.

On the other hand, those who sat in college for four+ years and know neither
the curriculum, nor any other skills, just barely squeaking by with the
necessary grades and forgetting everything as soon as the class is over, well,
they had every chance to realize they were in trouble and didn't take it. I
don't feel depressed about them at all. That such people exist is just a fact
of life.

------
donaldc
I'm not sure why that's depressing. You've managed to filter out the majority
of people you'd not want to hire with just a simple set of tests.

Perhaps a way to screen out many of these people without spending the time
administering the screen is to have a simple coding test that these people
complete before you'll even have them in for an interview. A couple places
I've interviewed with did this, and they both expressed high satisfaction with
the reduction in interviews of hopeless candidates that resulted.

------
vax_11
Part of it may be your questions, and part may be HOW you're asking (e.g. over
the phone, or putting someone on the spot).

2) Not a lot of people write SQL by hand anymore. I used MySQL for years, long
ago, and never wrote a join. Then ORMs came along and handle joins
automatically, so I've still never written one. So not being able to do it
without looking it up doesn't really tell you anything. I've written far, far
more complex things than an SQL join statement. A more pertinent issue I'd be
checking for is whether user input is properly escaped to prevent SQL
injection -- again, not an issue with ORMs, but an issue when building SQL
query strings by hand.

1) A lot of people use languages where recursion is unusual (i.e. NOT Lisp).
Even a senior developer (especially of something like PHP) might not recall
this, not having actually written a recursive function for years.

3) Representing a car as an object: A lot of software people don't know
hardware, so the confounding part might be recalling various attributes of
cars and how they relate to each other.

Also keep in mind that many introverted people get very uncomfortable when
someone is waiting on them. This can make it difficult to concentrate at all.
Being asked to perform on demand is normal for a musician but foreign to most
programmers. On top of that, it's usually in an unfamiliar environment, such
as verbally or on a whiteboard -- imagine asking a violinist to play on some
lines you drew on a piece of paper.

PG, RTM, and TLB would probably fail your test -- they don't use SQL, they use
flat files.

~~~
plinkplonk
"I used MySQL for years, long ago, and never wrote a join."

Scary! I wonder about your definition of "used", if you used an RDBMS in a non
trivial app "for years" _and_ never wrote a join! The database design must be
very ... unusual :-).

And this was in the days before ORMs :-D.

" Then ORMs came along and handle joins automatically, so I've still never
written one. "

I've never yet seen a non trivial RDBMS based app where the existence of an
ORM absolved developers from having a good grasp of SQL when they needed to
drop below the abstraction layer provided by the ORM.

"So not being able to do it without looking it up doesn't really tell you
anything."

It does, really! ;-).

"PG, RTM, and TLB would probably fail your test -- they don't use SQL, they
use flat files."

The interesting thing about this statement is that it shows one way out of
potentially being asked to code on a whiteboard/over the phone. If you have a
PhD' from MIT and/or have a strong track record of writing Open Source
software, creating and selling a startup, build robots (and electric unicylces
) for fun etc, in other words, if you can _show_ , well before the interview,
that you are as good or better than the job demands, you are very unlikely to
face the "find the longest subsequence in a string" or "write a join" type
questions. But then if you could do all that why would you look for a job?

In my case, after writing some open source software, (and posting an url as
the first thing in my cv) I've found that these kind of elementary questiong
simply drop away. When a few thousand people use your code everyday, people
_know_ you are good.

The OP's post is about when all you have is a cv that looks like the last 100
cvs you reviewed, and people who can't answer simple questions about how to
write a join (hence his depression).

~~~
jasonkester
"I used MySQL for years, long ago, and never wrote a join."

Wow.

Note to self: start asking candidates to write SQL in interviews.

It had just never occurred to me that people who spend their entire lives
talking to databases wouldn't know how to talk to them directly.

And really, you can get pretty much any piece of information out of a database
using just _four keywords_. It's the least memorization of any language in
history. SELECT, FROM, WHERE, and for extra credit, LIKE. That's it. That'll
get you any non-aggregate dataset you could want, joined or otherwise.

Learn IN and GROUP BY and you've got everything else you'll ever need for
reporting. It's just the simplest language ever.

~~~
vax_11
> It had just never occurred to me that people who spend their entire lives
> talking to databases wouldn't know how to talk to them directly.

You're begging the question. Most people who use databases don't "spend their
entire lives" talking to them.

Of course I've talked to databases "directly", tuned them, "alter"ed them,
indexed them, replicated them, and so forth, but just happened to use logic at
the application level instead of the SQL level for JOIN functionality.

~~~
james2vegas
Ok so hopefully the places where you used to work hired someone that wouldn't
write horribly inefficient client-side code to join two tables together when
its a SQL RDBMS, and sped up those applications|sites considerably.

------
anigbrowl
Flemlord, a lot of responses seem based on hypotheses about what a good answer
would have been. Could you follow up and give examples of what you'd have
considered correct?

My hobby-programming experience suggests...

simple - int add(int a, int b){int result; result = a + b; return result}

recursive - um, I like fibonacci sequences: void fib(int term){if term = 1
return 0; if term = 2 return 1, else return fib(term - 1) + fib (term - 2)}

SQL join: no idea, I haven't used SQL for years. I assume you mean something
like for(i = 0; i < table1.length; i++) do {newTable.fullname[i] =
oldtable.firstname[i] + anothertable.lastname[i]}

Car as an object: well I guess I'd have a struct with variables for size,
weight, power, turning circle and so on, and methods for accelerating, setting
direction [steering heading, speed (forward and reverse, or multipliers for
forward gearing)] and braking...then I'd have create multiple instances of
that object using a table of values...do you make racing games?

I'm not looking for a job, but I'm interested in knowing what sort of answers
you were after. The above seems super-simplistic to me, I'd feel nervous they
were trick questions of some sort and the obvious answers are actually the
failing ones.

~~~
philwelch
Your fibonacci algorithm is crap. You're tree-recursively calculating the
exact same values over and over and over and over again. Just try running it
sometime on a large n if you want to test your computer's ability to stall and
ramp up the cooling fans.

"do you make racing games?"

That's a question to ask first, not second. Maybe he's making a "Pimp My Ride"
style game where the actual driving part is less important but you need
detailed modeling of the seats and the ability to add a bunch of random shit
to the interior of the car. Maybe he's making a CAD tool for people to do body
design, so you need to be able to do computational fluid dynamics on it and
test weight balances, but also be able to visualize it in high resolution to
see if it looks cool and to print mockups for focus groups to say "this looks
like a kickass car".

~~~
anigbrowl
Agreed, but this is the very question I'm getting at - does Flemlord want to
know the quality of algorithms someone naturally comes up with, or is this
just to screen out the total BS artists?

Hence my mention above of how it feels like a trick question. I really don't
know whether the interviewer wants the autistic-but-informative answer or the
quick of-course-I-know answer. Trying to guess what sort of answer you are
seeking induces mild panic in me: I don't know you, but if your response in an
interview was 'what a crap algorithm' I'd be thinking 'this guy is a bit
Machiavellian...'.

I mean, I can think of a much faster way (than this) to get a given term in
the fibonacci series using a loop, but this one recurses, which was _what was
requested_ \- I had to quickly choose a problem domain in which to demonstrate
it. Towers of Hanoi would have been more impressive, but I don't know if I
could just rattle that off correctly, verbally. See what I mean?

~~~
philwelch
If I was interviewing you (I've never interviewed anyone) and you gave that
fibonacci algorithm, I would probably say, "Hmm, and how does that algorithm
perform? Can you think of a better one?" If you said "its performance is crap,
storing the previous two results and using iteration is better", I'd be happy.
If you said "exponential" and "linear" instead of "crap" and "better" you'd
get bonus points. Volunteering that information at the outset would get more
bonus points because it shows that you really care and that you don't even
feel comfortable writing down an exponential-time algorithm without loudly
warning everybody they shouldn't use it.

The standard interview question is "write a function to solve this problem
which has an elegant recursive solution", not "think of a recursive function".
But if you do want to think of a recursive function, try factorial.

    
    
        int fctl(int n){ return ( (n = 1) ? n : n * fctl(n)); }
    

(Don't use that code in an interview. It's questionable style for C, and the
ternary operator is scary to people.)

I was asked to write a fibonacci function once, and I actually said, "you can
do this recursively but the performance is awful" and wrote an iterative
solution.

The point of an interview is for you to show off. If you start going on and on
and on and on about unnecessary details the interviewer might stop you, but if
you launch into a talk about the space and time properties of recursion vs.
iteration and show some judgment for how to use recursion and how not to use
recursion, then I know how knowledgeable you are.

~~~
philwelch
Dammit, I made a syntax error. The factorial function in C should be

    
    
        int fctl(int n){ return ( (n == 1) ? n : n * fctl(n)); }

~~~
philwelch
(n-1) for the recursive call. God, I'm failing this interview.

~~~
Flemlord
Heh. Our actual interview question is similar but we ask them to add instead
of multiply. We ask them to add up all positive whole numbers equal to or less
than x. Good developers (who make it to the next round) bang out the answer in
literally 30 seconds. Ok developers (who still make it to the next round)
usually struggle for a while, maybe implementing it non-recursively first. The
other 70% just type nonsense until I cut them off after 15 minutes.

If somebody makes a mistake I tell them and give them hints until they fix it.
Well... it depends on the magnitude of the mistake I suppose. If they're not
even in the right ballpark I keep my mouth shut.

------
Leon
Steve Yegge's post on this is apt and may help those who haven't read it.

[http://steve-yegge.blogspot.com/2008/06/done-and-gets-
things...](http://steve-yegge.blogspot.com/2008/06/done-and-gets-things-
smart.html)

------
martincmartin
I have kids, and whenever I do interviewing, I get worried about their future.
I think the secret is to get them to feel passionate about something; as long
as they have the motivation, they'll want to learn. But how to do that? And
what if I fail? I shudder to think they'd have to stay in some of the crappy
jobs I've been able to leave.

------
edw519
Outsource Step 1 with an on line programming screen. There are many companies
that do this. Sure you'll have to pay for each one, but they will weed out the
bottom 90% pretty quickly.

------
andrewf
Yes. I used to do the chatty bit first before subjecting interviewees to the
"technical evaluation"; it seemed polite to engage people first.

That was depressing. At least I found out the technical evaluation was worth
giving. An astonishing proportion of people that came off well in less
technical conversation would utterly flunk the test.

I'm not talking minor errors caused by interview pressure. I mean people with
"5 years of C++ experience" writing void reverse_string(char* in) { char b[] =
new char; b = in; ... }

------
vaksel
I think the best solution is to ask the questions in as many possible ways you
can think of.

Some people are great at coding, but suck at whiteboard, others aren't
comfortable talking about their thought process.

So why not give them numerous ways to show their stuff? Have them find bugs,
have them code on whiteboard, have them write code in a compiler, etc etc.

Your goal is to find a good programmer, not a programmer who is good at white
board.

------
steveklabnik
Speaking of someone who's basically right out of school, I can tell you that I
probably wouldn't hire most of my classmates. It _was_ depressing.

~~~
kmano8
In reference to the post I made directly above yours, a classmate of mine got
an interview with my employer about 3 months after I started. The HR guy
approached me after his interview, and told me he left the entire linked list
question blank.. no pseudo code.. nothing. I was embarrassed.

------
rjurney
Interviewing leaves me depressed. I just spent most of an hour proving I can't
verify whether a tree is binary. Mostly, we didn't focus on anything relating
directly to the position or talk about my experience in any way. We proved
that there are holes in my education pertaining to basic CS, that I never
encounter on a daily basis as a web developer. I guess the rest of the stuff
was taken for granted.

I understand that these tests can be important, but I can't help but feel that
I will fail to qualify for any position that requires me to hold a host of
basic CS algorithms in my head, but that I could still do a good job in the
position - because I'm very good at learn as you go, and have learned lots of
higher level stuff while building elegant, complex systems - without knowing
how to verify a tree's binary nature.

------
akaGOMEZ
I've been working for 3 years as a front-end developer and somehow have
managed to pick up all of those skills minus the join (just started learning
SQL last week). Maybe your pool of samples is tainted. Where do most of your
applicants come from?

------
zackola
70% sadly sounds about right. end depression by making the screening process
as quick as possible.

------
jmonegro
Heck, I can do all of that and I'm nowhere near a "senior developer"

------
known
How will the interviewee know that interviewer is smarter than him?

------
mebigfatguy
My experience has been identical to yours. Ask simple questions, get constant
failures. In bad times, the signal to noise ratio is completely out of wack. A
recession is the worst time to hire people.

------
californiaguy
If you're going to put someone in the coding hot seat like that, at least give
them the decency of having 30 minutes alone with a computer.

