Hacker News new | past | comments | ask | show | jobs | submit login
Interviewing candidates (ericlippert.com)
119 points by daigoba66 on June 8, 2015 | hide | past | favorite | 81 comments

I feel like this is how everyone thinks they interview candidates, and that it doesn't really work. I wrote about this at length, specifically so I wouldn't write the same long HN comment every time this comes up. :)


Thanks for writing that up; really interesting stuff. I agree with many of the critiques you make. I suspect that it may be easier to get the sort of standard process you describe implemented in a relatively small organization that has a relatively consistent set of products and commensurate challenges than in a large, diverse organization like Microsoft, where I learned to interview. The Microsoft interviewer has the challenge of finding candidates that are good both for the specific team they're interviewing for, and also have the general skills necessary to work with other teams in the future. As you say, it's a hard problem.

But Microsoft is a large organization, with deep pockets! Is time spent crafting meaningful challenges for each team really that much more than, say, the man-hours that each team will have to spend to interview candidates?

I appreciate that you're tired of writing the same comment over and over, but I'd really appreciate your going into some depth about what "this" is that you're referring to and how it contrasts with your approach. It seems like the original article describes a very scripted, standardized interview that tries hard to put the candidate at ease. If you were to condense your article into a sort-of "Joel test" for interviews, I imagine that this process would score reasonably well.

It's based on the assumption that it's reasonably possible to put a candidate so at ease that they can solve interesting programming problems in front of an interviewer. But in-person interviews are inherently hostile, if for no other reason that they're timed and adversarial. It's better that we jettison the entire notion that they can be a venue for demonstrating programming ability.

I'm not opposed to interviews. I just think you need to factor out the programming from them.

Huge fan of that hiring post, by the way.

I'm currently working with a roomful of cognitive scientists on ways to unpick some of the bias from recruitment. Be good to have a chat at some point if you're interested.

I'm easy to get ahold of. :)

A very major difference that I noticed is that tptacek's process doesn't turn down a candidate if they don't have the requisite domain-specific knowledge; rather, they send them a bunch of learning resources and let the candidate resume the process at any future time. The OP's process would just reject the candidate, forget about them, and move on.

tptacek's process is also going to work a lot better for candidates who aren't comfortable in high-stress whiteboard coding situations. The OP's process is designed around the assumption that the candidate can be made comfortable; tptacek's process is designed around the assumption that some candidates can't be made comfortable, but you shouldn't reject them (and in fact you may desperately want to hire them if you could find out how good they are), so you need to find another way.

Although the OP's article states that he is prepared to offer a computer for anyone who prefers it, how many candidates are confident enough to tell an interviewer that they don't want to do whiteboard coding? My experience is that candidates tend to be incredibly submissive during interviews so they're not going to speak up about something like that.

Based on the OP's article, I didn't get the sense that they have a rigorous rubric or that they meticulously record objective data points for every candidate to establish a reliable dataset over time.

All that said, I do feel that the OP's example question is pretty good (for that domain of expertise, at least) and better than what I typically see in these sorts of articles.

These are good points.

The number of candidates who have told me that they want a computer rather than a whiteboard is small but not zero.

I do give the same few problems over and over again, and I do take extensive notes, but I am not keeping track of a specific set of metrics that I track across candidates. I have a pretty good sense of where the "middle of the pack" candidate is. The process could be more scientific, yes.

Do you worry that your brain is really, really good at reassuring you that you're making careful decisions? Mine is, and it makes me worry. I tried to design a process with that in mind.

Yes. There are so many possible biases when interviewing and it is hard to keep them all in mind at once. The practice of science is essentially one long story about overcoming those human tendencies to arrive at the truth.

I started my career being pretty terrible with interviews for reasons pretty well encapsulated in Thomas' article -- general nervousness, social anxiety, lack of confidence (in retrospect, I believe this was primarily an impostor syndrome issue).

I've gotten better at it with age, but the improvement in my interviewing skill is completely unrelated to my improvement in programming skill, though I believe both continue to improve still into my early 40s.

Speaking on behalf of younger me, and probably others like younger me:

Doing the work on a computer is scarcely better than coding on a whiteboard, and in some ways it is worse. It isn't my computer. It is set up all different, it would take me hours to set it up like mine, the keyboard is all weird, what the fuck is the ctrl key doing over there, fucking Lenovos, all of these are trivial differences in isolation but in aggregate they make the work seem just as foreign as doing it on a whiteboard especially because I already feel like I am "under the gun".

Also (and by far most importantly and relevant even if I had brought my own computer in): I'm being "watched". Even if you aren't actively watching me (or rather, younger me), I am effectively being watched if I'm sitting in some room at your company banging out code. I'm all up in my head about how this is taking too long, how long ago did I start, how long are they expecting this to take, etc, I never get anywhere even close to the "flow" that I commonly get into when I am doing real work, the whole experience is just completely unlike the experience I have when doing actual work and feels like torture.

And then lastly (drifting somewhat out of scope of both the original blogpostand Thomas' blogpost) another issue I've always had with these types of interviews is that I'm somewhat of a subconscious worker, in the sense that when I am presented with a thorny technical problem the way I often best deal with it is by not really thinking about it (consciously), but rather going for a walk, taking a nap or just otherwise zoning out and letting the solution sort of materialize out of my subconscious. This is obviously completely incompatible with any kind of traditional programming interview, where the interviewer is almost obsessively looking to "see the work" in how you're thinking about the problem and I can't just ask them to let me go take a walk for 15 minutes and get back to them with the solution they're looking for (generally - I'm sure some will respond saying they'd be okay with this, but practically speaking we all know it wouldn't fly in the vast majority of interviews). An extremely common occurrence for me back during the time when I did not interview very well is I'd be driving back from the interview and would just suddenly have completely fantastic answers for all of the questions that didn't go very well on-site, a technical interview version of l'esprit de l'escalier.

Having said all of this, I'm not one to suggest anyone who interviews in any specific way is objectively wrong (for their own needs). If whatever system you use generates enough hires to get the work done, then so be it; but I can really relate to a lot of the problems highlighted in Thomas' article, so at the very least I hope people interviewing with more traditional methods than he has been working towards realize they are certainly filtering out some really great hires (which is maybe okay for them, but like Thomas I believe ultimately bad for the industry if we continue with what we have now as the standard).

I recently totally bombed a programming interview, and I always feel like I've done terribly, even if feedback afterwards is quite good.

I'd be interested in hearing how you improved your performance in interviews, if you wouldn't mind sharing?

Partly it was just experience. Interviews (as done poorly in the tech industry) tend to follow certain patterns, the more you do the more you see the patterns and the more you can adjust for them.

Perhaps most importantly (and this is something that is often mentioned by people give interview advice but easy to ignore if you're a "meritocracy"-minded techie) I've gotten really good at just sort of taking control of the interview and leading it where I want it to go (while still being sure to display how I would be valuable to the company) rather than being a passive question-answer-er, but getting good at this is also pretty much just down to experience.

Thanks for this. Even when well-implemented, the 'standard process' for interviewing is rather unreliable and stresses the wrong skills.

Worse, it's almost never well implemented. It seems almost designed to crumble under the smallest misapplication - a badly worded question, a dry whiteboard marker, or an unsociable interviewer can destroy a candidate. None of this is conducive to good hiring.

The following is utter pedantry, but probably thousands of people are going to read that page. "Consligeri" (a few paragraphs into section 2) is plural; you meant "consigliere".

I'm not looking for a job, but I want to do the work-sample part of your hiring process after reading this post. Sounds like a fun time.

Then you will like the thingy we're about to launch a lot.

Don't forget to update us on the progress and the results after launch.

It would seriously change my worldview if it will turn out that Google, Apple, Facebook, Microsoft, Dropbox, Evernote, Amazon, Airbnb, Uber, Square, PayPal and countless others firms that hire their engineers using some variation of whiteboard coding would be proven wrong in their practices by one person.

I don't know how to respond to this. I was simply telling the parent commenter, if the work-sample stuff we did at Matasano sounded fun, the thing we're doing now basically extracts and amplifies all the fun we think was in that process and opens it to the public.

"How are things going?" is actually a terrible question to ask someone in an interview, if that's literally how you ask it. It is saying to them, in so many words, "tell me how you think you are doing, performance-wise, so far?"

Horrible, and about the direct opposite of a question that will put someone at ease. Questions that will put someone at ease are ones that have nothing that can be construed as something you are assessing their answers to, like asking them about their visit to the area or if they enjoyed lunch or something like that. Break the ice, tell them about what you work on, see if they need a break, and then start digging into stuff. I also like to tell the person up-front what we'll be covering in the interview so there are no surprises. Walking in the door and asking them a question they could interpret to mean to reflect on their interactions with the previous interviewers makes me cringe.

Well as I said in the article, the intention of the question is emphatically not a "gotcha" about how they are actually doing. It's intended to find out if there are any problems that need addressing.

Yeah it's a tough one I think to ask well -- maybe you've got the tone and wording down in a way that candidates won't misinterpret, but I've avoided that type of question because I'm not able to do so.

This is almost exactly the template that I apply to my own interviewing. However, I switched away from asking about a project on the candidate's resume because I found that that question - specifically meant to put people at ease - did not in fact put them at ease.

Instead, I found that people became surprisingly flustered - including one who said that "like a lot of things on there, it sounds cooler than it is" and went on to question his own commitment to the field. I asked about the project because it sounded awesome, and, when I pressed him to please tell me about his role, it was awesome.

I think that many people are not prepared to discuss every item on their resume. I don't think that's great, but I don't want to base a large amount of my decision on that fact. This question threw them off for the rest of the interview, so I've stopped asking it.

Some people -- i.e. introverts -- are uncomfortable talking about themselves and their strengths and accomplishments. Unfortunately that's kinda the point of the interview. Not sure there's much you can do about that besides (as you suggest) pressing them to answer questions and cutting them some slack for not being super forthcoming.

That's interesting feedback, thanks. I have found that most of the time people are happy to talk about their work, but I have also encountered the flustered reaction you describe.

As an introvert I can tell you that most of the time anything in an interview will be flustering if unknown.

Tell them to pick a project or two and prepare to talk about it ahead of time of the interview.

Actually many of the issues I have with the current "standard" interviewing process is all about the fact that its a "school test" environment instead of a "prepare for a specific task" environment. The latter being what real work is actually about.

"Red flags often surface at this point. I’ve had candidates with PhDs in computer science, for instance, who did not know that on a 64 bit architecture, pointers are 64 bits wide."

Because learning 64-bit architecture is impossible for somebody that have proven that they can learn a field, perform research, and successfully write and defend a dissertation? '32 < 64' is beyond them?

No, because we are actually building telescopes here, and astronomy PhDs apply, and when they are not actually clear on the difference between a reflector and refractor telescope you realize, hey, this person is going to need to learn so much about the very basics of the field that they're applying for that it's not going to be a good fit.

I've had candidates with PhDs in computer science who thought that pointers on 64 bit operating systems were two bytes wide. Do they believe that there are only 16000 possible allocations before the allocator fails? Not likely, because that's ridiculous. So how could they have that belief? because they understand so little about pointers that they don't understand the implications of their false beliefs.

I need someone who is going to command the salary that a PhD-level candidate will demand to be able to hit the ground running.

On your latter point (pointers) I've heard that argument before. I work with somebody that turned out not to know about hash tables. I could spin a tale about how this meant they were so wide eyed they'd just be incapable of doing anything valuable (an argument made so many times on here). Instead, I took 15 minutes and taught them about hash tables and how to do them in C++. A few pointed questions about run time and the constants implied by the promise of O(1) (i.e a good question), and from then on every time a hash should have been used sure enough, they used one. Even showed initiative and added a memcache type interface to some of our code that was performing very slowly due to poor access patterns.

I guess it is lucky for us that when they interviewed they didn't have an interview question that involved hash tables, because they would have been bounced for not hitting the ground running.

I bet I know a lot about a field that you don't. I wouldn't bet that you couldn't walk in and pretty quickly become useful, given that you've learned other things, and at one time I didn't know this field either yet somehow, somehow, learned it. I think you would do just fine; and I think somebody that can get a PhD in CS can handle the math that 2^16 < 2^64, even if they haven't thought about the implications re allocations.

Different perspective: I believe for most CS persons this would be a 5-minute-difference... i.e. it would take 5 mins for them to understand/research 64 bit OS pointers size. So basically you're penalizing potential good candidates for just 5 minutes of their life.

Yes, it may be only 5 min.

But it tells a lot about the amount of "just 5 min" things they don't know.

I was asked once what load average knows (and I didn't know it). Because my background wasn't in sysadmin.

The fact that you know one fact or doesn't is immaterial, but it's a good proxy.

Your hiring decision should not be based on only that, of course, but let's say it's a pixel in a picture.

Different perspective: I'd prefer to hire the astronomy PhD who knows the basics of the craft over one who doesn't. Reflecting vs refracting is pretty simple - astronomy doesn't really interest me and I'm aware of the differences. It's fundamental to the optics, and understanding the differences helps understanding why real data looks different depending on method of acquisition. Why not penalise the person that doesn't know the basics? You wouldn't forgive a carpenter for not knowing the difference between a hand saw and a table saw, yet they both cut wood and the difference takes all of a few seconds to learn.

For example: I'm a midrange DevOps/Sysadmin/Whatever-it-is-these-days. I will likely never be a senior. The (real) seniors are the folks who go home and mess with hardware and pull new toys apart to see what chips they were made with and so forth. I like to go home and play games. I do my job competently, but I don't have the underlying knowledge that those guys do. If I was up against one of those in a job application, then you'd be crazy to hire me at the same salary as a senior - there'd have to be a pretty big black mark against the senior, like being completely unpersonable or abusive or somesuch. Or the role used some of the other things I'm decent at. Perhaps a colloquial way to put it is that I live these things, but the seniors live and breathe them :)


Do you know who Eric lippert is or the work he does? This knowledge is absolutely very important for the kind of work for which he'd be hiring.

> You seem very entitled.

Personal attacks are not allowed on Hacker News.

I agree.

Ah, but the company builds static analysis tools. Dinging someone for not knowing the minutia of languages makes a lot more sense when the minutia of languages is your business.

I guess author assumes that if you don't know basic things like that, it is questionable that your degree is a proof that you "can learn a field etc."

Not sure that I agree on the choice of this specific "red flag"; on the other hand, having PhD in CS is a red-ish flag to me per se.

"... on the other hand, having PhD in CS is a red-ish flag to me per se. "

I would be interested in hearing your reasons for that.

So of course I'm generalizing, not everyone is like that, not everyone's CS degree is like that, etc. etc. Also, more true for recent grads.

But in my experience people with advanced "pure CS" degrees seem to be focused on the process not the result. I mean, they went to CS because they like to tinker with type systems, elegant algebraic concepts, nice abstract problems -- and not just as a hobby, they like it so much so they commited several years of their lifes to do that. Not that this is bad area of research, but generally when you look for a software engineer you look for someone not only smart but also pragmatic, who can get stuff done in a simple and efficient way, and this is kind of the opposite.

Of all PhDs, in my experience people who have degrees in areas that use programming as a tool not the goal in it self, e.g. physicis/EE, make the best software engineers.

You could say that about almost any kind of knowledge-based evaluation.

If I ask somebody ten factual questions and they know the answers to five of them, how should I interpret that? Sure, it would be easy for me to just fill in those particular gaps in their knowledge. But assuming my questions were well-chosen, it's reasonable for me to expect that they'll continue to hit similar gaps in the future, about 50% of the time.

That's not to say that a good interview consists of drilling someone on facts; more often, they come up implicitly in the context of solving a problem. And as an interviewer, I'll look much more favorably on a candidate who recognizes what they don't know, because I give them the benefit of the doubt and assume that in a real-world situation, they would be able to do the necessary research. But even so, if there are basic things they don't understand, it's a red flag because it tells me they're encountering these concepts for the first time.

He's not saying a PhD means you necessarily won't know that 64bit machines use 64bit pointers, he's merely pointing out that he has seen candidates where that's been the case, and that you can't assume that having an advanced degree means you understand the minutiae of computer architecture and how memory addressing & compilers work.

It had never occurred to me until I read this, but I have a sudden urge to ask my interviewers simple coding questions like "reverse a linked list".

In most cases it would be a silly waste of time, but it would also have saved me from one or two truly horrific jobs with skill-free managers.

This was my thoughts on the matter: http://seshbot.com/blog/2014/01/24/the-best-way-for-programm...

summary: ask the open ended question ‘what was the most recent interesting project in which you were heavily involved?’ and then focus most of your technical questions on the system the then describe. If a programmer can’t talk throughly about a topic which they chose themselves and are purportedly very interested, they probably aren’t right for the job (at whatever company I’m working for anyway!)

The benefits are that you wont be randomly hitting a gap in their knowledge, you get a better idea of what they feel are their greatest strengths, and because you can raise your expectations about the quality of their answer you can probe very deeply and find out how good they are at communicating, and whether they are actually blagging.

The problems you can encounter are: they might end up running the interview if you aren't careful; you must be very engaged and perhaps talk about things you aren't that knowledgable yourself, and it is harder to compare one candidate with another (which is VERY important at large companies where you must justify your decisions to HR)

I also wrote a thing about coding interviews, from the employee side:


"Second, what actually did they do on this project?"

A lot of developers I know when in interviews say, "Well, the "team" did this and "we" did that. You always want to use singular personal pronouns such as, "I did this", or "The team expected me to do such and such."

This is something a lot of potential hiring managers like to hear. If they have to ask what you did, many believe you're role or your work may not have been that important.

Yes, parsing "I/we" is a great way to filter out collaborative team players and hire antisocial narcissists.

Is that what you want, though?

I don't care as much about what the rest of the team did, because I'm not interviewing them. I'm more interested in what you, the interviewee, personally did to help the team achieve success.

I agree with what was said in the article. I'd say one half of programming in general is objective in the sense that either the program works or doesn't. The other half is completely opinion based - be it a design philosophy, choosing algorithms, future proofing APIs, hardening code against malicious actors, etc. This half is very fuzzy because there are no right/wrong answers as it goes more towards taste. IMHO you should try to evaluate the following - "What has the candidate done that makes their opinion worth something". If you can't come up with a very strong reason to give any kind of weight to their opinion, then I'd say do not hire them.

I feel like I follow this template as well. Except, I like to add a small self assessment at the beginning of the interview. Like:

      Q: On a scale from 1 to 10 (1 being low) how do you rate yourself on "Programming Language Du Jour"?
Then I can see if their self assessment is inline with mine. Which can help in reviewing the resume.

I also like to ask for them to choose a project in the past to talk about, because then it's easy to see what they are passionate about. Or easily tell if they are choosing to talk about something because they think, you think, it's cool.

I was asked by a hip, happening, popular web-app related startup to rate myself in C++.

I pointed out that Bjarne Stroustrup rated himself 7 out of 10 once. I wasn't as good as him, and I wasn't nearly as good as him, which left me the field of 5 and below.

I can guarantee that some ham-fisted chancer who'd flicked through the "Teach Yourself C++" chapter summaries put himself down as an 8, though.

Agreed - the answer to this question is really the result of how much one thinks one knows. Because a little knowledge can result in false confidence, those that rate highly will fall into two categories: Those that actually know, and those that don't.

If the interviewer can distinguish between these two categories, then what value does the question add? If the interviewer cannot and is relying on the answer to this question (even partially), then it's an example of relying on a false signal.

Whenever I'm asked this question, I always respond with the following questions:

1) What does a "10" mean?

2) What does a "1" mean?

3) What's the difference between 5, 6 and 7?

4) (Optional, this could easily come across as being snarky) What's the range of levels in your team/org/dept for this metric?

You are absolutely right, but your behavior would offend the kind of person who thought it was a good question in the first place.

I would welcome the challenge in what rating is what. Because I know they are highly subjective. This is not an adversarial question, it's mainly used as an indicator to gauge where to direct the interview.

I like to interview people across a range of subjects (as I have not had the chance to hire a pure specialist). So knowing what they feel confident in answering will help me tailor the questions to what their perceived strengths are.

So I think it gives me a feel of your confidence level in a language. If you flipped through a "Teach yourself C++ in 24 hours" book once, then rate yourself a 10 on a scale from one to 10. Then not answering basic C++ questions will raise red flags.

If they answer a "1" on something. (And I re-assure them that answering 1 is perfectly ok). Then I can feel a lot more comfortable giving them more hints, or a lot more supporting detail in a question that they have indicated they are not very strong in.

> I can guarantee that some ham-fisted chancer who'd flicked through the "Teach Yourself C++" chapter summaries put himself down as an 8, though.

It's very easy to filter out these guys though. I had one such come to my interview - on a scale 1-10 (10=best) the guy put 8s, 9s and 10s for several programming languages (C++, C#, Java, a few others); and that guy was fresh out of college.

Then during the interview it turned out the guy knows exactly nothing, even when I tried to talk with him about the most basic concepts of OOP (like, what is a class or an object), he couldn't provide any meaningful answer to lead the conversation further.

I have no idea how he hoped to pass an interview, perhaps he thought there wouldn't be anything beyond the online form to fill.

Talking about a project seems fine to me.

But the self-assessment on a scale of 1-10 just seems like a quick way out for the interviewer and can only serve to find fault with the candidate and to me, doesn't seem like it provides much information.

For example, what does 10 mean? Does that mean that they have mastered the language (and all of its minute details, implementation, etc.) and would find it impossible to learn more? Knowing that, the only possible individuals who would think about rating themselves a 10 are:

1) The language designers themselves. (Possibly not even - see the comment below about Bjarne Stroustrup)

2) Those that think they know the language so well that they would rate themselves a 10.

You're probably not interviewing people from group (1). (Or if you were, you wouldn't ask them this question as it would reflect more on yourself than the candidate) So, if someone rates themselves a 10, you can probably assume they are in group (2) or have otherwise misunderstood the question. I don't know for sure, but I feel there are better ways to rate a candidate other than asking them a question, that, if they answer "10" on, means they aren't suitable and are likely overrating themselves.

So where does that leave us? Most people who are comfortable with the language are likely to rate themselves a 6, 7 or 8, being conservative and not wanting to expose themselves to a tricky follow-up question should they self-rate too highly. Again - this seems to be leading toward gotchas, and seems adversarial.

If you're going to ask for the self-rating, at least limit it to something less fine-grained: Beginner, Novice, Expert/experienced/whatever, so you have an idea for the candidate's comfort level with a particular language. A 1-10 rating is way too granular (what's the difference between 3 and 4?).

But really, I think asking the candidate to rate themselves is really just a spin on the "What's your greatest weakness?" question - it's asking the interviewee to do the interviewer's job.

You ask them to rank themselves on a scale from 1 to 10 to see if they know their limitations. I used to answer 10 for some things when I was young and dumb because I had no perspective of what expert-level knowledge really was. Now that I'm older (and still dumb) at least I know how much I don't know.

When they do give themselves a midrange answer N, the appropriate follow-up is "What do you know that a (N-1) doesn't?" and/or "What would does a (N+1) know that you don't?" Either of these will let you calibrate your internal rating system with their self-assessment. It will also let the candidate demonstrate their understanding of the topic in a non-adversarial way. The follow-up questions might convince them that they should change their initial ranking as well, which is also a useful calibration of their knowledge and meta-knowledge.

So according to this logic, nobody should have hired you back in the day?

The solution to this problem is to follow up the question with "oh, so you're an eight, great, what is something that a seven would have difficulty with?" Make them calibrate the scale for you.

Why not just give them a scale and ask them where they fit? Quizzing them about in the structure of the scale is awkward game-playing.

Or flip it around, and ask them what they are going to do in order to get to 9. It shows how self aware they are of their own knowledge gaps and also calibrates the scale.

On the off chance they say 10, well, they better know just about everything.

(TL;DR: Multi-purpose languages are pretty large and used in particular ways across different domains. It doesn't even make sense to ask about the difference between a 7 and an 8. An honest interviewee with become bogged down realizing they're not a competent social science researcher who can design useful fine-grained rubrics, and will refuse to give any answer to these questions except a long-winded and hopefully not too ego-bruising "this is stupid/doesn't make sense/it depends/I'm not a 1. Everyone else in the 3-8 range is just bullshitting the +/-1,2 deltas. So in other words, the entire number and reasoning is completely just BSing.)

> what is something that a seven would have difficulty with?"

My honest answer is "this is an ill-posed question and there's no way of stating for certain that feature X or library Y is something every FooLang developer who's not as good/informed as me doesn't know."

But of course that won't get me the job. So I just pick based upon my perception of your personality/background from the 5-8 range. When asked for justification, I choose the most esoteric-but-not-inconsequential thing I know and bullshit for a minute or two while being personable. Whelp, that was a useful signal as long as your goal as to find a professional bullshitter...

Concretely, I've created stand-alone implementations and hacked on compilers for a couple of languages, but would not have a non-BS answer to this question for those languages. Mostly because I'm totally unfamiliar with the most recent set of popular libraries in that language's ecosystem, and also there are certain language features that I know how to implement but don't know the canonical usage patterns/idioms. Your standard FooLang dev could probably hack together certain programs in their sleep that would take a week for me to write. But then for other programs -- or especially if you needed a static analysis or optimization -- I could whip it up in a couple of days where the standard FooLang dev would probably take weeks just getting up to speed on the language spec and some particularly thorny parts of the implementation. Hm. Apples and Oranges. And what if you want a static analysis/optimization that exploits idiomatic use of FooLang's whizz-bang BarLib? Hm. Not sure who has the upper hand.

I know you'll say "but that's the point -- I want to hear exactly that reasoning so I know what you know/how you assess language familiarity." But this is all meta -- performance on this question crucially depends upon 1) ability to bullshit (i.e., come up with plausible-sounding hypotheses and give plausible reasons for them, quickly and on-the-spot -- this is not a particularly crucial skill in software development); and 2) ability to intuit the purpose of the question.

In conclusion, I'm not really sure what the point of this exercise is, except to select for bullshitters or people who read your HN posts.

> Make them calibrate the scale for you.

You're not just asking for a calibration, you're asking for a rubric and a scale. That makes a lot of sense as an interview task if you're hiring a manager who will be doing lots of interviewing or a business processes/HR processes expert/researcher. I'm not sure why this is a relevant skill for a software engineer, though.

> Leave the candidate with a positive impression of the company

I feel like I haven't experienced this enough in interviews. Mostly I assumed that if I'm applying then interviewer already thinks that I have a good view of the company (or otherwise I wouldn't want to work there right?), but it would be nice to get a better outlook of the company. Maybe I'm just asking the wrong questions.

> if we make a bad hire, that can drag down the productivity of a team for years

This is the essential cowardice of employers. Interviews are insufficient contexts to judge if someone can help move your business forward. Screen the incompetent, but just hire people and don't be afraid to let them move on if they don't work out after 2/=4/6 months.

Love this, very useful for me, especially the gradually elevated technical interview steps/details, though it's less meaningful for junior engineers, who normally can't think that deep yet be it at interview or on the job.

Why not do 2 phone screens to filter out completely incompetent people and then let them choose:

- paid work for a week

- writing patch for a project or making simple tool(proxy of the job) and invite them to talk about it.

Because candidates have lives outside of code. They probably already have a job, so taking a week off for "vacation" which will actually be a "paid interview" sounds like a crummy deal. Writing a patch-- similarly, it's eating into their own time. You're going to lower your hire rate from that.

If they can take half a day or a day out of their life to show up for an interview then certainly they can take the same time out of their life to write some code instead.

but it's paid so you can take a week off between jobs. As I said, give them an option. Interview can take a whole day, doing a project pretty much the same, but you can split it into a few day or do it over the weekend.

We did this for our most recent hire - gave them a short project to implement over a long weekend. I think we shot for a target of having it take the applicants at most four hours; I threw together the reference example in an hour or two. I think it was a success, and we'll probably use this method going forward. There's just no substitute for seeing actual code, and by extension, the way someone structures the problem in their mind. It also give you some easy jumping off points to discuss with the applicant - Can you explain why you used data structure X here instead of Y? Change this requirement, and how would you do this differently? etc.

I don't really get it.

This is sounds like a very pretty typical software engineer interview process. The fact that these kinds of interviews suck, don't produce good hires, and yield a ridiculous amount of false negatives has been beaten to death.

Unless you're Google (or Facebook), you are not getting thousands of applications a day. You don't need to emulate their hiring process. They do it for a reason (practicality) whereas it seems others do it out of pure snobbery. Whiteboard coding is worthless (barring simple fizzbuzz tests which can be done over the phone anyway). I mean how pompous does this shit sound:

"All candidates eventually figure out that they will need to add an auxiliary data structure that maintains a map from a 32 bit integer to a 64 bit pointer, because of the aforementioned ten pound sack. Do they know that there are off-the-shelf map classes? If not, do they have confidence that they could write one?"

Eric Lippert is originally from Microsoft though, so I guess that shouldn't surprise me. After all, they're the ones that started asking "Why is a manhole cover round?" in software engineering interviews.

Much respect goes to tptacek that outlines a much better and enlightened alternative.

Well, my intention was not to sound pompous, but writing is hard, you know? Thanks for your cogent critique of my tone.

Regardless of whether you or tptacek has the better method for hiring coders, I have much more problem with dvt's tone in his post than yours in the article. Hope this kind of response won't discourage you from posting more, as I've had a lot of value from your writing in the past and found this article interesting as well.

I second this. I've read a lot of your writings and have even watched your episode of 'Checking in with Erik Meijer', both of are very informative. Thanks for writing this article.

I hope you are being coy, because dvt's post was obnoxious and ill-founded.

Oh wow: now this is a solid example of HN fandom. Google, Apple, Facebook, Microsoft, Dropbox, Evernote, Amazon, Airbnb, Uber, Square, PayPal and countless others are hiring their engineers using some variation of whiteboard coding. Engineers in those companies built literally 99% of the technology the author of the comment above uses every bloody day. From the browser he used to leave his comment, to operating system to run the said browser and the phone in his pocket to notify him of new entries on HN to open with that browser.

For some, this is not good enough proof though. A local star running an obscure company is certainly more enlightened -- because, you know, he wrote a blog post on hiring.

Your point is logically inconsistent.

There's some evidence that the iPhone in your pocket, for example, may have been built via forced manual/child labor in Taiwan/Vietnam/China etc. The fact that the iPhone is a revolutionary (and generally game-changing) product doesn't somehow add to the virtuousness of how it was built.

> The fact that these kinds of interviews suck, don't produce good hires, and yield a ridiculous amount of false negatives has been beaten to death.

It has been beaten to death, but it is nowhere near to being established as "fact".

Here's how you hire a candidate, in two steps:

1) If they provide a github/bitbucket/sourceforge link, look at what's there. Skip step 2.

2) Literally anything else. It won't work anyway.

Best interview, I had, was:

"We need a web-dev, you know node.js?"


"okay, lets do this"

Assuming you just need a candidate to do grunt level coding.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact