Hacker News new | past | comments | ask | show | jobs | submit login
Why we don't hire programmers based on puzzles and tricks (37signals.com)
648 points by mapleoin on Feb 22, 2013 | hide | past | favorite | 442 comments

I remember when I was prepping for my Microsoft interview. There was so much lore online as to the types of questions they'd ask and how to prepare. I even found 'How Would You Move Mount Fuji?' in the library and started reading through it. It was an absolute waste of time as I, fortunately, learned weeks before the interview. As an example, one of the questions I was asked was 'how would you write the C function strcmp() without using any libraries?'

My gripe these days is that places I've interviewed with now that I have a few years of experience are still concerned about big-O notation and a lot of the more academic questions. I feel like I shouldn't have to go back to my college notes to prep for an interview for a job that I'd really be using my current experience for. I think it comes down to other developers not spending time really thinking about the quality of their interview questions. They just do a search for 'programming interview questions' and call it a day.

My most recent interview was with Amazon. I was asked the dumb big-O runtime stuff as well as how I'd count all of the stars in the universe. It pissed me off, but I did like the question I was asked about how I'd design an elevator system and then asked how my solution would scale. That's not too far off from what I'd be doing, albeit in an abstract form.

"What is the big O time of heap sort?" vs "Would you mind writing a function that returns true if any two numbers in the list sum to zero? For example, it would return false for [2,3,5] and true for [-3,5,3]"

Are completely different questions. The first is useless: It tests if someone remembers how a heap sort works, and whether or not they use the term "worst case" in the answer.

The second one is much better. Some could sort the list, (nlogn time, constant memory) then walk through it. They could use a hash table (idealy n time, assuming no collisions, but more memory). You also get to see if their solution handles the 0 case. If they answer it very quickly, you can move on to the three sum problem. If they recognize the question from university or something you can do a different one.

If someone doesn't know the big-O time of iterating through a list of integers once for every integer, then I seriously doubt their dedication to the field of software. This is not some pie in the sky doubt, it is of legitimate concern whether someone knows how to properly approach a problem of this nature with the right data structures, code, and communication.

Good interviewers put the interviewee at complete ease before they bring out the technical questions. From an economics point of view, it is better to hire the person that is in all other ways identical, yet interviews poorly, since they are less likely to want to leave.

Very good illustration, I must say. I strongly disagreed with the GP's rant about knowing O-notation, but I realized he might have faced the first type of questions a lot (e.g tell me the O-complexity of X algorithm).

My experience with Amazon was actually the opposite, but I might have been lucky. All the interviewers were very friendly and started from simple questions and went on to more complex ones. Never a "do you know concept a from language Y". In fact, one of them worked with me through the solution to a pretty tricky problem, and when we were done, he actually took a picture of my solution on the whiteboard! So yeah, it was fun :)

That was my experience at Amazon as well. Most of the interviewers were very good and the questions were a reasonable judge of skill IMO.

I didn't get an offer (understandably so) but I thought the interview was good.

That mirrors my experience with Amazon too. Although it was 3.5hrs of full-on whiteboard coding, I really enjoyed the process and felt like I learned a lot. 3 of the 4 interviewers were excellent.

Were you given an offer by Amazon?

I generally agree, most logic questions are useless, but Big-O notation I use every day. Programmers who don't understand the performance difference between Array#include? and Set#include? in ruby are dangerous!

I'm curious to know where you use the notation in daily programming. In code comments? Discussions with other developers? Other documentation?

I use the spirt of Big-O almost daily, but do the actual calculations rarely, and basically never use the notation. With some experience, you quickly get an intuitive feel for execution time, which is good enough in most cases.

If I'm reading up on a new technology it's useful to understand what they mean when they say 'operations are O(1)'.

Additionally, when writing an API for use by other developers its useful to say, this operation runs in constant time, that one in logarithmic time.

Redis, for example, mentions the computational complexity of all operations in its docs. Additionally, everyone should know that most decent sorting algorithms are O(n log n), and understand the implications of that.

The last time I talked about Big-O notation was three days ago, when talking about whether or not inserting/deleting from a list should be done online or queued up:

  Zarel: and for insertions, aren't they O(nm) for n insertions into a list of size m either way?
  Cathy: inserting n elements into a list of size m when done with queueing requires at most m iterations
  Cathy: when not done using a queue, it requires at most n*m iterations
  Cathy: a factor of n is the difference
  Cathy: of course, n might be pretty small, depending on how many insertions happen in the debounce wnidow
  Cathy: n might be 10-15
  Zarel: how do you insert n elements into a list of size m in m iterations?
  Cathy: iterate over the list of m elements
  Cathy: well, i guess you'd need to sort the list of n elements first
  Zarel: oh, I see, I forgot about sorting the list
  Zarel: so it'd be O(n+m)
  Zarel: which is admittedly much faster for our purposes
  Zarel: well, considering n is likely to be approximately 5, "much" might be an exaggeration
  Cathy: in any case, i agree that inserting/deletion approach is the right way to do this
  Zarel: incidentally if we wanted we could apply certain optimizations to improve an insertion from O(n)
  Zarel: for instance, binary search
  Cathy: it's not immediately obvious whether that is faster than the sorted queued
  Zarel: actually, that's not a bad idea regardless
  Cathy: but it's certainly faster than the naive single insertion
  Zarel: I mean, O(n log m) is significantly faster than O(n+m)
  Zarel: when n is ~5 and m is ~1500
  Cathy: true, and it actually makes it clear that using a queue is unnecessary
I don't know of a better way to discuss optimizing an inner loop than to use big O notation.

Actually using something like a math notation to solve a programming problem is asking for too much. What are those general problems which can't be solved by standard libraries, frameworks and IDE's these days?

And even if you are facing problems that mandate you to roll out your stuff how much of that requires these math tricks.

Given all this, if you truly want to hire a math geek to get some magic done. I guess you are looking at a very small pool of people to hire from and it should hardly be a problem to hire one.

Big-O notation is hardly "math geek" stuff. I'd not call this comment ridiculous, because I might have misunderstood it, but catch me in my cups and ask me for my true opinion and maybe I'd say that.

Big-O notation is pretty basic, and very useful. Combined with a memorized table of powers of two (you don't need all of them - I know up to 2^20, and this has proven sufficient - just enough so you can guess log N for a given value) you have a good chance at being able to make quick calculations about time and space requirements for things. Which, since we don't have infinitely fast computers with infinite amounts of memory, often comes in handy.

I'm not talking about Big-O. I'm talking about the general theme of these interviews.

Besides that I still don't understand this. I don't even know the last time anybody around me ever had a problem or a show stopper problem anything to do with Big-O or an algorithm.

Its always how the programmer can take pressure, how he can contribute to making the project happen, getting things done, how he can collaborate with fellow team mates, how he responds under times of crisis, how good he is with tools, what are his everyday practices, does he write unit tests, does he do code reviews, does he document the API's he writes, how does he package his code, how update he is with software world(tools, framework and techniques) etc etc.

Lets be frank, Its always these things that are relevant today in practical everyday software projects today. Unfortunately people hardly ask questions on these and then complain of bad software practices among people these days.

> I don't even know the last time anybody around me ever had a problem or a show stopper problem anything to do with Big-O or an algorithm.

Either your team does horrifically easy work, or no one has told you about their abstract work. I am sorry, but even writing CRUD apps I have to think about the informatics of the system to make it work reasonably well.

All one really needs to know about Big-O is that nested data lists can be exponentially harder to process. If you understand that then you understand the need for it and when the time arises for you to create these type of structures you'll calculate the time needed to process them accordingly using said method. In turn helping you predict future performance and avoiding bottlenecks.

In your example, I see Big-O as a side effect of using the right data structure for the job. If you need a data structure to hold items that should not be duplicated or ordered then a set (or hash which is likely what the ruby set is based on) is what you want to use because it is not only faster but conveys meaning to the maintenance programmer. If you need order maintained then move up to an ordered set. If you need to keep distinct duplicates then some sort of list is going to be needed (Array, Linked, etc...).

I guess what I'm trying to say is that Big-O between a Set and Array do not matter if you can't use a set to begin with. If the problem fits a set then using a set is right regardless of the performance implications.

I do agree with your overall premise of laziness though. Most programmers default to a list likely because it is the first thing ever taught, and for some reason rarely think beyond it. For me, part of the fun of solving coding problems is finding the best fit data structure :)

> and a lot of the more academic questions

It is on purpose. A <bigcompany> recruiter confessed. He said they are specifically targeting recent grads. So plenty of questions are of that type.

I guess they want to indoctrinate and take advantage of them while they are "fresh".

> did like the question I was asked about how I'd design an elevator system and then asked how my solution would scale. That's not too far off from what I'd be doing, albeit in an abstract form.

That would piss me off though too. "So does Amazon write algorithms for its elevators?". Why not just simplify a problem they solve. "So you have key value storage and you want to make it distributed". Why even talk about elevators?

Actually, yes...they do. I work in a building with an elevator control system designed by an internal engineer.

More importantly, the elevator control system uses mathematical expertise that is normally used solving variants of the multi-commodity flow problem, used extensively in the Supply Chain.

> Actually, yes...they do. I work in a building with an elevator control system designed by an internal engineer.

Well then that makes a good interview question then!

Heh, that's pretty cool. How does it compare to the average elevator?

Sib has it about right. It is an NP complete problem, and I'm pretty sure the solution is using a Heuristic, but it does seem to be pretty suboptimal during lunch time. Then again, maybe it just seems worse because you wait longer as a tradeoff to having a faster ride to your floor.

When load is not particularly high, it works well (seemingly efficient for passengers and in energy consumption). When load gets high (think ~noon), it tends to perform pretty badly in terms of latency.

> are still concerned about big-O notation

Time complexity is important, and hiring someone who doesn't think so is dangerous. That said, it's important to ask about an algorithm you've already discussed. Asking "what's the big-O of <generic algorithm>" also tests for whether you remember the specifics of that algorithm, which isn't terribly useful.

There are jobs where algorithms matter. Where someone picking the wrong data structure or algorithm can completely fuck up the problem, or where the company's secret sauce is doing something much faster than their competitors are doing it.

Yea this is where I'm at with it. If I walk into an interview and the first question is what is the difference between an abstract class and an interface my interest in the interview is probably going to decrease exponentially. How about asking me about my design approach to a specific problem, what frameworks have I try when presented with a specific problem and my thoughts on it, or what I'm currently learning. I get why they do it but at a certain level it's almost a sign of disrespect to be asked to fizzbuzz if you can verify I've been working at reputable companies for the last x yrs. Let's have a more nuanced discussion about technology so that you can confirm that I didn't just cram certification questions over the weekend and I can get a gauge on the type of problems I may have to tackle on the job.

You assume people working at programming jobs for years on end have a clue... that's not always true. I've worked with people who couldn't do fizzbuzz, or describe the difference between an interface and an abstract class, or any other basic programming stuff.

If they throw you an easy pitch, knock it out of the park and they'll ask you something more difficult afterward.

This is precisely right: I ask people questions like the difference between an interface and an abstract class as a gauge, because I've had so many people answer them really badly.

The question wouldn't normally come out of the blue, though, but as a follow-up from part of a design exercise where they would have just used one or the other in their design, and as part of finding out whether they could justify their choice.

This is something to think about.

The interview process is so badly gamed that these days its kind of a ceremony. In a recent interview, I gave a candidate a practical problem. He was allowed to use the sed man page, and then I gave him a file and search/replace problem. Nothing much! The candidate fails! Instead he asks me if there is going to be a algorithm/data structure interview. I told him, I don't know but as far as I was concerned I was only going to test him on practical everyday problems.

It was almost like I had committed a sin. He was like, every company has those standard one's so why was I doing like a practical test. These days all candidates do is go through most common puzzles/algorithms/data structure stuff. Its actually quite easy to get access to those. Just spend some time on interview forums and you can practically game any interview, in any company any where around the world today.

My approach is straight simple. Throw a manual/documentation at the person. Give him a problem common enough to test a practical everyday situation. Check if he can pass the the test of reading the documentation and figuring out the solution to the problem. Or give him/her their favorite IDE/text editor + documentation + problem and let them solve it.

If they can do it, or even get the approach right they pass the test.

Else if all they are going to do is vomit some mathematical theory from college text books, which isn't even remotely relevant to our everyday work- I'm hardly interested in hiring such candidates.

May I ask where you work? I'd love to interview at a company like yours.

I'm in the process of changing careers to programming in my late 30s. I dabbled in coding on and off for most of my life, but two years ago I started taking it seriously, read K&R, etc, and completely fell in love with programming. And so I start thinking about working full-time as a developer, and schedule an interview for a programming position for a company in the industry where I've worked for the past decade. I've always been a confident interviewer, but I totally froze in this phone interview. They started the interview off with a tough algorithm problem that I probably could have solved on my own in 30-40 minutes, but couldn't possibly solve on the spot in a conference call, with several people listening in and quizzing me. From there, things went downhill quickly, and I struggled to answer questions that I could have easily answered on my own at my desk. Since then, I've been burned on two freelance programming assignments (that I should have known better than to take given the level of desperation involved), which was quite a blow after spending a decade freelancing for dozens of clients without incident in another industry.

I love programming, and so I continue to code whenever I get the chance (and have two rather large projects underway), but these experiences have shaken my confidence. If most programming jobs involve similar interview processes, I'm not sure this would be the best move for me.

However, this comment has given me hope. If there are companies out there like this, then I stand a fighting chance -- I would excel at an interview where I'm given a problem, a time limit, vim, and then told to go solve it with no whiteboard or group of people watching. Let it be a hard problem, but make it realistic -- not some obscure puzzle or algorithm -- and let me use my own tools. I wish there were some job board where companies with this type of interview process posted jobs.


I'll even spend the extra time on it if we can do something similar to "Here is the problem and the docs. Email me back within the hour with some working code and we'll get on a call to talk through your thought process".

It's like trying to run a competitive 100m without shoes at some interviews I've been through lately.

I do something like that. My first test is ask to reverse a string (not using the reverse function!) in any language and/or pseudocode. Is incredible how badly the test goes with the people! I interview for myself or others more than 100 people (of any background, including university, tech schols...) and I think only 10/12 people do it correctly - barely-

To make it more fair, I always let them alone. And give internet and a computer, and remember them they can use internet and search. Not even with that amount of help the thing work well (I truly have find someone smart enough to copy-paste something and return to me in 1 minute or less).

Then i do the puzzle question, but know I retire it some years ago and some more question about real stuff...

This is great. My first thought was "perfect, Haskell!"

reverse "whatever"

"No reverse function? but we can use the internet? The Prelude source probably has the best implementation of a reverse."

foldl (flip (:)) [] "whatever"

Reversing a string is actually an incredibly common interview question. It's a bit like FizzBuzz (IME even more common, but I can't really speak about the world at large).

It might be an incredibly common interview question, but it's not an incredibly common day-to-day task, particularly when there are methods built in to classes that perform this function. An interesting task? Perhaps. But not reflection of the dev's skills, per se.

You're not asking about syntax with this. It's a problem solving question and at the end of the day, the only real metric here is not whether a dev could code this up, but whether he can code it up quickly, under time constraints, while being stared at, under pressure, with rent or a mortgage on the line, and maybe kids to feed.

I am hard pressed to think of even one real life dev task I had in my entire career where I had to solve a problem under those circumstances. For most devs, we take ownership of the problem and solve it at our desk under reasonable time frames without the spotlight. For bigger problems, we might think about it in the car on the way home. We might sleep on it over the weekend.

That doesn't apply to reversing a string, of course. But you have to consider that, depending on the dev's background and skills, he may be spending time in jQuery, maybe Spring and Hibernate (for Java guys), maybe even some CSS. Maybe he's a GWT guy? Maybe he was assigned to a team dedicated to (God forbid) EJBs. In other words, he may be multifaceted and spends more time working with an API than "just" raw language coding. The language is just a means for working with the API.

And the one thing you want to measure is not something he does daily. Now, if he can't use a for loop, feel free to mock him. But reversing a string? That's clearly a puzzle.

EDIT: I can't get this part out of my mind: "Is incredible how badly the test goes with the people! I interview for myself or others more than 100 people (of any background, including university, tech schols...) and I think only 10/12 people do it correctly - barely-"

So you continue to use that question? Despite the results? So that when you do hire a guy that passes, he won't ever do this on the job because you know you damn well he should use the reverse() method? Because you won't pay him to code it raw and waste time and money when there is a 1.5 second solution to this? That's sadistic. :-)

> So you continue to use that question?

Well, yes. Is the most simple one I can think that show if a person can declare variables, do a loop and show the result. If exist something better, I wanna know!

Before that question, I do things more complicated. Like do some CRUD, or some kind of task that was realted to the company I was before. But my first results with that lead me to simplify the interview a lot (in time and questions) because I see that if a person can't do this kind of task is pointless try with something better.

And I don't do much presure. I let them alone and give them all the time they want, with internet acces and that. I continue doing my work while they complete the task.

Is important to understand that in my past companies I was tasked to interview fresh graduated coders, and for a lot of backgrounds. And they pay badly. So no much oportunity of get "top players" back them. However, I interview several people with (in theory) experience or from reputable universities and that not change at all the end results.

P.D: Some point to the complications of do this task properly (because unicode). I don't even expect that kind of depth in the answer, looking more in the basic skill, code formatting, and the ability to explain the code writed...

I don't see how reversing a string is a "puzzle" - I'd put its difficulty on par (or maybe slightly harder than) fizzbuzz. In most mainstream languages, it requires 2 pieces of knowledge:

1) That a string is implemented as an array of characters.

2) How to reverse an array.

The implementation is straightforward and should be trivial for just about anyone who's done much development.

I don't know that the question is that great, but it certainly isn't bad as a fizzbuzz-type question to use as a filter.

Fair enough.

Here's my solution:

def reverse() {

    def a = "Can a fella get a job?"
    StringBuilder b = new StringBuilder()

    for (int i = a.length(); i > 0; i--) {
    print b.toString()

But it doesn't use arrays because quite frankly, I haven't used an array in years. I despise the noise of the brackets. :-)

Seriously, I use ArrayLists instead of arrays. But even on this FizzBuzz lite exercise, I got tripped up. My "i > 0" was off by one. And I started down the path of a.substring, but that was a bad move. I quickly switched to charAt.

But about the only relevant piece of this: on an IDE, I was able to place a breakpoint, see where I'm at, what my variables look like, and make adjustments.

That's real life. My original cut of this would have disqualified me. Actually, even the cut above would disqualify me since I didn't use arrays.

But I need to confess. What you see above is literally the first time I've ever reversed a string in this manner. It's just not something I do. And it didn't roll off the tongue so to speak. I knew I had to process the string from the end and work backwards, but it still felt like trivia and not a substantive inquisition into my skills or experience.

Trivial? A bit. It took me about 5 min. The off by one was killing me. It hits you in two places: the i > 0 and the charAt(i-1).

That isn't a bad solution. You obviously hope that someone you're interviewing can come up with something like that fairly quickly, but I certainly wouldn't disqualify you based on your method of solving it or an initial off-by-one error.

> What you see above is literally the first time I've ever reversed a string in this manner. It's just not something I do.

There is a certain amount of value in that - seeing how someone reacts to and solves something they've never seen before. Can you come up with a reasonable plan for attacking the problem, implement it, test it to see if it works in a bunch of cases, and make the necessary adjustments?

To be fair, the coding you do in an interview is a bit artificial - the problems may not be typical and you are without the environment you are usually coding in. The good news is that it is a skill that you can work on and become better at, and it is usually worth doing so if you are pursuing a job and you know these kinds of things will come up. I don't think it is wasted effort either - I've been trying to work on my algorithm/data structure skills by solving interview-type problems, and I've noticed that the work I've been doing to learn to solve them better has been beneficial in my daily programming as well.

Your solution is fine, you've passed, flying colours. I might poke a bit around the edges to see if I can get you to say the word "array", but mostly out of interest.

The important bit is that this is not an assessment of your ability as a programmer, it's a negative filter: Are you one of these people who simply can't program, because if you can't, let's not waste any more of each others time.

Arrays. (Uhhg...I said it but I feel dirty.) ;-~

I've never liked arrays due to the brackets. But in a twist of irony, I now spend all day in Groovy and I use maps, thus brackets all the time. Yet I don't even notice.

So after carefully thinking about it, I now realize its not the brackets. I can't explain it. There's something about arrays I just despise despite having no similar ill will towards ArrayLists. They just feel so 70s.

Personally I much prefer a[i - 1] syntax over a.charAt(i - 1). The extra "charAt" seems noisy to me. But that's probably just my C background showing.

And at 1), you went wrong, in any language where strings are in unicode. Combining characters, byte order markers, surrogate pairs... And this is why I dislike the 'reversing a string' question. Because if you actually know anything about real world string handling, you know that correctly reversing a string is impossible, because there just isn't any logical way to reverse some constructs. How do you reverse a string which contains bidirectional control characters? What is the reverse of an optional line-breaking hyphen? Even simple things like parentheses might trip you up.

Once you really start thinking about reversing a string, you realise why it's so difficult to do robustly: because there is actually no logical reason to reverse a string.

As an interview question, I would only credit an interviewee for this question if they basically objected to the requirement in the first place, and explained how bad an idea it was - but since I wouldn't expect most interviewees to do that it would be a poor test to introduce in the interview.

That is a really good point and it complicates a lot of string-based questions, which is why they are almost always constrained (either implicitly or explicitly) by assuming that the string is ASCII only, which seems reasonable to me.

There are probably better filter questions to use that don't require as many artificial constraints.

Of course, in ASCII you can just reverse the bytes. Oh, until you come across a CRLF... Or a backspace... and it still makes no sense to even do it.

Reversing a string was initially a good interview question because in C you can keep probing multiple levels of knowledge with it. You learn if the interviewee knows the algorithm, then if they understand how to do it in place with a pointer, then if they understand the XOR pointer trick. It's not a very interesting problem in a language like Ruby or Python.

For that kind of language, what simple question do you suggest? My objective is test if the candidate have basic coding skills...

Dood....I checked your HN profile which led me to your blog. You're talking about decompiling Java code and bytecode diffing. That's hot. See, now that is dev stuff. That's hardcore. Love it.

Now, that was a puzzle :) It was pretty fun to diagnose, even if the knowledge isn't incredibly useful and probably a lot less impressive than it might look at fist glance. Figuring out how stuff works (and why it doesn't work like you think it should) is pretty fun - I wish I had more opportunities to do it.

Damn right. Even worse than whiteboarding random algorithms are logic puzzles.

I can sort of see how implementing my own hash table fits into a dev job, but I why manhole covers are round, or light bulbs in a room has absolutely nothing to do with anything relevant in our field.

People who use logic puzzles as proxies for anything in an interview have my everlasting scorn.

My all-time favorite interview was one when I was interviewing for an SDET intern position at Microsoft. That interview (one in a day of four interviews) featured a great progression: logic puzzle, code the fixed input solution, code the n input solution, test your solution.

The puzzle was (apologies to the interviewer, but it IS a common puzzle) the eight ball and balance problem, where one ball is heavier than the other seven and you need to use a balance scale to find the lightest of the balls in the minimum number of weighs. I'd heard of the puzzle but never worked it out. I initially jumped to the 3 weigh solution, but the interviewer challenged me to keep looking, so I kept thinking and I found the 2 weigh solution. He then asked me to write code that could solve the 8 ball problem. Pretty straightforward. Once I finished that, he had me adopt it to be able to handle [edit: any number of] balls. This tested my ability to abstract my code. Finally, he asked how I would go about testing the program and we talked through that.

I cannot wait to steal that progression when I end up doing technical interviews someday.

Anyone who's studied information theory would destroy that puzzle though. In the same way that someone with a physics degree would perform way above average on a "count the stars" question. Or someone whose dissertation was on computational biology would excel at a string-matching algorithm question.

It is an example of the interviewer (or company/process) mistaking a specific problem as representative of all problems. Presumably they're looking for someone with "general problem solving skills", rather than "knowledge of astrophysics".

You can interpret the response more broadly, though. In this case, it was probably pretty apparent that the person hadn't worked through the problem before, so it is indicative of problem-solving and attitude. If he'd recited the answer from memory that usually comes across and then you have to pick something from a different area. If, as an interviewer, you literally can't find a problem that the interviewee hasn't already thought about, (i.e., he or she knows about computational biology, astrophysics, information theory, etc and none of this was obvious from their resume and previous positions) that can also be a good sign.

It's easy to game your answer as an interviewee though. HN actually had a post a while back about someone who "interviewed" for a living while on unemployment; by the time he was done, he had mastered pretending to think deeply about the answer to any interviewing question.

It's possible, but I doubt it's easy. I assume that the "mastery" of thinking deeply about answering interview questions was self-reported, right? I'd be curious to know how many of the interviewers would agree with him about it.

This is something that's pretty easy to test out in practice interviews with friends. My experience is that it's a lot easier to come up with questions that require thought and insight to answer than it is to answer them.

> Anyone who's studied information theory would destroy that puzzle though. In the same way that someone with a physics degree would perform way above average on a "count the stars" question. Or someone whose dissertation was on computational biology would excel at a string-matching algorithm question.

And...all of this is information the interview panel typically has available when evaluating candidates, as well as the result of any exercises done as part of the interview.

> It is an example of the interviewer (or company/process) mistaking a specific problem as representative of all problems. Presumably they're looking for someone with "general problem solving skills", rather than "knowledge of astrophysics".

If the process uses just one and not a set of exercises spread across problem domains, and gives it a fixed weight in the evaluation process and considers it independently of the candidates resume, the rest of the interview, etc., treating each component orthogonally rather than evaluating them together holistically, that might be true.

But that's not a problem with the puzzle as a tool, that's a problem with how the tool is used.

Anyone who's studied information theory would destroy that puzzle though.

I understand this comment in a general sense--the scale's behavior represents two possible states, so you can use it to filter the answer space in O(log_2 N) ala binary search, and you can get away with O[(log_2 N) - 1] if you partition the answer space the right way.

But what's the information theoretic interpretation of the problem?

It is a source coding problem. Kind of like a Huffman code. Depending on how theoretical or applied your information theory coursework was, you might have worked out simple codes like these in class, and that might be an advantage.

It's actually very similar to your tree reasoning above. The information theoretic outlook uses the probabilities of each of the N states to make a more efficient code, but since there are no probabilities in this case, the two approaches pretty much coincide.

Incidentally, there are three possibilities with each weighing -- scale tips right, scale tips left, scale stays balanced. So your logs should be base 3. In coding language, you are transmitting ternary valued symbols.

And, you should dispense with the O() notation, because the lead constant matters very much in this puzzle, and it happens to be 1.

Thanks! Haven't taken a formal information theory class yet, but I'd like to soon.

If you are mathematically inclined and good with probability, it could be a lot of fun.

Everyone from McKinsey to Accenture makes the same mistake.

I got to say, interview puzzles, while hated by the jobless, are lots of fun to think about. That sounds like a fun interview.

Re: hash table. I've been asking candidates to basically implement HashMap (for java positions) (without simply using Hashtable). It's an ongoing experiment. Although I've been told that this is a common question, I've actually never seen anyone do very well at it, and I have mixed feelings about it.

On the one hand, it gets at some important java concepts: hashCode() and equals() and the relationship between them. Depending on how they go about it, you can also get into issues of runtime complexity, memory/speed tradeoffs, etc... It's a deep question that offers a lot of room for exploration.

On the other hand, as mentioned, people usually do fairly poorly at it, and so my fear is that it's simply too difficult. I mean, I'd even be satisfied with solutions storing key/value pairs in a list -- inefficient, but workable -- but it's fairly rare that folks even get that close.

The point of any question isn't whether they get it right or wrong. The point is how they work through it.

I've been perfectly happy with people who didn't get the "right" answer to some question.

I would be surprised if many people knew hashing algorithms at all.

To clarify, you don't need to implement a hashing algorithm for this question. (Using hashCode() suffices). And you don't even have to implement a hash table. As I mentioned, a working linked list of key/values would probably be an above average answer, if it came with the understanding of the big-O performance characteristics.

My current job, they asked me one of those. I pretended to think about it for about 15 seconds, then asked for the answer (it had an answer).

The guy said "You want me to tell you?" I said "yeah."

I've been here a year.

the ability to just go with the already existing answer rather than constantly working everything out from scratch is quite an underrated skill IMO.


If the motive is laziness or the objective is to trick or intimidate a candidate, then I agree that the method deserves scorn. But I disagree with your position the technique is never reasonable. I like to hire people who know more than I do, and in this circumstance puzzles occasionally help learn about a person.

As a candidate I'm not particularly good at solving puzzles in this kind of situation, but I'm not bothered when people present them because the discussion and interaction about the puzzle matters more to me than getting the answer.

As an interviewer it is important to ask questions that a candidate cannot answer to see how they react. Since I'm a slow thinker myself, I must sometimes resort to "unfair" questions. I don't expect an answer, I expect a reaction.

After establishing a friendly rapport, finding common technical background, presenting an "unfair" puzzle, and discussing the puzzle a bit, then a discussion like this is golden:

Candidate: "Do you believe my ability to solve this puzzle indicates how well I could do this job?" Me: "No. Though I often enjoy puzzles, I'm not particularly quick with them myself." Candidate: "Then why ask a question like this?" Me: "Because I want to see how you react when you don't know an answer right away. In your case, I had to resort to tricks."

> As an interviewer it is important to ask questions that a candidate cannot answer to see how they react.

In the context of an interview, that would just make me horribly frustrated--because my default response to not knowing the answer to a question, nor even being able to perceive a method to resolving my ignorance, is to go out and research the problem.

Whether on the Internet, or by asking my coworkers, or finding an expert, or just running actual experiments and seeing a pattern in the large pile of data generated, this is usually "the" way to get a problem solved, and what I spend nearly 80% of each working day doing.

I don't rely on "sudden gusts of inspiration blowing through me" to solve problems, because the time that could take to happen is unbounded, and it can't be made to scale, unlike research, which can be delegated as you please.

But obviously, research is a tool entirely taken away from you in an interview. The only thing you can do to find out more about the problem is to ask the interviewer themselves--at which point you're basically playing a game of 20 questions, where instead of getting all the data you'd realistically expect in response to each query, you get dealt out precise quantities of informational entropy to string you along.

This contrived situation is not one that applies to the average day in the workplace, and being made annoyed by it is no useful measure of how one reacts to real non-obvious problems.

Answering "In the context of an interview, [I get] frustrated because my default response to not knowing the answer to a question ... is to go out and research the problem [and I cannot do that in an interview]" is a nice response to not knowing an answer and would get bonus points for honesty and self-awareness.

The best candidates will recognize and comment on the ways in which frustration can be either constructive or destructive.

"This contrived situation is not one that applies to the average day in the workplace," is a serious over-generalization. Enormous amounts of code are written based on contrived beliefs about the way business does or should operate. The world would be much better off if programmers were better at recognizing and constructively commenting on these artificialities.

I personally think that "I would go out and research the problem" is the best answer. Introspection about response to feeling sounds like a huge waste of time and something best suited for people's psychiatrist.

I think that question about why manhole covers are round is more of a urban legend than anything else. Or it might be a question asked to someone applying for a non-technical role, to see how they reason about the issue (though it is most definitely not the best question to achieve so).

I was asked that question during an interview for a technical role at Microsoft. Granted, it was 13 years ago at this point, but they did really ask stuff like that, and probably still do.

I haven't heard of those silly questions in the intern interview circuit. (2 at-school, 4 on campus)

My impression is that they are no longer in vogue - for a few years when I was in college (2004-ish) they were all over the place. Our industry is extremely faddish, even when it comes to interviewing...

You can still see them in some places I believe. I've heard anecdotally that Microsoft is still a fan of them.

When I was trained as an interviewer at Microsoft (circa 2006), they explicitly warned people against asking questions like that (and had been doing so for some time). There may be older employees who stubbornly persist, though.

I should add that my personal experience contrasts with yours - I did a lot of interviews around 2004 and got asked riddle-type questions by one, maybe two, people. My memory is that they mostly died with the original dot-com boom.

Entirely an informational note, but the manhole-covers question actually originated at Microsoft if I recall correctly. There are several references to it around the web, and at least one primary source confirmation by an interviewer that he used the question on multiple occasions [1].

That said, the fact that it was first introduced by a highly successful company doesn't lend it any merit in my book.

[1] http://blogs.msdn.com/b/bgroth/archive/2004/09/27/235071.asp...

Anecdotal, yes, but I have actually been asked the manhole cover question in an interview. Even if it started out as an urban legend, it does appear to be in use in the wild now.

The question never made any sense. Lots of manhole covers are rectangular. As long as the lip of the manhole is round and small enough, it still won't fall through.

Whenever I drill manholes, I always drill them square.

It is possible, but probably too expensive for you to buy.


Logic puzzle as the name implies are puzzles that can be solved using logic; typically deduction which neither of your given problems are.

It's important to make this distinction because skills like deduction are actually critical to developers, i.e. identifying the source of buggy behaviour.

Logic puzzles test how much the candidate prepared for logic puzzles. It implicitly tests how much the candidate "wants the job", but it also has the effect of turning away the stars who have options. As the best continue to have more and more options available to them where the team members understand that this kind of interviewing is counterproductive, companies that continue to use puzzles will end up with candidates that need them, instead of the candidates that they need.

I dunno. I enjoy solving puzzles.

Maybe I'm not a star :)

Logic puzzles are definitely useful ... how else can the interviewer feel superior (after all, he/she already knows the answer and has probably forgotten how they came by that answer).

If I was given a logic puzzle I would ask them "so how many times in the last year this company had to solve this particular puzzle here?".

And you would have gotten an instant fail on the "cultural fit" criteria.


Just as a logic puzzle might be a bad predictor of how you'll perform on the job, it's probably also a bad predictor of the quality of the company doing the asking.

I really don't understand why manhole covers or stars in the universe puzzles are considered "logic puzzles". Logic questions are not of the form "why?" and "how?". Their structure boils down to "is this statement true or not?"

It is a poor choice of word. I'd suggest the reason for calling them "logic" is that the process by which you explain what you're thinking and why to the interview gives them an insight into your ability to use logic. The puzzles themselves are categorically NOT logic puzzles.

Yes, you need a little logic. But for those puzzles it is (arguably more) important to know some key facts about the world which form a basis for your reasoning. I don't know, maybe reasoning based on wrong facts will satisfy some interviewers. Say, I can assume that half the stars in the universe are visible from earth and that there is only one galaxy and reason from that. The basis is clearly false, but I will be able to show off my "logic".

Interviewers ask puzzles and give you tricks because they want to see your problem solving process. Obviously, anyone who hires simply off a litmus test of such questions is not doing their job, but that doesn't preclude the use of such brain teasers/problems at all.

Asking a candidate a difficult brainteaser that may be even close to impossible can be extremely revealing. There isn't a canned response, you get to see them work under pressure, you experience live problem solving and you can understand how they move through their work better. All of these factors are just as important as competency when hiring.

A great programmer doesn't "fizzle" in front of a difficult interview question. He/she may or may not get the correct answer, but as long as they have the requisite problem solving skills and communicate with their interviewer well they will come out fine. I'd say that showing how you fail can be quite beneficial when trying to get a job, especially if you happen to fail gracefully :)

It seems like 37signals is just echoing platitudes to gain karma/publicity at the expense of cheapening the discussion.

People can be nervous on job interview and therefore "shut down" and fail simple tests. This does not reveal a persons real problem solving skills.

I don't consider myself especially nervous in job interviews and when taking tests and exams. But I still had a much harder time solving simple programming tasks than I would "back at my desk" with no-one watching. YMMW.

It would suck to want a job so bad you completely freeze up on simple interviewing tests :(

On the other hand, if I'm hiring someone for a high pressure job I don't want them to fail when under pressure.

I understand your point but I don't really buy it. I train lifeguards as a hobby/side-job (I'm in uni right now) and pressure is the number 1 reason they give us for failing their final pratical test. I can't give a kid a permit to work as a lifeguard if the pressure of an exam makes him screw up because the real life pressure of saving somebody's life is even greater.

The exact same applies here for all jobs where you expect the engineer to work in stressful situations.

"On the other hand, if I'm hiring someone for a high pressure job I don't want them to fail when under pressure."

But those are in no way the same kinds of pressure. The pressure of being able to meet a release date, design incredibly safety sensitive algorithms, etc, are completely different from the social pressure of having someone evaluate work you normally do by yourself, in real time.

This kind of interviewing would work well for the kind of stress a salesman gets, not the kind of stress a software engineer gets.

I completely agree and that's why that part of the interview shouldn't be the only one. As someone said earlier, it should be a negative filter of incredibly unsuitable candidates.

In a perfect setting the interviewer wouldn't evaluate the candidate's ability to solve the problem, but his/her ability to approach it, explore it, ask questions and take steps in the appropriate direction.

> On the other hand, if I'm hiring someone for a high pressure job I don't want them to fail when under pressure.

Programming is not like in those hacker movies, we don't have to solve problems in 60 seconds while getting a blow job[1].


Sadly that's true.

But in how many coding jobs is pressure a real and common occurrence?

Even working in front office banking roles, there's only 1 or 2 times per year where you really have to keep your nerve and perform under genuine pressure on timescales of seconds or minutes.

The kind of pressure where 'we need to get this feature out of the door by the end of the week' is a completely different thing altogether.

Dealing with outages is one time where I have to think on my feet in a tense and urgent situation, and occasionally write code.

I care about three things in a candidate:

Code Quality



... pretty much in that order. Brain teasers test none of those things (maybe a some communication)

I've never met a coder that couldn't eventually figure out a way to hack the code to do what the PMs want. But I have met a lot of coders that take way too long and/or produce horrific code that is a buggy maintenance nightmare. Those people often do ok on brain teasers (or at least do well enough that the interviewer doesn't feel like he can give negative feedback).

A more important point is that brain teasers are a waste of time compared to a million other things you could be doing with the candidate to determine their aptitude, like actually writing code to solve a problem.

"Asking a candidate a difficult brainteaser that may be even close to impossible can be extremely revealing."

I prefer problems that are ridiculously simple on the outside; however, allow for lots of optimization. Choose the next best move in chess or go for example (or design a simple sudoku solver). A completely simplistic solver just chooses a valid move. It becomes more interesting as one looks at how are you modeling state, how are you deciding what is best, etc. This way most people will get something that works, you can see their thought processes (where are they optimizing first, why are they doing that), and maybe learn something new.

"Choose the next best move in chess or go for example (or design a simple sudoku solver). "

You make the assumption that your candidate knows anything about those games. If you placed me in front of a chess board, said "these are what the pieces do" and then asked me to tell you the next best move, that move will either be A) blindingly obvious to anyone who has played chess for years, or B) too difficult to someone who has never played the game before.

Reapply the same paragraph to Go and Sudoku. You've created a filter for someone who plays games, not problem solving skills.

No...I don't make the assumption they know anything about those games. I start off talking to them about some of the above games.. One could just as simply walk through other games (checkers, tick-tack-toe, hangman, etc.). Or...if they've never played games, then you could walk through other somewhat classic problems (how should one load an airplane, the back of a car when travelling).

My point was largely that there are a variety of problems out there which have "no right answer" yet are fairly intuitive to get started on...and figuring out modeling of the problem space in combination with decision logic is an excellent example of problem solving which may help one see how a candidate attacks problems, optimizes, etc.

That's why the interviewer should ask if they are familiar with those games first.

Even Connect Four or tic-tac-toe can be a useful exercise to go over with someone.

Sudoku's kind of a fun one, because brute force works just fine. More elegant solutions use the same strategies humans use, but recursive brute force gets the job done just fine.

All good, but the success of this method also depends on how well the interviewer interprets/evaluates the interview. Unfortunately, the danger here is that inexperienced/immature interviewers can easily discard good candidates this way. This is similar to the skill required to grade a written test for partially correct answers vs. right/wrong ones. Hence the criticism on wide-spread usage.

I have heard of interviewers asking candidates to solve various levels on mobile brainteaser apps. Something like this game https://itunes.apple.com/us/app/smart-kick-pro/id590049769?m... could work well as it involves problem solving skills and is a great fun little ice breaker.

A great programmer doesn't "fizzle" in front of a difficult interview question

How do you know this?

A great employee is more than a great programmer.

> Interviewers ask puzzles and give you tricks because they want to see your problem solving process.

It depends on the "problem solving process" you are trying to get insight into.

These procedural "how does the candidate get to the answer?" questions are great for procedural thinkers. Even if they don't know details that feed into reaching an answer, they can show you how they'd get there.

Unfortunately, if your staff ends up entirely composed of procedural thinkers, you may find your company challenged at innovation. For that, you need candidates good at "Eureka" moments.

Your Eureka candidates will often do quite badly in the procedural "what's your thought process?" questions in the one hour interview context. Measured over a 24 hour period, they may do notably better.

"The problem solver initially has a low probability for success because they use inappropriate knowledge as they set unnecessary constraints on the problem. Once the person relaxes his or her constraints, they can bring previously unavailable knowledge into working memory to solve the problem." -- http://en.wikipedia.org/wiki/Eureka_effect#How_people_solve_...

What you're looking for here is both broad and deep domain expertise, coupled with an ability to understand the question well then let the subconscious mind pull together the insights needed for a breakthrough solution. This process cannot be talked through out loud. The answer may take longer (hours, a day or two, probably reached in the proverbial shower), but will often be an answer that gives you a long term competitive edge over a blunt procedural approach.

This said, it's not clear to me how to interview for this ability. I'd start by considering a way to understand the depth and breadth of a candidate's exposure to technology and domain knowledge. This is the opposite of interview processes using dives into the interviewer's area of expertise. I might consider a two part interview a few days apart, where the candidate gets the context in part one, and shares their thinking in part two.

> Asking a candidate a difficult brainteaser that may be even close to impossible can be extremely revealing. There isn't a canned response, you get to see them work under pressure, you experience live problem solving and you can understand how they move through their work better. All of these factors are just as important as competency when hiring.

Except that this situation is an artificial construct. You're covering their mouth and nose and saying, "Now show me how you breathe." You are favoring candidates whose problem solving is informed by the belief they can reach an emergency answer on their own through brute force, rather than those whose problem solving starts with the assumption that someone else has already solved or mostly solved the problem and who is great at finding the "shoulders of giants" to stand on, building an informed answer a priori.

I'm reminded of getting my deep water scuba certification. You're not interviewed for a diving salvage job by repeatedly seeing how you handle having your breathing apparatus turned off. Yes, a diver should be able to "perform under pressure", but what you really want are divers who help ensure nobody on the team ends up under pressure.

I'm a self-taught programmer. These stories about programming puzzles in interviews always make me nervous to look for a job.

My first job wasn't a programming job, but I did a lot of programming because the need arose and I was the only one capable of doing it.

I always felt too inexperienced to look for a real programming job. A friend pushed me to interview for the position I'm in right now (full time web developer). They didn't make me do any whiteboarding or puzzles. What I did do is give them a code sample and we discussed how what I could have done better and why.

I got the job and I'm doing quite well, despite my lack of a CS background.

Since I don't have a formal CS background, and most of my development experience is web development, I have basically no experience dealing with complex algorithms, but I've found I can solve a hard problem with enough effort (Google, co-workers, etc).

For web developers, wouldn't more appropriate questions relate to "please design a database for a website that needs to track the following", or "What would your process be for designing and implementing a REST API that would...". These would allow the developer to show you their thought process of working through a large problem without there being a necessarily right or wrong answer.

Exactly. I never ask those questions. I ask practical problems that we solve every day here. Some are simplified of course.

Also I don't try to be confrontational, I take the "let's solve this together". That often puts them at ease and makes for a better experience and a more productive interview.

Now it is still an interview and the selection process could be brutal but this way I get the most _relevant_ knowledge and information about the applicant in the shortest amount of time.

The "lets solve this together" approach is always great - I've had people simply give up and not ask any further questions - those are generally "pass" situations. I rather see someone be genuinely interested in how to solve the question - the more questions they ask, the more I will be willing to help them, and the better they will actually do in the interview.

The problem with this though is the meta-strategy of interviewing. Unless your interviewer tells you up front whether or not it's okay for you to ask an unlimited number of questions, you're basically guessing at it.

I know several companies that interview with a "hint" system, where every time you ask a question, they just give you a hint to the problem, generally regardless of the question asked, and then mark it down. Too many hints and you're passed over.

Interesting - while I never said it was OK to ask unlimited questions I have generally gauged how someone is doing and then let them know it's OK to ask questions or need hints. Some people have accepted the offer, and some people have simply given up - which just astonishes me.

How many questions they ask, how often and how relevant those questions are all taken into account when grading communication skills.

Some are just way to shy and never ask question. That's a slight negative. I am shy too, but I need to get the job done I'll go find out who to talk to, ask question, figure out specs, see who's done what before and so on. That is an important trait.

I'll give out hints as well, it is not too structured. I don't immediately subtract some point or anything. In fact I might hold it against them if they don't ask questions.

Sigh. This is the issue that will just never die.

Let's just summarize the points:

- Not everyone can produce real-world code. Most of what we do for a living belongs to our employer. Side projects, particularly in the US, can be an issue for IP reasons (California is somewhat of an exception here);

- Side projects, even open source projects, can be of questionable "real world" value;

- I've personally encountered many "programmers" who can talk a good game but can't code a for loop;

- tests like the FizzBuzz test [1] are, in my experience, remarkably effective as an early negative filter. This is an important point. If someone blows you away at FizzBuzz, it doesn't mean they're an awesome engineer. But if they can't do it, it almost certainly means they aren't. The idea here is to spend the most time with candidates who might potentially work out and wasting as little time as possible on those that probably won't;

- the problem with these kinds of whiteboard coding problems is that the tendency is for interviewers to think the problem needs to be "hard". It doesn't. In making it too hard (IMHO) you risk destroying the value of your filter;

- pop-quizzes of obscure language features, the kind that might appear in certification exams, are a waste of time. I have no argument with that;

- whiteboarding code by itself is not a great filter. It should be used in conjunction with a multi-faceted interviewing approach that involves testing fundamentals, the ability to construct a relatively simple algorithm, the issues of working on a team and on a production code base and systems design.

- the problem with simply talking about "real world" code, as the author suggests, is you're no longer finding a good engineer, you're finding someone you like, someone who thinks like you. This falls under the umbrella of cultural fit, which is of course important, but don't mistake that for engineering skill.

- I think we can all agree that "logic" puzzles like "how would you move Mount Fuji?" or "if you shrunk to 1cm in size and dropped in a blender, what would you do?" are stupid.

- testing "back of the envelope" estimation however can be useful. I mean things like "how much storage is required to store satellite images for Google Maps?" The idea isn't to get an accurate answer. It's to see what assumptions the candidate states and, based ont hose assumptions, to come up with a reasonable ballpark number.

The problem here is that there are many engineers who can't comprehend the possibility that there is someone being paid to be a programmer who can't code. But I assure you this is the case. It's shocking but true. Simple coding tests largely filter these people out so if you're offended by such simple tests, just do it and move on. I assure you there's a reason why they exist.

One final prediction: There's some guy here on HN who always posts the exact same huge comment on any hiring thread. I'm sure it'll pop up any moment now.

EDIT: let me add a point about trial periods and take home assignments.

Both of these are guaranteed recipes for mediocrity. Truly outstanding candidates need to justify the time investment for either option and very few companies have the kind of gravitas that would justify it.

Anything written without supervision will be of questionable provenance at best.

As for whether or not someone will work out in your organization, bringing them in for a day is (IMHO) of questionable value. Many engineers are introverts. I include myself in this. It's incredibly awkward as is to be in a new company or even a new team in the same company. I question the value of any such assessment over what you learn in 1-4 hours of interviewing.

[1]: http://www.codinghorror.com/blog/2007/02/why-cant-programmer...

One thing you're testing for is how well their brain works with an excess of adrenaline floating around in their body. All of the best interviewers I've known take great pains to make the interviewer feel comfortable and chatty - after all, they aren't going to be filled with interview anxiety when they're actually working there. A minority of your applicants are going to perform badly in that situation because the adrenal glands will effectively reduce the output prefrontal cortex below an acceptable level to solve a problem that they'd be able to solve handily in a situation that didn't involve so much performance anxiety with strangers.

I doubt that constitutes a majority of the people you pass on, but it's definitely going to be double digits.

I've never hired using a white board coding test, and out of 30 or 40 folks i've had to let 2 go pretty quickly because it was clear they couldn't do the work, but the rest did as well as somebody who would have passed one of your quizes - I'd expect we'd see similar long term failure rates in both sets.

I'm sure it makes sense at google where the hiring procedures don't leave much room for error, but I'm sure I've been able to hire some great folks that you would have missed. Arguably the best heads down brilliant engineer I've hired (well tied for #1) completely melted down on his first interview. Tried again with drinks and a show at Yoshi's and he did so well I would have hired him to be my boss. He easily pulled 3x his own weight for us and was very loyal and easy to work with.

So many people go on and on about how hard it is to hire yet they go around systematically rejecting all the same candidates for all the same reasons.

"A minority of your applicants are going to perform badly in that situation because the adrenal glands will effectively reduce the output prefrontal cortex below an acceptable level"

I have an off topic question. I don't see what the problem being discussed has to do with how glands reduce activity in certain parts of the brain. Sounds to me like a fancy way of saying 'people don't perform well under stress'.

You could of course ask - "what mechanisms control human behaviour, and how does stress specifically affect body systems?" Then, great, let's hear all about adrenal glands and prefrontal cortexes. When that's clearly not what the question was though, I find it a form of poor communication.

Anyone have any thoughts on this?

I've played musical instruments and sung in front of hundreds of people, it's stressful but I'm confident I could code under the same circumstances.

I have panic attacks if I feel too enclosed, particularly big crowds in small spaces. I would definitely fail to code FizzBuzz if I was having a panic attack! But if I needed to, I could disguise it pretty well. You might not notice that I was having a panic attack.

Stress isn't linear, so you can't really extrapolate from it that well.

Also, I would question whether absolutely everybody on your team needs to be exposed to lots of stress. Sure, some people have to care about live systems crashing (as one example of a majorly stressful event), but not everybody, surely?

Personally, interviews don't stress me out and I can do whiteboard coding tests no problem. A big component of that is because I went to the right schools and they rigorously trained me on that from a young age. I don't think it's fair that it's easier for me to get a job than someone whose grandmother was less well off (she paid for my schooling).

Sorry that you feel I was obtuse - it's a quirk and my editor side doesn't always step up.

Poor communication aside, I was attempting to make a finer point than you read to it.

Stress is an extremely broad term medically, and used broader still colloquially. Medically defined stress may include anxiety, but frequently will not.

Anxiety, or technically in this case "performance anxiety" is a much more narrowly defined state with significantly more understanding about causes it, what it causes, and effective treatments.

It's know pretty commonly as "stage fright" - that's what you eliciting in some individuals when you put them in front of people they just met to prove that they are smart. I think we'd all agree that most programmer jobs shouldn't be withheld from someone who has trouble with public speaking.

I chose to use neuro chemical terms not to attempt to confuse anyone - but to make the point that this is a biological process triggered by hormones and not anything related to raw intelligence or a weak personality.

As an aside, it's extremely likely that they wouldn't have any trouble at all at the interview if they took a 80mg of propranolol hydrochloride (a beta blocker) an hour before the interview. It's probably used by 50% or more of public speakers and musicians playing big rooms. Unfortunately it's expensive, on patent and that's an off label use.

If you're not looking for someone to do regular public speaking then detecting what you consider "stress" probably has little bearing on their job applicability.

> Unfortunately it's expensive, on patent and that's an off label use.

In the UK you can still get it prescribed for anxiety, and get it free on the NHS, so if you're wondering about betas it's worth checking what the situation is wherever you live.

I would strongly recommend that you do seek out medical advice regarding them, there are several very important caveats and contraindications.

Propranolol's patent expired decades ago, it is dirt cheap, and 80 mg is a huge overdose for most people.

Other than that you are right. Propranolol is an excellent calming drug.

Thanks for correcting me - I was working off second hand memory

I think it's a little more specific than "people don't perform well under stress." A software development job may well entail stressful situations—for example, "The founder got interviewed on Oprah and the servers are melting!" But that kind of stress is different from a interview stress. There are definitely folks that will step up to the challenge of melting servers, but go to pieces under the social stress of an interview.

Now, maybe getting into the biological mechanism by which they go to pieces is too specific, and trotsky could have communicated more effectively by eliding it. But it's an important point, and I don't think it reduces to just "people don't perform well under stress."

"I don't see what the problem being discussed has to do with how glands reduce activity in certain parts of the brain. Sounds to me like a fancy way of saying 'people don't perform well under stress'."

That's how I read it as well. But this got me thinking, why is it a good idea to test someone under interview stress when what they'll be doing 99% of the time won't be in interview circumstances?

I find some stressors don't have any negative effect on my performance but stress in an interview makes it much worse.

Agree. I have enormous social anxiety that completely wipes out my thought process... But give me a project to do with a due date (ex a school project due in 2 days) and the stress from that actually motivates me to get it done and does not mash my brain up like the stress of being in a social situation like a interview.

Unfortunately, there's no good solution to this problem. I recently had a company ask me to work on a 2-day project for them (in the office), for which I performed sub-optimally.

There are some other variables which might be involved (I was working with a new language, database & framework), but I never really found a "flow". Reflecting upon it, I think the pressure to create a product + learn the tools in a given timeframe threw me off.

Work is social.

What do you when a couple teammates walk up to your desk out of nowhere, abruptly interrupt your coding zone, and ask you why the build broke when you changed the FoobarService to return X instead of Y when Z? They can't release the latest features until it's fixed! What you were doing isn't important anymore, you're holding up everything!

If you want to work for a web services company: what are you going to do when your service goes down, you're oncall, and angry users, coworkers or your manager start asking you what's wrong?

Are you gonna meltdown? If so, I think you should address that first. My company, which does do whiteboard interviews, can wait to hire you until you won't freak out, and we likely will.

By the way... real work actually gets worse than this. Sometimes the co-workers involved aren't gentle about things in the heat of the moment and they're gonna say you did something stupid, that your mistake was obvious. You need to address their concerns anyway. Or during an outage, your nervous manager might start checking in every 2 minutes - you need to calm him down.

You're just lumping two different things together.

Everyone has performance anxiety at times - whether rare, mild, common, or occasionally overwhelming it's something everyone deals with. Something like public speaking to a room of strangers or flirting with a super model really doesn't share many physiological triggers with diagnosing code you wrote while people you're friends with look on anxiously.

Assuming that because something's easy for you (and most people) that anyone who has trouble with it is likely to be broken in lots of ways isn't too safe a bet.

If someone gets overwhelmed by vertigo on a skyscraper does that mean you won't be able to trust them to drive in rush hour traffic? If a guy pisses his pants when he gets a gun pulled on him does that mean he can't be trusted to be an airline pilot?

Those are exaggerated to make the point, but interviewing is an art. You have great latitude in terms of making people comfortable or uncomfortable depending on your goals. With enough effort I'd be willing to bet you could engineer an interview that would send almost anyone over the edge.

So now that it's just a case of degrees, all you have to do is decide whether you're hiring someone that needs to be able to solve programming puzzles in front of a stranger that's actively critiquing them.

> solve programming puzzles

Your argument all comes down to dismissing the content of the interview. Since you clearly assume the interview is worthless, you aren't even discussing this in good faith, so I'm just ignoring this post.

It didn't at all. It carefully explained why the two activities produced entirely different chemical reactions in the brain. If you're too invested in your process to even consider that a test of one thing may not translate into the performance on a totally different task then perhaps you're taking this all a bit too personally.

> What do you when a couple teammates walk up to your desk out of nowhere, abruptly interrupt your coding zone, and ask you why the build broke when you changed the FoobarService to return X instead of Y when Z? They can't release the latest features until it's fixed! What you were doing isn't important anymore, you're holding up everything!

This should have been caught before it was an emergency, through standard processes such as integration and unit testing.

> ... you're oncall, and angry users, coworkers or your manager start asking you what's wrong?

If your coworkers and managers are angry during an outage, or asking anything other than "what can I do to help", they're not helping.

> Sometimes the co-workers involved aren't gentle about things in the heat of the moment ...

Those coworkers need to be instructed in proper workplace behavior.

The time to be critical about work is before an emergency occurs. Institute policies to ensure code quality and IT robustness, define roll-back plans to be enacted in case of failure, and supply other policies, procedures, and evaluation criteria necessary to steer your employees to ensure that the chance of success is maximized (and the chance of outage is minimized).

When (not if) the shit hits the fan, you -- and all your employees -- are prepared for it. If a nervous manager is still checking in every 2 minutes instead of asking how they can help, it's your job to MAKE THEM STOP.

This strategy is how you prevent most failures, and make the ones that happen manageable. Your strategy is how you create failures, and then make them worse for everyone involved when they happen.

> This should have been caught before it was an emergency, through standard processes such as integration and unit testing.

Things always go wrong in the development cycle. Problems happen. Conflicts between individuals occur. I was giving an example. Are you asserting these issues don't arise in the workplace? Or that my example is of a rare situation in software development?

> If your coworkers and managers are angry during an outage, or asking anything other than "what can I do to help", they're not helping.

People sometimes do unhelpful things. Sometimes people even make emotional decisions about what to say to others. Are you aware of the fallibility of man?

> Those coworkers need to be instructed in proper workplace behavior.

They likely will be. Have you met engineers? A lot of them could use some workplace etiquette training! Until then, you need to do your job - you can't just go home because it's stressing you out. This is something that happens in every single workplace. If you can't handle it, you aren't a very valuable employee.

> This strategy is how you prevent most failures, and make the ones that happen manageable. Your strategy is how you create failures, and then make them worse for everyone involved when they happen.

This, and the rambling before it, is completely off-topic. I have no idea why you're rambling on and on about software development practices when the issue I'm addressing is that social stress always exists at work. If you feel my examples are unsatisfactory, but don't have any issue with my core argument, then you aren't being helpful. If you have an issue with my core argument, please address it instead of nitpicking my examples.

Your examples are all cases where someone creates social stress by reacting poorly to avoidable, stressful situations.

Your goal seems to be to hire people that will put up with these avoidable stresses.

My goal is to build an mature workplace where we avoid creating unnecessary social stresses by applying collective wisdom and experience to plan ahead.

This is one of the many difference between engineering and hacking.

Back in my early 20's I applied for a job (which I was basically invited by the manager to apply for) and was extremely nervous in the interview. I got the job and turned out to be one of the most experienced in the department. They later asked me why I was so nervous in the interview, I said I thought I just really wanted the job.

I've come to enjoy technical interviews. How often do you get to spend a whole day talking one-on-one with people in your field who you can learn something from? I enjoy giving technical presentations to groups of 1000 people or so and been not nervous at all. But I have to keep water with me because some reflex makes my mouth get really dry. So I'm fine with public speaking (the most commonly cited phobia).

On the other hand, the prospect of sitting down to pay a bunch of paper bills or do taxes makes my body want to hyperventilate. If, in a place where there are only people around I know well, and I remember an event in my past that involved a social interaction in which I acted suboptimally while at the same time someone is crinkling a plastic bag, I will startle to the point of crying out "Aah!". It's true :-)

My only point here is that people are complex critters who change with time and environment. So attempts to think of others as a box of predictable glands and cortexes (cortices?) can easily turn into a pseudoscientific projection of one's own preconceptions. (I find it very helpful when thinking about myself, however.)

Has anyone ever tried asking candidates to design their own interviews?

My thought is that it's a real phenomenon that occurs in a certain subset of people during a job interview, therefore, its obviously relevant

Lots of real phenomena occur during an interview. For example, you could go one level deeper and describe the molecular mechanisms between adrenaline and whatever 'receives' adrenaline. That would sound something like:

"What happens in job interview is that adrenaline binds to a variety of adrenergic receptors, which triggers a number of metabolic changes" [1]

It'd be just as real, also clearly connected, but also quite clearly not what the conversation is about.

[1] http://en.wikipedia.org/wiki/Epinephrine

I feel the problem is that, few of us actually being neuro-scientists; While saying something like "people don't perform well under stress" is something that is close enough to common knowledge, instead saying "adrenal glands will effectively reduce the output prefrontal cortex" leaves the reader with a whole bunch of unanswered questions:

* Does a stressful interview necessarily actually increase adrenaline? * Is there actually a link between incresed adrenaline and reduced prefrontal cortex output? * What role does the prefrontal cortex play in answering interview questions?

And while these may all have good answers, it just throws up a whole bunch of mental '[citation needed]' annotations all over the whole argument, that can just be skipped entirely if we aren't talking about neuro-science. And if we insist on talking about the bio-chemestry, we should probably go into the details.

Add major jet lag to that and you'd have one of my previous interviews.

>One thing you're testing for is how well their brain works with an excess of adrenaline floating around in their body.

Unless you are specifically hiring people for a death march kind of project (which I find unwise anyway), targeting people good at high-stress situations doesn't seem very productive.

If your programming projects are a series of high-stress, high-adrenaline runs, either you work for NASA or you're doing it wrong.

As a company you should always advertise if you want your employees to work under a lot of stress. I would for one never choose to work for such groups.

I'm not sure if I'm that guy or not... but anyhow, here's the hiring process that's given me some good results.

Note that this requires that your team does pairing.

Interviewing answers three questions: Can they do the job? Do they get along with the team? And, can they adapt when the game changes?

Before your phone interview, let them know that you want to do a short screen-sharing and pair-programming exercise via Skype, in their own personal development environment.

Start with a chat over Skype. Talk to them a bit about their preferences and experiences at other companies. Are they comfortable to talk to? Do they have opinions about the stuff they've worked with? This is important -- A Java developer that doesn't have a vocal preference on checked exceptions or static typing probably isn't very experienced.

Then give 'em the FizzBuzz.

At this point, do you want to hire them?

If so, it's time to invite them into the office.

When they come in, at the very least you should buy them lunch and pay for transportation. They're taking a day, not paid, to come and interview with you -- show some courtesy.

Have them sit down and pair with three of your more senior engineers for a few hours each, both as a pilot and a copilot.

At the end of the day, take your group of three, and see how your interviewee did:

1. Did the candidate demonstrate the expected level of knowledge and contribution?

2. Would they feel comfortable with the candidate as their direct boss?

3. Would they be upset if this candidate went to work for the competition?

If somebody can get through the phone screen and a day of pairing, they're probably competent, and likewise want to continue working with the people they paired with.

I don't know if this is a Silicon Valley thing or what, but the idea that someone would basically do a day's unpaid work on projects they know nothing about with three different people is just about the most foreign thing I can think of.

That being said, I've never done any pairing so maybe it would work better in that type of environment.

A job interview is going to take at least a day of your time anyway. What does it matter if that time is spent programming, talking over coffee, sketching on a whiteboard, or whatever?

Pretend it's not tech, and Starbucks interview consisted of you working as a barista for a day, or Goldman Sachs wanted you to trade stocks for a day, etc...

I will tell you, back when I bartended in New York (10 years ago), it was pretty common to spend a day or two being "tested" as you made drinks for customers, until they decided you were worth hiring. You wouldn't even keep tips, since they'd test you out just on drinks made for tables, or whatnot.

Yes, but during a trial period don't you get paid?

Nope. No wage, no tips. In my case it lasted two nights only, and was paid the third night (and making the big bucks on weekend nights within a couple of months), but I'd heard of it being as long as a week at other places.

The worst part was, you wouldn't even know how long the trial period was -- they'd just ask you, last minute, if you could come in again for another trial. If it's a place you wanted to work at eventually, you'd say yes.

>"Nope. No wage, no tips."

That sounds illegal, at least here in Canada.

Sounds illegal in most jurisdictions that I'm familiar with

The food service industry in the US has never had much interest in labor laws I guess

It's illegal in the US as well.

You still get tips, which generally outweigh your hourly rate anyway.

Starbucks isn't going to bother to fly a potential barista to their location, have several managers assess their ability to perform as a barista, and buy them lunch. As someone looking for a job in the tech industry right now, if a company was willing to pay my transportation costs to interview with them and buy me lunch, I would consider that a fair deal. Even if I didn't get the job, I would be out a day or two of my time (during which I'd be able to scope out the city they were located in anyway) and they'd be out anywhere from hundreds to a few thousand dollars. If they got any value from my work while being assessed, I imagine they would want me on their team.

I'm not sure that I have a constructive way of replying to this. Can you spell out your objections, instead of trying to jostle my imagination?

You act like no one's been fired after their first day at Starbucks.

My point is that having someone work a day for free would be great in any industry, but somehow only we in tech tolerate this, why is it different?

Having a random applicant "work" a day for free is a pretty crappy proposition in most industries; I suspect tech isn't much different. You gave an example of asking a job applicant for goldman sachs to "trade stocks for a day." Are you kidding? Do you realize how much money a bad applicant could lose in a day? If it's different in tech, it's only because it's (often) easy and painless to roll back an applicant's mistakes.

There are a few ways to approach a job interview. The one I like is to say, "I'm a professional working in this area; those people are professionals working in this area; I should learn what they're up to and we should see if it makes sense for us to work together right now" and right now is key, because sometimes the timing or the particular projects aren't a good match, but the people are and you'll want to stay in touch in the future.

One nice non-coincidental advantage of treating it this way is that "success" is a lot easier to achieve, so the interview's going to be a lot less stressful.

So now approach it from that perspective. You could spend a day (or two hours, or two days, or whatever interview duration) chit chatting, talking to HR people about benefits, describing past projects and whatever. But time is really short and the important questions are: what are you actually going to be working on? and are the people you'll be working with any good? Are their experience, personality, skills, etc. going to set you up to do well, help your career, get stuff out the door, and whatever other concerns you have. How are you possibly going to learn any of that without talking to them at length and in detail about their projects? And the best way to "talk in detail" is to pretend you have the job for a day and get up to speed.

So are you now "working a day for free?" Not really. Because, if the people you're interviewing with are people you'd want to work with, they've had a lot longer than you to think about their problems and know a shitload about them. If you contribute anything of value that day, they probably would have gotten as much value from talking their sticking points over with any bright, engaged outsider to get a fresh perspective.

In the rare occasions where the applicant really does provide high value consulting for free for a day, he or she should have known better than to interview there in the first place. If there really was no way to know in advance, just accept that you had a dud of an interview and go on with your life. You've still learned whether that particular company is one you'd want to have a business relationship with in the future, which is valuable, and whether any of their employees are particularly good or bad, which is also valuable; and, if the company has the right values, there's a good chance that they'd want to hire you for contract work. So even those interviews still tend to work out.

tl;dr - when you interview for a job, no matter what, you're working. You'll get the most out of it as an interviewee if you take it seriously as work and try to get as close to the day-to-day job as you possibly can. If the company materially benefits as well, even better.

Exactly. An interview is work, on both sides of the table, and I see no point in distancing that work from what we actually expect the candidate to do once they get hired.

Puzzles, whiteboard coding, and so on don't tell me whether or not somebody can actually build software.

If the goal was to get free work, we wouldn't sacrifice staff time on the interviews. We'd give the applicant a challenging "homework" assignment and ask them to do it on their own time.

It is tech though. So why does pretending it's not have any bearing on the problem?

Well with a "regular" interview, it's pretty unlikely that the company is going to make any money off of the direct output of the interview. When helping them develop a feature or product, it's slightly different.


I'm in academia. I've had job interviews where I've literally gone office to office discussing people's research projects and helping them come up with ideas. It was fun, and it never occurred to me that I was being taken advantage of. Worst case scenario, they might think of me as a potential consultant down the line.

Many interviews I've had are 2 hour affairs, including jobs I've received where interviews were 45 minutes.

I guess it depends on the position, but the point still stands. Given that you spend x amount of time interviewing, what does it matter how it's spent? (with the obvious caveats that if they spend, say, 3 hours screaming at you as a stress test, you probably don't want to work there, etc etc).

>I don't know if this is a Silicon Valley thing or what, but the idea that someone would basically do a day's unpaid work on projects they know nothing about with three different people is just about the most foreign thing I can think of.

Yep. That basically amounts to an application fee.

If you think it costs you something to be interviewed (thus your suggestion about an application fee) it certainly costs the company at least that much if not more. They are paying their staff to work with you on something that likely will not produce direct value for the company (usually a throwaway project or trivial task)

Plus they will spend time before and after the interview discussing your qualifications, performance etc. It comes out about even.

The candidate is taking a much bigger risk.

If the job doesn't work out, they lose small% of potential productivity, while you will lose 100% of your income.

Companies have numerous employees, but you only have one employer. You bet everything on your employer, they only bet a small part of their hand on you.

The company is expected to have costs when hiring, it isn't appropriate to push them onto the applicants by having them work it off.

What if a company gave you an itemized bill after your interview? You're still exchanging something of value for their time; is that reasonable or ridiculous?

By this logic the candidate should pay half of travel and hotel too.

Usually the candidate pays all of it.


Of the three job interviews I've traveled for in the last year, all were paid by the company 100%.

Seconding this; even for internships I've not once paid for travel costs. Everyone from small, YC startups up to MS/Google paid for cab fare, flights, and hotels.

This is highly unusual (if not unheard of).

A company would have to be pretty damn unique in order to justify having candidates pay for their own travel + lodging.

Most jobs are not provided by SV companies or Microsoft. Most prople are not in a career that typically hires from across the country at their own expense. Paid travel is a luxury enjoyed by a small segment of job candidates.

Expanding the scope of discussion to all jobs is disingenuous. It's pretty clear this thread (not to mention this forum) is fairly software centric.

Regardless, companies across just about all industries that hire from remote talent pools will pay for travel (consulting, finance, government, NGOs, banking, accounting, manufacturing, insurance, you-name-it).

Whether or not a given company expands its talent search to include workers in distant locations is bound by several factors, none of which are unique to "SV and Microsoft".

Which is strictly forbidden in most jurisdictions.

It depends on how it is structured.

I doubt there would be anything wrong with saying "we maintain open source project X and we only hire people who have made submissions in this project that we have accepted. If you want to be considered for employment, you must first get involved in the community."

There is more "soft skill" assessment going on with pair programming than "hard skills".

I doubt that work important to the company is done this way. Otherwise, they'd expose proprietary information. Of course, I could be wrong.

Spending a day doing useless work is worse.

I'm curious if you wouldn't mind sharing a few more details of your experience pairing a candidate in the office with your senior engineers.

- Are the candidates able to help make meaningful contributions to whatever the senior engineers are working on without having the background and domain knowledge?

- Do the employees have to spend a significant amount of the 2-3 hours with the candidate bringing them up to speed on what they are working on, just so that the candidate isn't totally in the dark? If yes, does this totally sideline the employees day/flow?

- What do you do if the employees don't have several hours of non-stop coding lined up for the day?

I don't doubt this method works well - but I'm curious how you work through some of the stickier issues of pairing a total outsider with a fully-up-to-speed engineer and then evaluating the outsider based on what was produced in those 2-3 hours.

In my experience:

- No - Yes - You have to put other things on hold and come up with something non-stop code worthy, which may be frustrating.

There's a solution I like though - work on a known bug or new feature in some open source project, potentially of the candidate's choosing, but preferably not one either the candidate or interviewer is already a major contributor to. That way both people come to the problem with a similar low level of expertise and have to work together to come up to speed (which is a great real-work-like exercise). It also makes it obvious what to work non-stop on for that time. Bonus points for being way more useful than a make-work wheel reinvention project and for not being free work for the interviewer's company.

I havent done pairing interviews but I have tried giving candidates some started code and having them develop a solution based on that.

One piece of logistical advice here: keep different kinds of keyboards/IDEs available. I use a mac and I interviewed a candidate who wasnt comfortable with the mac keyboard. In an interview situation where a person is already slightly nervous the last thing you want is for them to be worrying about keyboard shortcuts!

I think a half-done chunk of example code to be finished is a great technique.

It's a late reply, but sure.

The purpose of pairing in this context is not to see a meaningful contribution; rather, it's to see how the candidate interacts with a codebase he's never seen before, and developers he's never worked with before. Can they spot mistakes? Recognize patterns? Read and write code? Does the candidate know the ecosystem well enough to suggest methods or libraries?

Most importantly, pairing tells us whether or not the candidate can communicate effectively. Are they terrified to offer suggestions, or overly arrogant and controlling? Can they explain clearly? Do they clarify when the other half of the pair doesn't understand?

For example, many moons ago, I did some pairing with a candidate that gave me a suggestion on how I could make a section of my code both smaller and easier to read by combining a couple of iterators in an obvious way that I hadn't noticed. We ended up hiring that guy.

The getting-up-to-speed part happens as they pair, with a lot of explanation happening as they work on the system, and it's another important part of seeing how they work -- can the candidate understand new concepts quickly? Do they stop and ask questions when they don't understand?

This type of interview is a performance hit for that employee for the day, but this is fine, as I consider hiring to be very important, and losing a day or two of engineering time over the course of a month is a price I'm willing to pay -- keep in mind that the phone screens eliminate a lot of people.

Admittedly, I've never had a day when my guys didn't have work to do, but not all interviews involved the company codebase.

I make it a point to carve space out for my team to do miniature research projects, in a fashion similar to the way HP used to work, or Google's "20% time". These are timeboxed, and at the end, the result needs to be made available to the team; and yes, "this didn't work because of X and Y" is totally acceptable. This always pays off in terms of education, and every now and then you get something useful out of them, but they're also fair game for pairing during programming interviews.

Hopefully I've answered your questions.

At the last place I worked our interview was split into 3 sections; Technical, Personality, and Critical Thinking (mirroring your three questions). While it wasn't a programming job, it was a support job with a wide array of third-party and in-house software, written in a variety of languages. Because of the size and mobility, they preferred resourceful generalists over hiring people who were extremely specialized. When I was interviewing, the whole goal of any puzzles wasn't to find the answer, but insight into their process of assessing the situation and seeing how reasonable their through process was. If they were familiar with the puzzle, it was obvious because there wasn't any thought process before answering--which isn't a problem, just move onto the next question.

No, you're not that guy, tokenadult is :-)

The problem with hiring (anyone) is that it's really like "speed dating" - even having multiple interview sessions really doesn't give a greater insight into the candidate. The idea of "umbrella of cultural fit" is what mostly happens with hires.

37Signal's approach of trying them out for size is a good approach, but not practical for everyone else.

... someone being paid to be a programmer who can't code. But I assure you this is the case. It's shocking but true.

This is a shockingly true statement - it's almost beyond comprehensive when witnessed first hand.

What is shocking is the fact that in companies where devs have no say in hiring, managers hire those people and then ignore the fact that they do nothing (or try to promote them out of development) to try and cover up their mistake. It is incredibly demoralizing working ten hour days trying to deliver working code while someone else is producing nothing, getting ahead, and going home at 4 pm. This shit is by far the single most enraging thing in all of software development. That's why I 'keep it real' in interviews.

How do you "keep it real" in interviews?

Concerning the viability of firing under-performing individuals, how likely is that? Are there any implications with indirect costs such as unemployment benefits? (I ask because I do not know - any insight on this would be greatly appreciated.)

I ask difficult interview questions and require actual code to be written on the spot. If the questions can't be answered I let everyone know it. The viability of firing under-performing people is zero. Managers just want headcount and don't care about ability. They play the political game not the build-and-ship-software game.

Are there cost implications? I should think so since the fakers cost lots of money and are not producing any software.

They play the political game ...

Unfortunately, this happens far too often. However, I have completely bombed on "difficult interview questions" and "code to be written" - merely due to the pressures felt in the interview(s).

As said elsewhere, identifying the "fakers" is extremely difficult - perhaps that's why people who were referred tend[0] to be more successful? (But how do people outside get in?)

[0] Merely a guesstimate based on casual observations.

Many companies are terrified of being sued for firing and will resort to passive-aggressive behavior instead. Most stuff like this is settled out of court so I don't know what the real odds and risks are but anecdotally I've been in a few orgs that weren't shy about firing at all, and didn't suffer for it.

> 37Signal's approach of trying them out for size is a good approach, but not practical for everyone else.

If you figure in the possible downside, it is the only practical way.

Practical is too much of a general term, but I do agree with your statement that it's the most prudent choice versus the possible downsides.

My usage of the term not practical meant that the employer is spending time and effort on something other than generating revenue. And that also means the potential employee is (potentially) working without pay (or other benefits).

The statement "without pay" is an assumption since the book, "Getting Real", does not explicitly state compensation.[1]

[1] http://gettingreal.37signals.com/ch08_Kick_the_Tires.php

I've worked to two programmers who could not code. I mean literally not code. You cannot imagine how frustrating this became before the managers finally got rid of them.

While terrifying, I think the real danger is hiring someone who can code, but can't do it in a maintainable way. I'd much rather have someone not write a single line of code, rather than have them write 6 months of unmaintainable spaghetti code. Zero contribution is bad, but it's much worse if they have a negative multiplier on the future of the team (if what should take a day to do takes a week to do because of unmaintainable code).

How can this even happen? I mean, isn't it obvious in the first week or two, month tops? If not how can they hide their cluelessness for months until they are laid off?

I've seen it happen a couple times. They get everybody else to help them, so their assignments get done. They invite the CIO out to lunch every couple weeks so they're insulated there (helps if they already know the CIO).

Out of desperation, their immediate supervisor assigns them some non-programming task like kicking off a special report every morning. The manager resigns and now they're the only one with institutional knowledge about how to do that task. So the new manager isn't going to fire them. Besides, the CIO speaks highly of them, so they can't be all that bad…

tl;dr: politics

Month one and two they are 'ramping up' Month three and four they pick something inane like what kind of font the web site should use and schedule lots of one on one face-time meetings with higher ups. The higher the better. Month five and six they are now telling everyone their life goal is to be a project manager or that they are training to be a project/program/product manager. Their work is sort of like a project manager. They did the font project. Remember the font project? Simple game theory prevents managers from acknowledging the truth that they are loafing because that might make all management look like all they do is loaf. The guy's direct manager is overjoyed that all the higher ups are paying attention to the font project.

Aw, c'mon, I want to be asked how to move Mt. Fuji and what would happen if I dropped into a blender (that one's easy: I'm pretty positive that I would blend).

For Mt Fuji, how you'd do it is less important than your first question(s): "Why?" and "What constraints?"

If it's to make space for a hyperspace bypass, I'll just let the Vogons handle it. If time is critical, how much mining equipment and how many many megatons do I have access to? What's the size of the evacuation zone? Is glowing gravel OK? There are some companies in Appalachia that have plenty of experience removing mountains, can I call on them?

I think the point of the article is this:

Trivial puzzlers like:write a fizzbuzz program, swap two variable contents without using a temp variable, how do you count the stars in the universe

Just encourage people to game the system. It's impossible to get real insight from them, because you can't differentiate between those who worked it out, and those who have read about it. If you're in an interview by now and someone asks one of these cookie-cutter questions like: how much storage is required to store satellite images for Google Maps? or How would you count the starts in the universe?

If you're good at problem solving but haven't though it through before you might come up with a general ballpark solution, or you might freeze under pressure of the interview of course. If you're a slick liar and have been reading codinghorror, you'll have an answer down pat for this category of questions before you even enter the room, one which you cribbed off the internet, as you will for all the other common categories. If you're so incompetent that you're neither, I guess you'd be weeded out - not a huge win - I wouldn't expect such people even to get to interview, which is a vastly expensive process (in time and money).

In contrast a real world problem, if the discussion is led properly by the interviewer, offers the opportunity for real in-depth discussion about the sort of problems you're going to be expecting this programmer to tackle every day - what trade-offs should they make when designing data structures to balance normalisation and speed, which libraries would they use to solve a particular problem, which problems they have seen in the past which are similar and how did they solve them, etc etc. and it also deals with cultural fit. There is not a clear dichotomy here between logic (puzzles), and wishy-washy cultural fit (a chat about code) - an informal chat, if held properly, can provide far more insight than a set of questions.

This depends entirely on the sort of job you're hiring for of course, but if you're hiring a web developer then a good background in interview teasers is not really on the list of desirable attributes, but a good understanding of real code is. I guess if you don't control the entire interview process and/or work at a large organisation, this sort of filter might become appealing because of the standard of interviewees you receive, but that's a reaction to a failed process.

One hiring project which I really admire is the stripe CTF as it selects for a certain type of person that they're obviously keen to find. It's a puzzle, but one so original and unique that it's very hard to game, and also one which records the users' attempts in real time (if they keep logs).

As much as people say "well, everyone knows about Fizzbuzz by now," lots of people still can't do it.

"Swapping two variables without a temp" is a bad question, because it depends on knowing the trick.

"Find the 2nd-highest element of this list" is a basic question, and I can change it up it a zillion ways such that anyone trying to just learn all possible permutations will end up doing way more work than someone who can just figure it out on the fly.

I think the swapping two variables has a bit more going for it if they do not know the trick. Explain the concept of invertible functions. Hopefully they would be able to come up with plus and minus - even if you have to walk them through the a=f(a,b) b=g(a,b) a=g(a,b)

From there you can say that works fine with pen and paper but why might it go wrong in a program? So how do you avoid integer overflow? If the signs are the same then use subtract first else use addition first. After we have done that. After this step assume I have returned to the main flow. How do I check if I should use addition or subtraction? If a is bigger than b I must have added so use subtraction. Writing the algorithm this way is trickier than xor. The problems to be solved are to do with understanding concepts

Next you have the xor solution but more important than getting the answer to that might be to see if they can see that the two functions are like encrypt and decrypt where a is the message and b is a key and a + b is the ciphertext c - so that f(m,k) is encrypt and g(c,k) gives the message and g(c,m) gives the key. If they have had any exposure to cryptography then xor should present itself.

If they give xor straight off then you can work backwards.

You're missing the point, and this is how people end up at these "puzzlers". The point of fizzbuzz or the simple questions is to weed out people who can't code at all. If you transform it like you do above, then you're testing something else: how well people know low-level tricks like swapping variables using xor.

Those tricks might be valuable in your workplace, and may make excellent interview questions for you, but they are a very very different type of question than fizzbuzz, at least in purpose, even though they may look superficially similar. In a fizzbuzz type question, you are not looking for brilliance, you are looking to just weed out the people who can't.

Exactly. This is precisely the problem. It's not that any particular question is a "bad" choice, per se, it's that interviewers can fail to realize what they are actually testing for.

XOR trick? I'm not sure I'd want to work for someone who didn't know about the x86 xchg opcode.

I had never seen the XOR algorithm and after seeing the parent post grabbed a pencil and paper and tried to come up with something. Took a couple minutes and I consider myself a middling programmer.

It's impossible to get real insight from them, because you can't differentiate between those who worked it out, and those who have read about it.

The fact that they're involved enough in the industry to know about fizzbuzz is still a strong signal. It will only come up if participate in some sort of developer community, which, I suspect, is something that the completely incompetent don't do, as a rule.

Once again, it's not designed to find the top talent. It's merely a high pass filter that removes the hopeless from the set of people to consider.

But even if fizzbuzz is something that the hopeless start learning by rote, switching it out for any other trivial problem will still work. "Count the number of 'a's that come after 'g's in a string", "print the lyrics to 99 bottles of beer", or hundreds of other problems like that are just fine for the task.

It will only come up if participate in some sort of developer community, which, I suspect, is something that the completely incompetent don't do, as a rule.

A great thing a small company can do to find people is sponsor a users group of some sort. If your company does Java start a JUG, Ruby start a RUG, etc... For the price of some pizza you can informally meet people that by the nature of showing up give a strong signal as people you may want to talk to when looking for your next employee.

Once again, it's not designed to find the top talent. It's merely a high pass filter that removes the hopeless from the set of people to consider.

I'm sure it is useful in some situations, particularly in big companies with a broken hiring process. I imagine 37Signals don't find it useful as they have a more effective screening process, and people coming to interview simply couldn't possibly be that bad as they're referred or they've seen some code they wrote.

If you are really faced with candidates who are incompetent (i.e. HR has filtered candidates for you based on CVs or similar), I think I'd try to automate some kind of online quiz they can go and take before they're even allowed to apply (which can include fizzbuzz or whatever little puzzles you want, checks for cheating etc) - then you can spend the phone screen or interview actually talking about interesting stuff which is in the problem domain they'll actually be tackling and finding out what they'd do with it. I just suspect these questions can be gamed and are not necessarily weeding out all the undesirables, just those who haven't done any homework on your hiring process.

I've seen people hired who talked a good game and could have passed these trivial tests without giving away that they'd just read up on likely ones, but were lacking in all the skills necessary to bring projects in on time and competently - that sort of mediocre candidate who lies about experience and knows the basics is far harder to stop.

It's interesting how different people react to IQ tests or little programming tests though - some people hate the implication of jumping through hoops, and that ability/knowledge can be reduced to such simple set of tests, and resent being subjected to them. Others really don't care and find them interesting or at worst trivial. I'd find it kind of weird trying to guess in an interview if someone who asks for say 'select the second highest number in this list' just wants a quick solution like list.sort[list.count-2] (which in the real world would be the first choice until or unless you are likely to hit major performance issues), or if they want you to write a trivial sorting algorithm for them and explain that you're familiar with the content of an undergraduate CS course.

It's also interesting seeing the range of different interview questions posted here as examples - some are trivial filters as you say, some are more involved, and some require knowledge of basic undergrad stuff, so I'm not completely sure that all of these puzzles are intended as completely trivial, there's always the temptation to elaborate them into some kind of exam (like those posted on HN a while back with maths questions) for esoteric knowledge which is frankly useless in most people's job.

Google has a pretty good reputation for tough filters and still, questions at the Fizzbuzz level are enough to blow many or most candidates away. If you're not asking anything like it, you'd be amazed at how many otherwise intelligent-sounding candidates just. cannot. program. at. all.

'select the second highest number in this list' is actually O(n) and does not require sorting.

Does that mean I fail the test :) A sort then select for the second highest seemed the simplest solution off the cuff (perhaps not the quickest), and if the code is fast enough, there's no point in optimising. Of course you could write a loop and check each value against a stored memo - your O(n) right?, but why bother when the library sort is fast enough for the purpose? If you're dealing with genuinely huge data sets obviously this becomes important, but then often the data will be in a db anyway.

As the person who posted that question, if you gave me that answer right away, I'd say "good, but can you do better?"

"Sort and take the second highest" is a fine first answer. I'd expect most good candidates to start with that. I'd also expect them to say "but I think I can do better" on their own.

This is where I find questions like this a little confusing. To define better (at least in real world situations) the question requires more context. How many records? How are they stored? What are the characteristics of the storage? Which libraries are we using to store it and how do they behave (and in the real world there might be a few)? What are the other constraints? If we request a sort (say in a remote db) will that be quicker than the latency of requesting data in batches and iterating?

Presumably you mean better given infinite memory to store all the records and a very large number of records? Because for most small collections of say a few thousand, a sort then select would be better than an iteration IMHO as it is simpler and the intent is clear if performance is acceptable. Perhaps I have misunderstood the intent of the question though, I'd be interested to hear what you feel is the ideal answer.

I think "do better" is a pretty obvious description, but if the candidate is having trouble with it, I'd say "you have a few million records, and your current mechanism is too slow."

I have patience for candidates, but not an infinite amount. If they keep on trying to come with reasons why they don't need to improve their answer, or try to drown me in a sea of bullshit, that's a no-hire. I'm screening out assholes and prima donnas as much as I'm screening out people who can't code.

Upon being told that "sort and take the second element" isn't good enough, I'd expect the candidate to come up with a linear walk through the code, keeping track of the two top elements.

I'd then ask the candidate to generalize it to "the #n element" instead of "the #2 element," which will probably require a new data element. Keeping those n elements in a sorted list is a fine way to start. A really great candidate would ask if they could re-order the elements as they find them (doing a partial version of a quick sort) or use some variation of a heap-sort with limited space.

Ok, I think that clarifies your intent, the question is purely hypothetical (many records, infinite resources) and designed to elicit the platonic algorithm given no constraints save time. You surely recognise though that 'improving' depends on the constraints which were not given? I find it curious that you characterise questions attempting to relate your problem to real world constraints as 'a sea of bullshit' :) Surely no one tries to avoid answering by asking for clarification of the constraints, wouldn't you normally face this problem with records in a db, not all in memory directly accessed? I'm not sure this interviewing game would suit me.

If sort first was too slow and it had to be done in memory, I would have iterated once and kept the results in an ordered list but not sorted the records during iteration as you have suggested, didn't think of that. Thanks for clarifying.

I think this - I mean the "meaning of better" being obvious or not - is a major difference between most of industry and academia. In academia code readability matters less, it's asymptotic performance matters more and constants can be safely ignored. Reimplementations of things tend to be rewarded, because it shows the student's understanding. It's completely different in the "real world".

And this is why looking at actual code someone wrote is a good idea - because there are things one cannot check during the interview, like how well the candidate structures the code, how well he writes comments, how does he manage the complexity of a growing codebase, how quickly he can learn a new library or tool and so on.

In this particular case I'd probably know that "doing better" means algorithmic complexity, but I would never come up with "but I can do better" remark myself. It's also possible that, when asked to "do better", the first thing I'd try to do would be to write unit tests and docstrings. Is this unacceptable? Would I be seen as an asshole or prima donna?

Also, there's an issue of me being able to come up with a "correct", "better", faster in terms of Big-Oh algorithm. I am not a genius, I know this. I am not and I can not hope to become Dijkstra. I'm also not a computer scientist. I worked hard to be able to read and understand and then code algorithms presented in papers, but I'm damn sure I couldn't come up with even the simplest of them. For example just last week I had to implement a scheduling algorithm called "highest response ratio next". I was able to look it up in the sources, understand it and implement it in a product, but I am sure as hell that I would not invent it during an interview. Does it mean that I'm not qualified for a job? Would I be seen as dumb?

Posts like danielweber wrote above make me nervous. It's probably not what he had in mind, but it seems to me like he would not hire me - and I work in "the industry" for almost ten years now and I have been programming for twenty. I can brush this off as irrelevant - I have a job that I'm proud of and happy with, but what a young, aspiring programmer has to feel when he gets told that he's no good, no hire, because he can't invent algorithms on the spot?

The algorithm design is all said out loud. If I printed up a page of code and handed it to the candidate and said "make it better" I might expect them to work on style.

So I don't mind answering questions during the interview process. The candidate might be nervous and I should be clear about my expectations. I don't like mind games like picking a fight and making sure the candidate responds the "right" way.[1] I would hold no points against them for asking on clarification for what "better" means, as long as I didn't they they were trying to be difficult and accepted that faster was better.

I'm sorry if you don't think I would hire you. Maybe I wouldn't, but that doesn't mean you aren't a good coder. But I also think you shouldn't be intimidated by "algorithm invention." You probably do this a bunch already. In the order you write your code you are probably already doing implicit algorithm invention. I think you can write the code to figure out the biggest number in a list if you sit down to do it, and once you do that finding the second biggest just involves keeping track of one more thing and doing one more test on each pass through.

Often the question is figuring out what data structure you should be using.

Oh, I don't actually expect anyone to give me the perfect answer of how to find the nth-biggest element in the list. In fact, if a candidate did give the perfect answer, I would say "this person is more qualified than me, please hire." I'm not capable myself of giving the perfect answer even though I've thought about it a few times and it's essentially a question I've practiced.

[1] See question 7 at http://www.joelonsoftware.com/articles/fog0000000073.html

"The algorithm design is all said out loud."

Ok, I thought it was being written as a code, even if just on paper or a board.

"accepted that faster was better"

The point we - grey-area and I - are trying to make is that it's not always the case, or rather that the "betterness" of faster implementation is, in practice, dependent in large parts on other characteristics of the code. If someone deleted slower implementation along with tests and docs and replaced it with faster, but undocumented, untested implementation I would not consider it a change for the better.

And I think you would agree with this in general (of course there are situations when a change like this is necessary!), and I think this is just a misunderstanding on my part - I was thinking about actual code, an implementation, all along while you were talking about description of the algorithm with words. In this case - with one additional assumption that memory complexity is irrelevant or not changed, but that's nitpicking - of course faster is better! :)

"I think you can write the code to figure out the biggest number in a list"

Yes, of course: (foldl max (first lst) (rest lst))

And I can even explain what foldl is and how it works.

"you shouldn't be intimidated by "algorithm invention.""

I am not "intimidated". I know my limits. I wouldn't be able to invent quicksort during the interview and I wouldn' be able to invent this: http://www.jstor.org/discover/10.2307/2346806 even if my life depended on this and I was given all the time of universe to do this (but I still coded it).

This also depends on what we mean by "inventing". Does using some previously learned technique or adjusting other already known algorithm count as an invention?

But from what you say I gather that you wouldn't want me to invent quicksort on the spot - and then it's ok, it seems like I still have a chance of getting a job! ;)

http://www.jstor.org/discover/10.2307/2346806 was not something someone came up with after a few days at it. They had deep understanding of the statistics behind the algorithm, and an almost unfathomable knowledge in the domain, not to mention a large network to help them. I'm sure that most intelligent people would be able come up with it if they had the same circumstances working in their favor.

As a rule of thumb, if you feel that the correct answer depends on these details, ask questions to pin it down. Or, even better, just pick what you think is the most relevant for the company that you're interviewing at and (briefly) say what you're using as working assumptions.

It depends entirely on the person asking the question. Some people probably don't care, but there are some fans of the most efficient possible solution.

At my current job, we used FizzBuzz when we were interviewing potential senior devs. One of the candidates who looked good enough on paper to talk to could not complete FizzBuzz, and in fact spent 20 minutes struggling with it before I decided to end the screening.

It's certainly possible that some of the people who completed FizzBuzz are not good programmers, but it seems pretty clear that the one person who failed it cannot possibly be a good programmer.

Having people go through FizzBuzz (which generally took 5-10 minutes per person) saved us the trouble of a longer interview with more people for at least one clearly unacceptable candidate.

I have the same experience hiring. Fizzbuzz is one of the best filters I have personally used.

Come to think of it, swapping two variables in place is a horrible question - because you can't really do it! A statement like

a = a + b

requires the (invisible) presence of a register where a + b (or xor or whatever) is placed right before being written back to a. But then you might as well use this register directly for the swap instead of dabbling around with one-to-one functions.

Also, it depends on which language you're using...

    $ irb
    irb(main):001:0> a = 1
    => 1
    irb(main):002:0> b = 2
    => 2
    irb(main):003:0> b, a = a, b
    => [1, 2]
    irb(main):004:0> puts "#{a} #{b}"
    2 1
    => nil
(obviously this has the same flaw you mention, the interpreter is doing stuff for you. It follows the letter of the 'no intermediate variables' only.)

You can also follow the letter in C, Java, etc.:

  a ^= b
  b ^= a
  a ^= b

What does that do?

Thanks for the pointer.

Just to add a perl version for the ruby and python ones.

($a, $b) = ($b, $a)

python: a, b = b, a

>Trivial puzzlers like:write a fizzbuzz program

That isn't a "puzzler", in any way, shape or form. It is quite literally "write the most basic, simple, trivial program possible". There is no trick to it, no gotcha, no special information that makes it suddenly go from hard to easy. It is always easy, and is a practical test of actual programming knowledge.

I didn't mean to imply it was hard. As to a test of programming knowledge, I'm glad I'm not in a position where I'm exposed to people who would fail such a test :)

I get it. That said, isn't there concern about candidates just cramming on the top 100 programming puzzles just to be able to ace such interviews?

I take a bit of a different approach. I want to know how they think. Pose a problem and collaboratively work through it on a white board. No code. Just state diagrams, flow charts, whatever.

Not looking for a solution. Looking for the thought process.

Then put up a piece of code. Ask the candidate to explain it and critique the code. How would you improve it? I'd expect them to ask questions like 'How do you define "improve"?'

I tend to be skeptical of the top 100 programming puzzles because I have no way of knowing if they spent the last two weeks training for just that or not.

You can even take the approach of posing a problem that has, on first inspection, nothing whatsoever to do with their job description. A seat-of-the-pants example would be to ask someone being hired for back-end web development work how they would go about designing a small multi-tasking OS for an small 8 bit micro-controller.

Crazy? Maybe. If the candidate launches into a description without prompt you know that this guy (or gal) either has experience outside of the web realm or is inquisitive enough to read and get into other stuff. If they do not, then you get to see how they go about stepping into something they know nothing about. Do they ask you questions or do they cave?

I don't think there's a single answer to this problem. If you need to hire programmers by the hundreds then, by all means, puzzles are almost a must. If you are after a few uniques, I am not sure they are worth it.

"Not looking for a solution. Looking for the thought process."

This is the part I can't understand. When the person is on the job, the opposite is in effect. If you tell the candidate to solve a hard problem, you don't wan them to just spit out their thought process, do you? No, you want them to solve the problem. Why the opposite for interviews?

If I'm an employer and you are working for me, I am asking you to solve a problem because there is some value in the result.

If I'm interviewing you, chances are, the result has no intrinsic value - I already (hopefully) know it. What I want to know is "Can you solve other similarly difficult problems?" In this context, being able to come up with a solution to the problem isn't as useful in answering that as knowing how you solved it.

Probably because you have, at best, a few hours in an interview to determine whether or not someone can solve the hard problems. Many hard problems take more than a few hours to solve, so the interview process serves as a gauge of how likely a candidate can do this on the job; of course after other vetting methods were applied.

> whiteboarding code by itself is not a great filter. It should be used in conjunction with a multi-faceted interviewing approach that involves testing fundamentals, the ability to construct a relatively simple algorithm, the issues of working on a team and on a production code base and systems design.

This I can heartily applaud. However, I take issue with this:

> tests like the FizzBuzz test [1] are, in my experience, remarkably effective [...]

I'd never heard of the FizzBuzz test before, just tried it.

You may as well have asked me to sing the solution out loud. It was that bad.

Paper turned out to be a completely unfamiliar medium for coding, despite years of whiteboarding solutions with colleagues. I instantly felt like I was using a totally different part of my brain. I practically lost the ability to write with a pen and significantly lost the ability to code. My heart rate rose substantially with fear of the unfamiliar. This tiny test took me ten minutes, and when I checked it later I found a syntax error.

And that was in my favourite chair, at my desk, at home on a quiet sunny Sunday morning, working in a language I know inside out. Under pressure, with strangers, in an unfamiliar job interview room? I'd have just fallen apart completely.

Back at the command-line? 30sec in a one-liner in a language I'm still learning.

I will not be adopting this method to filter hires, having self-identified myself as a false negative. I am never out to embarrass someone in a job interview - I want them to show me their strengths! If they prefer it (they usually do), I'll give them a real computer and ask them to shine using that.

I don't know if we've gotten there yet, but Fizz Buzz specifically may not always be an effective weeding-out tool: once enough people have heard of it, they may prepare specifically for it. Perhaps a better approach would be to come up with your own small test of comparable difficulty, sort of like a blog that has a custom but otherwise extremely hackable way of testing if you're a real person.

> Not everyone can produce real-world code. Most of what we do for a living belongs to our employer. Side projects, particularly in the US, can be an issue for IP reasons (California is somewhat of an exception here)

Says who? My side project that I do on my own time doesn't use the same language as my day job nor does it tackle the same problem domain. How can this be construed as an intellectual property issue?

Says the courts in many US states who have upheld ridiculous contract clauses where an employer claims ownership of everything a worker makes, even on his own time.

It's entirely possible that your contract has no such clause, but they are disturbingly common.

That depends on the agreement you signed. It's common for the invention assignment agreement you sign when you start working at a software company to cover anything you do, even on your own time with your own equipment, if it "relates to the Company's business, or actual or demonstrably anticipated research or development of the Company", or similar language. If your employer is a large company with its fingers in many pies, this could apply even if the work has nothing to do with what you yourself do for them.

California is unusual in that it has a statutory protection against the most expansive assignment provisions, but even it allows that provision. http://www.leginfo.ca.gov/cgi-bin/displaycode?section=lab...

Your employment agreement may include a broad intellectual property assignment clause. If it does and you don't live in a jurisdiction that has legislatively restricted IP assignment clauses the programming language and problem domain of your side project is unlikely to matter.

I'm sorry but writing code on a write board, ever, is stupid. I'm happy to write out architectures on a whiteboard, but code, never. Asking someone to write code on a whiteboard is the equivalent to asking a UI designer to chisel wood to demonstrate their ability to develop a web UI. It's rediculous.

I'm sorry but you are incorrect. It is not equivalent. It would make perfect sense to ask UI designer to draft a design on a white board. Same applies to coding. You don't get the point. It's about seeing how the applicant thinks and communicates under pressure. The thinking out aloud part is more important than the white board part. Everyone knows this, except you?

You can determine how someone thinks without trying to code on a whiteboard. I've interviewed many people for tech jobs recently, I've never even considered asking them to code on a whiteboard.

Writing code on a whiteboard is massively obnoxious. The times I've been asked to write code on a whiteboard I've spent more time screwing with the 'interface' than I ever have spent 'thinking'. e.g. trying to keep a line of text level, figuring out how to get brackets to line up, and erasing lines to reorder or writing lines off to the side with an arrow pointing to where it should go. A whiteboard is the worst possible interface for creating strictly structured text.

You aren't testing how the person thinks, your testing how they deal will a situation you've artificially made as awkward as possible, why? What does this actually indicate to you about the applicant?

If you want to quiz a candidate on how they think when coding then hand them a laptop that is already setup to duplicate its display to a projector. Or give them a simple 'take home' project. You'll learn vasty more than having them regurgitate algorithms on a whiteboard.

The basic question is whether it is a good idea to ask questions which humiliate a sizeable portion of interviewee simply because this maximizes your personal utility?

I don't even know for certain if such an approach does maximize your time versus finding-good-programmers ratio but let's assume it does.

The thing is you are willing to maximize this ratio by doing things that treat some portion people who have ability as if they don't (even it's also treating people without ability fairly)?

I'd claim that doing that sets you for up an adversarial relationship with everyone in the workplace - the people hire you can feel they're under-the-gun. I strongly suspect that the "almost no one can code" crowd are the people who live in these adversarial environments, which produce bad code through all the emotional and organizational games this implies.

Well, he said "real code" not "real world code". I would imagine you can just write something simple for them.

I think all your points are incredibly valid. But what WOULD you recommend?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact