Hacker News new | comments | show | ask | jobs | submit login
Hiring without whiteboards (github.com)
423 points by sugarpirate 45 days ago | hide | past | web | 363 comments | favorite

I'm responsible for hiring developers at our company based in Berlin, Germany, and found it best to have a guided interview about the candidate's work experience and interesting problems that she/he solved. I never understood the whiteboard hazing/CS trivia that are so widely discussed on HN since it seems extremely disconnected from the actual work that's being done.

That said, I'm always surprised how many candidates cannot even point to one problem they worked on they found interesting or one solution that they're proud of.

We worked with an HR consultant to develop a interview guide in the form of certain questions that we make sure to hit during the interview in order to be able to compare between candidates and make an informed decision.

However, we're small and not in the US. Anyone have experience with other companies in Germany/Europe? How does the typical interview work over here?

> That said, I'm always surprised how many candidates cannot even point to one problem they worked on they found interesting or one solution that they're proud of.

It could be that this technique favors people good at telling stories.

Personally, I'm a horrible storyteller. If you were to ask me what I did over the weekend, I'll offer some facts like "oh, went swimming in a river and Bob lost his hat, but we found it later. The water was nice." Whereas Bob could easily regale you with stories about the epic hunt for his hat and throw in a punchline in the end.

If I didn't know that you'd be asking for solutions I'm proud of, I might draw a blank at that moment. Granted it's an interview setting and these are the questions one needs to prepare for. But if given the choice between speaking about myself or whiteboarding, I'll take the whiteboard.

I've always preferred math over history for that matter. It's easier for some of us to apply processes than to recite chronologies.

> > That said, I'm always surprised how many candidates cannot even point to one problem they worked on they found interesting or one solution that they're proud of.

> It could be that this technique favors people good at telling stories.

Or people who think they're impressive. I know I would've been able to point out many problems and solutions I found interesting and were impressed with when I was 15. Today, not so much. The more I know, the less impressed I am with myself, and more and more stuff just feels "routine" and nothing special.

It could be that I'm just not impressive. At all.

I'm not a "look at me!" kind of person, but here's what has really helped me with these kinds of interviews.

Keep a daily journal of what you work on. Nothing fancy, just stop by once a day religiously and add a few notes on what you did, what meetings you attended, who you spoke with, etc...

Save your evaluations and especially any award packages you or your team might get submitted for. At least where I work both are your supervisor's attempt to make you look as good as possible.

Then, when you're job hunting review your notes and bullets and collect the ones that sound the best and perhaps ones that have numbers assigned to them (size, savings, productivity, etc..)

Preferably you'd memorize these few stories about yourself, but if you must you could also bring a notebook with prompts to remind you.

My grandfather used to do this. He had notebooks filled with what he did every day of his career. It's really cool to be able to go back and look through his notebooks and see what he was doing in April of 1971.

My notetaking is not as rigorous as my Grandfathers, but I do something similar. Every month I make a new manilla folder with the month and year on the tab. Each week I write down every project I'm working on and every meeting I have. If someone brings me a new project I put it on the sheet. Every note I take a meetings and during projects goes in that folder.

At the end of each week I go through the notes from that week and write down anything interesting on that week's note sheet. At the end of the month I go through all the notes and write what I accomplished on the front of each manila folder.

It makes it really easy to keep track of everything you've done, without consciously keeping a journal in the moment. It's also a really easy organizational system. When someone asks, "What did we do for [x] in the past", all I need to do is flip through my folders looking at the front for anything that rings a bell, rather than trying to keep up with a tagged organizational system.

This probably doesn't work if you don't have a file cabinet though.

If someone brought along an notebook full of problems they'd experienced and how they'd solved them to an interview I was conducting I would be very impressed!

This is great advice, thanks. In the past six months, I've been better about keeping notes. Not specifically for this, but this is definitely another benefit of journaling.

At this point in my career I'd have to say I agree. Many of my team are at the start of their careers. I find it really interesting the number of times they seem impressed by something I did that I'm just "meh" about.

Oddly enough, the opposite is true as well. I'm constantly impressed at the level of ability they have at this point in their career as compared to my ability at the same point in my career. They just simply know more. Of course, there's more to know these days but still, I find it impressive.

Or, they just don't find the work terribly exciting. There's a lot of unexciting work to be done in software, and somebody has to do it.

There's definitely an industry bias towards "programmers as artists". If you don't love your work with a passion you're seen as less good.

It's an alright proxy for other things, but there's a ton of good developers that leave their ego at the door, and their work at work.

Good point. When I notice that people are struggling with finding something, I usually just pick one thing from the resume and try to go more in depth about that.

You sound like a good interviewer that adjusts to each candidate. I wish more interviewers would learn the importance of this.

> If I didn't know that you'd be asking for solutions I'm proud of, I might draw a blank at that moment. Granted it's an interview setting and these are the questions one needs to prepare for.

The key word there is "prepare". These are standard interview questions that you just have to have a prepared, rehearsed answer for that you can rattle off without thinking. There are tons of these types of "behavioral" questions. Tell me about a project that you worked on that failed. Tell me about a time you had to deal with team conflict. Talk about a struggling project that you had to help turn around. You can get a book full of them. Be ready with canned answers for as many as you can and practice them in front of a mirror.

Yup, whiteboarding and reciting tried and true stories are inevitable routines when interviewing. However, you'd also be surprised at how many people don't know how to prepare for this. My main concern would be the possibility of turning a great candidate down just because they didn't happen to interview well.

> My main concern would be the possibility of turning a great candidate down just because they didn't happen to interview well.

Too bad (for candidates) that that concern doesn't appear to be shared by... well almost everyone in tech hiring these days. The general wisdom of the day seems to be "it's better to eliminate 100 false negatives than hire one false positive."

Probably because it's "easier to hire than to fire".

I worked with some really great folks that had trouble getting hired and it was because of the flawed interview process. One person writes concise and clear Rails code that's easy to read, and makes some of the wittiest comments on Slack (subtle puns that go unrealised for two minutes, then you finally get it and makes you smile). But he stutters a bit. And because of that, interviews are difficult for him. Which is a real pity, because in less judgmental situations, no one would think twice about that.

(Don't worry there's a happy ending and he's now at a good company.)

I learned after my first university job fair the necessity of being more outgoing during these situations, but there's great engineers out there who still live by the myth of "they'll know me by my work".

> Be ready with canned answers for as many as you can and practice them in front of a mirror.

Practicing algorithms for a whiteboard interview sounds a lot more fun.

I think many will be able to tell the story if it is a story about something they care a lot about. Also, there is an opinion that good developers are good communicators.

You can be a good communicator without being a story teller.

Not being able to a story doesn't make you a bad communicator.

A story is entertainment not work communication.

Companies differ, and what they are asking of software engineers.

I was working with a brilliant guy in the same company. He was very technical and you could say that easily he was programming all day long non stop. His intelligence and technical expertise amazed me. Thing is he wasn't good with people skills, hence he wouldn't fit the company I am currently working for.

There are companies that hire devs to solve a problem and give them time, space etc to go all hardcore. Then again there are companies that require from their dev's to be able to communicate with the rest of the team, and take part on meaningful theory-crafting meetings.

So neither whiteboarding, nor tell me a story question is wrong.

I just find it that whiteboarding difficult questions should be asked from the very technical jobs, and I've found whiteboarding difficult questions when interviewing for a company that wants to move out of wordpress to their own website in angular.

> It could be that this technique favors people good at telling stories.

It does, but then again, maybe that's what they're going for (even if subconsciously).

We're in Denmark and currently hiring.

I'm not sure why there is such extreme hate for the whiteboard. Yes it has plenty of caveats when it comes to actual coding and recruiters should not expect any candidate to write precise code on that medium.

I do use the whiteboard for trivial CS questions limited to 5-10 minutes. Think fizzbuzz and string reversal. Candidates have the option to use my laptop (not ideal because the keyboard has a US layout) but they are welcome to use their own computer as well (if they thought about bringing it). It weeds out candidates who can't even produce basic code. And yes, a candidate with 2+ years of experience should be able to write a basic function on a whiteboard, a napkin, or whatever. If not, the interview is not lost but the candidate will have to prove his skills another way.

For most candidates, we also give a longer technical test which is to be done at home and takes 2 to 4 hours to complete. Candidates are given as many days as they want to complete it.

Whiteboard is an excellent medium however when discussing architecture and higher level ideas. It's also a tool that I've used during day to day activities with colleagues. Software based tools don't come even close.

I believe the hate comes from unprecedented and hard CS questions asked on the whiteboard.

I recently went to an interview that asked me to balance a binary tree on a whiteboard. It can be done, and I can do it. Thing is when you go to interview for that company that has 2 developers(small team, small company) and ask you that kind of question it puts you off thinking that those guys won't be great to work with (arrogance etc comes in mind).

I am fond of simpler questions, like you said fizzbuz etc. Obviously if you are interviewing a guy that has 2+ years of experience, he has to be able to pass the fizzbuzz test. When you are interviewing someone with 5+ years of experience for a higher up position I guess you do have to ask some harder question, but I personally think just speaking to the guy and asking him stuff about his past projects etc will give you a hint on if he has the skills he is talking about or not. Asking him to outline a hard task he took part and how he solved it is an amazing start. As a 5+ years guy personally would rumble about a few things and it would take me days talking about them. (That will give you an understanding if I've worked before or not on the things outlined on my CV).

I've been coding for about 25 years. The last time I had to write any code related to binary trees was in school, 15-16 years ago.

Given enough time and debugging I'm sure I could do it. I'm not going to be successful on a whiteboard though. If I actually had to do it I'd look it up instead of wasting my time figuring it out.

Unless you're interviewing for a position where they'll have to implement binary trees you shouldn't be wasting time asking about code for them. Questions should be relevant to the position.

Additionally: Nine times out of ten, you shouldn't be implementing the binary tree in the first place. There are already tons of tested, robust libraries out there that will do it a lot better than you can, handling all conceivable edge cases, available in a variety of license flavors. Many languages' standard libraries include support for binary trees.

You want candidates who can work through the buy-vs-build tradeoffs.

Reminds me of when I was asked to implement a filesystem API in C or C++ with file manipulation and path parsing and whatnot. The best answer I can think of is usually "With no weird constraints given (is this an embedded system, for example?) just use boost::filesystem and move on with your life." Not impressed with this answer, the interviewer would continue with "assume you can't use Boost!" Next answer is: "There are a variety of filesystem abstractions already written, far better than I could whiteboard. I'd go through each one and pick the one most suitable to the project." I thought it was a trick question at first. You really DO NOT want candidates who hear that question and launch into writing code!!

It's obvious that they're not going to use the code you wrote at an interview in production. So "just use a library" is not a relevant answer. The question was probably posed to give you an interesting engineering challenge to work through with the interviewer so that they could see your problem solving process. Instead you just argued with the interviewer.

The GP's point (which I supported) is that it's a waste of time to take a candidate through scenarios that are not at least similar to what they'd actually encounter doing the actual job.

But the point is that if you cant do something so simple, like traverse a tree, then you probably don't understand how trees work to begin with.

Similarly, I've never had to implement fizz buzz for real, actual, work. But I wouldn't trust a programmer who was incapable of doing fizz buzz without importing a fizz buzz library.

But the earlier comment wasn't talking about _traversing_ a tree, it was talking about _balancing_ it, which is a much harder problem and one I wouldn't want to do without google either.

This can also be an education problem. There can be big mismatches in what is easy and hard

If you're coming from a more FP background things like flipping a binary tree are silly obvious. If you spend a lot of time with C++, of course you know how to write a linked list with memory management. Python devs are probably way better at slicing and dicing data

There's a certain expectation of what's easy and hard, but it's really easy to have a mismatch given the diversity of education out there.

It's also that coders tend to... err... "offload" memory for the stuff they aren't immediately using on a daily basis, and only remember the general terms and names that would be enough to find and remember the details shall they be required.

Last time I've had to do something with the trees implementation details (IIRC, that was AVL trees) was, like, 5 years ago. Right now, I don't exactly remember anything about AVL trees except for they have that nice O(log n) for lookups, insertions and deletions. Would I need to code them from scratch (or remember their internal workings for any other reason, like debugging some core dump), I'd go find the algorithm description (or particular implementation's source) and do so.

At the same time, last time I've worked with balanced trees was just a few days ago, but that was Erlang's `gb_trees` module that did all the stuff and I just had to freshen my memory on the syntax details.

All I would be able to do on a whiteboard is stare for a minute, trying to remember stuff, then probably say "uh, sorry, I don't remember this stuff - haven't did that in a long while". Yes, that makes me a worse programmer, I guess, but are interviewers actually looking for those who remember everything in their head and write code on whiteboard? Looks like a weird case to me that doesn't have to do anything with reality.

Last year our lead tech interviewed a guy that couldn't even conceptually get FizzBuzz...

It took me 5 minutes on my laptop at home to code out a problem that I danced around for an hour on a whiteboard being stared down at by 2 engineers in silence. Yes, I had the benefit of hindsight, but unless you've actually done a whiteboard interview yourself where your livelihood is on the line for an hour, don't act like a trivialising nonce.

> I do use the whiteboard for trivial CS questions limited to 5-10 minutes. Think fizzbuzz and string reversal.

Why do people keep saying string reversal is easy? Text is one of the hardest things out there. Unless you expect your programmers to support ASCII only?

Why not reverse a list or an array? That does sound like something that is doable in 5-10 minutes.

This is why I like the question. As an interviewer, I will leave that matter open and see if the candidate asks for clarification. If so, bonus! But I will tell them to just solve for ASCII.

The worst-case scenario is a "self-hazing" where someone sees the encoding difficulties and freezes up instead of asking for clarification. I've never seen that happen though, in my experience though, everyone -- even the good ones -- just assume ASCII.

And I am happy enough if they do it correctly.

Thank you. A great response to "reverse a string" is "What's the encoding?" If it's anything but ASCII, you're in for a long white boarding session. If the interviewer doesn't know what you're talking about, back away slowly......

Even in ASCII you have multibyte sequences to worry about: what's the reverse of CRLF?

what makes other encodings hard ? The two things that come to my mind are byte length and comparison function. If the encoding had a fixed-length byte length, then it should be just swapping n-bytes at a time instead of 1-byte. What else is difficult about non-ascii encodings ?

e.g. in UTF-8 a codepoint is encoded in varying byte lengths (so you have to split into codepoints and then reverse), and, a lot more difficult, a sequence of multiple codepoints can be combined to form a symbol. Simplest case would be something like "ö" encoded as "o" (U+006F) followed by a combining diaeresis (U+0308).

Other fun special cases: 🇺🇸 is U+1F1FA REGIONAL INDICATOR SYMBOL LETTER U, followed by U+1F1F8 REGIONAL INDICATOR SYMBOL LETTER S and should if possible be displayed as a US flag (otherwise falls back to text "US"), should reversing it create 🇸🇺 (replacing the flag with the characters "SU"), or still show the flag? (I'm not even sure if there isn't a case where both are valid country codes and it would change to a different flag?)

Similarly, Emoji can be formed from a sequence with combining characters inbetween, which don't display correctly if reversed codepoint by codepoint.

Some examples: If you're dealing with UTF-8, which is very common, you need to handle variable-length characters. If you're working with UTF-16 you need to handle surrogate pairs. Neither are the end of the world, but the basic "array walking" string reversal methods you'd expect from a white boarding session wouldn't work.

Well, OK, if you are using C or something, and traversing through binary it is hard.

But if you are in python, and given a string object, and told to reverse it without using the reverse method.... Yeah, that's easy. You should be able to do that.

People are usually expecting the latter.

What is the field you hire coders for? Because for the mine (iOS) your ability to code fizzbuzz have little relevance. The knowledge of APIs, common patterns, ObjC vs. Swift, common means to handle dependencies, etc. etc. is impossible(?) to fake if you do not have relevant experience you claim you do.

Sure but if you cant program fizz buzz, that means you don't know what an if statement is.

Fizz buzz tests for "does this person know what a function is, what an if statement is, and the modulo operator". And frankly, I'd give the modulo answer away for free if the candidate asked.

All programmers should know what a function and an if statement is.

As an iOS developer, I would still get whiteboard questions ranging from "reverse this string" to "find all the anagrams in this array and print them in this specific fashion", usually with iOS specific trivia about view controller lifecycle, UIKit (whats the diff between frame and bound?), etc. sometime before or after the whiteboarding

Interesting! I actually love using the whiteboard to sketch solutions and architectural decisions. I believe the hate comes from having to code on a whiteboard - which is justified, since no one ever coded on a whiteboard except in an interview.

Also from Europe, and we have a similar approach, discussion with the candidate about previous work, interesting problems, any public code they've written, etc. Just to figure out what kind of challenges excite them, and get to know them better.

We also do a coding challenge after that, basically the candidate behind a computer, with all the resources they want accessible to them, implementing a simple example directly relevant to the type of work they'll be doing, while the interviewers are in the room and debate with them if needed.

To me, the most important quality in a candidate, once we determine that the basic skill level is good enough, is their ability to learn, quickly find the resources needed, and rely on the team when they need help. Whiteboard interviews don't seem to select for any of those qualities.

> found it best to have a guided interview about the candidate's work experience and interesting problems that she/he solved.

This. I advocate this method of interviewing here in the states every chance I get. The most common form of push back I get is from hiring managers terrified of being hoodwinked into making a bad hire. It's like they don't trust their own judgement enough to be able to tell apart those who know what they're talking about from those who just talk a good game.

> I'm always surprised how many candidates cannot even point to one problem they worked on they found interesting or one solution that they're proud of

Junior engineers, I assume? I had problems with those questions when I first started out. It's hard to say with a straight face that the thing I was most proud of at that point in my career was creating a very simple templating system in PHP. Of course, the interviewers rolling their eyes and saying 'is that all?' didn't help.

> It's like they don't trust their own judgement enough to be able to tell apart those who know what they're talking about from those who just talk a good game.

Which is usually the case in larger companies, I assume. The person doing the hiring in that case is far removed from the people and work actually connected with the candidate.

> Junior engineers, I assume?

You're right, that's surely part of it. Not only, however - sometimes you get the feeling that work is something that happens to people, not something they seek out and try to get better at. Which for many positions is completely fine.

> Of course, the interviewers rolling their eyes and saying 'is that all?' didn't help.

That says a lot about their maturity, not yours.

Generally speaking, someone's ability to recall things is very dependent on how they're feeling. If you're happy, it's easy to recall positive things, if you're depressed it's practically impossible.

A lot of developers also have imposter syndrome. After I've solved an interesting problem, within a few days i'll probably have dismissed it as not being a big deal and started to forget about it.

These kind of questions are useful, but candidates should be preparing their talking points, because the risk of forgetting the story that's going to get you hired is too high

Yes. I read about this idea a few years ago (tell a story about a project you liked) so I had to make an effort to focus on ONE story and tell it well. I've been doing it a couple dozen times so far (I'm a contractor who likes small projects, six months is starting to get too long) and I've had significant success with it.

(When I think about the problem "objectively", all I did was use SSDT to simplify deploying database changes from QA to production without having to shut down either. No big deal. However, it was something that the company had been doing manually for years, spending three error-prone days each month, and they were extremely happy with my solution.)

So true.. and this is a great interviewing technique. I basically ask one question during an interview: what was the hardest problem you ever had to solve? It may sound stupid at first glance, but it gives you insight into not only the breadth of their technical knowledge (since that problem may not be germane to the position you're hiring for), but also the depth of their abilities. The way they answer this question reveals quite a bit about the person. Not only technical, but also their interpersonal abilities.. how they manage their place on a team, how they manage expectations up and below, etc. I can glean quite a bit about the person based on how they answer this one question. Only asking questions about your particular domain doesn't necessarily indicate how good they'll be at solving those types of problems, only how much they happen to know about them. Of course, you can say that perhaps they shouldn't be interviewing for a position they may not be qualified for, but frankly the best people I've ever hired had very little experience in problem sets that were specific to my day-to-day responsibilities, but delving into details about things that they've done told me a great deal about how they solve anything. It may not be the best way to hire, because it requires a breadth of technical knowledge of the interviewer, but it definitely has worked for me. Asking someone to solve a ridiculous puzzle on a whiteboard..when they're obviously nervous about being there in the first place seems kind of dumb to me. However, digging into a problem that they've solved immediately puts them at ease.

I'm not a huge fan of this question. I've been programming for 20 years (professionally for 13). Is the hardest problem some noddy maths i worked out at 15 to convert real coordinates to screen coordinates? Is it the first app i wrote out of college when i learned that servlets aren't thread safe? Is it what i worked on last year to design my current company's authentication/authorization system?

I find it very hard to interpolate between these examples. And none of them are "hard"! I haven't written an OS or a more efficient linked list. All I've done is plugged away at something until it works.

"found it best to have a guided interview about the candidate's work experience and interesting problems that she/he solved. "

When I've been job hunting in the past I've also found that these have been the best situations for me. I find coding questions beyond fizbuzz totally silly and when something more is requested of me my attitude shows this.

We are currently hiring. Denmark. We received i think few hundreds applications, picked 20, discussed internally, picked 5. Those 5 received invitation for homework, simple functional Javascript exercise. We dropped 2, the rest received invitation for UI Javascript exercise, vanilla btw. Then we talked with them. Picked one. In my previous gig in UK we had 3 homeworks and then chat. Sometimes we did drop the third when candidate was exceptional. So far only one candidate refused homework next stage, but the first stage result was bad enough for us to not consider him anyway. All top candidates and hires were eager to do the homework.

Hi JonasVP, a little late but regarding your original question...

We were building a tech team at a start-up in Germany.

Our hiring process evolved and stabilized into the following process, which worked exceptionally well for us: 1. Review of written documents, accepting any kind, any style and any medium. 2. Telephone interview with recruiter from HR about motivation, personality and the formal aspects of education and experience. 3. Personal interview with VP of engineering. Future employee is asked to bring some of his or her source code to the interview, any language any style any form is good. We don't copy the code, we don't keep it, we don't use it in any way. This is the only technical interview. 4. Interview with the CTO, immediately following. This is the most important social interview, and it also contains the main salary negotiation. 5. Decision.

Between any of the steps, we would briefly exchange our impression of the candidate, and it turned out that it is wise to sleep a night before taking the decision.

My function was VP of engineering and lead architect, and I would conduct the interviews of step 3. My principles are as follows: - Keep the candidate at ease as much as possible. We want developers to crack hard problems with "feet on the table" as the Dutch say, most of the time. Stress distorts things, and I want to see the undistorted version. - Get to know the candidate, and what makes him or her tick. This requires respect, tact and genuine interest in the person and his or her achievements. - Estimate the fit with team and company. I can talk a lot about this---but in the end it either feels right or it does not. - Gauge the technical capabilities of the candidate. For this I let the candidate choose a piece of the source code at will and have her explain what it does, and why it is the way it is. For all our hires, some form of dialog on software architecture in general emerged. This usually takes between 10-30 minutes. - Do your best to get the candidate interested in the company. For this I do not advertise the benefits of the company but simply explain what the company does, what the software we create does, which languages and systems we use and maybe even demo some of it live. Also I explain the risks of working for a start-up company to the less experienced candidates.

I don't do programming exams of any kind. Interactive exams exhibit dangerous positive feedback: you start with something in the middle. If the answer is good you make it more difficult. If the answer is bad you make it easier. Repeating this quickly converges to either extreme, which means you missed the chance to learn something more profound about the candidate, and even risk loosing suitable candidates at split-second blackouts. I don't like take-home exams either, although they are all the buzz these days. A take-home exam is a large investment for the candidate, and most of the people I hired never even touched the programming systems we were using (Erlang, Python, Ocaml, GLPK to name few). All hires did exceptionally well after a few weeks, and with a little bit of guidance. So what could a take-home exam tell me? Even worse, take-home exams are supposed to reduce the risk for the company, at the expense of the candidate. But do they actually do that? At what price? In addition, the large German probationary periods provide an easy way to correct mistakes later, which I had to do only twice so far.

> one solution that they're proud of.

Pride is a very strong word, especially for those of us with impostor syndrome...

The things I'm proudest about are the things I should probably not be proud of at all as a responsible professional.

> That said, I'm always surprised how many candidates cannot even point to one problem they worked on they found interesting or one solution that they're proud of.

I have trouble recalling the details of things I worked on even say a week ago sometimes. I also don't find most of the work I do that interesting as I've been coding along time solving the same kinds of problems. However to play the game i would of course brush up for questions like that, and be quite convincing.

> I have trouble recalling the details of things I worked on even say a week ago sometimes.

No problem here, it probably wasn't very exciting but have you never encountered something which stuck in your mind? Or that you could at least recall some part of a project if someone asked "So, you've written here that you've worked on project xy and did this and that, can you tell us a bit more about it"?

Asking about project xy is different though, it's more guided. I think it's much easier to answer than the really open ended ones that ask you to pick an incident from all 10-15 years of your work history: the field there is so wide that unless I've specifically considered the question in advance I'm likely to sit there going ummm for a bit.

In general I think the question format favours people with a good memory for anecdotes and story telling ability.

Interesting. I've done a fair number of interviews for my last few companies, and while I've found that whiteboarding isn't terribly useful, likewise I've found that I really do need people to do some coding. I've definitely encountered people that talk a great story but don't have the chops behind a keyboard.

My usual process (which isn't to say it's perfect, but it's a process I regularly edit to address problems I see) is to give a few, simple problems, and have them actually code it on a system.

* I know that the computer and environment are unfamiliar to you, so I expect there to be chopiness and typos * I know that most people are nervous during interviews, so I don't fail people if they freeze on one of the questions or miss a concept * I don't give "trivia" questions - the problems tend to be fairly easy ones that cover your ability to approach basic problems. Example: "here is a nested data structure (like an array of objects). Write a method to pull these sorts of elements out." * I'm actually hoping you'll screw up - I'm far more interested in seeing how you deal with a bug than if you can dash out an algorithm flawlessly. Nonetheless, I keep the questions simple because I want to minimize the impact of nervousness * I inform people they are absolutely allowed to use Google, StackOverflow, etc - I'm trying to mimic the actual work experience as much as I can. I WILL judge people on their searches, but I'm pretty loose - I can only recall one person that I dinged for searching, and that's because they went to about.com and copied the answer there without trying to understand it (and it subsequently didn't work). Usually this ends up GIVING people points, because if they demonstrate comfort with finding solutions, I expect that we can hire them and know they'll improve over time.

Despite this, I still ask a few questions that are purely verbal. I want to see if you are someone that will force your preferences on others or are willing to bend. (I'm not looking for a doormat, but I've never met one, so that's a bit moot) I want to see that you are continuing to learn things, because the skills you have now will just not be the skills we need in a year or two.

but overall - I have to judge candidates based on the limited info I can glean in an interview. Of: * resume * whiteboarding skills * discussion skills * coding skills

...I find the latter to be the best measurement of the options, even allowing that it won't be fully accurate.

Imagine this. You are interviewing for a job, you walk in a room with 2 people who hand you over a sheet with a few problems. They ask you to write the solutions on the whiteboard, while they wait for you to complete.

Not a word is said, they are clicking at their laptops, and staring at the whiteboard, as waiting for the genie to pop out of a bottle. All the while your mind is frozen and stuck in a bad loop.

This lasts an hour, you are barely able to complete parts of the problems and are frozen. Of course this affects your usually creative and sharp mind.

The torture lasts an hour, time's up! You shake their hands, as a kiss of death, and head out. As you are walking back, all the answers to all the problems they wanted you to whiteboard, come rushing like a torrent in your mind. Too bad, another 'botched' technical interview.

This is my experience as a battle tested developer who is shipped many products and has been programming for the love of computers since the age of 12 (professionally for more than 15 years). I am not going to be working at Google any time soon (not that a Google job really matters to me).

It is also a cultural thing.

I observe that anglo-saxon peoples teach their kids to voice their concerns, needs, wants, and opinions, and to expect (!) results from this. (Compare how american kids behave at airports, train station, and public places with kids from other countries, e.g. french speaking countries).

This helps them approach situations in life with a healthy dose of self-worth and the baseline state of mind that if something is wrong it's not their fault.

In contrast, many other cultures teach their kids that it is impolite/crude/disrespectful to bother adults with your half-baked thought, or (God forbid) unexpected wants, or complaints.

I come from a culture where you are expected to moderate your verbal output when addressing your superiors (e.g. teachers, boss, etc.). You speak out only when you have something worthy to say.


I had trouble going to "office-hours" when at university in North-America. I always got the feeling that I am bothering the prof, and it is my duty to study more to understand the material, and I should not take his time when I merely did not understand something.

My anglo colleagues seem to not have such qualms. Their thinking is more along the lines of "I don't understand (not my fault), it is his job to teach me (most likely his problem), he has to explain this better, I pay for tuition after all...".

I am happy to have unlearned some of the above things.

But generally, I find the speak-your-mind-at-the-white-board-while-we-watch interview tailored for cultures where kids are taught to be vocal, and are difficult for equally intelligent people from other backgrounds.

What you're describing is Power Distance.

On the whole the West tends to have lower power distance than the East. And Anglo and Germanic countries tend to have lower power distance than Romantic countries, while Scandinavia has the lowest power distance anywhere.

France has quite a high power distance by Western standards, although it's still lower than most Eastern countries.

The US tends to have rather high power distance by Anglo/Germanic standards although if you look at all the cultural dimensions the US tends to be an outlier towards moderation in its cohort probably due to its amalgamated culture (the US is hardly a strictly Anglo country).


I recently started teaching a class to grad students, most of whom are foreign-born (China and India mainly). I have a lot more sympathy for my college instructors now that I see it from the other side: If students don't speak, I have very little information on whether they are learning the info (which in turn, reflects how well I'm teaching it). Homework is my main clue, and that involves a delay (plus I have a healthy bias against busy-work homework, so while I give assignments it does not count for/against your final grade).

I believe this is exacerbated by the sorts of cultural differences you highlight - I'm (relatively) loud and brash, many of them are reserved. I have no doubt there are lots of signals of confusion/understanding that I'm missing, but I don't know how to find them either.

I recently got my mid-term evaluations from the students (anonymous) - I was shocked to see many of them spent (or at least claimed, which I have to trust in absence of other data) 11+ hours/week on the work outside of class - triple what I expected. (and the answers formed a believable curve of results, so again, I have to trust it absent other data). I had only one student up until that point say how much time they were spending. Everyone else was silent.

People are downvoting this, but the difference in cultures is real (though I wouldn't say it is confined to anglos). But udev might be exposing a case where the "anglo" culture has a genuine advantage.

Engineers really should communicate clearly about technical stuff. People solving complex problems really should concepualise their approaches clearly, ideally so that can be expressed in words. People really should reach out for help when blocked.

And if Anglos find it easier to do those things, then good on 'em.

I don't think it's just a communication thing. It's also a confidence thing. When a culture places emphasis on self-censoring behavior, then the candidate is less likely to express themselves for fear of censure.

As someone of East-Asian decent, I totally get this. But I also can understand the counter-point: if you are joining a company of predominantly "anglo-saxon" peoples, does that company not have a valid concern that the people they are hiring are conducive with their current work process (i.e. communication, collaboration, etc.).

I understand your comment isn't advocating for what I am insinuating, but in general, I think the sentiment that companies need to adjust their culture for their candidates is problematic.

What's the best way to do hiring in your culture?

There's a lot of variation within Anglo-Saxon cultures too. In New Zealand, we're usually a bit more reserved, but I can see that we're more forward than in many Asian and Pacific cultures. Also people everywhere have variation in introversion/extraversion and in their self-confidence.

It would be great if there was a way to do hiring that could identify good candidates without bias among these different things. Probably the only good way to hire is to simulate the real work environment as closely as possible. That's difficult and expensive though.

In my opinion, the best hiring interviews for tech positions should include the type of interactions that will actually take place in said job, but also resemble the usual way in which people in the given culture do intellectual work (e.g. for someone out of university, that would be the way he would usually sit an exam).

The best I can think of is to have a 45-60min written test with technical questions/problems going from very easy (where failure to solve is a red flag) all the way to more moderate, some difficult, and maybe even one very hard problem (solving which is a green (?) flag).

No need to be in the room with the person. Can put a webcam to record test if cheating is a concern.

After the test, the candidate gets a 5-10 min break, a drink, while the hiring team of 2-3 people take a quick look over the result of the test.

In the second part of the interview they all go into a room to have a chat based on the problems in the test. The candidate gets a chance to defend his choices, show additional skills, while the hiring team gets to ask tangent questions (how would you speed this up? why this data structure? if space was at a premium how would you improve this?).

I think the secret is to adapt the tone of the discussion to keep the candidate on the edge but avoid intimidation. If the candidate is a confidence artist, test him with more and more direct questions that he cannot weasel out of. If the candidate is more shy, engage him/her with a brainstorming like discussion, and go of on tangents from there.

The written test plus the opinions/notes of the 2-3 people present at the discussion should paint a decently detailed picture of the candidate, and also provide a written record of his/her performance for reference and comparison with other candidates.

That sounds better than an interview, but a written test is not representative of many types of work. In most kinds of work you'd have days to work on a topic, access to reference materials, the ability to discuss with colleagues and ask for help, etc.

Test results probably do correlate with job performance, and it may be better than an interview (not sure) but it's say it's still quite a long way from giving an accurate answer.

Have you ever really asked them what their line of thinking is? You are like to get many different answers.

I sought help from my professors and teaching assistants in college not because I felt it "wasn't my fault" or even that "it is his job to teach me" or that "I pay for tuition after all" but rather my thinking was, quite simply, "I need to learn this and I'm finding challenges with the current approach I am taking... maybe there is another way" where "another way" just so happens to include pursuing some direct face-to-face help from someone else. The line of thinking was outcome-oriented (I need to learn this/accomplish X) and I'd do whatever it took to make that happen.

Whiteboard interviews should be re-framed. Instead of "what is the best method to sort through a list?", set a situation where two co-workers are having a casual chat. "I'm having this problem, I can't sort through this list, any ideas what I should do?"

That kind of discussion between co-workers happens all the time. You suggest ideas. Have you tried this? Yes, but it has that problem. Oh, so what about that other approach? And so on.

It's a lot less stressful than an interview where you are expected to find the right answer, and it's also much closer to your real job.

When you get stuck, start talking. You should not be silent at the whiteboard trying to flash out a complete solution all at once. Talk. Even if interviewers are not saying anything or ask you any more questions. There are multiple advantages of talking while writing code at the whiteboard. Firstly, you show the interviewers the way you think and approach the problem. It is as important as actually solving the problem. Secondly, if you get stuck or start making unreasonable assumptions about the problem, the interviewers will have a chance to quickly correct you or give you a hint. Thirdly, it shows that you can communicate. Nobody wants to hire a person who can't communicate with other people.

I don't find talking and thinking at the same time natural. 'Just think out loud' seems so unnatural to me, I never do it in daily life. I am used to thinking to deep house music not to interviewers staring at me like hungry wolves. Its hard to communicate with someone who already knows the solution to question because you are thinking together and feeding off of each others thoughts like in a normal work situation.

I think the idea of modern open office is built on the same idea that people would be thinking, communicating and collaborating at the same time, but I hate open offices for this precise reason.

It's a band-aid for an unfixed root cause. Solving problems by talking doesn't work for all people. It astonishes me how broken interview processes are. They just need to see if the candidate will be able to do the daily job of a current employee in that position. It's that simple. It doesn't require whiteboards, puzzles, 10 interviews. Put the candidate to work on a project that you currently facing and see his performance. Doing something other than that is bullshit.

An interview process is inherently asymmetrical- the evaluated is in a vulnerable position. So it feels like Mark Twain's not-quote([0]) would apply - "It’s better to keep your mouth shut and appear stupid than open it and remove all doubt."

What I'm saying is that there's an underlying fear of looking bad when spouting off your mental rough drafts, during an interview. What would be helpful if interviewers were clear about their expectations, as in, "You will be graded on your overall approach, not your initial ideas." or "We don't take points off for hints."

[0] http://quoteinvestigator.com/2010/05/17/remain-silent

In my experience, as an introvert and an ESL (even after living in North America for 20+ year and been educated here from high school onward), the need to verbalize thoughts actually compound on the stress more. I understand why I should be talking (I am sure all your points are valid) but I just can't help it. I'd say the only comparable level of stress and anxiety is dating.

I think this is generally good advice but it's possible to overdo it. I noticed on my last round of interviews that I was vocalizing my internal thoughts so much that it was hard to concentrate and think deeply. I'm still working on finding a good middle ground.

Devil's advocate:

How else will you prove that YOU shipped those things over that period?

* Github account? Ha, my code is on github in many places, mostly without attribution, none of them uploaded by me.

* Recommendations? Ha. Unless the person recommending you already works at the company, their trustworthiness is zero - they could be a paid accomplice.

* Resume? Unless you were the sole employee, there is no way to tell how much actual work YOU did.

* "Tell me about that one time you solved an interesting problem?" Ha. Can easily be a well-practiced lie.

(yes i have seen all of the above)

Turns out that the only sure way to tell if you can do X, is to make you do X and watch you yourself actually do it, in person.

Now, I am not accusing YOU of fraudulently trying to get a job, but people do. An in person interview is a good way to see.

(as always, i speak for myself only, not my employer, my cat, or our Great Lord FSM)

I also was in charge of hiring programmers at some point in my career. I made it a point to give programmers a simple project to take home and get back when they where ready (at most a week). The project was very close to what the hired candidate would be doing if we chose them.

I told the candidates that they did not necessarily needed to show me a working application (although that would have been great!), but simply to document what architecture, language and approaches they would use to implement the task.

Also the task could be documented in a few hours, and those that got close to coding a few classes might have spent a day at it. Now the argument could be made that I did not remunerate this people, but keep in mind the work they did was not expected to be more than a few hours.

For me, this is an ideal technical interview. Also, I ended hiring a candidate though this method, and he is by far the most productive and creative force in our team.

And how do you know that they did not farm it out to rentacoder.com?

Remember, I did not ask for an implementation. Only a document with their thought process on implementing it. Also, they can use whatever means they like, if we hire them we will quickly find out if they are incompetent. No interview system is fool proof, of course.

How fast could you then fire them? How much money wasted on that and onboarding? Compared to the risk of missing out on that one person who cannot whiteboard but is otherwise brilliant? Quite a gamble.

"Tell me about that one time you solved an interesting problem?"

I agree if that was the only thing you asked, you wouldn't get a good answer. However, if you then dug into their answer you can probably tell if it is BS or not.

I remember having this exact experience with a junior dev who claimed to worked on some interesting projects at Uni, but when you dug deeper, you realised he knew very little about the technology he supposedly used.

That said, my company isn't really interested in hiring people who can code an algorithm to balance b-trees. My company just need people who know how to ask questions, write maintainable code, and use a profiler.

Worked with a guy once who was a pro at this. Every day he did nothing, but when higher ups asked the team what was done and how, he managed to take credit and explain how things worked well enough to convince them that he had done it all. The rest of the team was not amused. Just listening in on a team (while adding nothing to the code or discussion) might be enough to go into great depth in such a discussion.

What makes you think a bullshit artist that good a) doesn't actually know his shit well enough and is just lazy and b) won't pass your technical interview by either actually knowing enough or absorbing enough test prep material?

Also, how common do you think these people are? IMO, not common enough to design an interview process around. Then again, I don't subscribe to the "one bad apple will torch the orchard" paranoia popular in this industry.

Give each team member a different assignment - this should avoid the mentioned problem.

> "Turns out that the only sure way to tell if you can do X, is to make you do X and watch you yourself actually do it, in person"

I don't write code on whiteboards for a living, nor am I applying for a job that will have me coding on whiteboards, so watching me do that will tell you almost nothing about the job you're actually hiring me for.

I agree it is not ideal, but the counterexamples I provided do explain why other suggested methods are too simple to fool.

Disclaimer: I work at Google but in no way do I represent my employer's views. These are all my personal opinions.

I interned successfully at FB, Microsoft and Google as a SWE in 2008-2010 and interviewed with a lot of other tech companies (not always but often successfully). Like you, I tend to freeze up in a whiteboard situation, and have most of my life, going back to college and prior (I studied in Romania where at the time you had to stand up and answer questions and use a whiteboard to answer them). I find that a few things help:

1. Practice practice practice. Simulate the interview with a few friends multiple times. It gets easier no matter how shy you are.

2. Smart interviewers and companies open with easy, friendly questions that put the interviewee at ease. It's a sign they understand this is stressful for everyone, no matter how confident, knowledgeable and smart they are.

While these interviews aren't perfect, with the right attitude they are helpful: it shows how the person thinks through a problem and communicates their ideas, and how they "work together" with the interviewer.

These are important skills that are important for all people regardless of their title.

I also believe a solid CS foundation is important even if coding is mostly putting things together. A systematic, mathematical understanding of complexity of Algorithms and Data Structures shows in one's everyday decisions.

There are only a few valid alternatives in my mind: 1. Take-home project work More realistic assessment of how effective a person is. Really good simulation of that, but no signal about their collaboration style, interpersonal dynamics etc. 2. On-site trial for a week or two This is probably the best way to assess someone's ability - internships and residencies are as close to real life as one can get. It's hard for people who already have a job, and expensive for a company to do. Simpler proxies/filters are needed.

Am I missing any other ways to assess candidates?

>> Am I missing any other ways to assess candidates?

Yes. You are completely glossing over the fact that the candidate might be bringing something special to the table.

You have literally not left any room for the candidate to showcase what they've learned (again, not minutia of algorithms but something completely different..like the candidate could be a specialist in how compilers do register allocation)

How about the interviewer improving their interviewing skills to be able to identify strengths rather than being a pseudo automata that docks points?

Of course computer architecture, compiler knowledge etc is valuable and part of the interview process. I was referring to the general process of evaluating- - thus the question was focused on methods not content.

Additionally, explaining how a system works is valuable, and I captured it when mentioned communication abilities.

>> Of course computer architecture, compiler knowledge etc is valuable and part of the interview process. I was referring to the general process of evaluating- - thus the question was focused on methods not content.

The general process is fucked up because the interviewers are incompetent enough that they can't tell who is actually a smart guy. This is why incompetent interviewers resort to binary decision making on programming questions.

Here's a way to think about it. Let's take two candidates with equal communication skills:

1. This candidate solves all coding problems fast, with great syntax, good style and discusses algorithmic complexities well

2. This candidate is smart, can go deep in their specialty (because they've spent years developing complex expertise) but couldn't do a medium-level tree traversal without hints or has bad coding style (on whiteboard)

Who do you see getting picked (at Google)?

I remember a friend of mine at Google (a Java programmer) docked points off a candidate (a C programmer) because the candidate passed length of array as an argument. I was like wtf? Seriously? You don't know C?

The data-driven committee-based process completely ignores that not interviewers are equal. Their feedbacks on paper are not equal.

There's a reason scouting still exists in Baseball/Soccer/Football because data-driven process has given mediocre results at best. Interviewing is much like scouting. You cannot use mediocre engineers to identify who's a real talent.

The process is not designed to find "real talent."

"Real talent" is innate and thus no amount of outside valuation will ever do it justice. No one else, but the "real talent" holder, can effectively judge his "real talent" so it is up to him to make it known to others that he does, in-fact, hold "real talent."

The process is designed to find people that are good enough to do an array of jobs. "Real talent" gets scouted, because it's shown that it "has got the right stuff."

This argument boils down similarly to the "smart but lazy" exasperations. "I'm talented, but no one realizes it!"

Candidate 1 is getting the job at my startup and Candidate 2 will be getting a referral if he ever decides to take up a teaching position. Deep specialization is useless unless you're doing consulting (in-house and out-house) or teaching.

Like I said I speak for myself not Google.

They're both valuable, perhaps for different types of work.

A balanced tree does not tell me much about a person's knowledge but how they are able to apply and learn CS concepts does. Deep knowledge of a problem domain is valuable, but also highly perishable - the world changes fast and good engineers need to learn and adapt. Furthermore, if a candidate speaks about a domain they have deep expertise in, we have two more problems: 1. The interviewer might not know enough about the field to assess the candidate. (Generic CS algs and data structures are a good common denominator). 2. A candidate's knowledge about a field doesn't tell me how good they are at reasoning. They may have spent 5 years doing that one thing - that's enough time to learn almost anything in a limited domain - say OS schedulers.

How would you structure an interview process?

Good arguments and we're getting there. You seem open minded unlike others and so I'd like you to answer the following:

-- Furthermore, if a candidate speaks about a domain they have deep expertise in, we have two more problems: 1. The interviewer might not know enough about the field to assess the candidate. (Generic CS algs and data structures are a good common denominator).

So the problem is with the interviewer and not the candidate right? The interviewer is unqualified to interview this person

2. A candidate's knowledge about a field doesn't tell me how good they are at reasoning. They may have spent 5 years doing that one thing - that's enough time to learn almost anything in a limited domain - say OS schedulers.

Couldn't the same be said of candidates good at whiteboard tiny algorithm puzzle solving? They may have spent 5 years doing that one thing - say algorithmic puzzle solving. The fact that books and coaching sessions industry exists should prove this right? Your process doesn't give as much weight to someone who has the passion to go deep in something productive as much as passion to solve toy algo puzzles

That's nice of you to say, though I'm not sure I earned your compliment.

Re experience - yes, if you're interviewing for someone with deep Java or ML knowledge certainly make sure the interviewer can assess that.

Logic puzzles are not perfect - I used to be really good at solving them and if I ever have to interview on them again, I'd definitely have to practice first, to get up to speed. But I do think that being able to solve them is a meta skill. If you get really good it entails that you are good at reasoning and solving complex problems, or at least a specific class. If you are good with trees you'll likely be able to answer a trie question. It doesn't mean you're just good at that, but that you can apply knowledge to new problems - which is exactly why you are hired. An equivalent real question I got in Java was how to access a private member of a class. If I knew the answer it would mean I have some random bit of trivia in my head. If I solve it using Java knowledge it means I can apply my domain expertise in a new way - that's compelling proof of problem solving skills. If I ask why would you want to do that, it indicates reflective and critical thinking skills.

A seasoned engineer in ML or Java should, imho also be able to solve puzzles. They are, in many ways, better proof of abstract thinking abilities and problem solving skills than specific knowledge. But I agree they are both important - if I want to build a search engine I prefer to have a seasoned ElasticSearch lady lead the project than an equally smart person with no background.

Re your friend, if it helps, I interviewed at a lot of companies and probably failed about 70% of interviews in the first week or two. I got much better after a few weeks, which proves your point that interviewing is also a skill. And I mean for both tech and behavior questions. That's normal, I think.

Re Google, while I don't represent the company, there are public statements that they'd rather have a false negative rate rather than false positives. It's better to say no to a great applicant (politely) than yes to a bad one. If you've ever had to let someone go, or deal with a bad team member, you know what I'm talking about. Of course there's the occasional interviewer who asks a silly question or gets something wrong. Do you never make mistakes? It's impossible to ensure everything goes perfectly in such a large company, but I can tell you, as someone who works here, that people try really hard to do a good job at it, and leave everyone with a good impression. It's the rational thing to do: if you treat people poorly 1. You're and asshole and 2. They'll hate you. Nobody likes assholes especially at Google, and I can share that I haven't met one yet. That's pretty amazing given the size. Thanks for engaging in this discussion, I really enjoy it.

Great reply. And I'd like to refute them :)

Let me foreword by saying that I am not against coding interviews in general. But, I am against the interview process as it is implemented NOW. As implemented today, the process is overweight on coding optimal solution with good coding style.

Translated to humanese: "It doesn't matter if you invented linux kernel at your last job. We can't hire you because you didn't solve this tree problem in time or had bad coding style as COMPARED TO OTHERS who don't know shit but can code puzzles on whiteboard"

What I think an ideal interview score looks like:

Score = w0(whiteboard-coding skills) + w1(algorithms) + w2(area of expertise) + w3(design) + w4*(lets work together to invent something new/passion for engineering)

As it stands, the interview process has the following weights: w0=49%, w1=49%, w2=1%, w3=1%, w4=chunk change

See what I did there? The way interviewers in these companies are hiring is by calibrating their favorite coding puzzle over a wide range of interview candidates. They simply don't account for the fact that that person might genuinely be good at something that the interviewer doesn't even know exists.

>> A seasoned engineer in ML or Java should, imho also be able to solve puzzles. They are, in many ways, better proof of abstract thinking abilities and problem solving skills than specific knowledge.

You keep asking for proof that the candidate should be awesome. Where's the proof that the interviewer can adequately judge based on the equation I gave above? The candidate could be seriously good at ML but if you send the average java coder at Google to interview them then gosh, how would they even recognize his genius? Now you'll say we have 5 rounds to eliminate biases. Buuutt, the way the hiring committee reads interview packet is, and I quote you:

"It's better to say no to a great applicant (politely) than yes to a bad one"

Which means, even if two interviewers think this awesome ML guy is GREAT, the candidate wouldn't be hired because the other 3 interviewers were incompetent and marked the candidate no hire.

What I'm trying to say is that the hiring committee based process encourages actively removing smart people and selecting the ones who solve whiteboard puzzles. The candidate could be a shit engineer in person but as long as they solve code puzzles in those 5 hours they get hired. Make sense?

Look, I'm not saying we need to eliminate coding rounds. All I'm saying is balance it out. A person who can solve a tree problem in one round surely doesn't need to be asked another stupid coding problem (which is asked just to make sure some random crackpot didn't squeeze in through the cracks). I mean, if you don't have confidence in your own interviewers that you need to keep asking those problems, what does it say about the process? That you don't trust your own interviewers? Isn't that the real problem?

You can choose to ignore all feedback I give here but then, I'd specifically call Google out and say that they're not a great engineering company until they figure out that their hiring is broken and they need to stop having their mediocre engineers keep rejecting the good ones.

Or...if you really care about working with great peers, show this answer to hiring committee and see what they think. They need someone to tell them, "Hey, why are you implementing a process that self selects mediocrity"

P.S: Why do I know it is mediocre?

1. Other companies have tried the same process and it didn't translate to anything

2. You know there are enough mediocre engineers at your workplace :)

One more thing: the interview process is as much about assessing team dynamics as skill/ability. I wouldn't hire Linus at my own company because he's kind of an asshole. I could be wrong but this is the perception I have from interviews etc. I love his work but can't say I'd like to work on a team that shares his style. It's important to stress that I never talked to him I just saw some interviews. His emails actually seem very reasonable and thoughtful, and he comes across as a regular person.

I agree with your general approach to a more holistic interview process. I haven't thought too much about the weights so can't comment there.

I'm pretty sure the weights vary depending on roles: interviewing a tech lead for ML requires deep experience in That field and will have more of those questions than basic CS algs. Or at least a higher weight than it would in a fresh graduate.

It's not my place to judge the average ability of Google engineers. I'd guess it's a normal curve somewhat skewed to the right since the bar is really high. In my limited experience they're on par with top silicon valley engineers but have a different profile than startup ones due to self-selection.

The point of many interviews is to reduce false positives, both for skill and professionalism. It works amazingly well. Every single person I talk to has something interesting to say and is at the very least competent at what they are doing. Not everyone feels brilliant but everyone does seem bright. And I'm not drinking the cool aid.

Hiring good people is easy if you just test for skills. It's much harder to make sure someone is a genuine person, or at least not an asshole. Google does a tremendous job here and I worked at enough startups and a few large companies to know how incredibly difficult that is.

That other companies failed trying to do this can be because of many reasons, from bad engineers to start with, lack of resources, mistakes in assessing ability or rush to get devs on the team asap. Again, not my place to comment.

Every company asking its employees to interview prospects implicitly assumed they are good, or at least competent. If they don't, they have a bigger problem to start with.

If they hired someone they already think they are good, right? And if they ask them to interview someone else they think they are able to judge, or don't understand that logic (In which case they have even bigger problems).

How does a mediocre engineer fail a stellar one? If the candidate is really better than the interviewer, won't they be able to solve anything they ask them to?

I think the reason multiple interviewers ask CS questions is to avoid the issue you mentioned earlier: asking one question the candidate just so happens to know really well. Random sampling should solve it.

I interviewed at Palantir a long time ago and was asked super hard CS questions over the phone and in the first rounds and I killed it. The last round I was tired because I hadn't slept well and tanked one problem. Just one, everything else I had solved really well. Yes I thought it was silly to reject me and wish they had given me another shot. If you are at Palantir and are asked to interview someone, think they are bad, and they get hired, it's demoralizing - you just wasted your time and the company doesn't value your opinion. Sure, you could have a score and average things but again, firing is way way worse than saying no to a good candidate who just had one bad question. And everyone I interviewed with was legit. (I had another series with them later, for a different role and got an offer :-) )

I agree that weights could be adjusted but 1. How do you know that would yield better results? (Show me the data - I don't think anyone has it) and 2. It can also be a cultural choice to bias more towards CS skills vs design or whatever else. Perhaps the company has great design patterns and can teach those skills v effectively. I'm not saying it's true, just an example. If you had a company and ran the interview process your way, I promise there will be someone just like you who vehemently disagrees with your weights :-). But if you could prove that process A vs B yields better outcomes, most companies would definitely follow suit (if they pay attention, are good at adjusting their processes etc).

I don't actually know how they read packets, I just know they work really hard to avoid false positives, and I wholeheartedly agree.

Btw I was rejected by Google twice before - resilience is part of the game ;-)

I completely froze on a phone interview once. I opened up a co-coding application and was asked to write code to balance a binary tree as a warm-up exercise and I completely froze. Couldn't back out of my head, even with some prompting from the interviewers, and I ended up ending the phone call and writing a note of apology.

The painful part is, I used to teach CS-101 and CS-102. I assigned that exact problem to my students and have graded several hundred solutions to it.

Now I warm up on lots and lots of coding problems before I start an interview process, but I always worry in the back of my head that I'll freeze again.

> I completely froze on a phone interview once.

Ah, the memories come rushing back. I once froze in an interview, and force myself to blather something inane so that the phone wouldn't be silent. It went horribly.

I called back later that day or the next (it's been a long time), and basically said "that's not me. I know what I'm doing!" We had a conversation about the subject that I botched. It went well in part because I had already accepted that I'd lost that job; I just didn't want to leave an impression anywhere that I was a loon.

I got that job.

That was a warm up question?

Had they not heard of Fizzbuzz? Or reversing a string?

I'm a senior engineer with a ton of experience, and it was an interview for a challenging role at a major; I've come to expect that sort of thing on warm-up. It's a bozo filter before the real questions, and I should have been able to do it in my sleep.

I once froze on a warm-up Fizzbuzz. Actually it was even simpler than a Fizzbuzz: print all the elements in a list.

Freezing can happen to anyone on any problem.

Sorry you had bad experience. But I'd like to note here that many technical interviews that involve white board problem solving are nothing like this. I've attended interviews from a bunch of companies that do white board interviews. In almost all of them, the whole interview process went much smoother than this. Most importantly, nothing like "Not a word is said, they are clicking at their laptops, and staring at the whiteboard, as waiting for the genie to pop out of a bottle". If that happens, they are just bad interviewers.

White board interviews are supposed to be interactive. When I solve a problem, I always verbally say whatever is going on in my mind and share it with them. This does two things : 1. Helps me focus on the next steps instead of letting my mind race through different solutions, context switch often and lose track of my own thoghts. 2. Gives feedback to the interviewer so he knows I am on the right track and if not, give me clues (with or without him wanting to) that I can pick up on and proceed further.

In the end, my white board interviews always went like a conversation, a deep discussion about a problem that involves coming up with a solution and attacking it to see its pros and cons. I have come to appreciate the process itself because it tests several things like communicating to the team, problem solving, being open to other ideas etc.

Edit : I'd also like to note here that I didn't get these skills in my first interview. My first several interviews went horribly wrong like you mentioned. It is a skill that you need to invest time in. And you have to believe that the time you invest in this will pay off eventually. It did for me in several ways.

> Not a word is said, they are clicking at their laptops, and staring at the whiteboard, as waiting for the genie to pop out of a bottle. All the while your mind is frozen and stuck in a bad loop.

I interviewed once with a couple of owners in a restaurant. They asked me something relevant, and I all but froze. I knew the subject, but it just would not reveal itself to me.

I candidly admitted "brainlock." They were very gracious about it, I soon got through that subject and we moved on. I was eventually hired.

The lesson being, for myself at least, not being able to produce "on the fly" is sometimes compounded by being afraid to show or admit a problem. Good people, however, will be forgiving and encouraging. They've been there themselves, we all have.

There have been a number of times, over the decades, that I have basically admitted ignorance or temporary brainlock, and it has always gone well, whether I was hired or not. I think, if nothing else, that it's refreshing and novel for the interviewer.

It really reduces stress if you allow yourself to be human. In turn, that's much easier if you don't put yourself in a position that "I must have this job."

I find people are quite reasonable in real life. Unlike the internet, they're not going to bite your head off.

Next time it happens, tell the truth.

"Oh, hey guys, I get really, really nervous about whiteboard coding. It's really scary for me and right now I feel like my brain has stopped and I'm going to fail miserably. Would it be OK if we do this another way? - maybe on a computer, or maybe I can just talk about it?"

If you need it, you can also say "Can I please have a 5 minute break to get my head straight? Maybe a quick walk to the bathroom and a glass of water?"

I would be utterly and completely shocked if a rational, intelligent human being denied you that. And if they did, would you want to be working for/with them anyway?

Only issue with this is that those interviewing you might not be who are hiring you or you would be working for. So maybe their lame, but your actual day to day or your actual boss are awesome. Though it's on the company for not providing a good interview experience.

> Imagine this. You are interviewing for a job, you walk in a room with 2 people who hand you over a sheet with a few problems. They ask you to write the solutions on the whiteboard, while they wait for you to complete.

Please replace these guys with HackerRank. It so much better than this.

What's with all the whiteboard backlash? I ask simple questions and expect people to be able to write code unaided to express their idea. Not "implement a linked list" or "write quicksort" but basic "You have two arrays - find if a number exists in both arrays" sort of thing, primarily to reason about runtime complexity, and to make sure they can actually write code.

If you tell a candidate to prepare, they should be able to come in and do this. If they can't write some simple code unaided, then I don't want to work with them. I was a copy-paste programmer once too.

Whiteboarding itself is a skill. What I learned the hard way:

- There is no insert. If you need to go back and insert a line of code somewhere you must either erase all the code below it to make room (then rewrite said code), or you draw a line from the insertion point to somewhere else on the white board and write the code there. For the former you waste a lot of time. For the later the code quickly becomes hard to follow.

- There is no context driven autocomplete or quick method lookup. If you spend your entire day coding in one language and have done so for the last >3 years you should be fine. If you code in multiple languages and spend time with other tasks (devops for example), you might not realize how much you rely on this.

- The white-boarding process works well (given previous two caveats) if you code in a linear style. If you code iteratively the whiteboard is not your friend.

My recommendation to someone starting the interview process. Go buy a small whiteboard and use it to code solutions to interview questions you find online.

All your points are good--and they are part of the overall problem with whiteboard coding: A whiteboard is not even close to a realistic modern development environment.

How many times, for your job, have you had to code up a solution to a problem, even a trivial one:

1. Just given to you 5 seconds ago

2. With no access to documentation (paper, textbooks, offline or online resources)

3. With no access to mentors or colleagues

4. With an editor crippled as you described

5. Have to get it right within minutes?

6. And then have to defend your code against someone who's had plenty of time to think about the problem.

I can tell you, throughout my programming career, the answer is "never".

This, to me, is the most important point. We make such a big deal about making sure our tests actually test the thing we care about; why aren't we doing the same thing with our interviewing processes?

I absolutely agree that whiteboarding is a skill that requires practice. That said, with a reasonable interviewer, the first couple things you're discussing really don't matter. The trick is to understand that no one cares if you're actually writing working code, and that the important thing is communicating your intentions and process.

> There is no insert.

This really isn't a problem. If you need to insert a line, in most cases you can just draw an arrow and then describe what you forgot to do there. Code may not even be necessary. Anyway, an arrow is fine.

> There is no context driven autocomplete or quick method lookup

Again, not really a problem. Just say "I forget the method name here, I'm just going to assume it's called 'append'" (or some other appropriate name). No reasonable interviewer cares in the slightest as long as it's clear you know what you're doing and it makes sense.

> If you code iteratively the whiteboard is not your friend.

This is absolutely an issue. Everyone codes iteratively though. Successful whiteboarders learn to do the iterative part of coding verbally instead of in code. You talk through the algorithm you're going to implement, and refine your plan in words, and then only when you've got a solution you're happy with do you pick up the marker. As a side benefit, this demonstrates your thought process much better than coding would. Actually, this is just a better flow even for development on your own computer. Once you develop the skill, you'll find that a little more planning time before you touch the keyboard almost always leads to faster implementation overall.

> The trick is to understand that no one cares if you're actually writing working code, and that the important thing is communicating your intentions and process.

This is absolutely not true. I've interviewed at many major companies (Google, Apple, Snapchat, etc.) and they all say at the beginning "Make sure you have working code". They don't necessarily care about syntax errors or spelling errors, but they absolutely want working code.

> They don't necessarily care about syntax errors or spelling errors, but they absolutely want working code.

I'm not sure how code with "syntax or spelling errors" qualifies as "working code." The fact that they don't care about those things is pretty much the exact thing I was trying to communicate.

Other things they probably don't care about: unimplemented placeholder functions (they might ask you to go back and fill those in if they thing it would be informative and there's time), or misremembered function/method names (though it's important to communicate that you know you don't remember).

I mean, obviously they do want code that demonstrates a well thought out solution to the problem. But no one expects that you could just copy it into a text file compile it, and have it run - that's what I meant when I said no one cares if you're writing working code.

I am assuming that you can use pseudocode

Exactly! The above comment assumes that a linter will be run against the core on the white board. Jeez.

> What's with all the whiteboard backlash?

As always, some companies spoil it for the rest; whiteboards are fine but the crazy Hackerrank kind of interviewing, coding, on a whiteboard, correct algo's you studied in uni 10+ years ago and so on got a lot of people annoyed.

Nothing wrong with having someone sketch an architecture or a brief plan-of-attack on a whiteboard. But demanding someone to write a correct Bailey–Borwein–Plouffe implementation (I read a few months ago someone got that question in an interview; find the nth digit of PI efficiently as you can. How useless.) or something from their brain without a computer is just rather insane imho.

Seeing if they can write code would be actually writing code. Behind a laptop with access to internet and time to do it. Also; previous work.

There is a legal framework for interviews (e.g: EEO).

Probably they didn't like the candidate and they dismissed that person through a question they weren't expected to answer.

If someone was making a habit of asking impossible questions to members of a protected class and not to others I very much doubt that would be legal.

What does that entail? I thought the US was not as strict on law for workers / interviews.

Dismissing someone using an objective/concrete criteria such as a failed programming test is not problematic.

On the other hand, if you dismiss someone because "is not a cultural fit" or some other subjective reason, the person can sue you, accuse you of discrimination and challenge you to prove your point over a legal mine field.

This point is over-parroted. Anybody can sue (almost) anybody else over (almost) anything in this country. A butthurt applicant is not going to be dissuaded from suing you, if so inclined, just because you gave them an "objective" reason.

Most companies avoid that by not giving candidates a reason why they were not hired.

Well, if you did relatively fine they might still contact you in the future for other positions.

I can't tell if you're being facetious or if you're really that naive.

No company that I have worked for has ever said, "You know what? Let's look at the resumes of previous applicants, call them up, and see if they are still looking for new jobs." Every vacant position is always filled by direct, active applicants. Even backchannel applicants have to send a fresh copy of their resume to someone, so that it can be added to the applicant tracking system.

Tech resumes have a short shelf life. If the person is any good, and actively seeking to change employers, they will already have a new job by the time you have an opening for them. And if they are constantly doing a low-key, back-burner search, just to stay tuned-in, they will probably apply again to whatever you advertise, if they are still interested in your company.

The only companies that will keep your resume forever and call you back later are recruiters.

It may be that I have never been considered good enough to get hired, but not quite as good as the people they chose to hire on the first round, but for most companies, I either get an offer or I never hear from them again. Those still willing to communicate after a no-offer have never contacted me again regarding another open position. I admit there may be a theoretical nonzero probability of this occurring, but it has a lot of zeros between the decimal point and any other digit.

I'll offer a counter amnecdote: I submitted a friend's resume, he was interviewed and my boss liked him, but his boss didn't (I don't know about other feedback). Some time later, we had a new opening with some urgency and my boss asked me to see if he'd like to interview again; he was hired after that round (boss's boss did not interview him)

Many of the usual hiring practices can be bypassed if you already know someone at the company.

I had a company call me up years later based on an old resume. I turned them down, but mainly because I only applied to them because I was desperate (don't like the company) and the job they were offering sounded pretty lame.

I can't imagine it's a very successful tactic, and it probably only happens when they're desperate.

I'll ignore the insults and proceed to refute your point.

I am regularly contacted by recruiters from companies where I've interviewed successfully (with offers) and unsuccessfully (no offer).

They do it because it's cost efficient.

The book "Cracking the coding interview" also speaks about this in the early chapters.

No insult was intended. I intended to disparage the hiring processes of the companies I have previously encountered. Your experience appears to have set different expectations than mine. Perhaps it is a regional difference?

I work mainly with companies with a significant physical presence in the US Midwest and near-South. The furthest west I have ever interviewed was in Denver.

A significant number of the companies I have interviewed with lack even the ability to respond to a follow-up message the day after an interview, so never mind contacting someone months later.

And anyway a generic reason is "we decided to move forward with another candidate."

Ah right, in NL where I did most interviewing you are not allowed to ask a woman if she is or plans to get pregnant for instance. Not exactly a subjective reason, but I think akin to what you are saying.

Part of this is that a lot of interview practices (including whiteboarding) exclude people that suffer from social anxiety.

Just because somebody freezes up when being watched and critiqued by an Evaluator, doesn't mean that they are 'copy-and-paste programmers'. There are probably a lot of extremely good engineers who find this style of interview difficult.

People that call 'whiteboarding' a skill don't really get it. If you have irrational fear over people's judgements and this manifests when you're being interviewed, you can't merely expose yourself to this fear often to lessen it. It is very difficult to change how your mind and body react to uncomfortable situations.

Of course, you need to test ability, but you should also accommodate people's preferences on the environment that they are evaluated in. You can negotiate whatever this is by speaking to them: they might have a Github with projects which they can discuss with you, or they might be happy to do a small project, etc. You need to keep your standards high, while also ensuring that they feel that they are able to do their best.

"People that call 'whiteboarding' a skill don't really get it. If you have irrational fear over people's judgements and this manifests when you're being interviewed, you can't merely expose yourself to this fear often to lessen it. It is very difficult to change how your mind and body react to uncomfortable situations. "

I disagree on a few levels. I can't back anything with data but I'll point to where you can find many people who will offer their anecdotal stories of desensitizing themselves.

Being able to talk and interact with people is a part of the role. Software is not written in isolation - it is done best with a lot of collaboration and critical review of work. Being able to communicate and give and receive critical feedback are cornerstone traits. Good code is not written in isolation and giving and receiving critical feedback is not comfortable.

If I interview for those traits and qualities, I hope that I'm not discriminating against social anxiety but I spent 5 years in toastmaster's throwing myself in uncomfortable situations until I could come out of it.

I feel like interviewing is becoming this PC thing where we try to form to evaluate people in the way that is comfortable for them, where we live in a world where people need to break out of their comfort zone and learn to shape themselves into their environments. Go to Toastmasters, speak in front of people, keep going and you'll improve your skills - yes the soft ones too. You'll become comfortable - you will - ask anyone who stands and speaks confidently in front of you in those rooms if they were always that calm and composed while speaking in front of people. They were not and they learned to speak 'off the cuff' on topics they were not prepared for or subjects they didn't expect in there.

Maybe I'm alone in my thoughts but I'm looking for people who will grow. I want someone who will stand strong if I give them a critical code review - I want someone who will hang their ego at the door and be able to talk about what we're working on objectively.

If I'm still coming off too hard on this, what if I say that it's a consulting company that I'm hiring for and that the staff will be in front of customers who will interview and drill them every day?

Ultimately, we can grow and change. I know we can because I have grown and changed. I want my staff to have a growth mentality, not a fixed view of themselves.

Except that interviews are different than any other form of workspace interaction, except, possibly, for presentations (but not really). When being interviewed, candidate is not [only] solving the task of getting things done and done well - they're struggling with the task of appealing to others.

Interacting - be it in a normal non-stressful environment, or in most stressful emergencies when everything's on fire - is almost entirely different experience. I guess, primary difference is that no one judges you (at least, not explicitly) but you're working with the team to reach some goal.

Maybe there are interviews where interviewers are there to actually help candidate to tell about their good and bad sides and even receive useful feedback and not just "thanks, we'll contact you [never]". Wish I'd be to one, because every one I was to (not much) felt like an exam with a taste of scrutiny.

>>Being able to talk and interact with people is a part of the role. Software is not written in isolation - it is done best with a lot of collaboration and critical review of work. Being able to communicate and give and receive critical feedback are cornerstone traits. Good code is not written in isolation and giving and receiving critical feedback is not comfortable.

There is a difference between collaborating and receiving critical feedback from a fellow coworker, and being evaluated by an interviewer (or worse yet, a group of interviewers) who might or might not offer you a job (since your livelihood depends on that). That's why people who are typically comfortable working with others can become extremely anxious during whiteboard interviews.

  > I hope that I'm not discriminating against social
  > anxiety but I spent 5 years in toastmaster's
  > throwing myself in uncomfortable situations
  > until I could come out of it.
That is impressive and I believe you, however I don't think exposure therapy is proven to work in all cases for everybody. Either way, I don't think that we should require everybody to have 5 years of practice in whiteboard interviews before they can get a job. That can't be right.

  > Maybe I'm alone in my thoughts but I'm looking
  > for people who will grow.
Perhaps they would like to choose what they spend their time on. Sometimes introverts don't mind being introverted, and don't want to take courses on "how to be the most exciting guy in the room". They like their personality as it is. They might still push themselves to grow, but they choose to grow in the areas that they care about.

I suspect the backlash is mostly because an interview is already stressful. Then, when you ask someone to code with a marker, all of the muscle memory associated with how they usually code is gone...adding to the stress. You may be missing out on good candidates that just don't "whiteboard well".

Of course, some interviewers are better than others, and can manage to do things to make it a reasonable experience. You outlined some of the things that you do that make it seem like you're in that camp.

I'm not sure there's any one-size-fits-all interview technique that is universally better than others. The homework approach seems popular here, but then there are others that are put off by it. Probably the best thing is an interviewer that's smart enough to change gears when they sense the approach isn't really testing the candidate's skill.

If somebody has been promoted multiple times in programming roles, and especially if they have a visible open-source corpus (not necessarily GitHub BTW), then you know they can code. Making them do it in a highly artificial environment is pointless at best. Other words like "insulting" and "demeaning" also come to mind.

Asking people to reason about complexity is great, but that's not the way such interviews usually go. More often, the interviewer asks about complexity only so that they can compare to their own solution copied from the best of a hundred other candidates and then hand-tuned for days. It's a gigantic ego trip, and not even a fair fight. Candidates can see that, and it tells them a lot about what kind of person the interviewer is. Hint: it's not a pretty picture.

I was fortunate to go through an interview process recently where I had to do coding interviews, and it wasn't just a big ego trip. Second time in thirty years, vs. dozens upon dozens (all passed) of the worse kind. I could tell the interviewers had actually been trained, because I've been on the other side of that desk. They didn't care about the code I wrote, so much as how I approached a new problem, dealt with edge cases, explained various decisions, responded to challenges, etc. That's how it should be, but doing this stuff right is hard. Very few who pretend to do it have invested the time to do it properly, so they get results that are worse than useless. That's why there's so much hatred of the approach. Most people have never seen this kind of interview done right, and it's not their fault that's the case. It's on people who went from copy-paste programming to copycat interviewing.

> If somebody has been promoted multiple times in programming roles...then you know they can code

That's not necessarily true. There are people in "tenure" roles as I like to call them. They're not ostensibly bad, but they're far from good. They get promoted to senior developer and have pay raises each year. They're not bad enough to be let go, but after 5 years, they're still around and earn a senior title. They're not really in charge, but they're technically in charge with time at the company and the title.

I'm not saying throw deep algo questions at them, but promotions don't mean coding chops.

You do have to look at the role. If their role is to code, and they've been promoted recently, I stand by my claim. If they've moved into more of an "architecture" role or the promotion was more than a couple of years ago, then yeah, they might have lost it.

My father often told me that there are 2 ways go get rid of a bad worker. Fire them, or promote them. Eventually, they'll be promoted into a situation where they'll have to be fired.

I don't take "multiple promotions" to mean anything in itself. If they can talk about what they did and why, then I consider that they're probably good at their job. And that's the point of interviews, IMO.

Government / military like to promote problems away. Helps to keep the person away from the work that needs to get done. The good thing about that sector is that there is a lot of room for lateral promotions to other offices and so then they become someone else's problem.

I think the primary problem with this sort of interview process is that most companies don't hire enough people to discover whether or not it actually works, plus an interview is so much of a multivariate problem that you'll never really know which part of the process is broken if a bad hire gets though. And that's compounded by the fact any false negatives (great candidates who would have fitted in but got rejected) are clearly never going to be discovered.

The problem I have with whiteboard coding in interviews is that there just seems to be better alternatives. Using a computer as other people have suggested, but also just talking to them you can figure out if someone knows how to code. If you ask someone how they'd check if a number is in two arrays you can immediately tell a developer from a non-developer (or an inexperienced developer) by the language they use and the questions they ask. If they mention loops then that's a bad sign. If they start talking about whether or not the arrays are sorted, how big they are, whether you could refactor the array building code to return the data as a object index instead ... then it's clear that the person knows how to code simple things, and all without needing to pick up a marker pen.

Why is mentioning loops a bad sign? The question is simple, and asks for only checking one value; that's a simple answer that, for one value, takes linear time, which is the best you can do (without assuming the array is sorted, but expecting someone to ask you if the array is sorted turns it into a trick question, rather than a real world one). I mean, yes, I'd prefer them to go to a library call for that, but I wouldn't expect them to infer additional complexity from how I stated the problem.

Sure, if someone were to start asking questions, finding out that we're doing this operation a lot against the same arrays, so it makes sense to rebuild them into a set, or we're doing it against only sorted arrays so we can do a binary search, or whatever, is great...but that's more interview technique, not actual coding skill. You're clearly asking a contrived problem, have provided no additional context (no business requirement is "search two arrays to see if a number exists in both"), so it's not really fair to assume a prospect will treat it as an ambiguous requirement that needs further refinement.

This is really the heart of the issue; you're not testing for basic coding ability, which is what you're thinking you're testing for; your own criteria is testing for whether they assume you're trying to trick them. Talking it out isn't testing for coding ability either; I've met people who could say "You would search through each array to see if you can find the item in question", and then literally could not code that correctly, in their alleged preferred language.

If you want to measure whether someone knows to clarify business requirements, give them a business style requirement. "We have an interface that allows users to provide a list of their favorite foods. We want to provide a way for users to compare their own list of foods with that of a potential friend, and see what they share". Okay. That leaves the underlying data structures up the developer (as they are in the real world), and describes a problem they can ask questions around.

...that's more interview technique, not actual coding skill.

The skill and art of "coding" is in finding solutions to problems, not "writing code". When I interview people I'm looking for someone who can explore problems, ask questions, and find good solutions. If they don't feel like they have enough information I expect them to question things. If they're working on assumptions (like 'what does the array actually look like?') I expect them to question those assumptions. Expecting someone to go deeper in to a problem by finding out more is not "trying to trick them", it's a discovering whether or not they can do a fundamental part of any development job.

The fundamental part of the job being reading your mind?

For every interviewer who asks how to find if a number is in both arrays and is expecting a detailed discussion of he to do a nicer job by not actually having arrays, there's another interviewer who just wanted to see if the candidate can write two loops, leave the loops early when the value is found, and not run the second loop if the first one doesn't have the value; expecting to do all that in five minutes so more complex questions can be asked.

The fundamental part of the job being reading your mind?

The fundamental part of the job being that writing code without having a good understanding of the problem is a waste of time and often leads to throwing things away. If an interviewer was annoyed that I asked for more information about a problem before writing any code I'd take that as a sign I wouldn't really fit in at the company.

I was a cofounder at a startup that was trying to improve requirements management for a couple of years. This might go some way to explain my attitude about this. I really hate writing code to solve problems without having enough information first. It's so wasteful.

> This might go some way to explain my attitude about this

Yep. You're giving bad interviews because you're expecting someone to have the same previous recent bad experiences as you. I would never ask about the context of the problem you gave because it is so obviously just a quick coding test. You want questions about the requirements, you need to ask a question that feels like a real world problem.

If you keep questioning assumptions, you'll fine that justified knowledge is theoretically impossible. Don't be ridiculous.

data structures huh ?? thats a ml Q ie what sort of machine learning algo would you use to do that.

> If they start talking about whether or not the arrays are sorted, how big they are, whether you could refactor the array building code to return the data as a object index instead ... then it's clear that the person knows how to code simple things, and all without needing to pick up a marker pen

Having worked with people like this, I have to call BS. There are a lot of people with the distinct skill of being able to effectively imitate the speech patterns of actual programmers. Asking questions, engaging in productive discussion, etc. But when faced with an empty editor, they simply cannot produce working code.

If you're already talking about coding a really simple problem like your array example, why not let them use a computer language to express a solution? After those 5 minutes, you can move on to richer parts of the interview confident that you're (probably) not going to hire a total bozo.

> Having worked with people like this, I have to call BS. There are a lot of people with the distinct skill of being able to effectively imitate the speech patterns of actual programmers. Asking questions, engaging in productive discussion, etc. But when faced with an empty editor, they simply cannot produce working code.

Having never worked with anyone like that, I will have to call BS on this because it is simply too hard to believe, without evidence, that someone can understand this level of detail and not know how to create working code. A parrot cannot engage in productive discussion, and aping speech patterns only goes so far. Plus, if they're so good at aping speech patterns, who says they aren't equally good at aping code patterns?

It's all too true. There are, in my estimation, probably twice as many people who can talk around in circles and make it sound like they know what they are talking about, as people who can actually knuckle down and get things done. There's a little bit of overlap in the Venn diagram there, but not as much as you might think, for whatever reason. Probably because the people who can actually do things are too busy implementing all the shit that the bullshitters get their teams on the hook for.

> Plus, if they're so good at aping speech patterns, who says they aren't equally good at aping code patterns?

One is a social skill; the other requires understanding. Even in a technical conversation, a very large percentage of the interaction is social if the conversation is taking place face-to-face. I have also worked with people who could talk the talk, but not produce the code. Without social skills, those people seem to fail from job to job while not really understanding that they are failing: things just inexplicably go wrong all the time. With social skills (which is much more common), those people bafflegab their way into being system architects, middle or upper managers, and executives, where a lack of detailed low-level knowledge is more of an asset that keeps them from getting bogged down in details.

I've worked with people like this, and it's still hard to believe with evidence. I've changed my interview style from asking questions about programming to one that has candidates do a small design, then write one or two small programs implementing the design they just did. A surprising number of candidates are able to complete the design part well and then unable to make any real progress on the code. Most of them were phone screened before my interview; some had multi year careers in the industry. This feels different than people who are nervous or otherwise having a bad day, as they are exuding confidence while simultaneously being unable to write a while loop.

I've had interviews where someone expected me to write "compiles and runs code" in languages such as Python, Java, and C. I've also had interviews where "solve the program on the whiteboard with any language you feel comfortable with" means "exactly Java 5/6/7" because that's what the interviewer happens to know best.

I'm decent with a whiteboard. I routinely use one for brainstorming and discussing problems at work. I don't mind writing code on a whiteboard, even specific language code and including stupid shit like semicolons. I also understand that despite the fact that I can do it (and even enjoy it), it's a miserable experience for some people and expecting everyone to remember minor details that their editor/IDE takes care of for them isn't looking for the right thing.

I've got pretty complicated exercises in whiteboarding sessions: some problems with graphs or dynamic programming. The way how I solve those in my daily work is totally different to the way I go in 45 min whiteboard interviews: in 45 min you always have to do it in top-down manner, otherwise you will fail by writing non-relevant parts of the code and running out of time. I usually prefer opposite: bottom-up approach. It's a skill. Very artificial and very dumb to my opinion since you never work this way, but interview supposed to test how you do you work.

>> What's with all the whiteboard backlash?

Here's the deal.

As a developer, I have about 30 repos out on GitHub with plenty of examples I've done in the last three years. If you want to interview me, then go look at my code first. I have a pretty specific style so all the code is consistent. Go look at my code, dissect it, ask me questions about it, ask me why did this instead of that, why I prefer some library over another one, etc. If you feel like my skill set is what you're looking for, then bring me in and then we can see if we're a good fit for each other. If you don't like what you see, then don't bring me in - it's really that easy.

Having one or two whiteboard examples will never, ever, show someone the depth of your skill set. You have to look at their code, which will give you insights a whiteboard interview never will. All the things you mentioned about "basic skills" are obvious when you look at someone's code. Did they use classic or prototypal inheritance? Did they know what closures are? Did they use a factory pattern? Are they using ES6 yet? What tools are they using for their projects? All this and a ton more simply from looking at their code.

And if you're a developer, you should have an account on codeplex, or github, or bitbucket where interviewers can go look at what you're working on and the stuff you're building. It completely cuts out the "coding" part of the interview and should simplify the whole process.

Your last paragraph here implies that a developers work is either A) able to be shared and not owned by their employer, or B) they spend their free time coding after doing it at work all day, instead of having other hobbies. If thats the bar an employer wants to set, thats perfectly reasonable. They shouldn't ever complain about not being able to find employees however, if they are going to artificially limit their prospective employee pool to people who have no interests outside of programming

Two things. . .

1 - I've been active in sports my whole life. I work 40 hours a week, play in several competitive hockey leagues and still have time to tinker and build stuff in my own time. I'm also studying to get my commercial drone license, and have a host of other hobbies that keep me busy outside of coding.

2 - The only way I feel like a developer can possibly stay in tune with how fast our industry is changing is to code outside of where they currently work.

A good example is at my current gig we're building apps with AEM (Adobe Experience Manager), Java and AngularJS (1.4). If I wanted to go work in a smaller startup, do you think my knowledge of 1.4 is going to be helpful when most of my friends who are working at similar places have been developing with Angular2 for almost 6 months now? If I'm not working on my own to keep up, my skills will be completely obsolete when I want to get a new gig.

That's just a fact of life. I don't have any friends who simply show up, work 9-5, spit out some code and then go home and not do anything. Likewise, they don't go home, crack open Visual Studio and code until they pass out on the keyboard. It's pretty easy to do stuff in your off time and still have a social life.

I say this from the position of someone who has side projects on their own.

The general feeling that an engineer can't be any good without working unpaid outside of their job seems like a self inflicted and unnecessary cultural norm we've adopted as an industry. I can't think of any other industry where someone who goes home and doesn't practice their craft after hours is looked down upon as much as it is in software.

I don't have the label for it. It's similar to how employers want to hire developers for junior pay but don't want to provide any training and expect them to have taught themselves on various tools while lamenting the lack of skilled developers. Someone else in this thread mentioned the view of software engineers being viewed as artists who have to have a burning passion for their craft and I think that's bullshit.

Yea there are always guys who actually have that passion and enjoy programming as a hobby, but they are the minority like any other profession. It's become some sort of virtue signaling. I've talked to a number of devs who go through the effort of having publicly available work to show off who have confided to me that they do it just so that prospective employers feel that they are passionate

A business wants to hire you to solve business problems. Coding is the manifestation of the solutions. To me the white board session is to understand if you can take new problems and understand them enough to sketch out a solution. As a hiring manager, I'm glad you've written code before, I want to understand on a day to day basis if you can transform business problems into solutions.

At most of the companies I have worked for, the difficult part of my job is taking the pre-existing code that barely works under the old internal proprietary framework and rehabilitating it into the new internal proprietary framework with a bare minimum of new development.

Showing off that I know how to write (or recognize and copy) good code from scratch does not signal for that. I can't tell you how many times I have had to code something that is objectively incorrect because the old version did it that way, and the customer doesn't want the new version to be different.

The problem with the whiteboard interview is that you can't show an old, crufty code base on it, and ask the candidate to sketch out how to implement a simple feature request without overhauling the whole thing to use industry best practices. And you can't show an existing relational database schema that is pathetically ill-designed and ask about a new report for the customer. Those are the sorts of problems businesses have. They don't suck it up and pay the big bucks for a software pro until after the amateurs have already tried fixing it with an Excel spreadsheet or an Access database. And they don't want you to mess up the solution they already have, no matter how much baling wire and duct tape is holding it together.

Not everyone that can write code can also work well within an existing codebase. The only situation in the wild that I would trust to indicate that skill for my company is someone who actively contributes to an open-source project with a lot of other active contributors. A one-person side project that shows off everything you can do all by yourself won't reassure me. In fact, if it's too good, I might be worried that you will want to overhaul working code just because it's really, really ugly, and you can detect multiple code smells when you look at it.

>> To me the white board session is to understand if you can take new problems and understand them enough to sketch out a solution.

Honestly? This should be ferreted out in a screening interview.

As someone who's hired developers both as a manger and a senior dev, you know what's really important to me? Knowing where your head is at as a developer.

What I really want to know is:

1 - When shit hits the fan and you're buried under a mountain of stress, how are you going to handle that? Are you going to fight through it, or just give up?

2 - I want to know if you're open to other approaches to coding? What about other frameworks or tools? Do you believe there is one way to achieve a result, or multiple ways?

3 - Does your code have be perfect on the first try, or do you get something working first and then worry about how sweet the code looks later?

4 - How do you feel about fixing bugs? What if I made you fix bugs for 6 months before you started working on any project, how would you feel about that?

5 - Do you have the courage to stand up and say something if you feel like it will negatively impact a project you're working on?

Knowing how to solve basic coding problems at a whiteboard can't substitute for an impending deadline and tons of work that needs to get done in two days. I'd rather have an average developer who can handle a shit ton of stress and get his job done, then some rock star developer who folds like a lawn chair when things get tough. I want the developer who sees a tough deadline as a challenge, rather than the dev who whines constantly about it. You can't figure that out by standing at a whiteboard. You need to sit down and actually talk to the person to get to that stuff.

exactly the code is just a means to an end ie solving a real world problem

We do technical interviews on conceptual levels. One manager, one HR, one programmer (not always all three at the same time) and the interviewee.

We ask them about their experience with the tech we use / plan to use and test what they tell us with specific questions. If you tell us you know ASP.NET, I might ask you to describe the default routing or some specifics about razor views. If you tell me you are a git expert, I might ask you about interactive rebases and cherry-picking to find out when and how you'd use them.

General know-how is usually more viable than the ability to write working code in a pinch. The closest thing to a whiteboard test we have is giving you a page of code to ask you what's wrong with it (although, we don't do that often).

What about just asking the candidate to do the same simple task on a computer that they own?

My office is a satellite of a larger one in Austin; we provide laptops for candidates to use for the "remote" part of the interview, when they interview with someone in Austin.

But in general, I think the idea of the whiteboard interview is a chance to see someone "think out loud", to see them reason through a problem. Getting a chunk of source code in an email isn't representative of that. (At worst, they paid someone to do it for them; at best, you have no idea what their approach was. Somewhere in the middle is copying and pasting most of it from SO.)

> But in general, I think the idea of the whiteboard interview is a chance to see someone "think out loud", to see them reason through a problem.

So thinking out loud while coding is a job requirement now? I have been an engineer for over 14 years, and a software engineer for 10, and I can count on one hand the number of times I have done that outside of an interview. I collaborate all the time, but almost never synchronously while actually writing code.

Reasoning about problems is a very essential job requirement. And doing that out loud is the one of the few ways people can see how/if you do that.

>>Reasoning about problems is a very essential job requirement.


>>And doing that out loud is the one of the few ways people can see how/if you do that.

Strongly disagree. I would posit that written explanations are a much better indicator of someone's ability to reason about problems, because:

a) They can provide hyperlinks, images and other supporting materials to aid their explanation

b) You also get to see if they are a competent writer, which is also a very important skill for an engineer

> At worst, they paid someone to do it for them

Which would easily come out at the interview when they fail to answer basic questions about their code.

> at best, you have no idea what their approach was

So? Did you get code that would pass a code review? Did you get code with good commits/history?

Not really.

It is much easier to talk about a solution, when the solution is right there in front of you, and you read an explanation of the solution online.

Regurgitating someone else's solution is fairly easy.

> But in general, I think the idea of the whiteboard interview is a chance to see someone "think out loud", to see them reason through a problem.

This is exactly why I do this kind of stuff as well when interviewing. It's not about syntax or the ability to recall (regurgitate?) a function, but how does someone approach a problem and where can the conversation go from there. If you sent someone pre-work to write an array matcher, you'll see a final result, but not how they think through it, and you'll miss potential queues to further the conversation.

I have used both ways, and i would say the candidates in general respond better to using the whiteboard. I do ofcource not require syntacitly correct code or anything like that. Im looking for ideas not if they are a compiler.

Using their own computer way to many struggle with creating a new project or other basic things. And that dont make them comfortable either. It also takes much longer then using the whiteboard.

>If they can't write some simple code unaided, then I don't want to work with them. I was a copy-paste programmer once too.

Lovely. Someone put up with you, but now you won't do the same and offer to teach.

If you want to learn on the job and are happy to accept an intern's salary, that's great. Most organizations will be able to accomodate you.

If you want $140k and can't code productively, most early startups won't want you. Larger organizations may be able to accomodate you.

There are a shocking number of candidates who apply to senior positions who flat out cannot code. I have hired in the past without doing the whiteboard exercise and been burned by this; never again. The whiteboard is a filter to eliminate people who misrepresent their abilities.

>I have hired in the past without doing the whiteboard exercise and been burned by this; never again.

You should consider offering multiple technical tests and letting the candidate choose. Ideally the choices would range from whiteboard, take-home project, quiz site (hackerrank), or pair coding.

Or maybe ninjakeyboard skilled up before applying for a job.

So you test their ability to write code on the whiteboard. I was once programming with pen and paper too (only had access to computers at university). They I bought my own computer.

Don't know why this comment is downvoted, because he has a point. Writing actual code on whiteboard is different, unless it's pseudocode without any syntax rules and standard libraries.

I'm not a newbie (like, 15 years coding). But I haven't even bothered to properly remember standard libraries peculiarities - since they're different across the languages anyway, and with modern software stacks (that are quite diverse) it's not unusual to write Python but do some C or C++ one day (speed-ups), detour to JavaScript, try out Elm, then have a day patching some Go code that an useful tool is written in, then go learning some Elixir. Is it strpos(substring, string) or strpos(string, substring)? What does base64.encode(blob) returns, again, ASCII bytes blob or a text string? Which module/header file contains/exports that symbol? Maybe I'm doing it all wrong but I honestly never remember such details that for longer than a week or another - a proper tooling always hints those kind of things to me, so I don't even recognize I tend to forget them.

On a whiteboard, I'd have some awkward moments.

Why does everyone assume that you're asked to write perfect code on a whiteboard? I've been through 15+ interviews and have never been asked to worry about method names being right or signatures of methods being perfect and such. Most of the time people are amicably against me worrying about it. This isn't an issue.

I think, it's not an assumption, but a fear that this is the case.

Adds some extra anxiety as interviewee still worries about their code being correct or not. No one wants to risk being frowned upon just because they've messed up some syntax fine point. And since people are different, sometimes it really happens.

This can be avoided if interviewee's specifically told they just have to write algorithms in pseudocode, but it's not how it always goes in practice.

Because sometimes you are expected to write working code on whiteboard. Once interviewer was typing code I wrote on laptop and interrupting me on errors he had.

Exactly! When I was hiring I was even asking question I did not know answers to (not as a canditate, but as an interviewer).

You'd better be looking for psuedocode, because if you're looking for syntax, no strong programmer with experience is going to take you seriously.

Honestly, whiteboard interviews are for lightweight startups with very little programming knowledge. You should be looking at logic, not syntax. You should be looking at how the person thinks, not if they can memorize a specific language to do a simple task.

Have you ever interviewed at Google? They care about syntax, psuedocode will not cut it in the interviews... and Google has a lot of strong programmers.

Google has the name to attract strong programmers. I'd like to see stats on how few (or many) strong programmers their interview process misses. Especially when that interview process is used at a company without such a huge name.

It simply selects for people who can write a simple code unaided in an interview situation while somebody is watching every letter you type. Many people don't have a problem with that, others have.

I'm not sure that's true (in terms of explaining the backlash against whiteboarding specifically). From the page:

"The companies and teams listed here instead use interview techniques and questions that resemble day-to-day work – for example pairing on a real world problems"

I would contend that if you're doing pair programming with a candidate, you're looking at people writing code in an interview situation while somebody is watching every letter you type.

In fact, I think hacking on actual code is worse than whiteboarding: everyone knows how to use a pen and board, but don't do so frequently. Different candidates being placed in front of the same computer and IDE, however, may be vastly more comfortable or not - they may know the bindings, etc. If they're not the "driver" in the pair, they still have to grok the codebase and the problem quickly.

Whiteboards may be abstract, and people may focus too much on algorithms with them. I don't think that there is much fundamentally different about pairing, and I would have more reservations with pairing personally.

I would like to see a compilation of companies that aren't requiring a "coding project" as part of their interview process. I much prefer to write on a whiteboard as part of an onsite interview than to do a "coding project" for every single company that "might" be interested in me.

My experience lately has been an immediate request to complete a coding project right after speaking to a recruiter but before speaking to anyone technical. And perhaps the height of absurdity I have even had automated emails saying "thank you for you interest we invite you to complete this coding challenge/project."

Are you really expected to do one of these now for every single company you send your CV?

The whiteboards seems far more respectful of people's time at least.

> The whiteboards seems far more respectful of people's time at least.

Some companies will happily bring you on site for a full round even though they have no idea whether there is an open position for you (infamous example, you can pass Google's hiring committee and then be told there's no headcount). And you've just wasted one day off.

On the other hand, take-home projects can be used in a way that saves everyone a lot of time. I don't disagree with you, they shouldn't be used for screening though.

>"Some companies will happily bring you on site for a full round even though they have no idea whether there is an open position for you (infamous example, you can pass Google's hiring committee and then be told there's no headcount)."

In practice though this is kind of rare especially for the larger SV companies. I'm not saying it's never happened but that Google incident sounds like an anomaly.

How do take home projects save time if they take between 4 - 8 hours? That's the same time as an onsite takes. The difference being that before a company brings you onsite there has probably been at least a technical phone screen or two so the candidate has ben "qualified" to some extent no?

Not everyone lives in the SV. If you interview at a satellite branch of a company that has a big and generic interviewing pipeline, it can definitely happen.

I can do a 4 hour[1] assignment on my own time, assuming I'm not given an unreasonable deadline. For a full round of interviews on site, I need to take a day off. That's a huge difference to me.

[1] I've rarely been given any assignment taking more than 2-3 hours, I would flat out refuse anything that can take more than 4.

Edit: oh and I can give you a very long list of companies that thought I was "qualified" after 1/2/3 phone screens, wasted a lot of my time afterwards, and ended up deciding I wasn't :)

I am interested in this too. I recently failed netflix code review because I used java 7 instead of java 8 which apparently showed them I don't keep up with new technology (even tough the question asked me to pick any language on jvm.) Other reasons given to me were that i used maven instead of gradle.


I guess you can consider yourself lucky, I would not want to work for such company.

> all my open source code is written in java 8 for him to look

That completely defeats the purpose of a small, self-contained assignment like the one you were given. And your code isn't even in Java 7 (no diamonds), I think it was a legitimate remark.

You think it would be helpful to let the candidates know what they would be evaluated for in the coding assignment.

I would have dazzled them with java 8 if I knew I was going to be rejected a based on that :D

I was under the impression that this was the usual algorithm/data structure/complexity rigamarole, not show us latest language features.

Yes this patently absurd. I am guessing you weren't allowed to discuss your code or choices with anyone either correct?

Yikes! They'd probably bury me if I used COBOL!

Presumably a lot fewer people send in completed coding projects, so you are competing against fewer other candidates.

If you are good at selling yourself, whiteboard interviews are awesome.

If you are nervous in interview situations, starting with a project could be an advantage.

Edited to add: coming in for a whiteboard interview will also take few hours of your time (or longer if you need to travel). A sample project can be done whenever you want.

I don't think that this is a proper justification for sending coding projects to everyone who submitted a CV. This is a big imbalance in time investment from both sides and quite disrespectful from the applications point of view.

It's not disrespectful for a company to ask you to write a little code for them. They're just as invested on the time front as you are since they have to read and analyze what you send them.

Now, if you were to tell me that nobody ever looked at the test after you sent it in, then it becomes disrespectful. I had several companies do that to me a few months ago.

Not that it matters much. I don't think code tests will stick around. It's too time consuming for the hiring managers.

A little time is fine but last time I was applying for developer jobs almost all of these tasks would take at least an hour to complete, probably more like two or three to do it in a way I'm willing to show to someone screening me on that basis.

By requiring these tasks you're implicitly screening not only people who can't code but also people who don't have three hours to spend on applying to your company. If all you want is 20-somethings with no other responsibilities then continue as you are, but if you'd like to see a bit of diversity then you're going to have to compromise.

But this is an endless cycle. At some point, the company has to apply some test to see if the candidate is appropriate, IE, can code to the required standard. That will require a time investment from the candidate.

You could simply get rid of the take home test; that will shift the burden to the resume and cover letter, to the advantage of humanities graduates like me.

You could mandate a full day on-site, which will bias against people from out of town who want to relocate and people in delicate situations at their current employer.

For the time poor candidate, there is always the whiteboard exercise... Whose biases we are gathered here to moan about in the first place.

I always feel in these threads that people think hiring is 'broken' and it can be 'fixed' with the proper application of 'one weird trick'. But then all the comments actually have contradictory complaints. So obviously it can't.

I'm not really advocating for getting rid of take home tests entirely. There's a lot of value in seeing how someone approaches a known problem and being able to compare that to other solutions.

What I am asking is that companies don't ask me to invest hours of time before they decide if they're even interested in talking to me. At the very least I want to have had a phone interview and the chance to assess whether I'm even interested in working for you. That's particularly relevant when looking for remote friendly employers, I've had several calls in recent days from recruiters responding to me saying I'm looking for remote work which I stopped on finding out what they meant was "at least three months fulltime on site, and then maybe we'll talk about some remote work".

This is exactly what I meant. Home tests are totally fine, given there is some reasonable interest from the company already.

Home test is totally fine after a phone call or a two. However, they are not fine at all just after submitting an application and no real contact.

>"What I am asking is that companies don't ask me to invest hours of time before they decide if they're even interested in talking to me."

I feel like this is the crux of it as well. I am not against projects or tests to show I am technically qualified. But this should be after talking to some potential team mates to see if there is even a fit, personally, work culture-wise, technical interest or any number of factors that you could glean by speaking to potential colleagues for 20 or 30 minutes.

An hour of time for a homework is completely fine. A whiteboard interview would consume much more of both party's time. Just travelling there and back can easily take an hour of applicant's time.

I used to invite people for an interview without any prior testing, but it was generally such a big waste of time for both me and applicant. What's more, quite often I can see from CV that someone is underqualified, but I do not want to reject them immediately without giving them a chance, so I send them a homework. Finally, you would be surprise how often even people with good CVs turn out to be quite bad.

Hmm. One of the best exercises I ever had to solve took me almost a day. [1] I have actually rewritten it twice, once in the manner they said they would have wanted it (same repo) and once in Python. (I didn't get the job, they didn't like the style I was using.)

I don't mind long coding exercises, it's what I do all day anyway. Better than watching yet another episode of NCIS or what have you :)

[1] https://github.com/mdpopescu/public/tree/master/SocialNetwor...

To the people who say spending "4 hours is more like 6-8 hours and it is unreasonable":

- How is this any different than spending an unreasonable amount of time studying balancing binary trees and similar trivial whiteboarding problems?

I'm of the opinion that studying all these variations of the algorithm problems is a bigger time-sink and waste of time than getting into the habit of problem solving a take-home project.

>"It's not disrespectful for a company to ask you to write a little code for them. "

Except in practice it seems that its not a "little code" its a 4 hour project that is usually closer to a 6 or 8 hour project in reality.

If you're hiring backend engineers, code tests can be easily automated away on the correctness and efficiency aspects. Add lint on top and and an engineer eyeballing the style for a few minutes. Voilà, 4 reports per employee hour.

If it's automated, or direct from a recruiter - chances are the company hasn't even read my CV. Or looked at any of the previous work I've included. Of course it's disrespectful.

Would you prefer them looking at your CV and immediately reject you without giving you a chance == test?


>"Why would a company look at your CV? At best, it's full of exaggeration. At worst, outright lies."

So by your reasoning CVs are not meaningful at all. Should we get rid of them? Why stop there though, should we not even discuss past projects we have worked on in interviews since those are also likely exaggerations and possibly outright lies?

Yea but the process is never just coding project... It's either just whiteboard interviews or a coding project and then whiteboard interviews. There's never a reduction in time here

> sample project can be done whenever you want.

And also take a few hours of time, and possibly more.

I would not do a project before a basic phone or email interview. They should be screening candidates before sending projects.

Do you have any examples of companies that have asked this of you? I'd actually really like to apply; that seems like a fantastic way to gain some experience outside of my daily job.

Zalando, Bloomberg(you won't get to speak to anyone, immediately asked for code), Soundcloud(speak to a recruiter but nobody technical and then asked for a code project before)

These are just off the top of my head, however theres no shortage.

I think coderpad.io is the modern day equivalent of a whiteboard.

It's nice that you get REPL and a keyboard, but the unnecessary pressure of being watched, critiqued, and timed are all there.

My mind just goes completely blank whenever I am put in this scenario.

Are we programming or defusing a bomb?

Hi. I'm the guy that made CoderPad. I bootstrapped the product out of nothing because of terrible candidate experiences older purely textual collaborative environments provided. I sincerely believe CoderPad gives you much more room to breathe than many alternatives. There are a few reasons for this:

1. You get to introspect the environment. I've had lots of candidates forget whether certain objects support simple methods (map, sort, join etc). CoderPad lets you perform quick tests to make sure.

2. You get to debug how you would normally debug. Whiteboard interviews basically rely on the interviewer saying something "Are you suuuuure there's no bugs on, say, lines 10 through 12?". I find this incredibly stressful as both the interviewer and candidate. CoderPad lets you run the damn thing yourself and see if it says what you want it to say.

3. Compared to probably the next most relevant comparison, graded homework assignments, I think CoderPad's free-form format is much more forgiving. As a candidate, I've definitely wasted hours trying to get some hidden automated test suite to pass with no feedback as to what sort of failure was encountered. CoderPad blends code evaluation with live feedback from the interviewer.

Yes, social pressure can be daunting for many candidates. I don't think this is something you can avoid in a live interview in general. I think you can get pretty good signal from project submission from candidates, but at the cost of greatly increased time spent by both candidates and interviewers. For many people this may be a worthwhile trade and I think additional options are good, too.

Regardless of how much your software has made me sweat, I still think it's great software. My comment was more about the interview process, not your software. Good job creating it. I wish it was free... Haha...

I think it's worse than a whiteboard. I have seen very few interviews that expected anything other than psuedo code on a whiteboard.

With a repl I could easily pin myself on some trivial bug that interviewers would ignore on the whiteboard...

I thought I disliked whiteboard coding, until I recently had an interview on a provided laptop running one of those JavaScript IDEs. It was miserable. Extreme unusable OS X mouse acceleration, an awful keyboard, an unfamiliar hotkey setup in the text editor, and a tiny 11 inch monitor split into 4 sections which means my editor was maybe 10 lines tall. I failed miserably at a relatively short and simple challenge that I am confident I could code on a remotely familiar and sane computer setup. Heck, I would have preferred writing valid JavaScript on a whiteboard. At least I'd be able to see a lot more lines at a time, which it turns out is pretty important for my ability to code.

Sounds like a company to avoid if they throw the improper tool for the job at you and expect you to do it with that, even if a pen & paper would be better.

</slightly offtopic>

Some guys I've met, with different occupations and age, sincerely believe that smaller screens are more usable. Like small windows on 14" CRT when everyone had 17/19". Now it's guys that claim that 11-13" ultrabook is enough for them and they don't want more ever. This honestly baffles me every time.

I use only a small part of screen when writing code, because it somehow forces me to write shorter, cleaner functions. Although debugging is terrible experience on small monitors, because I need more windows open at the same time - watches, callstacks, ...

I much prefer a whiteboard to these programs. Half the time I end up typing vim commands and appearing as if I can hardly type.

> unnecessary

I can't think of any way to do an interview without pressure. Pressure is unavoidable when someone's future job is on the line.

It's avoidable. Behavioral questions don't tend to freak engineers out. Neither do questions about their past work.

> Behavioral questions don't tend to freak engineers out.

Speak for yourself. I'm far more relaxed when diving into a technical question than when playing a guessing game about what social signalling I'm supposed to be doing at that moment.

You're not supposed to be trying to be someone you're not. If you have to put in that much effort during your day job then you won't be happy. You just need to relax and be yourself, it not only makes everyone more comfortable but it lets you and the company start to evaluate whether or not you're good cultural fits. You'd be pissed off if they mis-represented the culture there during the interview process, and they have just as much right to be pissed if you act completely differently (in a negative way) when you start working.

> Neither do questions about their past work.

If you feel you didn't accomplish much at your last job(which would be a pretty common reason for quitting), then a question about it would be pretty stressful, since the company is evaluating you based on work you didn't find to be your best.

Then talk about work that you do consider to be your best.

A terminal does not replace a whiteboard. I know some force candidates to code on whiteboard but that's not what it's specifically good for. A person to person discussion with a white board is the best communications platform for conveying technical ideas, IMO. You need the whiteboard, so both of you can define symbols, then discuss the meaning of these symbols by pointing at them, drawing arrows from them, expanding them, etc. The symbols can include fragments of code, of course.

So, yes, if someone wants to force another person to code on a whiteboard then certainly, they can do better, but it's like asking someone to draw with an axe on the stone floor. Waste of good axe and floor.

Well, these code/rpad/lity/rank sites are no different in terms of types of tasks... The problem is not only with a medium (whiteboard or *pad/lity/rank website) but also in what kind of problems you get to solve... Are these problems relevant to job? Is the condition/situation similar to what you'll be doing on the job? etc...

And don't forget interviewer's breathing over skype while you coding online, just to add more drama

XKCD 1168:

- Rob, you use Unix! Come quick!

* To disarm the bomb simply enter a valid "tar" command on your first try. No googling. You have ten seconds. *

- Rob?..

- I'm so sorry.

That's just silly, though. "tar" is pretty easy to use in its basic form, and I have dozens of commands ingrained in my memory.

    tar tf filename.tar
    tar xf filename.tar
    tar xfz filename.tar.gz   # Newer tars don't even require the z
    tar cfz filename.tar.gz directoryname
Heck, surely even

    tar --help
would suffice.

More interesting questions would be like: "which lower-case letters are NOT valid options to GNU ls". :)

I had untarred hundreds of files before I actually needed to start creating tar files on a regular enough basis to remember the syntax.

Defusing a bomb of course~

But yea that analogy fits coderpad well, along with all the other collaborative code editor + webcam websites.

It's funny, whiteboard interviews really would be a better discriminator for founders than employees. They test confidence rather than competence (or technically, competence + confidence). The former is incredibly important for founders, but the latter is probably what you care about more for employees. And yet founders almost never do a whiteboard interview, yet it's standard for technical employees...

My view is that you can't have competence at a certain level without confidence - unless you're just an easily outsourced code slinger. I'm often at a whiteboard explaining architecture, ideas, etc. How do you do that without confidence and some semblance of emotional intelligence.

There is really no comparison. Explaining architecture, ideas etc to your colleagues has pretty much no resemblance to a whiteboard interview, where you have several people staring at you and judging you. One is a discussion the other is a test, they compare like having a beer with a colleague is like holding a speech in front of a bunch of stranger.

When I'm the only developer talking to managers, directors, etc. I'm constantly being judged by whether I have good ideas and whether I can be trusted to architect a solution.

May be I am a little overconfident, but I always go into an interview and treat it as a formality to getting the job. By the time I'm in front of the whiteboard, I've already spoken to the recruiter and know what they are looking for, seen the job req to know what technologies they care about, and I've had a phone screen. Why wouldn't I be confident?

One thing I've always been curious about is why folks are so afraid of being judged. Most people are judging you all the time, yet recognizing this fact causes a lot of people intense anxiety. I'd rather look at it as a selection filter: if someone doesn't like you or doesn't want to work with you, that's a strong signal that you'd be better off not spending any more energy on them, and instead focusing on the folks who do like you & want to work with you.

When an interview is not a discussion, but rather a test, it is a bad interview.

Now that I think about it, I wasn't asked a single programming question when I was interviewed for the job I have now. I was asked architectural, process improvement, soft skill questions. That's probably why I chose the job I have over other offers. It was more of a discussion and by the end of the interview, I was joking around with the hiring manager and said something to the effect of "I'm already assuming I'm going to get the job. When am I going to hear back from you?" He started laughing and said soon.

Another interviewer told me at the end of the interview to submit an application and they can get an offer emailed to me by the end of the day. We did a collaborative white board design session and spent a lot of time just talking.

Either all of the job offers I've gotten have been mostly just discussions or I talk tech so much I can't tell the difference between an interview and a typical day at the office.

Most interviews I've been to have been the same. I was talking about the ones where you have to solve some tricky algorithm questions at a whiteboard while someone is staring at you. I have no problem doing that in front of my colleagues, but find it difficult to do at a job interview.

We've had similar experiences. For the vast majority of jobs across all industries, the last deciding common denominator question that seals the deal is "Can I see myself having a beer with this person?"

Well, for a random employee of a random company there is no motivation to not haze people on the whiteboard, it's mostly a power play for them and oversupply of candidates permits it. They really don't care how good of an engineer they hire and how much value he can bring to the team or to the company.

I don't think that's true, actually. I've been both interviewer and interviewee, almost all my friends do interviews, and in every case I've seen, the interviewer is earnestly trying to make her best determination of whether you can succeed at the company. Most interviewers - both ones I know as friends and ones that have interviewed me - get very, very happy when they find a candidate that they know will meet the hiring bar, because it's such a rare occurrence.

I think that the perception that it's just a hazing process comes from a failure to look at it from the company's perspective. Most people know that they're good programmers (whether it's true or not...), and so they're like "This is bullshit...why do I need to prove my worth at the whiteboard when I know that I'm a good programmer." But the cost of a bad hire is really, really high (both for the company and for the employee), and the applicant pool is skewed through adverse selection, and a company knows nothing about you when you come in to the interview. Many people with stellar resumes can't actually code; some that can code don't actually want to work with you and would be bad cultural fits; and some outright lie on their resume. They're trying to figure out, in a 50-minute hour, whether you'll be productive on the particular problems that that employer faces.

"I think that the perception that it's just a hazing process comes from a failure to look at it from the company's perspective."

Actually, speaking of looking at it from the companies perspective, in 2010 google and several other big name silicon valley companies were successfully sued because they all agreed with each other not to cold call each others employees to try to recruit them. They actively worked to try to make it harder to switch jobs from one company to another.

Right after that, they started doing whiteboard interviews, which serve the same purpose of making it painful to switch companies, but without the risk of being sued.

Sometimes it's worth taking action to diminish a perception, even if that perception isn't actually true. Has Google considered the impact of the perception that their interviews are hazing?

Especially in the context of Google's public commitment to increase diversity among their engineers, it strikes me that revamping their interviewing process to eschew whiteboarding ought to at least be considered - it would be amazing PR for Google if they swallowed their pride and innovated in this area. And it would have a real effect outside Google, since so many companies' interview processes mimic Google's.

I'm not certain that an interview process without whiteboarding - or at least some other form of coding test - would actually be able to detect the skills Google needs in their interviews.

I see comments on HN time to the effect of "Why do I need to know how to traverse a tree or sort a trillion 32-bit integers? Why should I care about big-O notation? When will I ever use those skills on the job, when I could just Google for a standard library function?" I used those skills all the time when I worked at Google. When your data set is measure in petabytes, you can't just reach for a stdlib function, you have to actually know how the algorithm works, in depth, so you can implement it across thousands of machines. And any algorithm slower than O(N log N) is not going to work. Even in frontend work, I used tree traversals all the time because JQuery's byte cost was too high to use on Google services.

They have been taking other steps to increase diversity, though, like redacting names & genders on resumes, ensuring that every female candidate has at least one female interviewer, and trying to source candidates from non-traditional (i.e. non-Ivy-league, non-brand-name-tech-company) places.

Oh yes, they absolutely need a coding test!

Has Google innovated much in the interviews themselves in the last, say, 10 years? My perception is No. I get the impression that Google's old-fashioned process is a point of pride that they hang on to because it works (at least, the business is succeeding) and they are afraid of change. But it's a missed opportunity for a technical leader to advance the status quo in our industry.

For example, onsite, Google interviewers will still ask you to write code or pseudocode on the whiteboard. That's something they could re-examine (perhaps by giving an option of whiteboard or computer, and ensuring a level playing field between the options). They could get some good publicity from re-examining this, I think.

Posting a reply for @tsion, who seems to not be able to post at the moment:

I interviewed on-site successfully at Google Seattle about a year ago, and I was given the option between writing code on a whiteboard or in a text editor on a Chromebook, being projected onto the screen in the room, so the interviewers could easily follow without awkwardly looking over my shoulder.

Of course, I picked the text editor. I can't imagine wanting to write code on a whiteboard. However, I did use the whiteboard voluntarily as a thinking and planning aid.

Unfortunately, I also heard that the text editor option was only available at certain offices at that point. I really hope coding in a text editor becomes a universally available option because it's just so much more natural. It essentially put my mind at ease and helped me think more clearly.

On a more general note, my strategy, which seemed to work well for me, was to treat the whole situation like a pair programming task. Even though the interviewer won't directly tell you what to do, they will often comment on things if you question yourself out loud enough.

The problem with whiteboards, is they're not reflective of real life. When was the last time you can honestly say you used a whiteboard to solve a programming problem instead of Googling and ending at a StackOverflow question with a great answer? I've never used a whiteboard for programming. I've used plenty of whiteboards for things one level back; planning, prioritising and visualising task related things.

The reality of programming is, these days everyone Google's their problems. Sometimes you end up at StackOverflow or sometimes you find yourself on a random blog or official documentation. Occasionally you ask a coworker for help, but the ego a lot of developers have to be smart means most feel intimidated to do so. And sometimes you just bang your head against your desk until things start working.

Whiteboard and highly technical interviews that expect you to know high-level computer science concepts are tilted in the favour of graduates who are freshly minted from whatever respective educational institution they graduated from.

I use whiteboards all the time.

The thing is, the timelines are completely different -- in real life, we usually write the problem down, work out obvious details and a plan of attack, then come back to it hours or days later when people have had time to research and think. Never have I had a coworker interrupt me after 15 seconds to "say what I'm thinking about" any time I pause narrating.

In real life, going "hmmm" and taking a couple minutes is acceptable. And you have people being constructive during those times, instead of interrupting you.

Everything depends on the project (e.g. programming games is different than making web apps), but:

> When was the last time you can honestly say you used a whiteboard to solve a programming problem

Every time there is some interesting hard problem which I want to share with the team. We used whiteboards almost daily.

> Googling and ending at a StackOverflow question with a great answer

I google like 5% of stuff I do, because:

* a lot of time is spent on debugging and you just can't google why your program crashes on in file X on line Y * a lot of stuff is too easy and there is no need for me to google it * architecture - you usually can't google "how do I architect system X"

  > We used whiteboards almost daily.
For coding or for displaying parts/relationships between parts. We do use whiteboards. We never write code on them.

Tech interviews are not reflective of real life regardless if there's a whiteboard present or not.

Whenever I've had a whiteboard programming interview, it's always been a trivial problem that's a smoke test for "can this candidate actually write code or is their CV faked". I think it took somewhere between 5 to 15 minutes, which I consider to be rather well spent time compared to spending hours or days on a take home project or anything bigger.

The only time I've been given a take-home assignment I refused to do it because I was extremely busy at that time (finals at uni). The employer gave me an offer anyway. Someone else made a better offer, though (based on a whiteboard session).

The whiteboard is not a good way to assess the level of programming skill of an individual. But it's a good binary indicator of whether they possess any coding skills. If the result is yes and their resume is good, experience is relevant and maybe there's some GitHub or other code they can share, that should be enough to consider a person to be an able programmer.

Like any tool, the whiteboard can be misused. The task should be rather trivial and the time should be short. The purpose should be to weed out the people who have faked their resumes or are applying to programming jobs even though their talent is something entirely different.

> Whenever I've had a whiteboard programming interview, it's always been a trivial problem that's a smoke test for "can this candidate actually write code or is their CV faked". I think it took somewhere between 5 to 15 minutes, which I consider to be rather well spent time compared to spending hours or days on a take home project or anything bigger.

That sounds pretty reasonable, but I don't think it's the norm. I had an interview over two afternoons with 5 or 6 different whiteboard coding challenges, each around half an hour and each much more substantial than a smoke test. They were things like parsing and evaluating a string with a simple arithmetic expression like "2+3*4+5" or taking a phone number and enumerating every possible alphabetic representation (2 is "a" "b" or "c", etc.).

Google has example whiteboard interview questions on YouTube, and one of them was writing a function in C++ that takes an array of integers and a target integer and calculates whether some subset of the array sums to the target integer.

Not the most difficult things ever, but hardly smoke tests, and pretty annoying to whiteboard in my opinion.

I completely agree with you. Once you go past the 15 minute mark, the whiteboard is no longer a good way to do programming aptitude testing. It's a good smoke test, but doesn't really work beyond that.

Of the three examples you mention, I think only the phone number alphabetic representation is a reasonable one to do on a whiteboard.

All the 5 to 10 whiteboard interviews I've had over the years have been smoke tests. This is in Europe, and I got the impression from this thread that it's a common smoke test in Europe and Americans tend to have longer whiteboard interviews.

"Whenever I've had a whiteboard programming interview, it's always been a trivial problem"

I've had a few, and it's never trivial problems. Here's a trivial problem: - Write a method that takes 2 parameters - an Integer, and a list of Integers - and return a boolean indicating if the int param is in the list of Integers.

Instead of something actually simple like that, it's trick questions designed to appeal to the interviewers desire to appear smart and clever.

> But it's a good binary indicator of whether they possess any coding skills

Also: can you discuss technical issues with them? Code quality feedback, finding bugs, explaining their thought processes, acknowledging mistakes, etc.

Imagine they're part of your team, and you come to them with a problem you're stuck on. Do you come away with a positive experience, and closer to a solution?

> The problem with whiteboards, is they're not reflective of real life. When was the last time you can honestly say you used a whiteboard to solve a programming problem instead of Googling and ending at a StackOverflow question with a great answer?

Yesterday. Although, usually I pull out a notebook instead, and work things out with colleagues (or on my own) on paper at my desk. That happens several times a day.

I scribble in notebooks a lot too, but it's very rarely anything resembling pseudocode or any representation of the actual algorithm I'm working on. It's usually just a todo list, with occasional things heavily underlined so I don't forget them. I often do something similar at the end of the day if I need to remind myself where I left off. For me, it's not at all comparable to whiteboard coding.

Not whiteboard, but I do doodle on paper to find a solution to a problem all the time. It's silly to assume that the problems that I deal with have already been dealt with by somebody else. Who needs programmers anyway if you can simply Google everything? How can you justify the wages of programmers who can't solve new problems on their own?

> The problem with whiteboards, is they're not reflective of real life. When was the last time you can honestly say you used a whiteboard to solve a programming problem instead of Googling and ending at a StackOverflow question with a great answer?

Programming problems, not so much. Domain problems, almost daily.

Last week. I have to admit that I used a whiteboard for solving a programming problem a few times. But most often, if I have to note down the problem (structure), I'll use a notebook or texteditor. Then on to the Googling.

Using the presence of whiteboard questions as a metric for interview quality seems rather pointless to me.

Whiteboards questions can be very good or can be very bad or anywhere in between. Just like any interview question in general.

I for one prefer whiteboard questions to generic open ended personal questions.

Good whiteboard questions can quickly assess whether someone knows the basics of programming or not.

Open ended questions only judge one's ability to tell stories.

Early in my programming career, I got a whiteboard question where, after being asked my hobbies, I was asked to diagram an electric guitar on the whiteboard. I don't know that it was a particularly good question. But all this sabre-rattling against the whiteboard in general just seems like a silly distillation of a legitimate critique down to its stupidest form.

Even when it comes to CS algorithms questions. Yeah, asking someone to whiteboard quicksort might be bad, might optimize for the wrong things for the actual job.

But should it be a red flag that someone expected me to be able to do a more basic operation, that I've more or less purported to do professionally and efficiently, on a whiteboard on the spot?

[Edit: At my company we don't really do whiteboarding. We pre-screen with a time-limited open-book, open-internet test in advance. This helps account for candidates knowing how to quickly read/research/apply openly available information.]

That reminds me of an interview I had for a sales position. The guy man asked me how to tie a shoe. I actually really thought it was an interesting interview question, despite not being strictly sales-related, because teaching is a really big part of sales.

I like that guitar question for a similar reason. Someone once told me that you can't program something you don't understand. Being able to break down a complex topic in to atomic parts is the difference between someone who can program, and someone who can develop applications that solve problems.

I'm an analyst rather than a developer, but most of my value comes from being able to communicate information to other people. I know there are people who are better than me at programming, or statistics, or modelling, etc. However, I know that I can be incredibly successful in any given role if I take the effort to learn as much as possible about whatever I'm analyzing, and if I focus on communicating the information as clearly as possible.

To that point, I'm really proud of the fact that the man who interviewed called me afterwards to say that I had given the best instructions for tying a shoe he had heard in 30 years of asking that question in interviews. That being said, I can see how someone who is a nervous interviewee would resent questions that seem designed to trip them up.

That isn't the metric. Read the tl;dr.

It really is terrible. Companies need to accept this. You're sitting behind me. Or beside me. I can smell your breath. Or you're on a webcam. It's voyeuristic. You've already solved this problem. Perhaps you even designed it. Your talking is breaking my concentration. This is not how I work. It's probably not how you work either, and if it is, I think we're done here.

Now, if a company insists on an adversarial process, there are better ways to do it. The nature of the problem is given in advance. Both the subject and the evaluator go in blind, and they both do it.

You know what I call it when a coworker is reading off a document and harassing me for not knowing the thing they're clearly looking up as they go along?

A toxic work environment.

Of course, I believe a significant number of those "interviews" are hazing meant to find excuses not to hire or perpetuate H1B scams.

I was in our spare room coding one day. My girlfriend arrived home and sat on the sofa behind me. I couldn't concentrate just knowing she was behind me and asked her if she could leave.

The main purpose of whiteboard is to decrease the liquidity of job search for engineers, thus depressing wages. After a few years of doing design docs and copying protobuf fields, the median engineer at a top company would need a month of prep to be able to pass whiteboard interviews again. Much more effective than secret no-poaching agreements.

While it may have the effect of decreasing job liquidity, I don't see how that effect could be the main purpose unless you believe companies are mass colluding. Because the company running the interview is highly motivated to find a good candidate quickly and reliably.

A less intriguing but probably more accurate reason is that finding good people is hard, nobody really knows how to do it, whiteboard interviews provide a rough filter, and they're what everybody else does, so we're gonna do it....

You mean collusion like this?


It was happening, only more blatantly than is suggested here. Of course they are all motivated to find good candidates, but they are also motivated to keep their existing folks. Decreasing liquidity and mobility serves the latter purpose at the expense of the former.

Yeah, that happened in 2010, they got sued for agreeing to not try to recruit programmers from rival companies. Right after that we saw these long tortuous white board interviews introduced which serve the same function but without them getting sued over it.

Mass collusion doesn't need to involve actual individuals making agreements in smoky rooms. It can just be customs that arise as fads (whiteboarding as the answer to '90s Microsoft brain teasers), gain mass acceptance, and become standard practice.

Interesting! This comment gives even more reasons: https://news.ycombinator.com/item?id=13785405

Why the universal hate for whiteboard interviews? I studied computer science at a German university and it was expected that in the exercise groups you are able to solve the exercises that you prepared on a whiteboard from the first semester on. If you did not solve it sompletely you still had to whiteboard-code the parts of the solution that you were able to and improvise the rest.

For more complicated algorithms I still love to draw lots of pictures before implementing them.

I can understand that people complain that they have to solve tasks (on a whiteboard) that won't have to do much with what they will do at the job. But this has nothing to do with whiteboard interviews, but with badly chosen tasks (say: data structure brainteasers) for the job interview.

I have a question. When you hire complete freshers out of school, how do you interview them without asking for CS stuff? In my experience, fresh undergrads usually do not have any real world experience.

I have experimented with short take home projects. This drastically reduces the available pool, since great freshers can get job in other places with whiteboards interview. Also, there is a sense that interview process is used to get people to do free work.

Also, non whiteboard interview processes are extremely time consuming. Non-whiteboards interviews will be better. But as a matter of practical experience, I have found it to be the only viable option for hiring fresh undergrads.

Ask them what they studied and what their favorite courses where. Ask them what they found interesting. Ask them what kind of problems they can see themselves working on. Ask about what kind of projects they did and what they found easy and hard about them. Ask them some general programming questions. And by all means ask CS questions as well, but don't only ask that, and don't make them carry more weight than any other questions.

You will be surprised how often people with good grades will have good answer for these questions. But can't write dual for loops.

I agree that asking people to write dynamic programming problems are not good interview questions. But if the job requires primarily writing code, I think it is fair to ask people to solve coding problems with loops, conditional statements, hashtables, etc.

OMG it's happening!

I was always puzzled by how unrelated metric used for hiring is to actual work at many companies. I don't mind algorithmic brainteasers - on the contrary I would absolutely love to solve such problems at work yet those seem to be scarce. I also don't mind other types of developer work - be it more research or architectural oriented or "just" coding.

But I am not happy learning something different for interviews every few years and then forgetting it over the course of next period of day to day work. Feels like a waste of time and is confusing in regards to what one would actually do in certain job.

Am I the only one who thinks the pressure a whiteboard adds is actually something valuable? In my last job there were a lot of situations where you had to solve a real problem under extreme pressure. People who couldn't handle it simply weren't cut for the job.

I don't think all interviews should use it but for us it was good for filtering out candidates who would not perform in those situations.

In my experience, if you're regularly coding under extreme pressure, something is wrong.

Whiteboard coding is also a different kind of pressure. I can't imagine a situation where someone who already knows the answer is watching over your shoulder critiquing you while you work on a problem.

> In my experience, if you're regularly coding under extreme pressure, something is wrong.

right? unless you are in downtime mode, there's really something wrong with your schedule/deliverables/tasks.

writing code is not a race, it's more of a marathon where you have to be at a healthy pace to get to the end, otherwise you will not be able to finish.

In an ideal world, yes, but for a company on a tight budget it's not that easy. When the alternative is bankruptcy you don't have much choice.

What real life pressure do you think a whiteboard test simulates?

I think it's unlike any real situation, and you're fooling yourself that one kind of stressor is like another.

What on earth were you doing that required solving software problems under extreme pressure?!

This sounds like an incredibly toxic working environment, where developers are either being allowed to create fires on production systems, or management is completely incompetent or both.

Also, being able to "handle it" has nothing to do with anything in this discussion. I'd much prefer to hire someone capable enough to not need to "handle it" than someone proud of the fact that they often s*it the bed but are willing to jump up and do the laundry under pressure.

Toxic or not it's the reality for many companies. You speak like everyone has unlimited resources. For us it was a combination of legacy code and working with big external parties that set the rules (government APIs etc).

I personally enjoyed it but it's not for everyone.

I find the pressure of significant code on a whiteboard to be very different even from producing under pressure.

I think it comes down to expectations - in an interview I have to focus on impressing my audience and that's always there. In real life, I can focus fully on the actual problem being solved.

Have you ever tried to bring a laptop to an interview, and asked to use it instead of the whiteboard? Believe me very few would object.

It was not really a whiteboard exercise, but I once had an interview at which we went through some of my previous work (code-ish) on a whiteboard. I was mostly for structure, and I think that's a good example. Also there was a good relaxed atmosphere, so it was more a point of discussion. Got an offer.

Whiteboard interviews are important. Reason: http://i.imgur.com/oM9yCym.png

Ha, that's great!

Just ran through 2 complete interview cycles without touching a whiteboard once.

Got a great offer from both and accepted one of them.

I strongly prefer a coding project to a whiteboard. It is still a bit artificial, but it is way closer to my actual work.

I'm seriously tempted to make one of our technical interview questions "you're creating a technical test for role XYZ, how would you structure it" and see what the candidates come up with!

Would be interesting tbh.

That would be a great question for a tech lead.

Make a pull request.

I'd like to have a separate list for companies that do paid real-world projects instead of CS riddles.


Main main gripe with whiteboards is that it's hard to add extra lines.

If you're writing a loop for instance, you might code up the per-item logic first, and then you remember you need a loop variable. Which there's now no space for, because in a normal coding environment you can just insert the line where it's needed.

It makes me feel like I need to have solved the whole problem before writing, and that's just not how coding happens.

As a follow up to the first interview, we give promising candidates a quite simple task to solve at home and then send back via email the next day. This is used both for some basic evaluation of developers skills (whether they actually solved the problem, did they catch the small oddities hidden in the data set, did they document and write tests, etc.) and as something to discuss during the second interview, in which the candidate talks to one or two developers.

I also did a second code test, which I got to do alone on an offline laptop. The test had a time limit and it was designed in a way that you should NOT be able to finish it on time. This was communicated to me several times before I actually sat down in front of the laptop. The goal with this test is to see if a person can solve problems under stress, and if the person writes decent code that works, or just hacks something together and moves on to the next step - leaving a trail of bugs.

Overall, I think this has worked out great for us, but it might not be a good process for everyone.

Some time ago somebody mention in HN a process which sounded pretty good and fair. For promising candidates:

1) Meeting where you are presented a problem and asked to immediately give your thoughts on how you would approach it.

2) Some days, for example weekend, to work on the problem. Candidate gets a monetary compensation for this time.

3) Next meeting where you present the solution.

Yes, there's always a risk that somebody will get somebody else to write the code, but having to explain before and after reduces the risk. Paying for the work shows you are serious about the recruitment.

> Paying for the work shows you are serious about the recruitment.

In that case candidates should also pay companies to show they are serious about the position

You expect people to write documentation and tests for an interview coding challenge? I sure hope you mention that expectation or at least strongly hint that you're looking for a production-ready coding mindset. It would never in a million years occur to me to write documentation or tests for an interview coding challenge.

This honestly surprises me. The coding challenges are pretty much always simple enough for a junior to complete. The only way I've ever seen to set yourself apart from the bottom half of applicants is to include tests, comments and even documentation. I also wouldn't expect that to be in the instructions, otherwise everyone would do it and you'd be harder pressed to find candidates to eliminate. I'm curious to know if completing the challenges without tests etc is commonplace. Does anyone else think it's an odd thing to expect?

So the company gives you a limited amount of time and no access to the internet and expect you not to hack something together?

If I have a problem to solve "under stress", usually it's just a one off to fix something in production until I have time to write it, refactor it, put a unit test around it, and a code review.

Yeah why not just ask a surgeon to perform in a janitor's cupboard. No gauze? No problem, improvise with a mop. Remember, the aim is not to actually perform the procedure, just see if you solve problems under stress.

I haven't used a whiteboard in five years. Some people seem to love them, but I just don't get it - in any situation I might use a whiteboard for, I have more effective tools that I can use instead.

There's nothing like having to pull out your phone and take snapshots of a whiteboard because people can't be bothered to use something persistent.

Is pair programming really that different from whiteboarding?

More specifically, I would say "interview pair programming" isn't really "pair programming." When pair programming, it's not one person evaluating the other, under an artificial time constraint.

Can you really have any other kind of pair programming in interview context?


I don't see a problem with whiteboard interviews. I've been to a lot of interviews where they will ask me to explain an architecture and I ask them to do they mind if I draw it out while explaining it.

I've spent so much time in my career explaining architecture it's second nature for me to diagram while I'm talking.

During a whiteboard coding session in an interview, I would do some code and some pseudocode.

But at this point in my career if I saw that the interview was focusing more on coding than architecture/development process/automated testing that would be a big red flag.

If it lasts longer than 20 minutes, you're doing it wrong. But more importantly, are we talking about hiring a security expert? Or hiring for a product development team? Or hiring for the devops team? Context is rarely given in the "whiteboarding issue" discussion, e.g. at HN. I currently hold candidate interviews for a solution architect role, and I certainly don't want to hire someone who can't hold a whiteboarding session with a client or their partner's SME.

Whiteboard should not be the focus. If interviewer just shared whatever technical problem they're trying to solve and asked the interviewer to share their insight as a peer, then it's up to the interviewee to use the whiteboard or not. Frankly, I think this is best done over email so interviewee has the time to think about the problem.

In short, companies should approach interviews like free consultations with an expert or peer.

I personally love whiteboards...but only for discussion of ideas. But interviewers seem to use it only to dock points.

"Oh this guy solved the problem on the whiteboard with good syntax but he uses new and delete. Must be a bad programmer."

I think docking points on whiteboarding just reflects how bad interviewers are at judging skills, possibly because they themselves suck at self-awareness and skills beyond their domain.

Fog Creek is listed. I once interviewed there and was asked to calculate the volume of a cylinder. Have they changed there approach?

Calculating the volume of a cylinder is trivial.

What does that have to do with solving problems? If you can't do it off the top of your head, a quick search will give you the formula and then you plug it in. Questions like this make me not accept a job offer for a place.

In general I don't like questions like this either, but this particular one seems designed to test your wits if not your memory: if you don't happen to remember the formula, you can quickly derive it. I bet if you said out loud "It's the area of the end circle times the height of the cylinder", you wouldn't have to go any farther.

No. I said exactly that, and provided the formula for the area of the circle, and was still asked to do the calculation.

Yeah, it sounds more like "do you understand the spatial principle in determining volume" more than anything.

Why are trivial questions useful during recruitment?

What's the purpose of them?

This is great. I would add https://toggl.com/, but I'm just a silly marketer and don't have a github account. btw not affiliated with toggl whatsoever - just a fan :D

It looks like many companies in this list use the take-home project approach. The take-home project however is not a replacement to the whiteboard session, as they're not meant to screen at the same level. Once you're given a take home project, you're almost in. It happens when the recruiter really likes the person and wants to dig further with a real life scenario to see how that person would perform.

The whiteboard session is a way to spot people who can't code early in the process.

I work remotely and sometimes interview new potential hires. Does anyone have some suggestions for how I could imitate whiteboard questions without actually having a marker and surface available to me or the candidate? It's a little awkward to ask them to write on a piece of paper and angle their camera so I can see what they're writing! Is there any good software that can emulate this?

I do this all the time. What I do is make sure that the question, and anything that I would want to write on the whiteboard, is typed up ahead of time. Then, I use a teleconferencing program that supports sound, screen sharing, and IM. (Gotomeeting works well for me, but there's plenty others that work well too.)

In the actual interview, I IM the written parts of the question to the candidate. Then, I ask the candidate to share his/her screen, and use a program like Notepad or Textedit to complete the question.

It's useful if the candidate copy/pastes my IMs into Notepad / Textedit. Often I've provided a sample API that the candidate needs to program with.

I discourage using an IDE like Visual Studio or XCode. (They get in the way when you're just trying to quickly express an idea.) Most candidates "get it" that I'm just trying to see how they think and approach a problem.

Some candidates just feel more comfortable in an IDE, though. Other times, though, when a candidate uses an IDE it shows that the candidate will get too bogged down in details to communicate; or that the candidate really doesn't understand the language that he/she claims to be proficient in.

Edit: In the context of the article, I'm a HUGE fan of whiteboard coding. It tells me a lot about the candidate. Being on both sides of an interview; a successful whiteboard interview has more to do with how the interviewer plans the question and his/her expectations about what the candidate will answer. A whiteboard interview isn't there to determine that a candidate can think of a tricky algorithm; it's there to judge coding ability for very obvious problems with well-known APIs.

Where I think the industry needs to go, though, is a universal certification that demonstrates competence. Doctors have to go through this, thus doctors don't need to demonstrate competence when interviewing for a job. That's handled by the impartial 3rd party.

Let's agree Hackerrank sucks. Develop a shity solution in a very limited time, using ES5 and no libraries for JS, waste time parsing your input where the first element means nothing and the other must have the type converted several times....

If you really want to test a candidate, give them unlimited time, a "real problem" and something that they don't know.

Why something that they don't know? (Actually curious)

Most of your work is to learn stuff, to figure out solutions for new problems. However some people keep themselves in their safe-zone, avoiding new kinds of tech or solution. I belief that a good developer will like the feeling of frustration when learning something that they don't know, that a bad piece of code will make then uncomfortable.

The skill that I look for in another developers is the ability to learn stuff that they aren't trained. This way I know that they will be able to solve new problems, to find better solution for old problems. By testing then in something that they don't know, I will see if they are open to learn new stuff and able to apply new concepts, and ask how they felt about it later.

"Build MVPs for startups", from Popstand, is amazing

Whiteboards are MacGuffins for hiring conversations.

So what's considered to be "Pair programming", compared to whiteboard?

There is absolutely nothing wrong in principle with whiteboard questions.

I was far better at these straight out of university, as thats the sort of short assignments we got given.

15 years later I am a far better coder (I class myself as a full stack software engineer now) but I haven't needed to implement the sort of solution that they ask for in years (except in interviews).

There's nothing wrong with white-boards, it's how they are used that matters.

I agree, although I prefer a physical computer or shared session to whiteboard. I always tell interviewees that they can use pseudo-code or make up functions as long as they explain what they are intended to do. This takes the pressure off of knowing specific language constructs and leads to better overall assessment in my experience.

Meh, if you can't even handle a tiny whiteboard situation how are you going to handle any deadline pressure? It's not about knowing the right answer off the top of your head, it's more about how you work and approach the problem.

Sure some bad companies expect you to be able to write something very specific on the spot, but thous are pretty rare imo.

In what way do you think whiteboards are like deadlines? I honestly don't understand this line of thinking.

It's not about "pressure", it's about the fact I'm an introvert and do poorly at composing while speaking. My brain requires time to focus inward to formulate complex thoughts, and interrupting any time I pause narrating doesn't give me that time to think.

I've literally never had it be an issue in my career, because deadlines happen on the order of days to months and in real life conversations it's acceptable to pause talking for 5 minutes. (Heck, in real life, it would be rude to talk the way you do during an interview, because it wouldn't give the other person a chance to share their thoughts about the problem.)

>My brain requires time to focus inward to formulate complex thoughts, and interrupting any time I pause narrating doesn't give me that time to think.

So basically what you are saying is that you are some kind of snowflake that requires it's own room to concentrate and be productive and you can't even think while doodling on a whiteboard.

In my books that counts against you, that doesn't mean you are straight out, but it definitely makes you a liability to some degree.

Not even remotely.

What I said is that I share a trait with about 25% of the population, more in software, where I can't do my best composing if I'm narrating at the same time. By "time", I mean 30-120 seconds without having to speak.

In actual development that doesn't matter: pauses to compose an answer are fine in conversations and most work time is individual anyway (even in open offices).

I do note, however, you didn't actually answer my question: in what way is whiteboarding like real work deadlines?

746F7475 45 days ago [flagged]

>in what way is whiteboarding like real work deadlines?

hurdur, what is stress, I dunno. Why is everyone on this site suffering from autism?

We ban accounts that post like this. If you can't comment civilly and substantively, please don't post until you can.



Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact