"Everyone who builds a team of developers, and I do mean everyone, rapidly gets used to people turning up to interview who cannot actually program computers, even under the most generous definitions of the term."
I've started doing a small whiteboard coding exercise during interviews, and over the past couple of iterations it's devolved down to something simple...and I mean really simple. And I'm watching senior developers fail the test.
For example: I'm trying to fill a bootloader developer position so I get a couple of senior embedded developers to the table. Passed the screen, HR likes them, yadda yadda. Some even have U-boot experience. Should be easy, I think.
So I'll ask them to perform a simple bit-flipping exercise on the whiteboard. Nothing insanely tricky and it's usually something that any register-level developer should be able to blurt out without thinking twice about it. And they are choking on the question.
It's kind of freaking me out. Is the test that hard? Is the developer that inexperienced or distant from doing this kind of work even though they're applying for it?
The whiteboarding in the problem setting might be the issue. How many engineers actually whiteboard regularly? It's as old fashioned as waterfall design. Everything has a REPL shell (even Java!) now. Hell, even CloudFormation stacks let you fiddle with entire architectures through a command line. Engineers have always poked and prodded at things to learn and refine them. Whiteboards are for throwing around ideas; not for solving problems.
I spent most of my grad school years researching learning and memory. The medium and environment of the fact recall/problem solving can greatly affect performance. Now compound that with the stress of your standard technical interview and you'll get some flubs.
You might want to take a step back and consider how you're asking them to show competence in addition to what you're asking them to show competence of.
Would it really be any better if I handed them a laptop with Notepad or some IDE and asked them to type in their ideas?
What I'm asking is the bitbanging equivalent of "store 2+2 in a variable", so setting up an entire coding ritual for 3-4 lines of code isn't really what I had in mind.
To answer your question with a question - Would you be satisfied if they could solve the problem in that setting? Is making a laptop available to them for problem solving that much trouble?
Or, rather than looking for _any_ competent candidate, are you looking for someone like you and your coworkers, who can give direct answers without a pause? I ask because in one case it's more like an attempt to hire capable employees and the other is perhaps a subconscious attempt to maintain a monoculture.
You know that's a good question and, to be honest, if they could only solve this at a keyboard then I wouldn't be satisfied.
We constantly draw things out and scribble over it again when we're designing. If you can't translate and communicate to others then it's not a good fit for me.
> whiteboard ... it's as old fashioned as waterfall design.
Call me old-fashioned then. I like to map out the data flow and process flow a bit. A little planning now, saves quite a bit later. Call it, pre-re-factoring.
You don't go scuba diving, hiking, camping, etc, without a plan.
> It's kind of freaking me out. Is the test that hard?
You didn't show us the test so let me flip a coin...
One can get pretty far in embedded just knowing how to OR flags and mask & shift values. I guess that's something most embedded devs working at that level should be able to just blurt out. Is that what your test was about?
Then again, I'm sure you can find "senior embedded developers" who worked on the browser based interface to an appliance that runs Linux... and, well, the last few times I used U-Boot on our product I didn't really need to play flipper with bits. (Well, I did, even if I didn't need to.)
Here's the test. It's one I stole from an online test practice site but IMO it actually works well on a number of levels:
Give me a routine takes in an unsigned integer, pick any size you want, and returns TRUE if the binary representation has two consecutive 1s in it.
Test data: 9 = 1001b = FALSE, 13 = 1101 = TRUE.
I'll even give them an example function prototype:
bool adjacent_bits(uint32 x);
As I see it there are multiple ways you can write this and I really don't care how you do it. If you start with a simple case (say, a for loop), get it working, then realize you can optimize it...that's even better. It even optimizes down further to a single inline math operation.
The key is that I can see you understand bit shifting and a bit operator in C. If you've done this kind of work as your resume claims, you should be somewhere in zone on this without too much struggle.
I've seen candidates write triple-nested for loops. Bizarre interlocking bools trying to make a state machine. Hard-coded numbers that kill portability. Calls to printf where I asked for a boolean return. Attack ships on fire off the shoulder of Orion...
Perhaps, like you said, this helps weed out the people that just played with the GUI to configure something when I need people that got a driver working down at the register level. So I'm sticking with the test for now.
Perhaps, some of the time. I doubt it's the whole answer, though. I suspect that some of the answer is stress, and some is fakers, and some is people who have been in embedded and never had to do that. (I haven't for 10 years... but I'm sure I can still do it.)
why is it 'high-stress' ? desire to look good ? fear of being judged as a fool?
The ability to temporarily NOT GIVE A SHIT about what others think of you is a pretty valuable psychological/leadership indicator IMO.
I would want such folks, who may not be able to code up a binary-tree search, but would work through it in the same manner they would at home in pyjamas, to be team-leads or managers over time.
They all had a direct answer or something approaching it without even pausing to think about it.
It gives me reinforcing information that the problem isn’t out of line, but you make a good point that one shouldn’t lose a problem that nobody else at the table can’t solve.
People who can't get a job interview far more often than people who have a job. Rejecting a large proportion of candidates at the first step is not necessarily a red flag.
Off topic, but my mind immediately went to WW2 German submarines when you wrote this. I was wondering how you were screening so many 90+ year old German men for about half a second. :P
I've started doing a small whiteboard coding exercise during interviews, and over the past couple of iterations it's devolved down to something simple...and I mean really simple. And I'm watching senior developers fail the test.
For example: I'm trying to fill a bootloader developer position so I get a couple of senior embedded developers to the table. Passed the screen, HR likes them, yadda yadda. Some even have U-boot experience. Should be easy, I think.
So I'll ask them to perform a simple bit-flipping exercise on the whiteboard. Nothing insanely tricky and it's usually something that any register-level developer should be able to blurt out without thinking twice about it. And they are choking on the question.
It's kind of freaking me out. Is the test that hard? Is the developer that inexperienced or distant from doing this kind of work even though they're applying for it?
What am I missing here?