
On asking job candidates to code - lnmx
http://philcalcado.com/2016/03/15/on_asking_job_candidates_to_code.html
======
hnrodey
I'm going through the interview process this week at two different companies.
One of the companies I found through an ad on Stack Overflow whereby they
publish two programming "exercises/puzzles" with the opportunity to earn $100
for a correct answer to each puzzle ($200 total for two correct answers).
While I spent more than eight hours on both puzzles, I found them to be
thought provoking and I learned stuff in the process so I don't feel like I
wasted any time and I got a check in the mail for $200. My code was reviewed
by their dev staff so they had immediate feedback on my programming ability
and high confidence to bring me in for an on-site interview. I like this
process and consider it worthwhile.

~~~
ClayFerguson
What these companies have successfully done is weed out all the great devs who
value their time very much. Those who are desperate for a job will spend the
8hrs it takes on these silly tests and make it thru the process. Good
developers who can take their choice of jobs will just ignore these companies
who think they are being clever to require a test, and just move to some other
company what is not showing signs that they will be a pain in the ass to work
for. IMO, the more difficult the interview process, the more of a pain in the
ass the company is going to be once you're hired. If an interviewer is not
capable of determining my knowledge level by a normal interview that tells me
right away something up, and I probably don't want to work with them because
they aren't too smart themselves.

~~~
rhizome
_What these companies have successfully done is weed out all the great devs
who value their time very much_

I'm not sure I understand this. A dev who valued their time very much would
already be working at a job they liked enough not to be looking. As job
satisfaction declines, time opens up to work on whatever you want to call
this, a technical feeler, perhaps. A dev who values their time wouldn't get
snowed under by a ton of these because they wouldn't be applying willy-nilly,
only to those places they want to work.

~~~
mamoswined
Not for those of us who have experienced the pain of terrible management and
burnout caused by this. The last search I did I was also working 50 hours a
week under very poor project management. I had to rule out any prospects that
involved meeting with recruiters or these screens. I got a job I liked.

------
jcadam
A couple of months ago I did a 'homework' assignment for an interview (it
involved writing a simple REST service in go, even though I have at least one
personal project demonstrating exactly this skill -- to an even greater degree
-- in my github profile). During the phone-discussion after turning in the
project, the mid 20-something lead developer couldn't find anything to nitpick
(I could have) and even told me he thought I was a better programmer than him.
The conversation went very well and I felt confident I would receive an offer.

Now, what do you suppose happened next? _I never heard from them again_ and
all of my attempts at communication were ignored. I'm starting to think my age
is becoming a factor (mid 30s) -- and also these kids have no sense of respect
and professional courtesy. I'd like to say this was an isolated incident, but
that would be a lie.

Stupid me just spent his Saturday doing another such project, though at least
this one presented a more interesting problem. I swear, if I find out I had my
time wasted again I'm going into consulting. Or maybe I'll go to truck driving
school. Or open a cafe. Screw this "employee" stuff.

The fact that so many employers treat candidates like this tells me that the
whole "it's hard to find good developers" line is a _lie_.

~~~
dominotw
I've noticed that your github profile doesn't even get looked at until you
jump through all the standard phonescreen/homework hoops.

Even after than many interviewers look at your resume like 10 minutes before
the interview there is no chance that they would read through your code on
github. github as your resume is overrated.

I am curious to know if people hire or get hired purely based on their github
profile without a technical interview.

~~~
vkjv
A couple of quick notes:

* If you don't provide a link to your github profile, I'm not supposed to look at it. This an HR requirement. Other companies may use this same policy.

* I definitely look at github, but it's not always easy to tell if a project is original or not. Quite often I'm presented with either 30 forks, or a few projects that seem like they started as some boilerplate.

* Don't just provide github, also provide a specific project to look at. Even better, point out a particular section that you are proud of.

* Even with a well-stocked github, I will still ask you to code. This likely isn't to tell if you can code, but to figure out what it would be like to work with you.

~~~
billmalarky
My issue with the whole "github resume" thing is all my best code is in
private projects because they are related to serious side projects I intend to
monetize. I suspect the same could be said for any other developer who is
serious about their side projects.

~~~
justinlardinois
Your situation is just one of the many reasons for why hiring by GitHub is
bad. It selects for people who are willing and able to work for free.

------
overgard
I definitely wouldn't call myself an expert, but in my career I've probably
interviewed a few hundred people. I think people way over-complicate this
stuff by trying to be too clever.

To me, the most useful question is this: "here is an example project that we'd
be likely to do, how would you build it?" Then you just let them walk through
the steps they'd take.

From there you can launch into why they made the choices they made, trade-
offs, philosophy, etc. When you hear people explain _how_ they would make
something real, you start to learn a lot of things about them without even
needing to ask, and so you're less likely to get scripted answers. They're
also probably in their comfort zone at this point, so you get to see them in a
more real way.

I don't necessarily mind "homework" style projects, but I tend to shy away
from that sort of thing as it's not always respectful of the candidates time.

Two questions I usually avoid (but see getting asked all the time): puzzle
questions, and language trivia. I don't ask puzzle questions because I've
never seen it correlate to something useful. Most puzzles require an "aha"
moment where your subconscious bubbles up some kind of answer, but the easiest
way to short circuit that part of a persons brain is to put them in front of
strangers with a time constraint. Puzzles might give you a clue as to a
persons overall IQ and confidence, but it won't tell you a lot about how they
can do the job.

Puzzle questions are also "expensive", in that they usually put a candidate on
edge, and they set off the candidates bullshit detector. Let's keep in mind
that we're going to reject the majority of candidates, statistically speaking,
but you still want them to think highly of your company. If they feel like
they're being rejected for a BS reason, you've just created animosity towards
your company. It's important that the people you reject still feel respected.

I don't ask language or framework trivia, as I don't think it's useful. I
might lob a softball about a framework just to make sure they've actually used
it, but I'm not going to ask something hard.

~~~
whatthefox
+1000 This is how a proper professional does it.

I always have turned down homework assignments. It just feels insulting and
disrespectful. Even whiteboard crap is useless but I'm OK with it considering
they've paid for my travel expenses etc.

If you think about it ... homework assignments are a classic case of B's
looking for C's. A's wouldn't need to resort to such silliness. They can
identify (as an interviewer) and exhibit (as an A interviewee) with just an
intelligent conversation.

------
MangezBien
Interesting. I applied to DO about ten weeks ago. I had an initial phone
screen, and they sent me a code challenge. I completed and turned in the
challenge. Then I heard nothing for two weeks. I reached out to my contact
there, and she got back to me a week later saying, "I've been waiting on
feedback - sorry it's taking so long! I just pinged the manager again this
morning." That was the last I heard from DO.

I feel like I wasted the half-day I spent on their code challenge. I don't
expect a job, but a simple "Thanks but no thanks," would be nice.

~~~
jayjohnson
Nice. I had the same experience with DO last September...The same month
Braintree asked me to write a ruby project that took about 3 days and then on
the call only asked about python sqlalchemy database queries. I don't know
what the disconnect is with these larger companies and their HR staff? By the
time that third one asked me to write a node.js rest api from scratch. I just
chuckled and said thanks for your time...I doubt I will commit more than a few
hours in the future to any throwaway project for just a call back. Why can't
we get paid for the time spent writing sample code like a freelancer...If it
isn't good enough pay for my time (up to a max) and both parties can move on.
Time is more valuable than the risk of getting passed that initial
callback/more screens.

~~~
Can_Not
You should send them an invoice.

------
im_down_w_otp
Weird.

I've created from scratch: new CRDT data-types for geospatial problems, new
highly-available transaction patterns that provide useful causal history and
atomic visibility, large-scale messaging platforms (400k ops/sec), high-
throughput machine learning pipelines, lock-free lightweight-process mailbox
implementations, and an end-to-end build pipeline for generating bare-metal
rumpkernels to be parallel deployed to N-number of Minnowboard Max devices.
Among other things.

If you asked me to do homework to prove to you that I can "program", I'd have
a brief discussion with you about how misguided that seems, thank you for your
time, and walk away.

I mean fully "hundreds of connections" to a socket pool? Are you serious? If I
submit a solution that can do "hundreds of thousands of connections" that
includes QuickCheck tests for your protocol and my implementation, Ansible
scripts for deployment and orchestration, and a Makefile that builds it, tests
it, bakes it into a unikernel, and drives the Ansible scripts to push it out
and start the end-to-end load test, do I get to be CEO or something?

Why is tech hiring so full of strange, meaningless obstacle courses? Do we
just collectively not have better ideas?

~~~
estefan
Because there are a lot of shit candidates. You might know you're worth
talking to, but try hiring and you'll see why people want to separate the
wheat from the chaff.

~~~
im_down_w_otp
I have tried hiring. Many times. As recently as yesterday.

Using a combination of (semi-)rigorous personality profiling (time
constrained), and mapping the abstract problem domain of their prior work to
the problems I actually have for them to work on, it's been possible to align
new people with the work such that they're intrinsically motivated, and make
sure all the bases are covered for the necessary roles (hacker, inventor,
finisher, watchdog, etc.) to carry a very large greenfield project from start
to finish.

I've never once given a cute riddle, puzzle, or coding challenge as a part of
the interview process, and actively dissuade others from doing it as well.

~~~
tangled_zans
Can you elaborate more on your middle paragraph? It sounds very interesting.

~~~
im_down_w_otp
Sure. To start with there are many, many models for personality profiling. I
tend to stick to "The Big 5" mostly because it's foibles don't sufficiently
affect the outcomes I care about, and I've gotten proficient enough at it that
with the right set of questions in the right environment I can typically get a
pretty good read on at least 4 out of the 5 factors, and given a follow-up
interaction I can usually clear up anything that looks like contradicting
signals by getting additional context.

Like for instance, if you were to meet me, you would probably immediately peg
me as high on extroversion. While understandable given the outward traits I
project, you'd have needed to really thoroughly created a controlled
environment during an interview (requiring a series panel, a day-long
engagement, and an intentional highly-social event in the middle) to figure
out that I'm actually introverted, but have just learned how to benefit myself
by projecting extroverted traits.

Now the richness of any assessment is bound by time, planning, training, and
practice. If you want to get really good at it, you learn to do it in realtime
by assessing people you know well, assessing yourself, then moving on to
acquaintances, and eventually onto strangers. In each step you need to be
aware of where your assessments are drifting from the model. Like if you
assess yourself or someone else as high on agreeableness because you find that
you're very typically conciliatory or trend toward diplomacy and keeping the
peace amongst your peers, but you also are prone to "blow-ups" where you drop
the hammer on people in rare, but consistent, circumstances (when you've "been
pushed too far"), then you're probably not as high on agreeableness as you
think, and are probably higher on neuroticism than you think you are.

Given enough inputs and hypotheses you can get pretty good at this. Good
enough to provide useful points for making decisions given otherwise limited
information. More importantly you can get good at how you setup your
environments and your interactions to get higher quality signals from your
limited information. For instance, say I wanted to assess someone's "Openness"
because I wanted to bring them onto a team where their primary responsibility
will be in driving phases of invention. The simplest pass would be to just ask
them about personal activities and look for signals of risk-taking. However,
one needs to control for the difference between risk-taking due to openness to
experience and risk-taking due to self-destructive traits. The former would be
awesome for the job role and the latter will bring down the whole ship. To
control for that you'd want to put additional emphasis on assessing
neuroticism and conscientiousness. Doing that can be as simple as leaving a
lot of awkward pauses in the conversation and see how they deal with it. Does
it make them uncomfortable? Do they feel compelled to fill the air with
something? What do they fill it with? Relatable stories, trivia about their
work history, etc. When they talk about anything, not just work-related things
(sometimes specifically not work-related things), what level of detail do they
provide? Is it consistent? Is it always TMI, always 30k ft view, or can they
move fluidly from one to the other?

Things like that. Different kinds of job roles require a different kind of
personality profile to be successful, and more important than that, the
composite of personality profiles filling those job roles is a requirement for
a healthy team where each person is competitive with themselves, but
cooperative with their peers, and has the right mix of people to own each
phase of the development and business process.

~~~
tangled_zans
Hey, thanks for the reply, your insights are amazing. Totally with you on the
whole "being an introvert who learned the traits of extroversion" thing too.

I've played around with the Big Five model ages ago and found it to be a
really good way to look at personality. I especially liked its usefulness for
self-assessment - like you say it's easy to self-deceive yourself into
thinking that you're much more open and non-neurotic than you really are, but
so long as you are honest with yourself it can be a good tool in understanding
your own strengths and weaknesses. I hadn't really tried practising it on
strangers, though, that would definitely be a useful exercise!

I'm actually in the process of setting up a crowdfunding campaign for a
personal project that I'm working on, and if that goes well then I'll be
thinking about bringing in more people on board - hence why I asked you about
this.

I have to say, the way you're going about this is really impressive in terms
of methodology. Most high-level advice in this area that I've seen is full of
hand-waving and vagueness, but you've got the whole thing down to a science.
Especially the part where you talk about counterfactuals! So many people
overlook that.

If you don't mind answering another question, I would appreciate more of your
thoughts on this. You've said that you look for particular combinations of
traits to fill in particular roles (hacker, inventor, finisher, watchdog,
etc.) - what are the traits that you look for in each of them and how exactly
do you envision each role's responsibilities within the company? (Is there a
set number of such roles that you discovered or do you find yourself inventing
new ones as the company evolves?)

Also, it sounds like you've been doing this for a while and probably have
quite a few people on board, which would make the dynamic of hiring someone a
bit different than if you were just starting out. If you were back to square
one and had to consider hiring your first employee or even finding a potential
co-founder, how would you approach that?

~~~
im_down_w_otp
Sure, do you want to take this conversation offline? This is just going to get
buried deeper and deeper and eventually become locked when it's stale enough.

~~~
tangled_zans
Yeah, good idea! My e-mail is zans.lancs at gmail dot com

------
orng
I really hate having to solve these kind of programming challenges. I've
already sent you my CV, with references from former/current employers and co-
workers. Call them up if you are worried that I'm lying on my résumé, don't
waste my time by making me work for the chance of maybe getting a job as only
payment. Doing multiple interviews already takes up enough of my time, seeing
as I'm the one who has to commute to your location.

~~~
CiPHPerCoder
The problem is: Writing good resumes and having a lot of experience with
programming is a good heuristic for "experience" and "knows how to play the
game" but not meaningfully useful for "excellent at programming".

Work-sample tests measure the latter.

A worker's experience is a heuristic for programming skill, but a heuristic of
a heuristic is like an average of a set of averages: Useless.

~~~
MiddleEndian
Just to play devil's advocate, being excellent at programming is also only a
heuristic for working as a professional programmer.

~~~
CiPHPerCoder
Sure, but it's O(n) rather than O(n^2).

Hiring decisions are hard to generalize, because different companies have
different needs at different times.

~~~
vonmoltke
Which makes it extra frustrating that hiring practices are the most often
cargo-culted practices in this industry. I feel like the root of hiring
problems is that most companies know neither what they need or how to identify
those features in a candidate.

~~~
CiPHPerCoder
Sounds like a lucrative itch to scratch.

------
neoCrimeLabs
I just want to create a well documented simple resful API for applicants to
use. If an applicant wants to apply for a job they have to write code in the
language of their choice to apply using the API. Upon successful application,
request their SSH public key and supply a git repository they can check their
code into. A hook script will email me upon successful commit.

A moderately experienced dev will not spend much time to apply, and frankly
would probably enjoy the interface better than most job application websites.
:-)

A lesser experienced dev will end up spending more time, but really if they
are successful we would still want to look at them.

~~~
ClayFerguson
And the GREAT devs like me will not waste their time on your antics and will
look elsewhere for a job at a company that is less of a hassle to deal with.

~~~
steven777400
Great devs never apply for any job using the traditional job application
(except maybe as an after-the-fact HR formality); they are always recommended
and hand picked by their network.

As a intermediate (after so many years, should I just replace that with
'mediocre') dev I think a test like this would be a fantastic way both for a
company to judge me and for me to judge what a company would expect from me.

There's a huge variety in dev roles ranging from challenging to code monkey
and it's sometimes hard to figure out from a job posting or even conventional
conversational interview where along the spectrum the expectations are, but
establishing those expectations on both sides is critical.

~~~
odonnellryan
That isn't true for all devs.

~~~
steven777400
I shouldn't have said always and never. .. how about "most of the time"?

------
sixtypoundhound
The real problem here isn't coding interviews; its how we (hiring managers)
approach the entire question.

The goals of a technical interview should be: \- Confirm the credentials
presented aren't complete BS \- Validate they know enough to be immediately
helpful \- Assess their long term potential (brains, social skills)

Coding challenges tend to be good at the first two points but very
questionable for the third. Incidentally, given that I'm seeing a 50% washout
rate from a basic SQL code question (supported by confessions from candidates
that they fabricated credentials/experience), you would be silly not to
include some form of practical test.

The third point (potential) is best measured by talking with them about a
project they really cared about. For those who cannot share work examples,
pick a good side project. Don't get hung up on the content - focus on their
passion. You'll get a better view of my capabilities if you chat with me about
building an ad server, mobile optimization, or SEO analytics (real world
projects I care about) than with a contrived example.

As for the github profile of code _shrugs_ why? Seems like a bit of a cult to
me. I'm happy to sharing anything that's useful (and do so on my blog) but
don't have the free time to spend developing code for the sake of "work
samples". My work samples run on production servers as commercial projects,
thank you very much..... Suspect many other good candidates are the same.

~~~
dasboth
"you would be silly not to include some form of practical test"

Yes, I'm coming to the realisation that it's idealistic to assume that anyone
with N years experience will be at a certain level of ability.

"(potential) is best measured by talking with them about a project they really
cared about"

I agree. I've always thought that once you've made sure that a candidate's
programming knowledge isn't fabricated, the best way to evaluate them is to
get talking about code. Whether that's proprietary code they've written but
can't share, or it's on a GitHub profile for all to see shouldn't make a huge
difference.

~~~
cdibona
One of the best interviews I 'gave' was when the interview was for a position
programming on a topic I knew almost nothing about. I had the person bring
their laptop and walk me through their implementation of the topic at hand,
then asked how they'd scale/improve it given more cpu/memory/etc and what kind
of constraints the larger environment would impose upon the work.

It was super neat. Totally hired that person.

------
tptacek
Like the author, having run a hiring program for several years based on coding
challenges designed to mimic the work the team does (ie, work sample tests), I
do not understand why any tech company does developer hiring in any other way.

I particularly like how this team iterated on their uploader problem,
abstracting it out to a socket server and providing a test suite for it.
Reacting to ambiguity by _abstracting_ the problem was clever. And, of course,
a great illustration of how a standardized hiring tool can, like any software
project, be iterated on and improved --- unlike a free-form whiteboard
interview.

~~~
jasode
_> (ie, work sample tests), I do not understand why any tech company does
developer hiring in any other way._

Because _every_ method of hiring has its own biases and tradeoffs. Requiring
work samples biases against programmers who aren't interested in doing a
"homework assignment". For example, I don't enjoy whiteboard interviews but
I'd rather write some fragments in front of the interviewer and "think out
loud" instead of working on a multi-hour (or multi-day) coding project.
Writing on a whiteboard is just not that big of a pressure cooker to me.

One could argue that the _majority_ would prefer self-paced work samples
instead of whiteboard interviews and you're optimizing for a wider candidate
pool. That's fine but it's still a tradeoff. I think you're running a smaller
scale boutique firm so work samples makes sense for you.

For a company like Google Inc who is more selective[1] than Harvard
University, I don't see a compelling need for them to rely on work samples.
That company is so desirable for programmers that any "work samples" Google
tries to standardize on would get leaked and endlessly discussed on
stackoverflow (see FizzBuzz for precedence.) If Google has to keep changing
out the work samples to stay ahead of the street knowledge, you've lost the
ability to _compare_ work samples over the years which was one of the
motivations to use that method in the first place.

At their scale and selectivity, the "study your algorithms book and prepare to
be grilled on the whiteboard" seems to work best for them. Yes, they reject a
lot of false-negatives but it's offset by ... their scale and selectivity.
They can _afford_ to reject false-negatives. If their screening methods were
truly terrible, I don't see how the label "ex-Googler" would have the currency
it does in the marketplace.

[1][http://thenextweb.com/google/2010/09/14/one-half-of-one-
perc...](http://thenextweb.com/google/2010/09/14/one-half-of-one-percent-of-
google-applicants-get-hired-heres-how-it-works/#gref)

~~~
tptacek
The "no homework" thing is such a red herring.

Companies that do work samples _minimize on-site coding interviews_ , because
the whole point of offsite work sample testing is that most candidates can't
provide an accurate picture of their aptitude in an on-site coding interview.

No interviewing software developer prefers an extra 6 hours of on-site
interview to an off-site coding challenge.

The problem developers have with "homework problems" is really a problem with
bullshit companies that won't provide timely feedback on their submission.
Nobody wants to do homework that goes into a black hole, only to find out 6
months later that they were in consideration for the role. That's a problem
with _all_ hiring companies, and it is orthogonal to the structure of the
interview itself.

~~~
jasode
_> No interviewing software developer prefers an extra 6 hours of on-site
interview to an off-site coding challenge._

Sure, Google/Microsoft have the legendary "6 hour" marathon interviews with
multiple teams with lunch in the middle but for smaller scale of companies,
they don't have bandwidth to mess around with all-day interviews. It's ~2
hours. In this thread, it shows I'm not the only one who prefers onsite
whiteboard over homework projects so the "no developer prefers" claim is too
absolutist.

 _> The problem developers have with "homework problems" is really a problem
with bullshit companies that won't provide timely feedback on their
submission._

Regardless if the feedback is received within 10 minutes or 10 days of
submission, if I apply to 4 companies, _I don 't want to do 4 homework
projects_. I simply don't. At least with 4 onsite interviews, I see glimpses
of the office and meet the interviewers.

~~~
tptacek
This is just not true. I talk to interviewing developers every day, and very
few of them are interviewing at Google.

The final round of on-site interviews usually eats a whole day, and is
preceded by a whole battery of time-wasting phone interviews, which themselves
eat as much time as a coding challenge and often involve coding _with someone
else over the phone or Skype_.

There are companies that are not Google that have _multiple rounds of on-site
interviews_.

~~~
pyb
Are they not only getting very desperate candidates ?

~~~
tptacek
No. Everyone interviews this way. The very most marketable candidates leverage
personal brand and connections to dodge a lot of it, but you have to be good
at sales to do that.

~~~
pyb
Perhaps in the US, but here in the UK, in my experience, companies have
shorter onsite interviews. (except for the American ones...)

------
jetcata
I am in the midst of my job search, fortunately I'm not currently working (due
to an international move) so I have more time.

So far, I have completed around 12 1 hour phone coding tests, the same amount
of 30 min chats with recruiters, I have 8x 5 hour on-sites scheduled, have
spent about 4 long days on one take-home assignment, 6-8 hours on another, and
have two more assignments to complete, both of which I estimate will take over
8 hours each.

I haven't had a weekend for the past 2 weeks because I've been working on
assignments and I'm pretty stressed and exhausted.

This process has made me consider changing careers because it's so exhausting.
When companies give you take-home assignments I don't think they realise how
many things you have going on at the same time.

Additionally, I've found that a one hour coding test has allowed companies to
move faster in the process with me than those who have given assignments. When
I receive an offer from a company I am happy with then I will tell those
companies with assignments I haven't finished that I won't be moving forward
with them and they will miss out on a great candidate!

~~~
ElysianEagle
> This process has made me consider changing careers because it's so
> exhausting.

Me too. I'm so put off by the entire process but it seems that to stay
relevant, you have to get lucky and end up at a place that keeps your skills
up-to-date with opportunities to learn and use the latest tech, or you jump
ship every 2-3 years meaning you're once again dealing with this insane
pressure of interviewing. Throw in ageism once you get past your mid-30s or so
and the mere thought of staying in the industry starts to look quite
depressing :-/

------
dandersh
Sorry but I will not write any code for you.

Furthermore, after providing ZERO feedback in the past when I have provided
github links, offline code samples, personalized cover sheets, etc. to
prospective employers this also will not be done. I'm not going to worry and
beat myself up because you can't be bothered to provide feedback even when
requested to do so.

Homework for programming jobs is just like homework for school: it is not used
to judge competency but to demonstrate that your time is not your own as it
belongs to someone else. In the case of programming jobs the homework
assignment also acts as a filter to weed out employees who will not be
properly subservient to the employer.

------
innertracks
Last week, I finished 2-3 hours of on-site interviews which required a flight
and an overnight stay after an initial phone screen, a phone technical
interview, and a short technical challenge online.

I was surprised how a couple really basic concepts evaporated from my head
during the in-person white boarding. The questions though were reasonable and
the interviewers were friendly. All in all a positive experience just an
incredible investment of time it seems for all concerned.

The take away is I need to practice solving problems on a white board with
just 3 hours of sleep. The sleep deprivation part might be the most important
factor to simulate.

Hm. Maybe bourbon? Or a good cider?

~~~
grp
Linden tea, orange blossom tea or mix of both. Have a nice deep sleep.

~~~
innertracks
Noted. That sounds good. Thanks!

------
gorpomon
On paper (or blog), having a candidate write code for an interview is better
than a BS whiteboard session.

In practice, it means I spent 20 hours on my last piece of sample code I
submitted to a job I was excited about. Only to be told no thanks, with the
feedback of:

"Your app meets what we spec'd out, but it differs from our style."

And to be clear, after I repeatedly asked for what they were looking for in
this code, they replied "we are purposely providing nothing except functional
requirements to see what you produce".

I could have given them exactly what they want, but I guess they'd rather just
wait and hope that someone magically codes the same way they do?

Honestly I should have billed them after that.

------
aprdm
A startup in Berlin gave me a code challenge that would take at least 20
hours, I had even to deploy in specific version of the libraries, they give
the datasets and etc.

I was very suspicious of being an actual feature development. I've said I
would only do that if they would pay me to.

They answered that they feel very sorry for me because a lot of the candidates
feel the coding challenge very challenging because they can learn new stuff
and they love it

go figure

~~~
colmvp
I had a pretty famous startup in Toronto decline a second interview because I
flat out refused to do a take home test that probably would've been at least
worth a day's exploration.

In comparison, a YC company (who eventually got acquired) actually responded
very positively to my suggestion that I get paid to do the test they required,
not dissimilar to bringing in a freelance contractor for a day.

------
Tunabrain
> As brain-dead as I was after eight hours at my project, [..]

Is giving out an 8+ hour programming project as part of the application
process really considered good hiring practice? If you apply to five jobs,
you're expected to spend an entire unpaid work week writing code that doesn't
benefit anyone? If a company receives 20 applicants, their ideal candidate
selection wastes a collective work month?

I don't really understand this ideal. What's the point of maintaining an
online portfolio and a github profile full of code if companies prefer to
waste your time writing a pointless uploader?

~~~
philbarr
I got laid off recently and was applying for all the jobs I could find. So I
had to do a bunch of these programming projects. I don't mind them per-se, but
it takes so long to do each one and you have to be ultra-clean with each one.
I don't think companies take into account that you might not have that much
free time. It's particularly stressful when you don't get through.

On pair programming tasks, I had a pair programming session with a guy, he
gave me an empty project and a poorly specified brief, verbally, then gave me
15 minutes to crank something out. It was pathetic. I didn't get the job and
the feedback was, "he didn't know some of the keyboard shortcuts I'd expect."
In hindsight, I'm glad I didn't get that one but at the time things were
getting desperate.

The best process was the one where I finally got a job. We talked on the phone
for an hour and a half and by the end you could just tell we had that,
"developer bond". His approach was the best I'd seen. He just talked through
some of the problems he had, I proposed solutions and we talked through pros
and cons. He treated me as an equal from the start. After that we had the
face-to-face HR questions (how do you resolve conflict, etc.) which they have
to ask because it's a big company, and another one with another manager but by
then it was pretty much a done deal.

And I think that's the way to do a decent hiring process. Don't behave like an
authoritative dick, and use phone interviews to screen candidates. Programming
projects don't work because it's going to take you forever to review them, and
pair-programming doesn't work because there's not enough time, and it's kind
of horrible for the candidate.

~~~
rbritton
I'm actually in the hiring process at a place currently, and their process is
reasonable for someone in my position coming from freelance:

1\. Phone screen amounting to about 45 minutes.

2\. Skype interview with small code samples between two interviewers at 45
minutes each.

3\. 2-week paid trial doing real work.

I had #2 yesterday and am currently waiting to hear back from them about
moving onward.

~~~
ctdonath
It's #3 that makes all the difference: a task that means something, and makes
it worth one's while.

Duolingo wanted a functioning nontrivial (albeit sparse) app, expected to take
10 hours to write. I found it amusing on its own, but interviewed & got
multiple offers elsewhere in less time.

~~~
ThrustVectoring
That's actually harmful for getting quality candidates - "hot" prospects nope
out of the process when they get an offer elsewhere.

A clever option is emailing the recruiter at Duolingo with something like
"hey, I've got a competing offer, so the 10 hour coding challenge is really
not something I have time for at the moment. Is there any other way to proceed
through the hiring process?" Either way they answer, you're okay with.

~~~
devonkim
I did that before during an interview with a pretty good company overall and
the employer refused to accelerate the process or make an exception and just
stopped the process.

------
Hermel
In my experience, asking candidates to read and explain someone else's code is
a much better indicator than letting them do some whiteboard programming.

~~~
vinw
Similarly I've found that talking through some deliberately broken code, and
what improvements could be made to it, gives a good idea of what a candidate
would be like to work with on real problems. It's more about the discussion
than the code changes they come up with!

~~~
cwilkes
Agree 100% especially since that's most of the daily job is fixing up old code
that was written in a hurry or for a different circumstance.

The ability to see what some code was trying to do and how to fix it and make
it extendable and or understandable for future generations is invaluable.

------
jessegreathouse
I think a coding test is the wrong approach. Typically a coding test is too
simple to be meaningful, or too difficult to be practical.

Instead I would have a Mock PR in a github style discussion forum. Have it be
a contrived example of some things that could be improved or some things that
could spark really good discussion. Give the mock PR link to the candidate and
ask them to Peer review the code. Make the discussion and comments of the the
candidate the basis for which you determine their aptitude. If they have a lot
of depth in their critique then you can be relatively confident how they would
interact with the team in a real work scenario.

~~~
LankyDataGeek
This is a great idea!

------
dhd415
I can see the value from a hiring perspective in asking candidates to code,
but it's no panacea. The bias in the interviewing process that so many
acknowledge still bleeds through into many of the coding projects in the form
of prescribed languages or frameworks. I suppose there's some merit to that if
the intention is to screen for experience in a particular language or
framework, but it does run counter to the commonly-expressed sentiment that
companies are looking for smart developers who are able to learn new
technologies rather than developers who have happened to use particular
technologies.

As a candidate going through the job search process now, though, I have to say
that it gets tiring to do a project or programming test for every different
company. I can juggle more traditional applications (phone screens, on-site
interviews) at a time than applications that require coding projects as I'm
limited by the number of 4-hour (or 8-hour or whatever) blocks of time I can
devote to completing a project for each one.

------
dpeck
I just instituted a coding assignment with our hiring (exceptions made for
those with extensive github/similar open source contributions), and so far
only have had three candidates do it, but I'm a fan.

Mine is crafted so a really competent developer could get it done in 30
minutes, and intermediate 2 hours and a junior probably not finish. I tell
them to send what they have in after two hours that I'm as interested in
seeing the structure and thought process than the final result. It seems much
better than having someone do it at the office or one a whiteboard for
accessing basic competence.

Yes it sucks, but after you get bit once or twice by hiring people who knew
all the right words but can't actually produce something useful you get a lot
more open to it. I'll gladly take the time to take you out to lunch to discuss
the job and code assignment and give feedback afterwards, investing a couple
of hours into finding your next job shouldn't be this huge ordeal and gives
off a diva vibe I don't want in my organization anyway.

~~~
jmiserez
_> I tell them to send what they have in after two hours that I'm as
interested in seeing the structure and thought process than the final result._

While this seems like a good idea on paper, you _know_ that people are not
going to stop after 2 hours unless you force them to (i.e. by doing the
assignment on-site).

 _> get it done in 30 minutes, and intermediate 2 hours and a junior probably
not finish._

What kind of exercise could a junior programmer _not finish_? Give up or
produce miserable code sure, but not being able to finish? Do you require some
really strange domain-specific knowledge?

PS: The website in your profile doesn't seem to be working.

~~~
dpeck
well with 3 attempts one went over and kept working until he got a solution
(took a while), another stopped at around 2 hours to submit and asked for
feedback, 3rd finished in time and is a badass.

You've be surprised. Maybe I expect too much, but the assignment is
essentially network aware k/v store with a single get command and set command.
Extremely poor mans redis and a step up from an echo server but for some
people doing anything over a network that isn't HTTP breaks them.

Thanks for website, my consulting/blog site that pretty much fell into
disrepair after taking architect position at a startup.

------
verelo
A lot of people here seem to suggest it is unreasonable to ask someone to do
some coding in an interview given that you've shown a series of projects and
provided references. Unfortunately I've been through a variety of interviews
where I feel the conditions which I was asked to code were unreasonable
(whiteboard, overly complex problems while under pressure, using a language I
was not comfortable with or just academic problems that are rare in the real
world), but I really think confirming their ability to write code, beyond
things like github repos etc, is an important step of hiring a developer
(because i've seen great github repos and references, but ended up with
developers that are very, very average)

I've only hired tens of people, and while that is not a massive sample size,
I'm still astonished by the simple questions we ask that people get stuck on.
The one that continues to shock me:

"In any language of your choice, write a function that takes an array of ints
and returns the sum of those ints (If you use a language such as php do not
use array_sum)"

I'm just looking for something like:

function sum($ints) {

    
    
        $sum = 0;
    
        foreach($ints as $i) {
    
            $sum += $i;
    
        }
    
        return $sum;

}

Then we follow up with something like: "lets say your input array contains
[4,-7,0], walk me through the code"

And we finally ask "talk to me about some things that could go wrong at
runtime that might cause issues with this function". Expecting people to talk
about overflows (maxint?) or if its a dynamic language like php, checking for
non-integer values.

I estimate that around 50% of people get ruled out during the "coding"
exercise mentioned above. It is astonishing that these people even apply for
developer jobs.

~~~
tomjen3
Imagine there are twenty candidates who can't code at all, so they apply
everywhere. There are also twenty candidates who can code, but are presently
unemployed because they all worked for SpoonRacket.

Now the entire lot apply for jobs at your company, and you hire two of those
who can code. Within 3 months the rest of the good ones have found jobs, but
30 others have temporarily been unemployed and none of the bad ones have been
unemployed.

So you interview again and roughly 40% of the programmers you interview can't
code. But that doesn't mean that even 5% of the programmers out there can't
code, it just means it is the same bad programmers that interview in lots of
places.

It is the same when employers say they are getting ten or twenty job
applications for every open position - that doesn't mean anything because
everybody who is unemployed applies to more than one place, but from only one
side of the fence it sure looks like that.

~~~
Chris2048
I'm not sure I understand,

> but 30 others have temporarily been unemployed and none of the bad ones have
> been unemployed

Is this 30 new programmers plus 20 of the original candidates who can't code?
Should that be "none of the bad ones have been employed"?

~~~
tomjen3
Yes, sorry.

------
bewe42
If a company asks me to do 8 or more hours of artificial coding test I usually
refuse, unless I really want to work for this company (because it is well
known in the community, which are few).

However, I'm happy to work on any real task for a reduced or limited fee. It's
a win-win: the company gets to know the candidate under real conditions, has
an actual task solved at a reduced price, and the developer does not give away
his/her time for free. Plus, if it works out, the candidate has learned
already the first steps about the company's system and process and can
directly build on it.

I'm always wondering why not more companies do this.

------
herval
The process I've seen working best to this day was at Pivotal Labs: pair for a
half hour to one hour with 3 different people in the team you're being hired
for. You get to measure everything, from communication skills to cultural fit.
Perhaps pair on an internal tool, or even better, an open source project (in
which case the candidate gets an extra incentive for not wasting their time).

If you're dealing with too many candidates, issuing a simple 1-hour coding
challenge will weed out the uninterested (and won't favor people that have 4-8
hours to devote to a huge coding challenge).

~~~
aprdm
I've interviewed there. Only two people I've paired with, one in the morning
and other on after lunch. I had two problems:

* The 2nd guy I've interviewed with was completely non interested on doing it. He was checking the phone all the time and getting off and on. I felt so awkward because of his lack of interest.

* I didn't get _ANY_ feedback at all, just got refused. After 6hours in their office...

~~~
herval
That's an orthogonal issue, though. I had the same experience during a
whiteboarding somewhere else...

------
thedz
I don't mind a 2 hour or even maybe a 4 hour assignment, but a project
estimated at 8 hours?

Yeah, either you pay me for that time, or I'll just look elsewhere.

(And granted, that might exactly be what the company is looking for: a way to
reduce the amount of applicants they have to sift through)

~~~
p4wnc6
The problem is, there are no realistic 2-hour, or really even 4-hour, coding
projects that faithfully map to on-the-job skill. 2 hours is too long for a
data structures riddle, but way, way too short for anything of even a tiny
degree of complexity. Let alone writing some unit tests, mocking up a design
and then scrapping it for something that makes more sense.

Timed coding tests are just a waste of everyone's time, whether it's a weekend
take-home thing, or a 30 minute whiteboard thing, or anything in between. They
do not account for individual developers' needs, like privacy, their own
editor or IDE, and they exist in this ether between bullshit trivia and an
outright working internship that ought to be paid.

The other huge thing is that in a lot of firms (I doubt Digital Ocean is like
this, but many firms are) "programming" is a low-status, junior activity. In
these companies, if someone asks you to write code in an interview,
_especially_ if you are an experienced candidate with a GitHub profile,
references, and project experience, then even merely _asking_ you to code is
more like saying "dance, monkey, dance" and setting the whole atmosphere of
your negotiations and interviews in such a way where you are the lucky low-
status person who should be so grateful as to receive a job.

Unless you get a really great cultural feeling from speaking with technical
team members first, the best general advice is to simply reject employers who
ask you to write code. If they ask you to write code before even speaking with
you, absolutely reject. Full stop.

I also think all candidates should fully reject internet-based timed "exams"
as well. Things like HackerRank are designed to commoditize software labor,
which is a creative design activity and if it even can be commoditized at all,
it surely can't be boiled down to data structure trivia that you have to code
around in a browser-based IDE with a literal clock ticking down.

Anything like HackerRank immediately screams "quantity not quality" and is
straight rejection-worthy, no questions asked.

------
vblord
When we hire, after a few phone interviews we give the programmer an in person
coding exercise that is very basic. Open a file and parse data, etc... They
can use the internet. It's very basic. 80% of the candidates cannot do that.
I'm not exaggerating. Maybe I'm missing something here, but it's pretty
astonishing to me.

~~~
autotune
Do you require them to do it in a specific language or let them choose which
one they are most comfortable with? How do you decide which programmers to
bring onsite? Have you considered giving them different options in the from of
doing a take home test or coming in and coding inperson instead?

~~~
vblord
We require a specific language... the language we use for the job. The phone
interview asks technical questions, a few design questions, and some
personality geared questions. The technical questions are basic and cover some
of the basic concepts of the language and basics of OO.

We haven't considered doing a take home test for 2 reasons. 1) We want to see
how well the person can get around windows/keyboard. I believe this is
important. 2) What we are asking to code is very basic. If we sent something
home it would have to be harder.

Are you thinking that the in-person coding would make them nervous? I ask
because we are clearly doing something wrong if it's hard to find candidates
that can do basic programming. Even if the person is entry level, that would
be fine as long as they are an A player at the entry level... if that makes
any sense.

~~~
autotune
Valid points as far as the language goes, but I'm not sure I agree with your
points about workflow... the primary goal should be to see if they can reach
the solution using the tools required for the job over _how_ the tools are
used in my opinion, and how much guidance they require to do so. I am thinking
that the in-person coding might make them nervous. As a personal example for
my specific line of work coding is more of a secondary requirement over having
knowledge of how systems work, and despite having contributed code to some
widely used open source projects and having enough knowledge to get by given
language given enough documentation, I would likely freeze up in an interview
if asked to code.

If I were you I would look for candidates that actually have some projects
they can showcase that proves they know how to code, and offer them the chance
to either do an easier in-person test or a take-home project. During the job
search I personally jumped at the chance to do take-home projects since I view
them as learning experiences, but for more experienced people I can see them
being viewed as tedious. Most truly passionate entry level programmers looking
for their first job and some direction I hope would jump at that chance to get
some more directed experience as well.

------
afarrell
One thing which I've wanted to see a company try is to give someone code to
read and asking them to walk through how they would go about understanding it.
I've done this once when I worked at MassChallenge and was trying to hire a
more senior engineer to work above me and I felt like it gave me a good idea
of his ability to talk through problems and work with others, but that was
only one interview and I'd like to see a larger sample size.

------
randcraw
Calcado describes his wish that the candidates be "more T-shaped", and not
just worker bees with narrow skills. But I see no mention of ways his more
successful hires "branched out" and expressed wider interests or capabilities,
presumably in in domains outside computing.

Here, do the T branches apply only within computing, like knowing more than
one software tool? If so, I think that's not what Tim Brown meant.

Presumably an interview seeking signs of "T-branching" would propose higher
level questions, like business acumen or domain curiosity or spontaneous Q&A
to better understand or elaborate the requirements needed to solve a
representative problem. Otherwise the "T" just equates to broad software
skills.

------
tetraodonpuffer
not sure why there is so much resistance here about doing an 8 hour test, I
might be in the minority but for a traditional whiteboard interview I would
spend a lot more than 8 hours revising algorithms and so on, and have a lot of
stress due to whiteboard coding not being anything like real coding.

Being able to take a Sunday to work on a challenging problem in my own editor
/ computer / dev environment sounds great! I wish that all companies added in
their job ads something like 'if you want to skip the whiteboard send us your
solution to this problem together with your resume' where the problem is a
representative problem of the type of work they do, then you spend the
interview discussing your solution as opposed to playing algorithm-roulette.

It would also cut down the uncertainty when you see a job ad where they ask
for all sorts of different languages / skills, if you have fun doing the
coding test, it's likely you'll enjoy working there, and vice-versa (in which
case you might decide not to even apply)

~~~
ryandrake
> not sure why there is so much resistance here about doing an 8 hour test

If you're applying to just 5 companies, that's a full (well 40-hour) work week
of effort for a small chance of going forward with the next stage of the
interview/hazing ritual. Might be good if you are already unemployed, but
incompatible with candidates who already work.

~~~
AnimalMuppet
Which means that, if you're hiring, _and you want only candidates who are
currently unemployed_ , it's a reasonable practice. But the best candidates
probably already have jobs, and you're not going to get them. So maybe that's
not really the way you want to do things...

~~~
tetraodonpuffer
not necessarily, if I am not unemployed I am not likely to apply to tons of
jobs, but be a bit more selective, and I don't see finding more than 1 job a
week that I would be interested in... of course I am not a typical developer
(C, python, back-end, security, networking, ...) so there are a lot less jobs
in my niche than say if I was a front-end person

This means that taking every sunday to work on cool / interesting self-
contained problems would be great, and much better than spending time revising
red-black trees and avl tree rotations and sorts and so on for the n-th time
to prepare for the whiteboarding

------
Toenex
I like the 'simplified job description'. All too often a job description is
simply a list of technologies the last guy claimed to be using/have used/knew
something about.

------
bryanrasmussen
I've only had to do 4 of these, as they aren't that popular in my corner of
the earth yet, but I didn't like them, I have a kid and stuff to do and I
can't spend my time doing 4 hours extra on the weekend unless it is a job I
really, really want.

First out of the 4 I did what was asked but got refused because I didn't clean
up the code afterwards ( I thought I'd been cool for demonstrating some lesser
known functionalities of the language and API I was dealing with). If I've
spent 4 hours doing it, I don't want to spend 2 hours making it really pretty.

Second out of the 4 I misunderstood the instructions so I went in and 'fixed'
their code to make it work they way I thought they wanted it ( was maybe too
tired since it was late at night after a day of working on a personal project)
This was actually a job that was close to one I really, really want.

Third out of the 4 I can't remember anything about it now.

Fourth out of the 4 they really liked what I did and we had a great interview,
and then I got turned down because they thought I wasn't really that
interested in their company (which was true, I was just fulfilling a job
search requirement while getting my startup going)

Aside from that I interviewed at a social analytics startup one time and we
ended up talking data mining and they asked me to do a 2 weeks project with
them to do a POC of sentiment mining Twitter/Facebook. I gave them what my
consulting fees would be, but they seemed really interested in me doing it for
free as conditional for pursuing employment there. So I never got to try that
coding exercise.

------
mjt0229
I wonder if maybe instead of asking people to code on the fly, it would make
more sense to ask people to review, analyze, and explain some existing code.

------
justinhj
A few years ago I was given a programming test after an initial phone screen
with a HR person. They said do it in any language, and it had a fancy
framework that let you edit and test the solution before you sent it.

I chose Clojure since I was learning it at the time. The problems were easy
and I made working solutions, but the framework wasn't working. I gathered
some call stack data and sent it to the people that make the tool. They
thanked me for the feedback and later fixed the bug.

I sent the completed code by email to the company with a note about what had
happened. Showing clearly that my code works in the form of some tests.

Then, on the basis of this they didn't proceed.

------
bcg1
> we would ask you to use just your language’s standard library, no third-
> party libs or frameworks... With the problem description, we sent to
> candidates a functional test suite, a binary that when started would try to
> connect to the candidate’s server implementation, open lots of sockets,
> sends lots of messages, and verify the results against what the problem
> description stated. The candidate was instructed only to send their
> submission once it passed the functional test on their local box.

Incidentally, this is exactly how the "Sun Certified Java Developer"
certification was structured. You were only allowed to use java.* and javax.*
packages provided as part of the JRE, and even among those things like the SQL
packages were off limits. Instead of sending a test suite, they provided an
interface you needed to implement and then when you submitted it, they had
some sort of test suite that tested your code for conformance to the spec (and
if it failed, you failed). If it passed, someone would manually review your
program to make sure it made sense with respect to the requirements (which
were intentionally somewhat vague, like real requirements) and if you passed
that step, you had to write an essay about WHY you made certain design
decisions (like sockets+serialization vs. RMI, etc).

This was by far the most challenging and fun task of its type that I've done,
and to this day if I see that someone has that certification I know that they
have what it takes to get things done. This methodology should really be
utilized more, although for many companies the overhead of designing such a
challenge would be way over the top. Makes me think that things like
[https://www.stockfighter.io/](https://www.stockfighter.io/) have a future in
our industry.

------
makecheck
Well, you can’t give people stuff to do offline because they cheat (you’ll see
the whole thing show up in no time as a Stack Overflow question). And you
can’t avoid coding questions during interviews because it is astounding how
much people lie on résumés; seriously, even people who describe their
background as “Advanced Knowledge of $LANGUAGE”, can choke when asked to do
the most trivial things in $LANGUAGE.

I try to ask questions that aren’t completely ridiculous, but definite tests
of knowledge. And in my mind I judge correctness not so much on remembering
obscure syntax but on how they approached the problem, and how much time it
seemed to take them to start doing something. I _always_ ask about things like
how they approach debugging, testing, etc. because these are important too and
they tell you a lot about how likely the person is to be able to handle any
problem you come up with.

------
lfmunoz4
Good for initial screening, but should not take more than 30 minutes. Else
IMHO laziness or incompetence asking a programmer to do an assignment. Any
good manager/developer/interviewer can sit with a candidate for 1-4 hours and
determine if his skill level and personality match the job requirements.

------
zby
"we sent to candidates a functional test suíte, a binary that when started
would try to connect to the candidate’s server implementation, open lots of
sockets, sends lots of messages, and verify the results against what the
problem description stated"

Codility.com is a software as service company that automates this.

------
LoSboccacc
we use this [http://play.elevatorsaga.com/](http://play.elevatorsaga.com/)
because it's extremely easy to understand, is open to multiple solution and
immediately sort out both who doesn't understand queues, priorities, closures
but most importantly who has completely no empathy and win level pushing
numbers instead of satisfying clients.

it also takes very little time to introduce and solve which is a plus since we
prefer to give it out in-office. but we do tell the candidate beforehand that
there will be a 15 coding challenge so they don't panic.

------
logfromblammo
You don't have to determine whether I am good at writing software. I put that
right on my resume.

You just have to determine whether I am a _liar_ \--or, more generously,
simply unable to assess my own abilities accurately.

------
joantune
"It turns out that most people would put together a Rails app with more lines
on their Gemfile than lines of actual code they wrote. Worse"

\- how is this worse then rewriting the wheel? write code only when you must

~~~
Detry322
I would argue it contributes to huge bloat. People pull in huge powerful
libraries to format a single date. In addition, a vulnerability in a library
you pull in could compromise your application. Even deeper, a sub-sub-library
(i.e. one you didn't even add yourself) could have a vuln that affects you.

------
yandrypozo
This article sounds pretty good, we need more people like you, but I think the
next step for DigitalOcean is not hire for resume, or at least giving a chance
to make one of those coding tasks

------
BBlarat
I like Jeff Atwoods approach[0], hire the candidate for a short project (2-3
weeks) let the candidate a feel for the product and team while getting paid.
And at the same time let the team get to know the candidate. I would love to
go through a hiring process like that :)

[0][http://blog.codinghorror.com/we-hire-the-best-just-like-
ever...](http://blog.codinghorror.com/we-hire-the-best-just-like-everyone-
else/)

~~~
ryandrake
Unfortunately, that approach filters out already-employed people whose
employers do not permit them to work simultaneously for someone else and/or
impose IP ownership rules (We own everything you write inside and outside of
work).

------
encoderer
I don't usually ask candidates to write code for free. They're professionals
and I expect them to _review_ code with me -- some of theirs and some of ours
-- but not write it. I find that asking a candidate to write code for no true
economic purpose that I will lightly read and then throw away does not inspire
somebodies best work and I can gain more insight by an exercise where i have
them read an explain code to me.

------
dmitrifedorov
That article is the best advertising for not working for DigitalOcean. The
management doesn't understand what makes a good programmer.

------
lmm
"Language standard library" seems like a very arbitrary standard to impose.
Some languages have a very large one, others a small one. Wouldn't it be
better to specify what "primitives" you wanted the code to use? (e.g. must use
raw sockets vs. may use a http library vs. may use a basic json-parsing
library vs. may use a full json object mapper)

------
p4wnc6
As I mentioned in another comment on this thread, one of the biggest issues is
framing programming as a low-status, commodity activity.

I doubt Digital Ocean does this since they want good people, but a lot of
firms absolutely design their hiring process to create an upper hand and lower
your status before you even walk in the door -- especially if you're an
experienced engineer.

Short timed tests or whiteboard hazing are the worst version of this. These
are the most blatant attempts to commoditize software labor down into a fixed
set of "primitives" like data structure trivia or riddles. This is what
organizations like HackerRank exist to do: allow big companies to suppress
wages by creating these sorts of commodization filters. They don't attempt to
capture the value of creative problem solving at all -- because the company is
not pricing the value of creative problem solving into the budget for the
position, since it's a rank-and-file commodity job.

For this reason alone, you are better off flat out rejecting anything like a
HackerRank test or similar online, interactive, short timed test. I would
absolutely go so far as to even refuse to write code on a whiteboard if asked
to do so in an in-person interview. It's fine to _talk_ about how to solve a
problem and spec things out. But the minute it becomes focused on actual
fucking syntax, it's game over. You are now a cog.

Longer-form tests are a lot harder to evaluate from the candidate's point of
view. Now you are asking for a significant chunk of my time, without paying
me. And I am still at the mercy of whatever you happen to think constitutes a
valid test. _I_ might look at your take home test and thing, "wtf? this has
nothing to do with real world work." Where does that leave me? Of course I
want to impress the hiring staff, but I also don't want to get roped into a
bad job, and a team that hires me to do real world work by using contrived
coding examples is probably not a good place to be.

Personally, I think it's a lot more useful to talk about code that already
exists. Either example code form the candidate, or example code that you give
to them along with some time to study it.

Engineers read code more than they write code anyway, and right when you hire
someone, even if you're a bleeding edge start-up, their short-term impact is a
lot more predicated on their ability to read code and quickly learn a new
system, not so much writing a bunch of stuff from scratch.

It's also pretty hard to fake competency when reading and discussing actual
code. If you don't know how something works, you just don't know. You probably
can't just google how the internals of some highly-specific ad hoc code works,
unlike data structure trivia (which is yet another reason why it just doesn't
matter how much data structure trivia you have memorized).

Basically, my overall conclusion is that when someone asks you to code in a
short, timed setting, they are basically lighting a cigar, fingering their
handlebar moustache, kicking their feet up on the table, and saying "dance
monkey dance."

Unless you are literally desperate for the job, you should reject it right
away. You are being positioned so that no matter how well you do on the test,
you are but a lowly code typist and they will not take your attempts at
negotiation seriously.

------
pnathan
My main hope when talking with a recruiter person is that we avoid the kabuki
theater. at this point: I'm good at some stuff, poor at others. My resume is
an adequate description of things I am good at. I'd like to talk personality
fit now, thanks.

------
merb
Flavio is a nice Guy! He wrote a great scala Database library, lately. (quill)

------
CaRDiaK
Not writing code for a potential employer or being asked to in an interview is
a warning flag to me personally. I feel better if they do, as they've seen the
goods to influence their decision on the hire. It's more about "How does this
person think, how do they approach the problem". How can you measure that
unless you're there? I see a lot of "I shouldn't have to do that". Well yeah,
ok. But if you're any good, you shouldn't have a problem with that either.

~~~
alistairSH
_Well yeah, ok. But if you 're any good, you shouldn't have a problem with
that either._

Whiteboarding, sure. Homework assignment that takes 8 hours, as the author
experienced? No way. I'm good, my hourly rate reflects that, and I don't work
for free.

~~~
stutsmansoft
See I'm the inverse. I'd take the 8 hour project over the whiteboard.

Thinking in a quiet, relaxed environment beats trying to reason through a
problem while also thinking about how you are "performing."

------
ebbv
I understand the appeal of having candidates write code as part of the
application process, but in my view it's useless and worse, it will drive off
some of the best potential candidates you could get.

First, why is it useless? Because it is an artificial and contrived exercise
and not real work. Even if you design your requirements to mimic your real
work environment, there's something key missing; the candidate has no
knowledge of your team and your environment. How I would write code for my
current team vs. how I would write code at any other job I've been at is
different. You're basically asking the candidate to take a guess at what kind
of paradigms your code reviewers like and what they don't like and hope it
works out. One team might find use of closures smart and great, and another
might find it unintuitive and adding unnecessary complexity. Your applicants
have no way of knowing and just have to hope they write code your team likes.

On top of that, one contrived exercise is not a good way to get an idea of a
persons' actual skill and knowledge. I'd much rather have a 30-90 minute
conversation with someone where nobody writes any code than to review a
contrived example. I get the appeal of the programming task, it can be done
asynchronously and thus doesn't require any developer time until you go to do
the review. But is it really saving you much time? Anyone who's code doesn't
pass unit tests or any other automated level of screening wouldn't probably
make it very far into a conversation either.

If I have an applicant spend 1-4 hours writing a code example I've had them
use at least as much and probably more of their time for far less benefit than
if I had simply called them up on Skype and talked to them about programming.
The conversation will reveal not only a lot more to me about their actual
skill set, but also their personality and how they might fit into my team. Now
I realize teams doing exercises aren't skipping the conversation step, but I'm
advocating for skipping the programming task step and going right to the
conversation if someone's resume is solid and they can intelligently answer
some basic questions (a handful of questions that should take no more than a
couple minutes to answer, not an essay.)

Second, why would I say it drives off good candidates? Because I know a lot of
really good developers (including myself) who skip any job posting that
requires writing code as part of the application process. Why do I do that?
Because it tells me the team behind this application is immature. They haven't
yet gotten to the point in their careers where they realize these exercises
are a waste of everyone's time. That means there's probably going to be a lot
of other young dude bullshit in the team that I'm not interested in dealing
with.

------
ClayFerguson
I think the reason so many companies resort to asking candidates to do a
programming project for the interview is because they are lazy and don't want
to take time to interview someone, or they suck at interviewing, or both.
Bottom line, if you can't judge someone's ability by talking to them it's YOUR
fault. You suck at interviewing. Last time I was looking for a job a company I
was interested in gave me this massive assignment and I just told them to take
a hike. I'm not spending all day writing some code for you for free sorry.
Sure there are lots of horrible programmers out there, and you'll end up
hiring them if you suck at determining what their skill level is via a
conversation. But I hope other GREAT developers follow suit with me. Refuse
these stupid programming challenges.

~~~
msy
Have you considered that part of the purpose of the challenge might be to
filter out particularly arrogant applicants who might be problematic in a team
setting?

~~~
qzxvwt
Could be, but that'd be an assumption at best, and "negging" at worst.

Also, one could just as easily see a "massive" assignment as arrogance from
the employer.

------
TheOneTrueKyle
I recently spent 4 weeks of my life interviewing for one of the bigger video
games companies for their web dev team.

After those 4 weeks, I didn't get the job. Around the 3rd week I felt like my
time was wasted. I want to work for this place, but if I want try again later
in life, I probably won't because it took up way too much of my time with
little to no value.

It sucks that I'm required to sacrifice my personal and vacation days to find
potential new employment.

------
mixmastamyk
Watch the recent coreos post/video, the CTO is probably a smart and nice guy
but he looks about 16.

~~~
dang
HN is not a place for discussing personal details about somebody, like how
they look. Moreover, that weird "but" makes this comment read like an attack.
Please don't post anything like this again.

We detached this subthread from
[https://news.ycombinator.com/item?id=11290180](https://news.ycombinator.com/item?id=11290180)
and marked it off-topic.

~~~
mixmastamyk
Sorry, wasn't meant to be an attack, just noticed that almost every startup
about page and video makes it clear that grey hair is not welcome, even at
exec team level. Can't delete now.

~~~
dang
Ok, but singling out a single company, let alone a specific person, isn't the
direction to go. Think of HN threads like a Markov chain: each thing you add
biases what others add next. A comment like the one you posted leads to an
ugly place.

~~~
Flenser
> Think of HN threads like a Markov chain

Fascinating! It's like Blindsight in reverse.

------
sauronlord
The best candidates are too busy kicking ass and getting lured away with huge
sums of money - and not waste time working for free.

