
A Method I’ve Used to Eliminate Bad Tech Hires - zerr
https://medium.com/@ayasin/the-one-method-to-eliminate-bad-tech-hires-630d539b2e1d
======
mrcncpt
> I will purposely challenge their solution to see how they react.

> If they get defensive, it’s an immediate no-go. Remember, there’s a
> difference between defending which is good and being defensive which is bad.
> The difference is that the former is based on rational facts, the other is
> based on emotion.

Are you always able to challenge them based on "facts", or are you going to
challenge them with your current world view? I can see someone getting
defensive if your challenge is perceived as deliberately provoking.

~~~
pc86
If you're submitting something for review which is admittedly going to be used
at least in part to decide if you are "good enough" to hire, it would be
foolish not to expect to have to defend the choices you've made.

Using the author's definition, "being defensive" based on emotional responses
is more a character trait than anything else, and a negative one at that.

I would much rather work with a slightly above average or average developer
who is thoughtful and rational than a rock star who gets personally offended
if you question their use of $DESIGN_PATTERN.

------
tempestn
We recently started using the 'assignment' alternative to interviews and have
had great success with it, although only a couple hires to go by so far. One
issue I have with it though is that it asks a lot of candidates in terms of
their time. Paying for that time seems like a great solution, as well as
providing the other benefits you mentioned. Will definitely try this next time
we're hiring.

~~~
pc86
I did a(n unpaid) coding assignment for my current job several years ago.
Estimated time to complete was 1 hour, but I probably spent half my weekend on
it because I wanted to make a good impression.

Once I was in a position to offer advice and assist in the interview process,
getting rid of that altogether was the first thing I did. It took time, I got
nothing out of it, and all it showed the company was that given 40-50x longer
than necessary I could produce a working product.

~~~
tempestn
Yes, this is another problem with it. However, from my (albeit limited)
experience, it still beats classical interviews. Bringing people in and
enforcing a time limit could help, but many people get very nervous in that
type of 'exam' situation, so you lose a lot of the benefit (which is primarily
seeing how they work in the real world, out of a pressure-filled interview
situation).^

Personally, I prefer to make the assignments less open-ended than the ones in
the OP. That way hopefully there's only so much time that could really be
reasonably spent on them. It's still an issue, but (over-) paying for the "2
hours" helps too. The remaining downside is worth it in my opinion for the far
better filtering it provides. (Also, classical interviews are no picnic for
many people either. I know plenty of people who would rather spend all weekend
at home working on something than be subjected to a regular interview.)

^Also we're all-remote, so in our particular case we really couldn't do that
even if we wanted to.

------
awjr
I would qualify this better. Most people take a day off for an interview. If
you explained to the interviewee that they were to turn up at 9am, would be
given a small project using X technologies, to be completed within the day to
be ready for a code review at 3:30 and presented to the team at 4pm. Lunch
would be provided and they would be paid for their time.

------
junto
Another way to do this is to already have an _existing_ mini project with a
bunch of unit tests.

It should be functional but contains known bugs, of which, some are easy for
junior developers, and some are much harder for senior devs. Whilst I don't
like checking broken unit tests into source control, leaving a broken unit
test in the source code is also another good test of the candidate, to see if
they can fix the code to match the assertion.

It would also contain methods that a senior dev should be able to look at and
then appreciate that there is a optimization that could be made to improve
performance.

Everything is in a private Github repo that the candidate can be given access
to and a week to play around with, giving people time (some people have
families and busy lives) to complete.

They should issue a pull request when they are finished and their changes
should be well documented.

Most companies have an existing product and developers are needed to support,
fix and extend those existing products. It isn't often developers get
greenfield development. They need to fix code written by _other_ developers.

The really good developers:

\- figure out how to clone the repo

\- comment their changes well

\- fix all the bugs

\- figure out how to run the unit tests

\- fix the unit tests that don't work

\- optimize the poor performing methods

\- manage to issue a pull request

If the developer is junior, then they might miss a few bugs and the potential
optimizations, but if they can get to the point of issuing a pull request with
some bug fixes and good documentation, then they have done pretty well.

Most importantly, check the developer's knowledge beforehand. For example,
they might not have had Git experience before. If they can figure things out
for themselves, then it shows they can solve things for themselves without
someone else having to hover over them constantly.

Finally, if they ask questions to validate their assumptions, the developer
isn't going to be one of those devs that goes off on left-field for 3 days,
only to find out they got the wrong end of the stick. People that think they
understand but don't, are dangerous and expensive to your organisation in the
long run.

~~~
ahstilde
What company uses this methodology?

~~~
twic
e2x did exactly this. We had a tiny web app with one controller, and the
candidate had to implement some small feature. Looks easy. But some tests are
failing, some are passing incorrectly, there's misnamed code, dead code, and
various other traps. We did quite a bit of rescue work on projects screwed up
by other consultancies, so it was even somewhat realistic.

At Pivotal Labs, we have an interview project where the code works and is
tested, but is a horrifying big ball of mud. We ask the candidate to add a few
features, but really, they need to refactor heavily first.

------
andrewingram
For someone who is a full-time employee with no freelance work on the side,
this would add significant complexity to my income tax calculations compared
to what I currently I have to do (which is literally nothing). I'm not sure
i'd go through the trouble of that for a job interview, i'd probably actually
be better off financially by doing it for free :)

That said, it is a great approach.

~~~
blazespin
Yes, personally, I wouldn't do anything like this for less than a few thousand
unless for some reason I found the assignment intellectually interesting.

~~~
pc86
Two hours of your time is almost certainly not worth "a few thousand" dollars.

~~~
blazespin
Oh sure, but $200 is not worth the accounting / liability / legal hassle. Plus
it depends on the problem. If it's a tricky problem that wants me to solve
something that they're trying to solve (I've had that happen a number of
different times), than I'm not just putting in 2 hours. I'm providing them
with the wealth of my entire experience.

------
CrLf
By using this method, you exclude candidates that cannot (or are unwilling to)
allocate the time necessary to do the assignment, plus the time needed to take
care of tax-related issues just to be able to write you a receipt.

In other words, you are excluding people who have a job.

Also, I fail to grasp this idea that potential candidates must prove their
worth through numerous hoops to compensate for the interviewers lack of skill
in judging character and competence (i.e. seeing through the bullshit).

In any hiring process not only the interviewer is interested in avoiding a bad
hire, but the candidate is also interested in avoiding a bad employer. Both
stand to loose from a bad choice, but that's why even the most strict
employment laws allow for an experimental period.

Certain hiring requirements tell a lot about the culture of the company. Avoid
sending candidates the message that you have a culture of distrust. You may
end up loosing the ones you wished to hire.

~~~
Ologn
> Also, I fail to grasp this idea that potential candidates must prove their
> worth through numerous hoops to compensate for the interviewers lack of
> skill in judging character and competence

They say:

>> It’s very easy for someone to embellish (to put it generously) their
resume, and coding trivia can be memorized

There are arguments about the efficacy of interviews, but judging technical
knowledge is not one of them.

If I was hiring for a Java programming position, I might start out with an
easier question, like what is the difference between an interface and an
abstract class. Or ask what a class was, or an object, or a constructor or
method. Or what a static class variable was. Plenty of people with resumes
that look good will stumble through these questions, whereas others will give
clear answers.

Perhaps you can't tell if someone knows all they need to know or can do a good
job, but it is certainly possible to see who does not have the level of
technical knowledge you want.

------
chayesfss
Personally I'd ask anyone who asked me to do this to also complete a project
over the weekend. Just to see how committed they would be to his employees
problems.

~~~
gaius
I'd ask them to do my laundry and my shopping and entertain my cats. Cuts both
ways right?

------
bambax
> _but it’s really hard for someone to fake actual skill_

That's a funny way of putting it. If one can fake actual skill... well, hire
them!

~~~
johnchristopher
As a sales representative I suppose :).

------
blazespin
Giving them the weekend / take home is a bad idea. You might get some
enthusiastic fellow who spends 10 hours on something but claims to have done
it in 2 hours. You really don't want that. You want someone who can do it in 2
hours and not lie about it, otherwise you end up with a workaholic who will
burn out in short order trying to live up to unrealistic expectations.

Give them a cube and pay them to work in the office. Stop by to see how
they're doing and interact with them.

~~~
enraged_camel
Unless you pay your programmers by the hour, how long they take to complete a
project should not matter (within reason, i.e. a weekend). Judge the quality
of their code, not their coding speed.

In fact, intense time pressure has a very adverse affect on cognitive tasks.

~~~
Ensorceled
Well, except for the fact that taking 10 hours to perform a 2 hour task
doesn't scale when I want them to do 40 hours of work a week. I really, really
do want to judge people, in part, by their coding speed.

------
sfk
"If they get defensive, it’s an immediate no-go. Remember, there’s a
difference between defending which is good and being defensive which is bad.
The difference is that the former is based on rational facts, the other is
based on emotion."

And whose call is it to make that distinction? Presumably the author of the
optimal-hiring-procedure #122327 that has been posted here.

~~~
pc86
A pretty good example of "being defensive" would probably be accusing the
interviewer/author of not having sufficient knowledge/skill/ability to make
that distinction.

------
DanielBMarkham
I like this a lot. Works well for both parties.

Pro tip for candidates: after you solve the problem, when you share it with
the team this is a good time to show how friendly and open you are. "Gosh
guys, I've never been much good at TDD, but I limited the code to less than 50
lines, so I thought for such a small project I could squeak by" or "What a fun
project! Yeah, that screen design sucks. I suck at UI design. From a technical
angle, though, it was fun and I enjoyed playing with the problem"

People want to know both how confident/competent you are and how you handle
weakness. If you're an ace on the technology, can talk a good game, _and_
enjoy having people help improve your work? You're a keeper.

------
platform
I do not like the methodology. I was subjected to it 3 times in '09 during the
recession when nobody was hiring. I went to the next level of interviews in
all the cases -- but did not get those jobs at the time.

Hope will never need to waste my time on that again.

It does not test the speed and quality with which a candidate can absorb new
methodologies/ways.

Therefore it is perhaps, reasonable if the job assignment to maintain an
existing code base in XYZ tool chain, or participate in a larger team where
the architecture/design approach is done by somebody else.

------
timwaagh
well one shop i interviewed with found it necessary to put me to the test this
way (except them being cheap did not offer payment). I disliked the long
process and the fact that they would make me do work for free. so when
opportunity presented itself i signed with my current employer, who offered me
a slightly worse contract but trusted me enough to sign immediately on the
first interview. i think the method is only appropriate if you are looking for
high level people and offer a corresponding salary.

~~~
troels
In all fairness, the author explicitly stresses the importance of paying for
the work.

------
rdl
I would probably make the work product open source (voluntarily), and maybe
over a longer period than a weekend. Would still pay 50-150/hr for the desired
number of hours.

------
fred2133
Interestingly, this is exactly the same method I use to weed out bad
employers.

Maybe when the market's a bit worse I'll be forced to jump through one of
these hoops. But right now...

------
awjr
This is one of these articles that has changed my thoughts on recruitment and
makes sense when hiring mid/senior talent. Not sure the approach works if you
are recruiting junior/graduates but even then I see this as just a case of
extending the development time at a lower rate with lower expectations (which
hopefully will be proved wrong).

Front loading the cost of recruitment brings major cost savings in the long
run.

------
dvh
This gem:

> I may say please use functional reactive principals in JS to solve this
> problem, but the decision of using Kefir, Bacon, or RX is left up to them.

------
hliyan
I'm all for this technique (I might actually use it myself next quarter). But
there will always be a few bad apples who would go home and get someone else's
help (or worse yet, get someone else to do the whole thing).

~~~
olau
They would probably have difficulties answering questions about why they did
something in a particular way, though. The conversation you can have
afterwards is probably the most interesting part, also for the good apples.

------
davidgerard
I wholeheartedly support the approach of "get them to build something".

It also works really well for sysadmins: give them a subtly-broken system, get
them to fix it documenting their thinking along the way.

------
dvh
> did the person write a test for every line of code? No?

Does anyone expect this for 2 hour assignment? Seriously!?

~~~
Ensorceled
Yes. I've twice been criticized on "short" interview coding assignments for
not having _enough_ unit tests. I had unit tests, just not enough in their
minds.

One position was CTO and the other was a Software Director position.

Tech interviews are getting crazy.

~~~
pc86
Why is someone applying for CTO performing coding challenges?

~~~
sundaeofshock
Because the company is not really hiring a CTO. What they are hiring is a
senior developer who will put in a full week of work coding, and then will be
asked to put in another 40 hours dealing with management tasks. The "CTO"
title is there to try to make up for the fact that the position is woefully
under-compensated.

~~~
Ensorceled
Exactly. "Well, it will grow into a CTO role".

They also didn't have an answer to "How often does the board meet and how long
is my part of the meeting, typically?"

------
bambax
> _The solution addressed your problem using the technologies and techniques
> you proscribed_

Prescribed...?

~~~
junto
Yep, he meant "stipulated" I guess.

~~~
redblacktree
He certainly didn't mean "proscribed," which would mean that the candidate
used prohibited technologies.

------
brianmcconnell
I am with the commenters who have a problem with this being a weekend project.
How about assigning the problem on a Monday, and reviewing the results toward
the end of the week?

If they are asking you to do your interview homework over the weekend, it
seems likely they'll be "asking" you to work unpaid overtime, weekends, etc on
a regular basis.

------
mootothemax
The only thing I dislike about this is the short timeframe of interviewing on
Friday, and having to work _that immediate weekend_ on the problem.

I'm a busy guy, I have kids, and I like spending time with them (on top of the
2 hours I try to get in every day). That removes the weekend daytime.

Weekend evenings - well, realistically once everyone's fed, bathed, and had a
bedtime story, we're talking 7:30-8pm in my household - are then for eating,
spending some time with my wife (shocking, I know!), taking care of household
tasks, and then going to bed, ready for the inevitable 6am kiddie wake-up
call.

Yup, weekends are by far the busiest time of the week, and that's even when
everything goes right!

Give me a week, interview on Friday, deliver code/presentation by the
following Friday, then I'm absolutely game for it, will get it done, and will
get it done _well_.

That said: I guess this may be a deliberate filter against people like me.
Always difficult to know.

~~~
dagw
_The only thing I dislike about this is the short timeframe of interviewing on
Friday, and having to work that immediate weekend on the problem._

I suspect that that might be a secret feature. If you cannot work _that_
weekend due to other, more important plans, then that is a strong indicator
that you might not be able to work other weekends in the future.

~~~
mironathetin
"I suspect that that might be a secret feature."

Yes, agreed. It may also test, if you are really serious with your
application. If you are, there will be a way to free some time.

This goes both ways: companies blow time to interview candidates, who just
want to see how far they get. Or who plan to negotiate better terms with their
current employer (happened to us recently, we paid flight, hotel, lost 3
months and a whole application round).

Candidates blow their time by implementing stuff for companies, who just want
cheap code or who want the best solution for a problem they cannot solve, or
simply re-decide their plans after you invested your time for the coding.

Both sides should find out how serious the others are. Payment for code is a
good step. But if you pay, you can also ask for something, right?

~~~
pc86
Is your "application round" limited to a single candidate?

~~~
mironathetin
No, we had a couple, but after the one we offered the job to finally let us
know he wouldn't come, the whole shortlist had already found other
opportunities.

------
Kiro
In many countries you can't simply pay people unless they're incorporated and
can send invoices. You need to hire them. Otherwise it's tax fraud.

------
gaius
This is ridiculous. Getting paid would make taxes complicated and would count
as taking a second job, something forbidden by many employers. On top of two
precious days vacation. This might work for student interns but no one already
employed would bother jumping through these hoops.

~~~
tempestn
Is it really that complicated to claim a couple hundred dollars in
miscellaneous income on your taxes? (At least here in Canada it would be a
non-issue.) I'm sure applicants are welcome to turn down the pay if they feel
it will be an issue for whatever reason (including current employment
contract), but I can't see how that part is a negative. Certainly calling it
ridiculous doesn't seem productive. I do agree that the potential of blowing
your weekend is an issue that should be addressed.

~~~
gaius
Yes it is, in countries where normal people do not have accountants.

