
A Method I’ve Used to Eliminate Bad Tech Hires - nickfrost
https://mattermark.com/the-one-method-ive-used-to-eliminate-bad-tech-hires/
======
zachd1_618
This. 100% this.

As many others are saying, I find that this is the most comfortable way for me
as an interviewee to demonstrate that I do in fact know what I'm talking
about. Put me in front of a room full of people and a whiteboard and throw
something at me, and I'll freeze. Freezing in that situation and being a bad
engineer are wholly separate things.

Anecdotally, when I get one of these problems, I spent 20 minutes reading the
prompt, looking at resources and making an outline of my code. Then I step
away for the afternoon and don't think about it. Let the subconscious
internalize the problem for a while and when I come back, I've got a much
better mindset to approach it. Solving new problems on the spot is just a
skill I don't have. I prefer to think that this is what makes you valuable as
an employee and engineer, solving a problem by the end of the day. Not while
under scrutiny. Personally, my resume and experience don't get me far
interviewing by the big companies, but on the occasions that I get take home
code assignments, I tend to get pleasantly surprised and pleased reviewers and
it takes me to the final rounds (where I face the dreaded white board and
actually-there-is-a-correct-answer situationals).

My two cents, this just really resonated with me. Does anybody else happen to
solve problems this way?

~~~
throwaway4job
This isn't a viable approach for much of the hiring pool, as a majority of
working engineers have contracts which explicitly forbid work for contract for
other employers.

As such, if you hire any of those, you're hiring somebody with a proven
willingness to ignore their contract, which is a strong anti-pattern.

~~~
zaroth
IANAL, but that's not what this is.

"Work for contract for other employers" could literally mean I can't help my
neighbor mow his lawn in exchange for a beer. In this case what it really
means is a non-trivial amount of work done for meaningful profit.

If I fix a friends computer in exchange for a nice meal, I am using my
relevant skills to perform work, but I'm not actually violating a contract
forbidding outside work in a way that could ever possibly result in damages or
firing for cause.

For a potential employee to be so _rule-stricken_ as to concern themselves
with performing a coding interview, that would be the potential red flag for
me. So this could actually be another positive outcome of this approach.

~~~
mcpherrinm
I fear deportation, being unable to ever enter - never mind work in this
country again.

You can call that a red flag, but it is the reality I live in. Maybe most of
your candidates don't have that problem, but I would definitely be eliminated.

~~~
zaroth
I mean, you could always just ask that the payment be donated to a charity
instead, right?

But since almost everything we do is in some way technically illegal, it is
important for everyone to be able to function under that premise.

I mean, just to cut the check technically the employer would need a W-9, a
work for hire contract, and who knows what else. Let's call the lawyer and
spend $5k deciding whether we can do this, and then decide 6 months later it's
too risky.... Or, we do it, get better hires, execute on our plans better,
meet revenue targets, and succeed in the market, by not following _all_ the
rules, just the rules that matter.

Knowing the rules to follow and the rules to forget is the hardest part, but a
crucial skill in life and business. The more you push the envelope, the faster
you can run, and the more likely you are to combust. See _Zenefits_...

------
tdeck
Love the idea of paying people. That's great. However, consider the
perspective of someone who already has a job in the industry. If every company
did this, I wouldn't be able to interview at more than one per week. Not only
that, but the Friday and Monday onsites mean I miss work on TWO separate days.
This would be a non-starter for me if I had any decent pool of interest from
other companies, although it's great for people trying to break into the
industry.

Another problem, depending on your philosophy. Specifying the exact
technologies someone has to use means that you've closed yourself off to
generalists, unless they have time to learn a whole new stack that weekend.
Not that it's rocket science to do so, but that turns a two hour project for
me in say Angular into an eight hour project in your favored stack. I can
always learn your stack on the job, as any good engineer can.

~~~
BigJono
I feel like it's in the best interest of the employer to leave the choice of
tools fairly open and see what the potential employee elects to use. For the
example given in the article (a basic web application), I'd be very interested
in whether the interviewee decides to go with a heavy stack, does it
completely in vanilla JS, or something in-between.

~~~
oddlyaromatic
I wonder about this. I'm pretty new to js frameworks and I see the advantage
when things need to scale or get very complex, and I get that companies use
these things. But for a simple assignment like this, is it better to use
vanilla js and maybe handlebars, or "show off" that you can implement all the
routes and whatnot in Angular, even though that's overkill? Or maybe the
interviewee's justification of their choice teaches you that they understand
the trade offs.

------
pauljaworski
This is literally the only way I've gotten job offers. I'm a very competent
programmer, but when I'm asked to solve some silly puzzle with someone staring
over my shoulder, I freeze under pressure and can not think at all.

Luckily not everyone thinks solving leetcode problems is a good way to hire.

~~~
nolite
I've been in situations together with clients deploying a product, them
presenting it for their managers, orpreparing for a big demonstration, and
something goes wrong due to some unforeseen environmental factor you couldn't
have imagined or predicted, and breaks in a way you don't expect, despite the
weeks of testing you've done.

You've got your main client staring over your shoulder, arms crossed saying
fix it, and every two minutes asking "is it fixed yet?"

You damn well better be able to code under pressure... And like a freaking
ninja

~~~
aianus
Most tech jobs aren't like that. If the job you're hiring for is one of the
few that involves high-pressure coding in front of strangers, then by all
means test for that.

~~~
edblarney
If you are coding 'in front of customers on the spot' \- you at the wrong
company :)

Coding under pressure is quite different from coding on a white-board in an
interview, often on an arcane CS problem.

~~~
ellisv
> If you are coding 'in front of customers on the spot' \- you at the wrong
> company :)

Or a procrastinator ;)

"Hi, one coffee please."

"Sure, just one minute. Ok, first I'll pip install stripe…"

------
mi100hael

        For example, I may say please use functional reactive
        principals in JS to solve this problem, but the decision
        of using Kefir, Bacon, or RX is left up to them.
    

These days I'm utterly unable to tell whether someone talking about JS is
being serious or if it's parody.

~~~
duderific
No, this is real...however, using reactive principals in JS is a bit esoteric
and definitely not something that needs to be covered in an interview (IMHO).
At my company when it comes to JS interviews we are more interested in proper
separation of concerns, understanding scope, using proper data
structures...you know, the basics.

------
ceronman
This is a great way of hiring, but unfortunately it doesn't fit in all
scenarios.

First, this model alone doesn't scale. Big companies interview hundreds of
candidates every week. It's not possible to design and evaluate a weekend
project for each one of them. This doesn't mean that they can't have projects,
but early filters are needed. That leads to the classic phone and F2F
interviews to rule out those who can't code. If you freeze in one of those
because of the pressure, you're out.

Second, for simple developer positions, such weekend projects might be very
good. But if you are trying to hire a senior developer who is going to solve
problems at big scale, then the ability to create a simple movie catalog
doesn't tell you anything. And there are no big scale weekend projects. You
have to ask about algorithms, data structures and other CS topics. These
things are important!

Bottom line, this is a nice hiring technique suitable for small companies at a
small scale. But it might not be that useful for big companies.

~~~
pgl
> _It 's not possible to design and evaluate a weekend project for each one of
> them_

Can't you just design and evaluate one project that they can all do?

~~~
ceronman
Even if you design only one project. You still have to evaluate them, have a
discussion, etc. That takes time. It doesn't scale to hundreds of interviews
per week.

~~~
user5994461
It's absolutely doable to evaluate one hundred take home tests in a week. It
is a full time job though.

------
lacampbell
I wish more companies would give example problems to solve in your own time. I
did one for a fairly large Australian company, and they rejected it. By the
time they got round to letting me know (very long turn around time) I had
already found other employment. But still, their reasons for rejecting my
solution - the output was not formatted to their liking, and they didn't seem
to know what an anonymous function was - was hugely beneficial to me. I would
not have been a good fit at all.

~~~
redwards510
I don't mind homework, but companies need to start specifying their salary
range up front. I've been having a lot of time wasted lately by getting
through Tech Interview #1 or Homework #1, only to find out their max salary is
laughable for San Francisco.

"You should just find that out upfront" I hear you saying. Well that is not
always possible. Recruiters either don't know the range, or lie about not
knowing, or you have to submit a big packet of info (cover letter, history,
resume, homework) before you can even talk to a human about salary.

~~~
peteretep
Recruiter here with a history of working as a developer

    
    
        > but companies need to start specifying their salary
        > range up front
    

The costs outweigh the benefits for employers (encouraging existing employees
to ask for more money, losing candidates who will accept significantly less
but wouldn't apply at a lower range, etc) -- so don't hold your breath.

    
    
        > I've been having a lot of time wasted lately by
        > getting through Tech Interview #1 or Homework #1, only
        > to find out their max salary is laughable for San
        > Francisco.
    

State it upfront when you submit your application, rather than "find that
out":

"I am interested in roles paying above my current salary of $xxx,xxx only, and
am looking for an improvement on that to reflect my increased experience and
the risk of moving roles"

    
    
        > Recruiters either don't know the range
    

Yeah they do.

    
    
        > or lie about not knowing
    

"Could you tell me about offers you've previously made for this client?" They
may also be lying about actually being authorized to recruit for the role(!!!)
so consider asking them for a copy of the technical specification. If it's a
larger company, ask what the salary banding and official job title for the
range is, then head to Glassdoor.

    
    
        > you have to submit a big packet of info (cover letter,
        > history, resume, homework) before you can even talk to
        > a human about salary
    

I had a hunch that roles like this weren't worth applying for when I was a
developer. As a recruiter, this has been almost completely confirmed.

~~~
MAGZine
> "I am interested in roles paying above my current salary of $xxx,xxx only,
> and am looking for an improvement on that to reflect my increased experience
> and the risk of moving roles"

Are you really advocating that people disclose their salary to a recruiter,
voluntarily?

What terrible advice. "How to immediately throw away your best bargaining chip
101".

~~~
peteretep

        > Are you really advocating that people disclose their
        > salary to a recruiter
    

Yes

    
    
        > What terrible advice
    

No

    
    
        > "How to immediately throw away your best bargaining
        > chip 101"
    

If they're an internal recruiter, they have a banding you can be paid in, and
they don't really care where you fall in it. Decisions above that salary
banding will need to be referred to the person whose budget you are.

If they're an external recruiter, they have no control at all and are
incentivized to get you paid as much as they can, as they're paid on
commission.

~~~
greglindahl
External recruiters get paid more for getting more candidates hired with the
least effort per candidate -- they are not interested in spending a lot of
time getting 10% more salary for a candidate, instead of getting another hire
done.

~~~
peteretep
Most external recruiters don't make much more than a couple of placements a
month, so they are absolutely incentivized to amp their commission up as much
as possible without losing the sale. Was this not the case when you were
working as a recruiter?

~~~
greglindahl
As a hiring manager and candidate dealing with external recruiters, that's not
what I saw happen.

~~~
peteretep
How did you get people you didn't hire to share with you their actual salary
preferences, and companies that didn't hire you their actual salary ranges?
Sounds like that would be super useful info

~~~
koide
This is outside the US, but I just ask at the first call: please give me a
range on your expectations to see if it overlaps with ours and that it makes
sense to move forward and avoid wasting time. Everyone answers, most giving a
number directly and few counterasking for our range for the position, which of
course I disclose at that point.

------
paxy
This is one of those "aha" solutions that people keep mentioning, but
completely ignore how it would work in a non-ideal world.

• What 2 hour project can you assign that would let the candidate go into
enough depth for you to judge their code, design decisions, architecture
skills?

• If you give them more time (e.g. a weekend like in the article), is it still
financially viable to the company, especially at $100/hr??

• How do you filter out the candidates that aren't serious about the job but
just want to make a quick buck?

• What about all the candidates out there who you can't legally pay (e.g.
workers on a visa)?

This works when you are a small company that needs to pick 2 out of 6 total
applicants, but otherwise completely unfeasible.

~~~
ohitsdom
Exactly. Their example:

> "Create a single page app that lets me enter movies in my home movie
> collection, store them in offline storage and search through them. I want to
> search by Genre, Title & Actors."

Significantly more than 2 hours, unless you are looking for a quick hack and
for the candidate to repeatedly explain "I would have done it this other way
if I had more time..."

~~~
ebrenes
That's only if you're expecting a candidate to provide a perfect solution; and
don't make it clear that the time limit should inform their design and not
just measure their speed of implementation.

Part of software development is tailoring a solution to the scope of the
problem, and this includes taking into account the time and effort available
to come up with a solution. Someone who looks at that problem statement and
designs something that will take a week has missed a very important
requirement.

You could reasonably scale up any problem to take an arbitrarily long amount
of time, by adding onerous requirements (must have 7 9's of reliability). What
I've found more rare and precious is the developer who can limit the scope so
that items that are not up for negotiation like time (2 hours) & resources (1
developer) are respected.

I fully expect a developer to state their assumptions up front, so that I can
judge their design based on the stated assumptions and not some unstated
imaginary assumptions. It also lets me see how or if they prioritize.

------
shandor
This seems like a pretty popular take on the interview process here on HN.
However, there's one pretty glaring flaw that I've never seen discussed: if
the interviewee can take the problem home, what prevents them setting up a
hackathon with their friends at their place, and making the interview problem
a team effort?

A couple of knowledgeable friends can do wonders, and if you also prep with
them a little you can probably answer all relevant questions and beyond about
the code on the Monday on-site interview.

Sure, that's cheating. And even if it wasn't, I would never do it for the
wrong expectations it would set upon me if I actually got the job. But the
ease of "embellishing" one's CV always comes up in these threads/posts, and
while I wouldn't do that either, I kind of fail to see how this home project
approach solves that problem.

Anyone able to ELI5?

~~~
deepGem
One view of this behaviour is cheating, another view is hustle. I mean, if you
could arrange a hackathon to solve an interview question - you are hired my
friend :)

~~~
maccard
Do you expect this guy to organise a hackathon to solve every problem you give
him? And what if that havkathon now involves the people at your company
working on his problems as well as their own?

~~~
deepGem
Of course I don't expect him to organise a hackathon. If that's his way of
solving the problem, I'll actually appreciate his effort. Shows great
leadership skills. In fact, most of your success in a company comes from your
ability to muster a team. Individual skills only go so far.

------
ahh
This system is interesting but has a crucial flaw: I legally can't take part.
I have a job now, which claims ownership over any tech-related IP I create.
There are processes for getting exceptions, but a) you won't get them for
actual job-related things b) I'm not asking Google for an exception so I can
interview somewhere else.

This is true for nearly anyone with a standard SV job. So if you're willing to
restrict your pool to the currently unemployed, this can work, but I think
you'd rather not do that.

~~~
metaphor
>> I have a job now, which claims ownership over any tech-related IP I create.

Is there an omitted qualifier to this, e.g. ...while on company time; for IP
with sufficient correlation to your primary role?

I'm having a difficult time swallowing the legal enforceability of "all-day-
everyday-any-tech-related-IP" outside of military personnel subject to the
UCMJ. Would that suggest that you technically couldn't contribute to an open
source project?

~~~
sethammons
I have to get exemptions for any open source contributions or personal
projects. My employer gives those out readily, but I'm expected to follow the
process.

------
donretag
I very much like the paid aspect to this approach. Not that I need the extra
money, I am already financially compensated enough by my current employment,
but it reduces the chances for the interviewing company to flake out after
such a programming assignment. By offering some form of compensation, they are
investing in the candidate and will probably not ignore the candidate after
this stage of the interview. My assumptions at least.

~~~
daxfohl
Excellent insight. I just finished a weekend project for a company (and really
the onus was more on my wife who had to take the kids by herself all weekend),
and all I got back was "nope".

------
eloff
I wish I could filter jobs by this criteria. Can we do this in the who's
hiring thread? I'm sick of the interview process and massive unpaid work
assignments _cough_ Compose _cough_ only to find out after investing thirty
unpaid hours that they can't offer US market rates to a Canadian for 100%
remote job. Oh and they also want to own anything you do in your free time.

Sorry for ranting, I'm just pissed off at how an industry that brags about how
they pamper their employees can treat them like garbage during the interview
process.

------
eranation
This is great, except the over the weekend part. I have a wife and kids, I
rather be with them than solve (even paid) challenges. I rather take half a
day off, solve it right there in the environment and team that will potential
work with you. I would encourage candidates to also reach out with questions
during the challenge and see how they communicate ideas and react to
suggestions. Other than that, completely agree.

~~~
jon-wood
I'll second that. When interviewing for jobs recently the sheer volume of
tasks being handed out to prove I can program was a real problem - I work five
days a week, my wife works Saturdays and I look after our son. That leaves an
hour here or there available in the evenings (when I'll do a bad job because
its 10:30pm), or taking a chunk of time out of the only real time we get
together as a family.

If you apply for say 4 jobs and they all want 2 hour work samples you've just
taken on an entire extra work day to apply for jobs. I can't even just take a
day off work and batch them because of the way applying for jobs works.

Honestly, I don't know the right answer. From the other side of the table I
like this approach because it lets me assess how someone thinks about solving
problems and writing code, but its just reinforcing a culture where only young
people without any real responsibilities outside of work can get into the
industry.

~~~
mcv
The advantage of being paid for this is that you should be able to afford
taking time off from a paid job.

~~~
jon-wood
Its not really about the money. I get paid a salary, whether I work or not
I'll be paid the same amount (I'm aware this isn't true of everyone). As I was
saying, if I apply to 4 jobs that do this, I'm now at a full day, but I can't
take that as a full day.

What am I going to do? "Hey boss, I need to book another 2 hours off as
holiday so I can apply for jobs elsewhere."

------
deepGem
DuckDuckGo follows this process. They in fact do a paid 3 month assignment.
May be that's taking it to the extreme and perhaps works very well for them.
If I were looking for a job and this process was followed, I'd definitely be
game. I will most likely be following a similar procedure while hiring as
well. Just come to the office hang out with us for a day - we'll fly you in,
put you in a nice hotel + pay you a per diem. All we'll do is work through a
problem, not through a series of daunting hypothetical interview questions or
some algorithm nonsense that you'll never use in your job.

~~~
kctess5
In some sense many companies do this via 3 month internships. They're
basically just low cost future employee vetting opportunities. Of course, they
are generally limited to college students.

------
a_imho
Problem is, many companies are only playing interviews.

When you want to hire people who gets the job done it is no harder than:

1\. Please tell me about a project you have done, a sw book you read, a hard
bug you solved, really anything you like about computing etc.

2.a (This guy seems smart) Let's discuss compensation and you can start your
probation period asap.

2.b (This guy is not smart nor passionate) Thank you for coming in, but we
don't want to hire you.

edit: I used this method with great success, imo programming exercises are
just an indirection that only adds noise and loosely correlates with on the
job performance anyway

~~~
arpa
Exactly. Also you have no idea what kind of cultural fit you're having if you
only rely on technical skill. And besides, why do you want me to solve
$problem? Take a look at my portfolio and public code - it's telling enough.

~~~
huehehue
My employers don't want me open-sourcing the work I do for them, so this only
works if you open-source your side projects.

I see a lot of comments here about how they would never waste their time on an
exercise like this, but a small project for the purpose of an interview can
probably be done in less time than any of your public projects.

------
morgante
This interviewing method suffers from the tragedy of the commons. It works
great when almost nobody is doing it because if most of your interviews are
traditional it's easy to squeeze in 1-2 homework assignments a week.

However, it would be totally unworkable if most companies were to adopt it.
Try finishing a dozen homework programs during a typical job search.

~~~
RandallBrown
If it's supposed to be 2 hours of work, then it wouldn't be too big of an
issue.

Even a simple phone screen is going to eat up 2 hours of my day, much less the
full day interviews that most big tech companies do.

~~~
morgante
In practice, I rarely see assignments that are actually 2 hours. Even this
article suggests giving candidates the weekend.

~~~
patrickdavey
Yes, which is interesting, as 2 hours != the weekend..

------
hslee16
Having candidates work on small projects simulates the communication required
to complete actual/mock work projects. This will definitely foster better
rapport between people involved.

Personally, I'd rather be given a weekend to work on a smaller project rather
than be tested on algorithms that are available by common libraries of most
programming languages.

------
edpichler
Digest: pay for a task before deciding to contract.

On the article, Amir wrote: _" They refuse to do the project because “someone
will hire me without it”._

On book "Smart and Gets Things Done: Joel Spolsky's Concise Guide to Finding
the Best Technical Talent" Joel said that the _" the best developers usually
have lots of proposals, so they don't spend time doing tests for interviews"_.

I think Joel and Amir are right.

~~~
StavrosK
I came here just to make the opposite point. They aren't making you do a test,
they're _hiring you for a project_. It's fine if you're too busy even for paid
work, but why would you turn it down otherwise?

I think Joel means "unpaid tests". I routinely turn interviews with unpaid
projects down, because each interview process gets a number of hours of my
time for free (depending on how much I want to work for that company) and
unpaid tests/projects usually way way overrun that time.

Or maybe that's what you said and I misunderstood?

------
stove
Does anyone have experience with this style of interview from the candidate
side of the equation?

I want to move to this style of hiring; however, I fear that because it's
against the grain I may miss out on good candidates. Is that unfounded?

~~~
ubernostrum
I do from both sides, albeit without being paid to do the exercise. Take-home
exercises are great.

When I was looking for a job last year, I liked that it was something simple
and low-pressure (compared to typical coding-interview practice) which
approximated how programmers _actually work_. It was OK to look something up
in the documentation if I didn't have an API memorized off the top of my head.
It was OK to change my mind partway through and realize I could do it more
cleanly another way. It was OK to pause for fifteen minutes and get something
to drink.

Now that I'm on the other side of it, I like that it's a more natural thing. I
like that I'm not having to loom over someone or try to fill awkward silence
while they live-code. I like that I can read their solution before I go into
the interview and have some prepared comments or questions to get discussion
going. I like that it emphasizes the collaborative nature of programming in a
team.

Good candidates are beginning to revolt against traditional coding interviews.
Take-home exercises are one of the alternatives that people actually seem to
like on both sides of the interviewing equation. So you're not going to miss
out on good people by switching your coding interview to a take-home; if
anything, you're more likely to start attracting them.

~~~
StavrosK
I want to second that, with the caveat that, if the exercise is unpaid, I am
much less inclined to do it (mainly because I am too busy with paid work to
take on more, especially unpaid).

I've interviewed with companies that gave all kinds of exercises. I have no
problem with take-home exercises, if they're implemented like the article
suggests. A few times I've gotten "do this exercise and we can maybe pay you a
bit for your time", followed by them never paying and me never asking. If you
decide to pay someone for exercising, tell them how much beforehand, and pay
them as soon as they hand it in.

The best interviewing experiences have been with paid onsites for companies,
even if I didn't end up getting hired in the end (I got to meet the team, work
with them, they took me out to dinner, etc, it was nice).

------
hydandata
I have offered exactly this to multiple companies who interviewed me, but so
far none of them was willing to devote time to do it. I guess industry is just
too used to the established practice, and "hey, if Google is doing it why not
us?" thinking. It is not like I declined to do their normal interviewing
process either, I offered this /after/ their process was finished. It is the
best thing to do for both sides. And this was where they were trying to
recruit me, not when I applied.

f.e. I really like my current job, so I would benefit from getting a better
look at your workflow and interact with the team to decide if I really want to
work for you.

Hopefully the situation will improve in the near future, but I am so taken
aback with common practice that I currently see no other way but to start my
own company, I just cannot imagine working for such stubborn and close minded
people. And for darn sure when I hire, this will be the process.

------
mahyarm
This is the coding exercise/homework interview.

A bunch of place do this in some form or another, usually without paying the
person, and as part of a series of interviews on an interview day.

I myself don't like homework questions, mostly because I still have to do an
onsite, and I only have so much time to do these things. They are usually much
longer than 2 hours too. If you communicate upfront that it should only take 2
hours and would remove the need for an all day onsite, then I would definitely
be pretty excited. It would probably let you interview a segment of employed
engineers who don't want to sacrifice a vacation day to do interviews.

I really like coding exercises as a general interview type although, I feel
like it's the most important & real part of an interview.

The paid version doesn't work for people on a visa or are worried about IP
assignment agreements, but I like the charity idea brought up by another
person to solve this.

~~~
verbify
I agree, maybe I'm a poor programmer, but 2 hours seems incredibly unrealistic
for:

a) Creating a frontend form to submit movies b) Creating the database to store
movies c) The 'search to find the movies' feature

Maybe I could do it in two hours, but it'd be pretty bad, and there's no point
showcasing poor quality code.

My favourite was when they asked me to produce examples of things I had coded.
Like any other creative industry, a portfolio is normal.

~~~
mahyarm
We only expect you to make what is possible in 2 hours, and we've seen a bunch
of similar exercises. It doesn't have to be poor quality, just basic. Usually
the UX isn't expected to look nice at all either.

~~~
patrickdavey
So in 2 hours (according to the article, not this comment) you expect a
candidate to deliver a functional reactive single page javascript app, with
localstorage offline storage, searching (perhaps with filtering over a
"category" model also), responsive, ideally with some form of test coverage &
tested on multiple platforms.

I have a long way to go it seems! It would definitely take me more than 2
hours to do that to a standard I'd be happy with. And there's no way I'd
submit "what I had" in 2 hours if I could sneak off and spend more hours of
the weekend doing it & return something polished.

Do you penalize people who give something "too good" or query them on whether
the _really_ only spent 2 hours doing it?

~~~
mahyarm
I have no idea, since I'm an iOS dev, not a web dev. We've seen a lot of these
coding exercises btw, but the ones I've done have been onsite, so there is no
way someone could of spent more than 2 hours on something. Usually the
standard is if your able to complete the exercise, that's pretty good! I've
only seen tests written once or twice, mostly because there was no time. You
also see a lot of shortcuts due to the time constraints. You go through and
explain your code, so you can point out 'this is a shortcut, normally I would
do XYZ".

Usually when someone makes something 'too good', it usually means they've
leveraged libraries that they already know. If they make a lot of code, I know
they probably spent more than 2 hours. By making it a weekend project, maybe
they are subtly hinting that better candidates will spend more time on it,
which I think is shitty and one of the reasons why I don't like homework
questions.

I'm assuming for all of the things you've just said, it means you are using
libraries that make a lot of those things one liners. Like read/write local
storage, search a data struct, reactive UX changes, etc.

------
galacticpony
Quote: "I may say please use functional reactive principals in JS to solve
this problem, but the decision of using Kefir, Bacon, or RX is left up to
them."

That sounds like something a bad tech hire would say.

------
Jgrubb
I was actually hoping for a post on how to cull bad hires after they'd already
happened. Any advice there? Asking for a friend...

~~~
GuiA
Set up a proper culture of mentoring and training in your org to turn people
who are a little too green or inexperienced into effective team members.

Not as catchy as "fire fast", but infinitely more sustainable.

------
geebee
I'm coming late to this. I've been through two take-home projects. Both of
them resulted in a no-hire, and each has refined what I'm willing to do and
what I'm not willing to do.

The first one was a pretty egregious experience, honestly. I never spoke with
anyone other than a recruiter. The first was a 2 hour Java test. The next was
a take-home that I was supposed to spend 5-7 hours on. I did it and sent it
in. I knew it wasn't great, but I put a hard stop at 7 hours. It took almost a
month for me to hear back, from the recruiter, "we've decided not to continue
with your application at this time..."

For a while, I said never again. However, after refusing (and passing on what
might have been good opportunities), I did another. Circumstances were _very_
different. I had two extensive interviews with developers prior to the take-
home, and I was interested in the job. Furthermore, they did reply with
feedback, though it was through a recruiter.

The feedback was mixed, some things were good, others were bad. Unfortunately,
though, it was a hard stop. I didn't get and presumably will never get a
chance to defend my choices. Although I understand the company probably
doesn't want to open up a debate, they've moved on, it rankles that someone
got to say those things about my code and I never got a chance to reply.

So, am I at the point where I'd never do it again? Almost. I think I would
insist on an opportunity to defend my code base, at least one back and forth,
before calling it quits.

It sounds like this company (mattermark) does this, which is a positive thing.
You do get a chance to argue (though maybe they do cut some people off when
the project is terrible, and who knows, maybe the people who rejected mine (no
connection to mattermark) were just taking it a little easy on me, but were
genuinely not interested after seeing it).

I don't actually care about getting paid for the project time, if it's a good
opportunity. I have to spend as much time re-studying my data structures and
algorithms book for whiteboard interviews anyway, and I actually my learn
something from the take-home.

I'll probably talk to a buddy I consider to be a really good programmer who
works in this area, and see what his take is on my project, asking him to be
pretty brutally honest about his assessment.

~~~
ebrenes
I'd probably follow up on talking with your buddy. It's often invaluable to
get an outsider's feedback on how others look at our code.

From what you typed, it seems like there wasn't a lot of communication prior
to doing the coding exercise. I'm also not aware if you had channels to ask
questions or discuss technical design options. If available, those are often
"part of the test", to see if you make use of such resources.

When I'm reviewing code in such scenarios I'm often looking for other
indicators aside from the code's correctness. Things like: naming conventions,
structure of the code, patterns used, how over-engineered it is. Just as
important is seeing if the candidate picked up on less explicit instructions
or hints in the coding exercise. Finally, I look for code that follows the
conventional patterns and standards for that programming language; I expect a
candidate to discuss with me the deviation from those standards prior to
handing in the code.

Anyways, I wouldn't completely give up. This might be an opportunity to grow
and learn, but you'll never know unless you follow through on asking your
buddy. Likewise, I've seen people here on HN volunteer to review code and
provide feedback.

~~~
geebee
Thanks for your reply.

Your instincts are good here - one cited problem was indeed the departure from
standard conventions. I had my reasons, not saying they were good ones, but I
didn't get a chance to explain them either.

I'd like to, but it appears I won't get that chance. I gotta say, that is the
part of this all I don't really like. It was vastly better than my only
previous experience with a take-home project, where I didn't hear back for a
month and got a one-line brush off from a recruiter. In this case, I am
certain that they looked over the code reasonably carefully - they did provide
a short bit of feedback, though it was delivered by a recruiter, and there was
no back and forth here. I'm ok with not getting a job, but I would have liked
an actual follow-up where we reviewed the code after that level of effort and
time invested.

Lastly, the job was fairly senior, so it may have been a high bar, and that
wasn't the only problem with the code. Then again, how much are you really
supposed to work on a sample project?

Overall? I'd be hesitant to do this again. I might ask in advance what the
policy is on a review. Maybe a 30 minute conversation is a requirement for me
to do this. Still mulling it over.

------
MorePowerToYou
This method is great. It's worked for me when seeking new jobs.

Regarding the "algorithms trivia via whiteboard" style interviews: I think
this type of interview is, unfortunately, the best large companies can do
given their constraints. These constraints include hiring lots of new
graduates, interviewing pipelines that include technically illiterate
recruiters and engineers who lack the executive function to accurately assess
candidates' viability. Lastly, the people involved in hiring at large
companies are clueless about what the new hire will actually work on. It seems
so obvious that tailoring the interviewing process to the actual fucking work
that the person will be doing is a good approach. Once again, the contraints
in a large company prevent this from happening in general.

------
Tistel
I did CS and have been a professional programmer for more than 15 years on
pretty much every platform under the sun. I am very introverted. I still panic
during the whiteboard phase of the interview about 75% of the time. Like go
blank on simple stuff. Some problems need collaboration but I find I can only
really think clearly when alone. I have been debating taking one of those
public speaking classes because I probably need to transition to more of
management role (love programming, but, my hair is starting to turn grey and I
hear we kick everyone out of the club when the grey happens). The public
nervousness has not been very helpful in my life. I wonder what it's for? Why
have that feature? Maybe it's a bug.

~~~
titanomachy
If you are very introverted, then some of the companies that do collaborative
whiteboard interviews would probably be a poor fit for you. The reason they
interview that way is because they are collaborative work environments, or at
least they like to see themselves that way. Extreme introverts, even highly
skilled ones, might not fit in.

However, if you're experienced and skilled there will always be work
available. You'll probably do OK. Overcoming your social anxiety would
probably open some doors, though, and not just in management positions.

------
msimpson
We use a similar system which involves:

1\. A phone interview to validate the applicants resume.

2\. An in-person interview where the applicant is given an hour to complete
level 10 of Blockly Maze ([https://blockly-
games.appspot.com/maze?level=10](https://blockly-
games.appspot.com/maze?level=10)) after being briefed on the wall follower
rule (if they are unaware).

3\. Completion of an outside example project in the applicants strongest
language, for which they are not paid to avoid non-compete clauses.

This usually is enough to evaluate whether the applicant is embellishing on
their resume, capable of analytical thought in a completely alien language,
and their ability to deliver a deployable project given specific requirements.

~~~
sundvor
That sounds like a great way to do it; if I were an applicant, I would have
liked process. (I am currently looking for jobs, and do have a recent
experience of an on the spot test which I just hated).

I thought I'd try your step #2: I have no idea of the wall follower rule, and
no prior experience in maze solving, but did bring it home in 18 minutes.

That included figuring out this particular Blockly UI (have admittedly seen
similar UI with Dash and Dot, only at a much simpler level of doing non-
branching, sample tutorials with my 6 year old boy), trying to design a simple
program, reaching some dead ends / observing effects, and then realising what
needed to be fixed.

I did use all the blocks for 17 lines of JS though, so perhaps better ways to
do it, but thanks for that interesting little challenge at any rate. :-)

I'm guessing really smart people would have it done in just a few minutes..
Also monitoring son's cough (night time in Australia) to make sure it didn't
get worse, for a bit of distraction.

(Edit: Brought it down to 1 spare block / 16 lines having a 2nd look after
writing this comment.)

~~~
tthbalazs
Hey!

You might be interested in a solution that uses less blocks (i would be). I
might have been lucky, took me 2-3 minutes to find this.

[https://blockly-games.appspot.com/maze?level=10#taj4p4](https://blockly-
games.appspot.com/maze?level=10#taj4p4)

Does anyone know of a better one?

Edit: just as i posted this i saw that you dont even need the first "Move
Forward", so it is 10 lines total with 4 blocks left

~~~
sundvor
Had another look before I saw your comment, down to 3 blocks left now (12
lines). Thought my previous one was a bit messy, much better now, not sure if
it can be optimised any further. Will check your link next. :)

EDIT: Nice. I was doing it slightly different to yours, [https://blockly-
games.appspot.com/maze?level=10#foif9x](https://blockly-
games.appspot.com/maze?level=10#foif9x) , two if statements instead would have
fixed that, and the pattern becomes perfectly clear. Cheers. :)

~~~
NLips
5 blocks left (using 8 lines): [https://blockly-
games.appspot.com/maze?level=10#9zdu9f](https://blockly-
games.appspot.com/maze?level=10#9zdu9f)

But that's code bingo - fewer lines, but less efficient for the mouse.

------
stevefeinstein
So you only hire experienced developers? Where's your commitment to paying it
forward? Where will we get the next generation of experienced coders if you
don't give a few less than perfect candidates a chance? I guess if you get
yours then screw everyone else, right? Or am I being too harsh and bitter?

When did we stop smart people who may not have fit precisely into the perfect
little niche we've carved for them but showed a good ethic, and an ability to
get the job done and learn new things as technologies evolve? We're really
screwing up this industry in favor of getting it done fast, today.

~~~
umanwizard
Companies aren't charities. Their goal is to make money, not help anyone out.

------
andy_ppp
I liked the article but I'd rather people just come in and do some work rather
than fitting things in (2 days in the office). I suppose with American
holidays this isn't possible to take a couple of days but in Europe we get 25
days. I like the idea of paying for people's time too.

Probation is also important; I've used it when I don't like companies and I
think companies should be willing to say to people it's not going to work out.

Three months is enough to know; people can look amazing on month 1 and lose
their way. It is also important to never underestimate a manager who actually
thinks through what they want from their staff and asks them to do it :-)

------
rnovak
Having been on the hiring side of this, I'll note that it does require a bit
of start-up cost (i.e. work) on the employer's part, if you want to
standardize the process (i.e. to make sure you're giving the same exercise to
every candidate, the expectations are clear and consistent, etc).

For example, I set up the project for my team in 7 different languages (Ruby,
C, Python, Perl, C++, Java, Go), with the appropriate project structure, etc,
for each. This took considerable work on my part, but the end result is that I
can judge the output based on _whatever_ the candidate is most comfortable in.

Granted, that's all assuming my skills in each language doesn't completely
suck (hopefully).

------
Gustomaximus
Payment option is really good points. With take home problems, there is risk
their brilliant buddy helps them out, and teaches them about the solution to
help them talk about it. For this I feel its best to do any tests onsite.

~~~
jon-wood
Is this really a thing, or are you just optimising for a problem that could
potentially happen? Maybe I'm just being naive, but I'd like to think if
someone did that I'd quite quickly realise either at interview or on hiring
them that they're not actually capable of doing the job by themselves.

~~~
Gustomaximus
> I'd quite quickly realise either at interview or on hiring them that they're
> not actually capable of doing the job by themselves

Not sure. The whole idea of the test is to get around people that talk well vs
do well.

And assuming the friend told them about how/why they did something the
interviewee can talk about this solution as if they did it. How would you
know? See I'm assuming the 'friend' adds value to the solution to make it
amazing, and the person who is interviewing still has average knowledge so its
not like they are completely winging it.

> are you just optimising for a problem that could potentially happen?

Absolutely. You should optimise for problems that could happen if they have
reasonable likelihood and will become significant issues if they do happen.
Not going to rule out all issues. Thought this seems a likely risk over a
reasonable number of candidates.

------
silverbax88
I think the current interview process fails but...this idea of seeing if
people can code a small project isn't much better - it might be worse. It's a
lot different giving someone a tiny project to knock out in a weekend and
working with them for weeks during crunch time, or getting estimates, or just
if they are a good workmate. You're basing an entire hiring decision on
whether the person can write some queries and a page?

I mean, I like the idea of proving what I can do, but software development -
as in most jobs - are far more than technical ability.

------
voycey
I am 100000% behind this - I point blank refuse to do code tests because they
are asinine and I probably have the exact same one in a private GH repository
from one of the million other unimaginative tech hirers.

You want me to prove my worth? Fine, you can pay me for it and get something
useful out of it at the end, as a senior developer I really shouldn't have to
"Write a palindrome detector" to prove that I can code, I have a degree, we
did that shit repeatedly!

Nice to see other people are coming around to this way of thinking

------
perseusprime11
This is good for testing for skills but does not help with team fit and
behavioral issues down the road. I hired a person a while ago who provided
with a clean solution to the problem with good separation of concerns and unit
testing. I hired this person but after few months this person starts yelling
at other engineers, won't complete the tasks. Ultimately, I had to let this
person go which was very unfortunate given the quality of code. The headline
is don't miss out on testing for team fit and other cultural values of your
team.

------
physcab
One thing I enjoyed about consulting is the opportunity to work with so many
companies that gave me a glimpse into their work culture. There have been
companies where the experience I had interviewing completely shielded the
shitshow underneath. And other companies where the interview process was
really boring and routine, but the people inside were dynamic and amazing to
work with.

I like this approach of merging consulting and interviewing because it gets
interviewer and interviewee closer to the source of truth about how a real
work relationship will exist.

------
mcv
In 2004 I got a job at a company that used basically this method, except I
didn't get paid for it. I think I got the assignment on Monday, and had to
present the results on Friday, so it wasn't over the weekend. Though I could
be wrong on that. I could ask the CTO questions (it was a small company), and
on Friday I presented it for the other devs, who asked questions, and later
voted by email on whether I should be hired. I was, and it struck me as the
best hiring process I'd ever seen.

------
vog
Interestingly, this is exactly what you do when you start to work together
with a freelancer. First, you give them a small, isolated task. If that works
out well, give them more tasks.

------
wallunit
I like that approach, and we tried doing something like that in the past, but
here is why it didn't work for us, unfortunately:

We work remotely, so is our hiring process. While it would be possible to give
a programming task to a candidate, we won't have any control over whether they
did it themselves, or how much time they actually spend on it. Well, the
former is less of a problem, once you discuss the solution you see pretty
quickly whether they did it themselves and understand how their solution
works. However, candidates that want the job will try hard to do as best as
possible, so some candidates that you give a 2h task to, tend to spend much
more time on it. Others might not have as much time to spend on the task,
since they usually still have a full-time job. So the results are not
comparable.

Besides that, I find it quite difficult to find a task that can be done in 2h,
but gives a sufficient insight of their skills, however you cannot expect
somebody to take on a more elaborate task, whether paid or not, because most
candidates are in a full-time job when they apply.

Also, this might be different in the US, but here in Germany, if the candidate
isn't a freelancer/contractor at the moment, but permanently hired by an other
company (this is normal), there is no way to pay them officially.

------
bbcbasic
For me time is money is less important than time time. 100 hours aggregated
over interviewing for a few shops would yield $10k which is nice but not
extremely useful, whereas those 100 hours taken out of family time is a real
pain in the butt.

The solution to this I feel is a bit of a prisoner dilemma because the entity
that can make this easier is the company the candidate is a planning to leave
(!). If they have flexible working, make it easy to take vacation on short
notice etc. then that ironically helps.

------
datashovel
I won't promise anything, but have been contemplating an idea for a relatively
long blog post about exactly this.

Having lived for the last several years in a very rural area of the midwest I
would say that not only do you need the right mechanisms in place to sort
through / vet candidates, but you also need to be willing to constantly
venture outside your comfort zone. Why are you always recruiting locally? Why
aren't you willing to look in all locations throughout the country? To be
frank, I would guess that almost never do you find a "diamond in the rough"
outside of large metropolitan areas, but my personal experience is that as
soon as I work with a team in a metropolitan area I realize I'm almost always
substantially ahead of the game in terms of best practices / capacity, etc. I
would probably argue that I'm an exception to the rule, but (not to be
political) the recent US election leads me to believe that the "norm" is not
the "norm" any more. People are thinking outside the box and we (as recruiters
/ employers) need to realize this and not instantly discount a person just
because of where they live (btw I'm pretty strongly Dem living in a mostly Rep
world).

------
ohyoutravel
Is there a list of people who hire/give interviews based on take home
problems?

------
raoulw
Trials are how we do things at Automattic. Not only do we get a feel for the
candidate, but the candidate gets a feel for us too.

~~~
Kluny
I've actually got a technical interview from Automattic sitting in my inbox
right now, that I'm planning to work on tonight! Any tips? Is it
possible/encouraged to write back and ask questions during the test?

~~~
dep_b
The thing I liked most about their hiring process that it seemed to be done by
permanently overworked people that didn't bother to give me any kind of answer
after I worked all that hours and pushed a working solution.

------
stutsmansoft
I'm a huge fan of the "demo project" interview. It really allows me to shine
where whiteboard gameshows don't.

I don't need or expect payment, either. If I'm legitimately interested in the
company, I am happy to spend a couple of hours showing them my stuff. Hell I'm
proud to do it. I do enjoy programming, after all.

------
ryuker16
Being a freelancer in all areas of tech, I've offered this to every employer
and 100% refused or ignored it.

Closet thing I've been offered ironically was rather tough sys administration
tasks for no pay for a few gigs (5 hours of work) or asked to solve an obscure
ntp issue on a Linux server they couldn't figure out.

For programming jobs? Never.

------
elcct
This approach is much better than something like solving another puzzle in
Hackerrank. It also working both ways, because by the quality of assignment
you can see through what kind of company you are really applying to. I
personally find most of the time such assignments to be a good thing and
enjoyed doing them, but also had one case where company asked me to do self-
contained module that solves one of the problems they had and on the follow up
interview they asked me to refactor it so that it could be embedded into their
product. Which I did and they liked it and then decided not to proceed. I
could sense they didn't actually need a new developer, but needed someone to
extend their product for free.

------
deepaksurti
>> They refuse to do the project because “someone will hire me without it”.

This is wrong in certain cases. The candidate may have significant
contribution to an open source project or may have shipped an open source
project as an owner.

Why should that candidate not refuse? You already have the code in front of
you. If you as an employer cannot take the time out to go through that and
evaluate what you would otherwise with your take home assignment, then
something is not right.

This one size fits all rigid process won't work. Given a process, the
employers must learn to be flexible, all the while expecting employees to do
so, is insane. So certain qualified candidates saying 'someone else will hire
me' is not arrogance, but fact.

~~~
codeulike
But the method proposed is only 20% about code. It's mostly about seeing how
someone approaches a piece of work and discusses a solution in a meeting. You
can't get that from browsing GitHub.

~~~
deepaksurti
Agreed. The Github code is a substitute for the take home assignment. If his
GH code is good enough as per the employer standards, invite him in office to
explain the work he did on that project.

Now employers don't want to do that most likely I assume is because no one
wants to go through the pain of figuring out the details of GH code. OTOH, the
employer is expecting the candidate to understand the problem assigned to him
and come up with a solution!

~~~
codeulike
I don't think you understand how getting a job works. It's entirely
appropriate for the amount of effort expended by two parties to be
asymmetrical.

------
nunez
My current employer (ThoughtWorks) interviews like this. I had a week to do
the problem they gave me (it was unpaid).

It was the best interview I've ever done. I rode a huge wave of emotions the
entire time and learned A TON.

Technical minutiae interviews like The one I did at Google really suck

------
nchuhoai
I really like this, and have been trying to implement this at my company. One
thing we are struggling with is to find the right challenge/project, in my
case for the frontend. Two issues:

1\. What is a reasonably sized project? I.e. often times two hours is not
enough to produce something of substance/complexity, yet asking the applicant
to do more becomes prohibitive. 2\. I see a bunch of people asking candidates
to do greenfield applications, while virtually all work most future workers
will do is refactor and maintain. The latter however requires additional time
investment to get to know an existing system/codebase.

How are you guys handling those issues, and trade-off time complexity against
insight value?

------
shams93
Paying people to do demo work is revolutionary, I love this idea. I can't tell
you how many hundreds of hours I had to put in over the last 25 years doing
these "evaluation tasks" where essentially the company is getting me to put in
20 hours of work for $0. Then there's the places that use puzzles from CS
courses to exclude anyone who hasn't been through CS school. Practical
experience is often treated as worthless by our industry. They don't make a
lawyer who has been practicing law for 25 years take some bullshit test to get
a project. but in our industry the more experienced you are the more the
industry diisrespects you.

------
illicium
Or I can have the candidate code with me for an hour during the interview, and
save $100. They get to ask me questions about the problem directly, and I can
see where they are struggling, and how they debug (trial and error?
Google/SO?)

~~~
Kluny
It sounds like you would get a lot less out of this method than you would from
the one described in the article.

------
IloveHackerN84
This should be the best recruiting process adopted worldwide. Unfortunately,
at least in Berlin, companies go for stupid quiz and questions they even don't
know by themselves but that they've read from books.

------
doppel
I think one flaw is that everything happens over the weekend meaning the
developer has Friday to understand everything and then be ready for Monday.
Personally, I don't know half the issues I might run into when I first start a
new project, and if they expect me to just "make assumptions" the solution I
end up with could be miles from what the wish for (both in direction and
size). If it was Monday to Friday (or Monday to Monday), there's a chance for
me to ask questions, get smarter and iterate on the solution - same as I would
if I was actually working.

------
amai
Why give them an artificial task? Let your candidates bring their own pet
projects or let them show their portfolio of projects and discuss about it.
Designers, Artists, Photographers always have a portfolio of their work. Why
shouldn't programmers have one, too. And no programmer is shy when talking
about her/his own projects. See also
[https://blog.codinghorror.com/a-programmers-
portfolio/](https://blog.codinghorror.com/a-programmers-portfolio/)

------
avodonosov
How to invent an interesting problem which can be solved in 2 hours, which can
not be found in the Internet and would allow candidate to demonstrate
something?

And you need to produce such problems relatively often, right?

~~~
jon-wood
Ideally you only produce one good one, which allows you to benchmark answers
against previous ones. The other important thing is that you should do the
task as well - that both ensures it is actually doable in the allotted time,
and gives you an initial benchmark of what you're hoping for.

------
zaroth
The 2 hour version of this is what I've always done in actual interviews,
except where we peer develop the solution together the whole time, and I let
the candidate drive and lead as much as they want to.

The take home version might be useful beyond that, but I've found that solving
some fun toy problem like, "let's build a little app for X" in a couple hours
you absolutely know if the candidate can code and have a good feel for what it
would be like to work with them.

~~~
zzzcpan
No, yours is different, basically an exam, that requires a lot of preparation,
good sleep, etc. to perform well. Very few would be able to do it.

~~~
zaroth
I think it's different from an exam because the candidate is working on their
own laptop, with their own tools, with full access to Google, StackOverflow,
etc.

If for some reason they don't have a portable dev environment I guess we could
provide it but that does hamper the process somewhat because it's not really
"in their element" anymore.

I don't know how you really prepare for such an interview, other than actually
having the portable dev environment and being well rested.

The only thing we use the whiteboard for is understanding the problem
statement and sketching out the steps to creating the solution, as in, things
a whiteboard is actually meant to be used for.

------
draw_down
Remember: if you do something that completely eliminates bad hires, it
probably eliminated some good ones too. Of course, lots of people wave this
away by saying bad hires are more expensive than bad rejections. OK- but how
much more expensive? And does this practice make sense in light of that
figure?

Not to say you shouldn't do this but it's very important to remember a lot of
these kind of things when it comes to recruiting involve tradeoffs and you
rarely get an unalloyed good.

------
lsiebert
Wow, so you want me to work weekends is what you are telling me.

I suppose it's nice that this is paid. A lot of companies give me HW
problems... and frankly it doesn't scale.

------
daxfohl
Neat, but it still measures only technical aptitude. Anyone can look
impressive playing with a fun project over a weekend. But how do they do at
the grind when the important tasks may be much more mundane? In my experience
this is what separates the good hires from the bad.

------
dominotw
Sounds great. Let's add this on to our 50 step interview process that we
copied from google/apple/MS - Tech Hiring managers everywhere.

------
danielrm26
The reason this works is because it uses Evolution Thinking vs. Design
Thinking to find good people, which we need more of in tech.

[https://danielmiessler.com/blog/why-you-should-hire-using-
ev...](https://danielmiessler.com/blog/why-you-should-hire-using-evolution-vs-
design-thinking/#gs.pyFsGzk)

------
yexponential
How do you find companies that follow this or a variant over the traditional
process. I've been interviewing at companies over the past couple of years and
I've only participated at 1 take home task (startup) and know about 1 other
company (on my list) that follows that.

I, like others stated above, tend to perform much better in my own space and
time than under eyes on a whiteboard..

------
Joof
I never really know how much time to spend on take-home problems. I could
spend a lot of time on it and it should be better -- possibly demonstrate more
of my skills -- but often that ends in it being over-engineered and ultimately
bad or otherwise unrepresentative.

Getting paid could provide the ideal constraints for the problem at hand.

------
yAnonymous
>I will purposely challenge their solution to see how they react.

Exactly like Google, who tell applicants that their solution is wrong when it
isn't. Most people complain about how bad their application process is and
that interviewers don't know anything. Nope, they just weed out smart asses
like you that are a pain to work with.

------
happytrails
We do a similar thing at our office. Phone interview to make sure basics are
known and then a "homework" assignment. This weeds out a lot of folks who
can't cut it. Then the interview itself we can get the social aspect out of
the way.

Sometimes a candidate is a good coder but a horrible communicator or human :|

------
lifeisstillgood
But ... both sides have to really really want this.

This is the end of the process final wash

And so you probably won't do it with those candidates that fail your normal
process. If you are biased against old, black, women you won't give them this
test

It reduces your false positive rate. But won't ever help your false negative

------
Quarrelsome
This is what you do when you're looking for people on eLance/upwork/etc. I did
this on my last start up and it worked amazingly. Its more expensive than
getting lucky but cheaper than getting unlucky and allows you to find amazing
people to work with.

------
shoefly
I use this process and it works great. We give potential hires a 1-4 hour
project to do at home, they bring it into our office to defend. We don't pay
them to create the project. We don't look at education. Some of our top coders
are self-taught.

------
fredophile
One important thing to realize with paying people for code they write as
applicants excludes a large number of potential hires. People on work visas
are not allowed to moonlight. I wouldn't risk ending up on the wrong side of
ICE just for an interview.

~~~
throw_away_777
Well then those people can refuse the pay and still take the interview.

~~~
fredophile
No they can't. If they did they would be doing volunteer work for a position
that is usually paid and this also violates US immigration law (at least for
the visas I'm aware of).

------
darrenshrwd
Thanks for this, makes total sense and I'll remember it. Can I ask what you
use for your blog, I like the look and feel and the share bar at the bottom
and would like to re-use it if I can.

------
halfear
Something that we experimented with was pointing candidates towards one of our
public repos and asking them to make a meaningful PR to be discussed at the
interview. The PR would be discarded afterwards irrespective of the interview
outcome.

------
zenincognito
I am afraid this is just another marketing for a paid product. There are a lot
of assumptions being stated as a fact.

> Since the candidate is getting paid, the candidate treats it as a legit
> consulting session and therefore gives their best effort in solving the
> problem.

If a candidate has applied for a job then it is in his interest to treat that
application as his career depends on it. Best effort is wagered not on the
fact that he is paid but on the premise that his life will become better by
getting this job.

> It gives the candidate a real life sense of how you interact with people on
> the team.

Where is this coming from ? This is just another assumption the author is
making.

> The biggest issue I have seen with tech hires is that they can become very
> defensive over their solution. I will purposely challenge their solution to
> see how they react.

You are asking us to be non-judgemental in the paragraphs above and then being
judgemental on the candidate becoming defensive.

> Pay them immediately on Monday

Why ?

~~~
minitech
> > It gives the candidate a real life sense of how you interact with people
> on the team.

> Where is this coming from ? This is just another assumption the author is
> making.

It’s coming from the fact that you’re giving someone a project and discussing
the result with them.

> > The biggest issue I have seen with tech hires is that they can become very
> defensive over their solution. I will purposely challenge their solution to
> see how they react.

> You are asking us to be non-judgemental in the paragraphs above and then
> being judgemental on the candidate becoming defensive.

That’s not being “judgemental”. It’s just using regular judgement – evaluating
a person’s willingness to listen to criticism. The earlier points were about
hearing rationale for choices without previous criticism.

> > Pay them immediately on Monday

> Why ?

It shows that you’re well-organized and keep your word.

I think most of your arguments were coloured by the mention of a paid product.

------
phillro
I know its only a small fee, but legally wouldn't this require issuing a 1099?

~~~
eseehausen
It's under the limit ($600): [https://www.irs.gov/businesses/small-businesses-
self-employe...](https://www.irs.gov/businesses/small-businesses-self-
employed/reporting-payments-to-independent-contractors)

But even if it wasn't, a company of any significant size (enough to pay even 1
person's salary, say) should have a low effort and cheap way of tracking
contractor payments and issuing 1099s.

------
jlebrech
the downside to this is that potential hire may have a few interviews lined up
and you might not be the first, so the "best guy" might get an offer while
you're still undecided or looking for someone better. It's a good strategy if
you're not yet desperate for another developer, but if you're more established
and desperate for more than one dev you could hire a few and later let go of a
few (contract to perm is more friendly and you can hire them again as a later
point). in a way the presented solution is a micro-contract to perm.

------
foobiekr
$200 is cheap compared to the benefit of basically eliminating the ability of
candidates to field multiple offers.

Which I mean sure has nothing to do with why this idea is being marketed so
hard.

------
CSMastermind
There's no chance I would consider a job that required me to do this.

~~~
lolsal
Why not? I find your perspective interesting because I much prefer this style
of interview.

~~~
CSMastermind
I'm busy. I both work a lot and have a life outside of work.

If a company were to give me a take home assignment I'd see it as them not
valuing my time.

When you do a traditional interview the company has skin in the game. They
have to "spend" the same amount of development time as I do in the interview.
That person interviewing me could be working, if they're interviewing me
instead, that's the company signaling that they think I'm worth the time. You
don't have that in a take-home problem. It's asymmetrical. I'm investing time
on this and they aren't. That's not how I want to start a business
relationship.

Then there's the marketplace. I have a lot of options for employment. I'm
confident that if I quit my job tomorrow I could have another one, just as
good, by Thanksgiving. The job interview goes both ways. I'm interviewing you
as well. I want to talk to your current developers. I want to get a sense of
your office culture. And I don't want to complete a take-home assignment
before I get to do that.

Finally the people I like to work with are like me. They're busy, driven, and
good at what they do. I can't imagine they would work for a place that used
take-home work for interviews. So even if I did go through with it and get the
job; what type of people am I going to be working with?

------
ma5ter
The company I applied for last week did similar thing for a devops position -
They asked me to deploy an open source web application which is less popular
and almost 0 documentation available on internet.

------
ronilan
Any employer should be required, by law, to pay candidates interviewed, an
hourly wage equal to that of an employee in same position.

It will be good for candidates, good for employers and good for the economy.

~~~
omribahumi
Do you mean pay them for the interview time? People will start interviewing
for a living.

------
peteretep
Looks reasonable from my experience. Don't think I'd offer money for up to two
hours coding test, but I've certainly offered money for something that took a
whole day before.

------
a13n
I find this pretty reasonable if you're interviewing with a handful of
companies but I remember doing 20+ interviews in 2-3 weeks in university and
would simply not have time for that.

~~~
countingteeth
I'm all for pounding the pavement if your network doesn't line you up, but 20+
in 2-3 weeks suggests a very different assembly-line approach. I'm having
difficulty imagining how it would be logistically possible for me to line up
20-30 interviews in 2-3 weeks where I had predetermined that they were all
places I wanted to work and could make my case. I would hope for a batting
average and negotiating leverage where if I got in the door at a lot fewer
places I would find something that worked.

Then again, everyone is different.

------
tummybug
As someone who is looking for a new role at the moment and absolutely hates
the standard interview process I would definitely apply for a role at any
company that had this process.

------
jbrozena22
Question to all - would you consider paying for access to a platform that
organizes the interview process Amir described? It sounds useful to me but
would like to hear other opinions.

------
grandalf
I've tried to do this before but CEOs are typically not on board paying for
throwaway code. It does make total sense for all the reasons the post
mentions.

------
tulika_47
The concept of paid interview is pretty innovative and interesting to judge
the technical skills of a candidate as compared to the traditional method of
interview . But before implementing such method the company infrastructure,
availability of resources, mentality of the candidates in the market etc
should also be considered as a factor to understand whether the company can
afford and establish such method or not.

To implement any new method , system infrastructure is very important.
According to me paid interview is not that much beneficial for a mid-level
company as it will be for some MNC or IT-giants because the number of openings
matters. The article suggests to bring a group together on Monday to discuss
the solution to a particular problem given. In that case the number of opening
& scheduling has to be huge in order to arrange such group for the G.D which
is not possible in a mid level company where you have only 1-2 openings at a
time and the number of candidates who apply to take the interview is also very
limited i.e within 5-8 in a week out of which only 2-3 appear for the
interview in a week.

According to the Indian mentality Paying against just appearing for the
interview might not be that fruitful in absorbing the right talents in the
company . If such concept of interview is out in the market people will take
this as a scope of earning rather than joining. We might not get genuine
candidates who actually wants to join the company. Candidates will just appear
only to earn that money and will leave. Here what we can do is we can only
provide the money to the candidates who gets selected and then joins.

Here we are allowing them to write the problem down and take them home. Here
comes the question that what if the candidate is taking someone else's help to
solve the problem and how much honesty he is maintaining to solve the problem.
If the candidate is appearing for the interview in the office premise such
factors won't be an issue as he will be monitored constantly.

If we are following such process of interview then we constantly need a team
who needs to change the question for every session of interview as the
question is going out therefore we will always be in requirement of new set of
problems/question which is again tough for a mid-level company to maintain due
to resource crisis and that to only for 2-3 openings at a time.

We are providing the candidates constant supervision and guidance to address
the question, also we are providing them no time limitations and if they clear
the test a group is there to judge his perception and the result instantly on
a one to one basis which is less time taking and prompt as compared to the
process suggested in the article.

------
amgin3
No company is going to pay someone for an interview project assignment when
they know they could easily make candidates jump through hoops for free.

------
kinkdr
The Triplebyte folks are doing something very similar. But I got the
impression their clients don't understand the idea yet.

------
ryanSrich
Great post. In summary:

Pay people to complete a real life project and then bring them in to talk
about it.

Why is this so hard for most tech companies to understand?

------
kevando
Wow. I did this! And you're right it worked great! However, the candidate was
a student with no experience so I did't pay him. I said here is the git repo
to [http://www.wookietranslator.com](http://www.wookietranslator.com) now go
make some changes. 2 weeks later he figured out git, python, js and I was very
impressed! Kudos.

------
gigatexal
Anyone know of places that interview like this? I'd love a shot to interview
with a company that does this.

------
laxk
It would be nice to collect here the list of companies who is doing interviews
in that way.

------
samlittlewood
One possible source of tasks for this process is the bug backlog from open
source projects.

------
zwischenzug
Why not have those two hours + discussion in your office and call it an
interview?

------
ranveeraggarwal
Is someone out there actually using this method? How do I get an interview?

------
fowlerpower
I think this is a great system that reduces false positives, significantly. It
helps candidates shine and it helps employers get good engineers.

I would not rely solely on this as no process is perfect. However, I do weigh
this part of the process the highest for my hiring.

------
jkingsbery
My weekends are just as busy as my weeks. I have a wife and two daughters that
I do stuff with on weekends, and other family members I like to see. This
process seems to presume that I somehow have "free-time" on weekends.

~~~
darkstar999
If you don't have time to scout out new job opportunities, then you are
comfortable enough to not need a new job. A 2 hour project like the article
suggests is reasonable.

------
dschn
All you need to look for is passion. Worked for me.

------
sargun
How do you avoid people outsourcing their projects?

~~~
teach
I'm a high school teacher, so my experience isn't _directly_ relevant but I've
certainly encountered my fair share of cheating.

The student that wrote the code can always explain not only how it works but
why they made certain decisions along the way.

The student clueless enough to copy someone else's code usually can't even
explain its function, much less explain tradeoffs.

------
teen
Coding on a whiteboard works. There are just a lot of people who can't code,
and complain & make excuses. I'm so tired of these clickbait posts.

~~~
kasey_junk
How do you know?

------
honkhonkpants
No mention of the tax implications? Paying every candidate sounds like a
wonderful way to increase paperwork.

~~~
yoloswagins
Collecting a W-9 takes 2 minutes, and is only required for payments above
$600.

------
gregjor
I'm glad to hear this works for you. But I think the process emphasizes
technical skills over what really matters when hiring: how the person works
with the team and within the business.

Giving someone a toy project unrelated to the business tells you nothing about
how they will perform in the real environment. Letting the candidate take the
work home to work on it in isolation tells you nothing about how they will
work with your team, or within the company.

In his study "People and methodologies in software development" Alistair
Cockburn found that “People’s characteristics, which vary from person to
person and even from moment to moment, form a first-order driver of the team’s
behavior and results."
([http://alistair.cockburn.us/People+and+methodologies+in+soft...](http://alistair.cockburn.us/People+and+methodologies+in+software+development)).
Other studies have come to similar conclusions.

In a technical job, actual skills and experience are necessary but not
sufficient for success. Programmers fail in their jobs because they can't work
with or fit in with the team, and the larger organization. Any competent
programmer can learn a new language, framework, or technique like TDD in a
week or two. But people who don't pay attention to the business environment
and requirements, and can't work with their team, group, etc. will fail, often
costing a lot of money.

Presumably you have weeded out the unqualified before offering to pay them to
work for a few hours, but this seems like offering an unnecessary incentive to
learn very little about the candidate's likely on-the-job performance.

I don't know any scientific way to measure how someone will fit in, and I
don't think anyone else does either. That's why having several people from
different parts of the organization interview is the norm -- the interviewers
compare notes and reach a consensus because everyone understands they are
going mostly by their subconscious (gut) feelings.

I spend time describing the business requirements and then ask candidates how
they might solve an actual business problem. I don't expect a complete or deep
answer; what I'm after is finding out if the candidate can think about a
business problem at all. Examples: Our inventory system frequently doesn't
match what's in the warehouse. We get a lot of people abandoning their cart at
checkout. We have customers gaming our promotion codes.

Testing for things like "functional reactive principles" is an example of
ignoring the business environment and favoring technical trivia. No business
ever had the problem "We need 2,000 lines of Ruby code by next week," or "We
need a system to organize a small movie library, you have two hours." What I
find is that far too many programmers don't even hear business requirements
and can't do basic analysis or think of the right questions to ask. They want
to start writing code. Having expensive people writing code that doesn't
address business requirements is costly, and very common. If a candidate isn't
asking questions about the business and the team that's a big red flag for me.
Programmers who just want to be left alone to do their own thing are easy to
find but they aren't necessarily a good fit, or productive.

I also spend time getting as many people in the organization as I can to chat
with the candidate, so we can compare our impressions. That is just
recognizing human nature -- we all tend to form opinions of other peole within
a few seconds, and then look for things that reinforce our impression. Of
course I want to see evidence of actual technical skill, but I can do that
with a few actual programming problems -- even something on the level of
FizzBuzz will tell you if the person can solve a problem in code. And I want
to see how they solve the problem, and what kinds of questions they ask.

If someone is uncomfortable working alongside another programmer, or solving
problems on a whiteboard, or talking to a group, I get it, but those are all
necessary skills too. Just like learning a new framework, a motivated person
can learn to work in a group, listen and ask questions, speak to a room. They
may be a genius when left alone in a corner not interacting with anyone but
they probably aren't going to be a good fit in most organizations. I often ask
candidates to tell me how they would go about hiring someone for the same job
they are interviewing for -- suppose we have two positions open and you get to
hire a peer. That can tell me a lot about what they value in a co-worker.

------
EnderMB
The place I used to work had actually tried something like this, but it
completely fell apart, and I imagine when I tell this story you'll realise
why.

One guy came to us from a recruiter highly-recommended. He had a few years
experience, but he was widely respected online via his community
contributions. He was also out of a job due to redundancies. It was enough to
get him an interview, and he could talk well, so my bosses got the dev manager
to set up some basic problems on a client website to solve.

He seemed like a nice guy, and he had a lot of questions, which was a good
thing. He had a nice chat with some guys over lunch, telling them that he'd
been unemployed for several months, and he was down to his last savings. He
was able to fix the problem purely through looking on some forums for the
answer, and that answer helped him fix the problem. The dev manager was happy,
and the PM's were happy. The devs that worked with him were brought into a
room to see his solution, and everyone in there said he should be hired. We
had a new dev!

He was instantly given a project from the backlog, which was billed out, and
would take around six months. He clearly knew what he was doing, so he was
left to his own devices. To cut a long story short, the code he was writing
was absolutely shocking, and he'd engineered a solution you'd need a PhD to
understand. He was also finishing tasks really quickly, but they were bounced
right back to him because his code was full of bugs. He stayed with the
company for a while longer, but was ultimately let go. That project went over
budget by six figures. It turned out that he was let go from his last two
places for similar reasons. He was a junior developer that believed himself to
be senior, and as such he disregarded any advice from anyone but managers. He
had claimed open source contributions, but he was widely considered because
he'd made thousands of posts on an open-source project forum, and due to their
community contribution rules these posts deemed him a candidate for MVP.
Despite thousands of posts, he'd never really solved a problem for anyone.

I looked through the code he had written during his trial, and it was largely
the same mess, but obviously smaller and self-contained. It wasn't an optimal
solution, and he'd written 500 lines of code for what should be a 20 liner.
The dev manager used to be a programmer, but hadn't written code in a while,
and didn't see a huge problem with it. When the rest of the devs were asked
about it, they all said that they thought his code was messy/crazy, but the
managers wanted him, and the guy really needed a job, so they said yes.

I'm not saying that this solution isn't a good one. If you can do it, I'd
argue that it's the best way for a typical company to hire a developer. What I
would add is the following:

1\. Management should not publicly offer their opinion on a candidate before
technical advice is given. If the managers weren't praising him, our devs
might have had the confidence to say no.

2\. Any technical test should be fully QA'd before a "Yes/No" is given. You're
not just looking for someone that can write code. You're looking for someone
with domain knowledge of your tool-set that can offer solutions to problems.
If you're shown a bunch of code with little context. Funny enough, the code
that he submitted was actually fixed by another dev on the following Monday.

3\. Make sure they're not working on something that actually needs to be
delivered in the next week. This guy was brought into the office on a weekday
because he was unemployed, but it turns out most of the pressure for this was
from a PM who said that the problem he needed to work on was actually required
in that sprint.

------
pmiller2
There's an inexplicably [dead] comment in reply to your comment here. I'm
reposting it so more people can see it.

@throwaway4job, you appear to be hellbanned for no reason I can discern. :(

> throwaway4job 8 minutes ago [dead] [-]

> This isn't a viable approach for much of the hiring pool, as a majority of
> working engineers have contracts which explicitly forbid work for contract
> for other employers. As such, if you hire any of those, you're hiring
> somebody with a proven willingness to ignore their contract, which is a
> strong anti-pattern.

~~~
geofft
My guess to reasoning is that they called a startup founder a sociopath who
would kill people to make a profit.

That said, that was a completely defensible and evidence-based thing to say in
context (the founder threw a fit about regulators who wanted to know what the
chances are that people would die), so unless HN is cracking down on the
casual use of "sociopath" by people who aren't making professional medical
diagnoses (which, to be fair, I do find objectionable, but not to the point of
hellbanning), it seems like it was for no sensible reason.

~~~
dang
It isn't always possible to figure out from the public data why an account is
banned.

When a banned account's comment gets vouched for and rescued, we look at the
account history and unban it if we see that we banned it unnecessarily (we try
not to do that but it happens) or if the commenter has reformed.
Alternatively, there may be a good reason for the account to stay banned, but
if it posts good comments, HN users will vouch for and unkill those. This
system has proven to work well, even better than we hoped when we introduced
it.

------
cloudjacker
Pretty good but rule #4 is totally totally opinion without any consideration
about why someone would be defensive.

So given that Amir of June is the only person doing this quirky interview
style (where you actually pay them for their time), the interviewer still has
absolutely no idea of this psychological gimmick Amir has baked into his
process. Right at the very end!

Amir Yasin admits that he is elevating them into a contracting gig status, and
contractors ARE THE EXPERTS! They have to be the expert! They have to make up
stuff on the fly to defend their implementation, "its a feature".

But for Amir's warped interview process, of which the candidate is being
bussed around on a literal auction block at other companies before having the
luxury of getting his groundbreaking interview style, he still has this one
random assessment.

Look, Amir is the one that elevated his tiny sample size of engineers into
gospel. There likely was a literal 2 people that formed his and his company's
entire opinion about Rule #4.

So I think it is worthwhile to point that out.

------
throwaway_3452
I had my first ever whiteboard interview recently. I severely underestimated
how difficult it is to write code as compared to a terminal. At the end of it
I came out feeling deflated and rather angry as the same piece of work on a
machine would have been trivial.

------
mkoryak
I a single non tech question that is a very good heuristic for hire/no hire.

What is your favorite 4 letter word.

I usually ask at the beginning of interview and expect an answer at the end.

Answers like fuck and shit === no hire.

The other ones tell you a lot about the person

~~~
wayn3
The only honest answer to this question is "i have no favourite four letter
word".

but you cant say that because idiot hiring managers who ask idiot questions
would insist that you do.

sorry, but if you ask shit questions, dont expect non-shit candidates.

~~~
zimpenfish
Another honest answer would be "I thought I was applying for a development
job, not a place in kindergarten. Good day, sir."

~~~
mkoryak
and this is good, because I wouldn't want to hire someone without any sense of
humor

~~~
wayn3
needs a sense of humor but the right kind of humor. "shit" too edgy.

