
Hire by Auditions, Not Resumes - _pius
http://blogs.hbr.org/2014/01/hire-by-auditions-not-resumes/
======
jasonkester
I like the idea of an extended audition, but not for peanuts. It needs to be
my actual rate or a significant fraction thereof or we're not really operating
as equals now, are we?

I actually negotiated an extended audition as part of a recent contract. The
decision makers were starting to say things like "we definitely want to hire
you, but we need to flesh out requirements on the project" and "we'll get back
in touch in a few months to get things kicked off". Code for "wow, that's a
scary looking bill rate" perhaps, but certainly not the sort of talk that
tends to lead to gainful employment.

So I suggested that we do a quick R&D project at a reduced rate just to see if
the thing they wanted to build was in fact buildable. We'll just run for a few
weeks and see where we stand, then reevaluate whether we want to work
together.

The nice thing about doing it this way is that they get to see the "other
variable" in the salary calculation. It's not just Bill Rate * Hours. It's
Bill Rate * Pace. A fast, expensive guy can turn out cheaper in the long run,
if the Fast bit outweighs the Expensive bit. 3 weeks to a Month is enough time
to see whether a given dude is in fact good and fast; not just in bursts, but
steady state.

Granted, that's one datapoint. And it just happened to work out nicely. But I
think it's a good way to prove to a company that you are in fact worth the
rate you've quoted them.

------
ryanSrich
$25/hour flat rate? Why? That's not even half the market rate for an average
developer.

I somewhat agree on having "auditions" or at least a quick try before you buy,
if you will, but at least pay a decent wage.

What would work best imo is to technically vet the candidate early on and over
the phone. Perhaps a 30 minute pair programming session done in stypi where
you're both working on some code. After that do a group skype chat session
with some other teammates and the candidate to see how well a group
conversation goes. Nothing extreme just a 30 minute to an hour pow wow about
the job, the company, etc.

If the candidate is still in the running then bring them into the office, if
you actually have a physical office, and give them a couple hundred dollars to
work on some code for 3-4 hours. See how they fit with the team and how well
they perform in person. You'd be surprised at how quickly you can get a feel
for someone. It doesn't need to take weeks. If you're remote you can do the
same thing, in fact it'd be much cheaper without having to cover the
candidates travel expenses.

So how many people would even make it past the first round? 10%? lower? After
that you've got a nice number of qualified candidates and plenty of
opportunities to weed them out before anyone wastes their time.

~~~
accountoftheday
>$25/hour flat rate? Why? That's not even half the market rate for an average
developer.

If over half the developers are worthless and you are buying an unknown
quantity this seems equitable.

~~~
nl
It's equitable in theory, a great deal for useless developers, and a bad deal
for good devs (the only ones you care about).

You shouldn't care about the extra money, just finding the right developers.

~~~
danielweber
Gresham's Law: Bad Money Drives Out Good

------
pixeloution
I've always found this theory of hiring interesting but I think it eliminates
some of the best possible candidates. People who are really good at what they
do don't need a 20+ hour try out to find outstanding jobs - especially not in
software.

If I have a choice between working for the company that wants to try me out
... at a rate grossly below market, and a company that can look at my
experience and examples of code I've written - well it's a no brainer.

I feel like a company that hires like this has no respect for the potential
employee.

~~~
nairteashop
You could look at this another way: _you_ are trying out the company, and they
are paying you to try them out.

A few hour interview may not tell you much about the candidate, but as the
candidate, it doesn't tell you much about the company either. The future
boss/colleagues could appear pleasant for the hour you get with each of them,
but could be real jerks when you actually work with them. Maybe they just have
a different style of work than what you prefer. Or maybe they said "agile"
when you asked them, but they are anything but. I could go on.

For me, I really like companies that hire this way because it also allows _me_
to try out what it's like to really work at that company. And get paid to do
it. No one wants to be the guy who quits after 2 weeks coz the job sucks.

~~~
georgemcbay
"and they are paying you to try them out."

But if you're a talented developer then opportunity cost comes into play.

If they are paying me $X an hour to audition but I could be making $X*3
otherwise, the fact that they are paying me for the "try-out" isn't much of a
positive factor.

~~~
nairteashop
Sure, but I see the payment as just icing on the cake. If you're a talented
developer, you probably have a few offers in hand from pretty good companies.
What better way to pick one than actually work 1-2 weeks at each to see what
it's really like? It's a reasonable investment, given that you'll be there a
good chunk of your life in the coming year, likely more.

Edit: BTW, I disagree with the rate (flat $25/hour for everyone) mentioned in
the article. IMO it should be closer, or equal to, what the candidate will
eventually make at that position. Compared to what companies pay recruiters
and the lost money/time on dealing with a bad hire, 2 weeks of full pay is
nothing.

~~~
jiggy2011
If you agree to work for less than your regular rate then really you're paying
them.

Besides are you really going to get a good feel for a company by working
remotely and are you going to be doing your best work at 9PM after a full day
of work and other obligations?

------
ronilan
Article failed to mention that prior to Automattic's "auditions" they
administer a fairly long refactoring take-home test. It is _unpaid_. But it is
in PHP so at least you get to see a lot of $ signs :)

~~~
nawitus
If there's something I want to do for free, it's refactoring PHP code for a
Wordpress company.

------
thedufer
> They can do the work at night or over the weekend, so they don’t have to
> leave their current job in the meantime.

Based on many of the terms of employment I've seen for creatives
(designers/developers), this suggests that you don't legally own parts of your
code. That's kind of a dangerous position to be in.

> Automattic employs 225 people...maybe 10 people leave the company, and
> another 25 or 30 we’ve let go

Is the audition really working if you fire 10% of the people you hire? 10%
feels really high. That said, the rate of people leaving is lower than I'd
expect. Together, those numbers make me think its working for the employees
but not the employer.

------
001sky
Hire by internship, not interview. This I agree with. But the model is not
robust to many stages of career development. Crown jewel employees are not
going to have the time or the supervisory freedom to do this (unless the rare
case of them being uneployed or between jobs). True entry level jobs should be
done with interns (paid, IMHO)...which are proper but short term assignments.
That leaves what for ths use case here? Junior to mid-level peopel with time
on their hands? I'm not sure that is either a real market or a segment that
has the most problems. The biggest area of problems seems to be managing the
funnel...and that takes throughput. Which means active auditions for
multiple(s). Which takes alot of time and energy, and questions the coherence
of applying this model to people where turnover is the issue. On the way-
out...are these people going to slow down because of this? If not, what is the
ROI on all of this additional time? Should you spend more time on a better
funnel, better internship, or better "crown jewel" employees who can recruit
previous colleagues? Its a tough question, because I like the notion and the
approach, but the devil is in the (many) details.

------
latch
It'd be nice if they gave an example of a trial beyond "work on engineering
problems." What's the expected amount of time it should take to complete the
trial? 4 hours? 4 days? 4 weeks? 4 months?

There's a huge difference between "here's something that should take you a day
or two, but take your time" and "come work for us for $25/hour, with no
benefits; but as a perk, you don't have to quit your current job"

~~~
rralian
My trial was to add a drag/drop file upload UX to their (or now our) internal
yammer-like communication system (basically a very customized WordPress site).
It wasn't a big time commitment, was pretty low-stress, and I didn't mind it.
Was nice to get a taste for working at what is a very unique company (everyone
is remote, nobody uses email, no product managers). I didn't think of it as a
crappy paid gig. If anything I thought of the payment as a little perk.
Basically a way to make this little project approach legal. On the other hand
I already had some good insight into the company, having a couple of good
friends work there, and having chatted with Matt a couple times. So I was
predisposed to think this is a good place to work and worth the effort. I was
also really intrigued with the idea of a fully remote workforce and was very
interested to learn how they manage that. So I actually saw value in doing the
project beyond the payment or the whole "get a job" thing.

I think the other key part is that they're not just evaluating you on the bit
of work that you're doing. It's largely about your communication during the
project. If you just took the project and banged out a perfect feature without
talking to anyone or asking questions, I'm guessing you wouldn't get an offer.
In order to make an all-remote workforce work, communication is incredibly
important. "Communication is oxygen" is the mantra at Automattic. And the
channels we use for it are IRC and our yammer-like system. We rarely talk to
each other and almost never use email. That's really non-standard, and you
can't easily get a feel for that in an interview or with pair-programmed
fizzbuzz.

In many ways, I think working at Automattic is somewhat similar to
contributing to an open source project (to be fair, I haven't done a ton of
this). And that makes sense, given its roots in and continued investment in
open source. In fact when one developer asked if we could open source a
particular component, Matt's response was that he was pretty much ok with open
sourcing everything except our passwords. Given how open source works -- your
voice in a project basically equals your investment and contribution to it --
I think the paid-project interview makes some sense in that light.

Anyway, I'm kinda responding to a few posts here, but I hope that gives a
little more background into the process.

------
ZanyProgrammer
Why don't we all be honest about programming interviews, and say that we don't
care about resumes, references, or GitHub accounts/personal projects/etc
(because interviewers on here have often mentioned they don't have the time to
actually look at those). It _always_ boils down to the whiteboard and asking
algorithms and data structures (and maybe a few other questions) to potential
applicants. And for larger companies, it can be a multi day process.

~~~
vonmoltke
I still do not understand this attitude in the SV tech world. The rest of the
engineering world doesn't care much for book knowledge (except for college
hires), while SV thinks the ideal employee is a walking CS textbook.

------
kasey_junk
There is a lot to like about this style of hiring and it is similar to
processes I have endorsed before. There is one unfortunate problem with it
though. It rules out developers who have moonlighting clauses, which is a very
large percentage of them (even if they don't know it).

At the end of the day the only system I've been able to think of that gets
around many of these problems is a contract hire (3/6 months depending on
prevailing employment conditions) that at the end of it has a lump sum payout
of the equivalent time (so a 6 month contract would end with a lump sum payout
of another 6 months) regardless of if the contracted employee continues with
the company (either by their choice or the companies).

The advantage of this system is that it will not remove candidates from the
pool who would normally not take on the risk of a contract position, yet
allows for an honest assessment of the candidate/employer relationship.

This seems expensive up front but is often just accelerating bonus payments
for employees that are kept, and is a small price to pay for removing bad
employees quickly.

------
codex
A shorter audition: ask people to write code on a whiteboard or a laptop
during a day long interview.

~~~
JoeAltmaier
Yeah no. I won't spend a day of my time for free, because you distrust my
resume and references. Maybe if you paid me?

~~~
dsymonds
If you had seen lots of resumes and references, then interacted with those
people later, you'd distrust resumes and references too.

~~~
alxndr
Out of curiosity, could tech screenings via phone/skype have helped in
avoiding that situation? (Or perhaps they have already not helped?)

~~~
dsymonds
Yeah, phone screens help a lot. But people can still sound convincing enough
on the phone for half an hour without being a good programmer, so that's just
another filter layer before an on-site interview.

------
simonsarris
I am always fascinated by answers to the "hiring" question. (I don't claim to
know any good ones, perhaps that's why it's so interesting and why we discuss
hiring so much).

You're still beginning by submitting a resume - they are replacing _parts of
interviews_ with auditions, though they suggest there are interviews too, so
it's more of a supplement than a replacement.

So Automattic is adding something to the process, something non-trivial in
time commitment. I think I can show that this hurts them.

~~~

Employers need to keep in mind that there are some vast asymmetries between
employer and the (prospective or not) employee. For instance, some common
cases:

* When an employee is fired, they are at 0% productivity until they find a new job. The employer on the other hand is at 99% or 100%[1] productivity. This is in short why we have things like unions, because negotiating power in terms of productivity lost to both parties (employer and employee) approaches equal when the employer has all employees at stake, not just one.

* When an employer wants to fill a position, they have one spot and can try out several candidates (even with auditions) simultaneously. When an employee wants to find a job, they need to spend time choosing to fly out to places to interview. These decisions are hard, because there's opportunity cost involved: When you fly to place A to interview, you're spending that time not looking for other, potentially better jobs and/or flying to place B.

This ups the ante of the opportunity cost significantly. It's becomes an
awkward gambit: "I'll pay you $25 an hour to pause your search for a job
(you'll do our tasks instead), and after an indeterminate amount of time,
perhaps offer you one." That would get some strange looks on a job board. It
becomes an odd form of speculative work.

You're trading a lot more of your time (that you could be spending evaluating
other jobs) to hone in on this one, and then they may not accept you. It seems
to be a good deal _if and only if_ you're _certain_ this is the job you'd
prefer over the others you'd seen so far, or that you have hopes of seeing.

Otherwise, this is the one prospective job you'd want to entertain last.

So this might look good to Automattic since they are capable of handling the
load, but they are forgetting that such a load is not preferable from the
employee's perspective if you have many decent job offers. A shorter, more
streamlined hiring process will probably win out, since they're likely to try
interviewing _anywhere else_ first (easier) and since the opportunity cost
anywhere else is less. I'd like to think we're at least a little rational
about this, at least those that feel somewhat time-constrained when job
searching, like the currently unemployed.

I _think_ this gives other companies a slight edge against Automattic, at
least for prospective employees who have options. I think by doing this they
will lose good candidates, even ones who might (eventually) entertain their
lengthy interview process.

Am I off base here? Or does this seem sound?

~~~
bigchewy
I don't think you are thinking this through all the way. You are optimizing
the transaction (hiring / taking a new job) rather than the process (a long,
mutually beneficial relationship).

First, As an individual evaluating opportunities, how much more likely are you
to learn if there is a not a good fit and therefore avoid months in a sub
optimal job?

Second, the concept of a company still being at 99% or 100% productivity
doesn't make a lot of sense to me. If you leave a job, the company is at 0%
productivity for that specific job you were filling.

Third, based on the comments, this will push certain people away from
Automattic (e.g. "I don't do anything for free") and into the hands of the
competitors. Automattic gets a double win: lower probability of hiring people
who aren't willing to put in effort to find a good long term fit and they
raise the probability that their competitor does. Awesome for Automattic and
for the entire team, who gets to work with higher caliber people.

Finally, from a behavioral psych perspective this has some similarities the
Zappos "pay you to leave us" approach. When someone goes through the effort to
get the job, they are more likely to stick around AND feel happy about it.
It's just human nature. Google "The Ikea Effect" for another example.

~~~
maqr
I'm legitimately curious how you arrive at the conclusion that the "I don't do
anything for free" crowd is less likely to put in effort and find a good long
term fit.

It seems that somehow in the startup community, doing things for free (unpaid
overtime, unpaid internships, unpaid working 'interviews') is considered a
badge of honor. I don't understand how that became the expected and accepted
situation.

~~~
dagw
Being a 'sucker' is seen by many employers as a very desirable trait in an
employee. From an employer point of view, why wouldn't you want the guy who
happily works for free over the guy who is 'difficult' and insists on paid
overtime for every hour after 40 hours a week.

And as someone looking for work who is just very good you need a way to
differentiate yourself from everybody else who is also very good. A history of
being willing to work ridiculous hours for free is a pretty good
differentiator.

------
atmosx
This topic is so subjective and has been so widely discussed (having github vs
not github, having issued foss code vs not issued foss code, prior experience
in the field vs non-prior experience in the field, etc) that I'm not sure if
there's anything that can be taken _for granted_.

Of course it's good to have experience, (good) github repo, foss contributions
and so on. But which one is better and why I guess depends on the specific:

If Google needs to deploy large private Git repositories, they will search and
hire Git core-team members. No matter if they have experience or they are
students. They are able to work hands-on, from day one.

If MS wants to compile the universe, it hires D. Robbins, creator of Gentoo
linux which has a soft spot for compilations (joking of course, but you get
the idea ;-) ).

So it's a case-by-case scenario, not some generally accepted truth that
applies here.

IT is a huge field with very diverse technologies coming into game
(programming languages, frameworks, network structures, databases, robotics,
etc.)

------
usablebytes
I appreciate that the auditions are actually paid ($25 an hour, irrespective
of the position) and as they claim, you can complete the assignment at your
convenience, sounds really reasonable.

Where I feel hesitant is - they hire "all" their employees on contract basis.
Wouldn't a great work place want their 'employees' be their 'people'? Would
you feel like the product belongs to you? And the tone of the whole article
also makes it a little dry - check out the comment by "Eugeny Brychkov" and
may be you'll understand what I'm saying.

------
awjr
From what I gather the article seems to indicate an candidate is given access
to their code base to 'do a fix' remotely. Is this a viable strategy
initially? Learning the application takes time. Also there is some risk that
the candidate will run off with the codebase.

I'm sure they've thought of this, but I can see a situation where you default
to a 'standard' audition task in it's own sand-boxed environment just to
mitigate any theft issues.

~~~
Clorith
The article stated you would be working in the area you wish to work in, so if
you are a developer you will most likely be working on WordPress or something
relating to it (I may be wrong, just a presumption).

Considering that Wordpress is open source and any candidate applying at
Automattic as a developer will most likely be familiar or know of it, the
chance of having a developer not being familiar with it in one way or another
are slim, even more so if it's someone actually wanting the job.

If I was looking to be hired by a company working with open source, I would
most definitely do my homework up fornt and familiarize my self with their
"flagship product" as it were.

------
jbb555
I imagine they find it really hard to recruit. I certainly wouldn't put up
this. It's WAY too much of my time for a job that I may or may not get.

------
MyNameIsMK
Bad advice for small business.

