
Everyone sucks at interviewing. Everyone. - jaf12duke
http://www.humbledmba.com/everyone-sucks-at-interviewing-everyone
======
tomkarlo
I'm not sure what the OP suggests is feasible in most situations. It's
disruptive to the existing team and you have to have "throwaway" projects
around for the person to do that won't cause problems if they're late or
poorly implemented (and if that's the case, why do them?) Also, you're letting
every guy who "interviews" learn about the internals of your systems, code
base, security, etc to some degree, which is not something most companies want
to do.

Interviewing is hard, but it's clearly not totally broken, given that some
companies obviously do a far better job of it than others. Do we really
believe that the big consulting companies or banks don't know how to filter
for the better candidates given how much their business depends purely on
having the smartest people? Are Apple, Amazon and Google really just more
lucky at hiring?

One non-interview thing that I DO like to look at is publicly viewable work
like open source contributions, blogging, social media activity, etc. If
someone wants to work on web sites, and they're not doing anything web-related
outside of their job, I have to wonder how passionate they really are about
the domain.

~~~
JoeAltmaier
No, not throwaway projects. Real projects. And if they are late or poorly
implemented, well, they would have been that way had you hired the sap anyway.
But now you can just quit paying him, and try again.

~~~
tincholio
You're assuming you would have hired him anyway. If he was such a sap, and
you'd done a half-decent job of interviewing him, that shouldn't happen...

~~~
JoeAltmaier
I guess the whole point of the article is, that is a myth. Interviews don't
work.

Anyway, companies that have the time and resources are turning to this more -
the summer internship program at microsoft is actually a 3-month interview,
where the intern works on real code and gets a real review.

~~~
ghaff
Internships work pretty well for everyone at entry level. Not just
programming: lawyers, investment bankers, consultants, etc. But you don't
really have that option for more experienced applicants, especially those who
are employed elsewhere.

------
agentultra
Hiring is hard because understanding people is hard.

There's also the problem of selection bias. Your technical interviewers are
going to look for people that resemble themselves. In a broad sense this is
because they think they're smart and everyone else is dumb (and rightly so).
The problem is that this strategy can be far too successful and you will
invariably turn away a perfectly suitable selection of candidates along with
the unsuitable ones. It's human nature and difficult to detect.

There is another form of selection bias in the interview process. You need to
know that the candidate you're going to hire is going to be competent,
assertive, and talented. However the exact match of skills, abilities, and
personality traits that fulfill those broad categories are going to be based
off of those skills, abilities, and traits you believe have helped you to be
successful so far. When interviewing someone it is far too easy to check off
the features a candidate is lacking and miss the ones they do have that you do
not. A good hire, IMO, is someone who has some of the skills and abilities you
already have and some you do not. Yet all too often, we look for people who
have ALL of the skills we already have instead.

I think strategies such as the one in the article would at least by-pass many
of the definicies noted above. However I think it might be impractical in some
scenarios (ie: when the candidate is already in a position at another company,
or when they have received attractive offers from other companies). It's a
start though and I think alternative strategies should be considered more
often.

~~~
btmorex
It's not just impractical. It's a complete nonstarter in any competitive job
market (like the bay area).

First, contract to hire is a pure negative for the employee. People can "try a
company" out just fine with a full time offer and then quit if it's not a good
fit.

Second, it's far easier to find good people by poaching from other companies
than it is by traditional means. No one is going to quit their job for a 3
week contract even if they're really unhappy at their current job. So, with
this approach, you've just removed the biggest pool of talented prospects
there is.

~~~
erikpukinskis
You didn't read the article. They offer 5 day (few days off plus weekend)
contracts, or even weekend contracts. They will wait until you have a
vacation.

If your company is a total bore and no one in the bay area thinks what you're
doing is at all exciting, then you're right. The ONLY thing you have to offer
is a signed contract with a very high number on it.

But if you're actually doing something personally exciting to someone, they
will absolutely take a few days out of their routine (with compensation) to
explore that. Honestly, if someone isn't excited enough about my company to
put a few days into it without a full time contract, I don't want them in my
company.

I've put a few YEARS of my life into my company without such a contract. I'd
expect my employees to have at least a fraction of that passion.

~~~
mattmanser
IANAL but as far as I can see this strategy is not without risk.

After all, they are already being paid on those vacation days, are they not
technically working for their existing company?

What happens when your new employee's old company turns round and says 'Hey,
we just found out about your wonderful scheme about testing out our employees
when they're still under our contract. That means our intellectual property is
in your code base. That employee has no negotiation power and you clearly knew
that, _our_ price for this bespoke job is $100,000. You've already got the
goods, payment terms 1 day. Thanks.'.

~~~
kurin
I know there are some asshole IP contracts out there, but I think even so a
company would have a hard time claiming ownership of a project an employee did
for someone else, in his own time.

------
chollida1
> Sometimes, a talented person can't, for whatever reason, commit to a 3 week
> project.

I would think this applies to pretty much anyone who is in demand.

> Maybe he can take 3 days off his oyher job and work half a week and a
> weekend with us.

Can someone else explain, why i'd take time off of work to do this job
interview? Why not take a day off and interview with Google/Apple/Microsoft
instead?

~~~
dominostars
How can anyone expect a candidate to burn 3 vacation days to work for a
company they are unfamiliar with? For people who only get 10 vacation days a
year that's almost 1/3. The money you get may not even be enough to compensate
for the money you lose by using that vacation (since accrued vacation is paid
to you when you leave your job). This is a huge risk for an already employed
candidate. Why take that risk when there are plenty of interesting job
opportunities?

~~~
swalberg
The author says the person is paid "a reasonable contractor fee for the work",
which I would hope is equal to or greater than the current salary. If not, it
really doesn't make sense, as you say.

I would be curious to know what happens before this 5 day test. Do people just
walk in with resume in hand and sit down to work for 5 days? Probably not.
There would have to be some sort of discussions-that-aren't-an-interview which
would lead to a yea-nay decision on the work term. What sort of funnel is
there?

I agree there is a big commitment on both sides to do this 1-2 week work
stage. There has to be more to the events leading up to the stage than what's
in the article.

~~~
franklindholm
I see it like this, if you are using paid vacation days for this, then you are
in a way on your current employers payroll, I think it is a bit dishonest to
the current employer who pays you to rest/recover from a years worth of work.
If you take unpaid leave then it would be more reasonable, but then the fee
for the work done during the leave should be generous.

As it is with my current employer, it would be very hard to just take 5 days
or even 3 days of for something like this, paid or not. If it was with a
couple of months notice then I would think it wouldn't be a problem, but with
shorter notice than that it would be hard.

------
cygwin98
Perhaps Jason's approach may come out of his intuition, I'd like to explain it
from a more "academic" perspective. In economics, the labor market is often
suffered from information asymmetries where the employer has little means to
determine the productivity of prospective employees. Therefore, if the
employer is willing to pay average wage, it will obtain below average workers,
as workers who has higher than average productivities won't accept the offer.
Such a phenomenon is also called "the Market of lemons" in the context of used
car market.

The root cause of such a market failure is because workers' productivity is
difficult to measure. A certain measure has to be introduced to indicate the
worker productivity indirectly (often termed Signaling in economics). For
decades HR/Recruiters have been addressing this problem by using different
metrics as the signals/indicators. Popular signals include education (GPA,
university prestige).

Programming may be a bit more challenging as multiple factors can affect
programmers' productivity, e.g., intrinsic intelligence, problem solving
skills, the speed of learning. Our hard-working recruiters/interviewers have
introduced some new signals --- brain teasers, coding tests, etc.

What Jason proposed in this blog post is that we don't need all those signals,
they are all inaccurate and can be fooled around by a well-prepared
interviewee, why not directly measure their performance by working with them
for a short period, say, three weeks.

That does sound like a good idea. IMHO, that's basically what internship does
for students, but not sure if that will work for full-time employees, as it
will incur extra opportunity cost for them.

------
TheloniusPhunk
I was once given thirty seconds to come up with synonyms for information,
which I did. Then I was asked come up with antonyms for fast and furious
starting with the letter p, I came up with pudgy and pleased. Then I was asked
to tell my life story in twenty seconds. I refused. The interview ended.

~~~
spottiness
That's funny. Are you serious? Was that all they asked you?

~~~
TheloniusPhunk
That's all I remember being asked. I guess they saw themselves as pioneers of
creativity, but as they were something of an obscure start-up, this wasn't
totally clear to prospective interviewees. So I went in there expecting a
serious interview, but got nonsense. It's funny now, but I was pissed at the
time. Drove eight hours, booked a hotel, and wore a suit for nothing.

~~~
spottiness
LOL. I wish the writers of South Park read this... This is material for a very
hilarious episode...(Cartman would be the interviewer :)

------
yannickmahe
I like the idea, but as a developer there is little chance I'll do contract
work before getting a job when I'm looking for a "real" job. Right now,
finding a job is easy enough that I don't think a lot of people will jump
through a lot of hoops before getting the job.

~~~
bluekeybox
It's like accepting a lower starting salary hoping that, since you love doing
what you do, you will prove it to everyone that you're good at it and that you
will get a raise eventually. I got burned with this twice -- believe it or
not, low starting salary coupled with uncertainty about whether there will be
an eventual raise played a big role in me actually stopping loving my work
(which at the time was a good thing since I acquired more productive
interests, but that's besides the point).

~~~
hkarthik
I'm looking for Rails work now and I've been offered substantially low
salaries compared to what I make now. I was thinking maybe I need to adjust my
expectations but your post makes me think twice. Thanks!

~~~
artmageddon
I will repeat the parent's sentiment - do NOT accept a lower salary. I was
nearly caught in this trap a year ago, trying to get out of my current job
that's not exactly fulfilling. I was given an offer that would have forced me
to take a 13-18% pay cut and nearly had my vacation time slashed in half. I
politely declined. I stuck it out for a little more than a year and just got
offered a position that's going to pay 55% over what I'm getting now. Keep
plugging away and keep looking..

Edit: Not sure about the vacation thing with the new role, but this is on a
6-month contract. I'll assume long vacations are out in the mean time, but
it'll pay well enough that I'll be living ok for a little bit if I don't go
full-time after.

~~~
acconrad
I don't totally agree with this. If you shift from big company to startup,
you'll almost certainly have to take a pay cut, but usually you're doing this
in exchange for seniority and equity. I wouldn't agree with a blanket
statement that you should never take a lower-salary job.

~~~
kelnos
Yeah, this is a tough one. I recently took a (7%) pay cut to join a new
company, but my situation was a bit different. I had quit the previous company
outright without anything new lined up, and I was being paid a bit above-
market (their attempt to retain me despite the fact that I was unhappy). I'm
sure I could have eventually found a company willing to match or beat my last
salary, but I would have had to pass on several interesting opportunities to
do so. So you can't just make the blanket statement that taking a pay cut is
bad for any reason.

------
blauwbilgorgel
I like this approach of really testing out the waters before committing.

I find there is also a divide between HR/recruiting and the lead developers.
Once I had two interviews at a company. The first one from HR and a second one
from the lead developers.

The attitude of the developers made me think they didn't have much say into
the whole process. They did ask the best (read hardest) questions. This
interview order seems fine, though questions like:

how many lines of codes did you write in language X? If we ask you to build Y,
could you, and how would you go about it? If there is a problem in the
weekend, and we call you, would you come over to fix it?

Could perfectly be asked in the first round of the interviews. And if you
really want to be sure that a person will come to the office, if need be, then
maybe plan the contract meeting at midnight on a saturday :)

One thing I noticed while last searching for jobs is the apparent
inconsistencies in job listings.

Pre-requisites like: PHP and Ruby, Web standards and Flash, thorough
understanding of javascript (jQuery plug-ins), Photoshop or Illustrator and
version control, familiar with Linux and .net.

At first I ascribed these pre's to unskilled job listers, but maybe this is
the start of the negotiation process? \- "I do know X, but have to work on Y"
\- "That is fine, have you thought about salary yet?"

Is this really a thing in recruiting, or am I seeing things?

~~~
pkteison
I interviewed with a firm that said the initially advertised salary was too
high for my offer because I refused to describe myself as an expert in
silverlight and opengl. If they find an expert in opengl, silverlight, c#, sql
server, web services, c, and desktop applications, I wonder what they think
that would be worth.

So yes, I do think it's a negotiating strategy. I hope nobody good will fall
for it, I'd like to see this behavior punished by only being able to hire bad
developers.

------
cypherpunks
My experience has been the opposite. Best employees aren't desperate to work
for you. They're gainfully employed, and you have to poach them. No one I know
who is productive would give up this much time to something. Almost everyone I
know who is unemployed and desperate would.

The key to finding good employees is to do what Google does. Find successful
people. Don't have them come to you -- go to them. How do you do this? Talk to
professors and see who top students are. Read publications and books in your
field. Hire whoever wrote them. Find neat free software projects, and hire the
authors. The list goes on. People like that generally won't want to work for
you, and the trick is to recruit them somehow.

------
pnathan
I'm sorry. If I was currently gainfully employed and looking for a job, I
don't think I'd be on board with your system. I appreciate the idea of getting
to know a company, but, I'd be applying for several companies every day. Even
given the current interviewing speed (several hours), it'd eat up time.

Doing part-time contracting is just not going to cut it. I've done
moonlighting before: no one was very happy with my work, including me. You
can't hire me this way if I have a job already.

If I was unemployed and looking, I'd be more interested, but you would not get
a cut-rate from me: you'd get a full consulting rate & contract.

 _In my opinion_ , if you want the best engineers, you need to know them and
offer massive bait. Because they aren't just going to jump for anybody or any
old normal reason. You have to offer them what they want - and more than their
current job does.

~~~
true_religion
The problem with articles like this is that they talk about practices that
realistically (regardless of what they say) would be reasonably applied to the
mass center of the bell curve.

However, everyone who reads them gives criticisms as if this hiring strategy
will only apply to the "best engineers".

To make a music comparison---not everyone is a proven rock star. Some people
are garage band singers who're dying to take _loans_ to produce their record
just for a _chance_ to make it big.

~~~
pnathan
I am not a 'best engineer'. I am a semi-junior engineer who is quite likely in
the 1st standard deviation of engineers. This article's company hiring
practices would be very difficult for me to work with.

------
spottiness
Finding great talent is hard but identifying talent is not difficult. Yet,
finding and identifying talent is just half of the story. The other half is
determining if the person can focus in what you need, be motivated, take the
initiative, and deliver great work. That's the difficult part. More often than
not, very talented individuals have a lot of stuff in their heads, such that
mundane but essential work ranks low in their platonic priority list, and that
affects their capacity to concentrate and deliver.

------
hkarthik
I really like the approach of "Contract-to-hire" developers but I've often
found it to be difficult in markets where most good developers have multiple
offers at any given time.

Also it's tough to do this when you're boot strapping and literally every pair
of available hands can make or break your first big deal that helps keep the
lights on.

~~~
jaf12duke
my experience has been that the very best developers often prefer this.

I've had a few top-notch developer friends request recently that a startup
allow them to work as 1099 contractors for a month first. They see their time
as by far their most valuable asset and they want to learn more about a
startup before committing.

In reality, with work-for-hire employee contracts, anyone can leave and anyone
can fire for whatever reason. And that's what Paul English does. Contract-to-
hire just takes all the expectations and paperwork and emotions out of it.

~~~
silverbax88
I would add that it also takes out all of the benefits, which employers never
mention. No benefits, no unemployment payments. Having hired many people over
the years, I always get a kick out of this argument that they are the same.
They aren't the same at all! Much easier and cheaper to hire contract
employees. Much harder to get good employees. I don't hire contractors unless
it really is a short term job. Contract work isn't for finding your long term
employee partners. Contract work is for one-time or short term work. If I need
a long term employee, I look for a long term employee and I don't pretend that
I'm trying people out by using the guise of contract-to-perm.

~~~
hkarthik
I agree, using short term contract work to identify good long term employees
sounds contradictory.

------
mhp
Tl;dr try someone out part time before hiring them full time.

Ignoring the link bait title of the post, I think the OP's suggestion is good.
However, it's usually not possible. People are working full time and don't
have extra time to work on your side project.

Interviews should include the same activities the interviewee will be doing
during their job. Programmers need to program. Designers need to design. Sales
people need to sell. It's actually quite easy to do this effectively in an
interview and plenty of companies do that quite well (i.e. They don't suck)

------
mangala
After interviewing for months I basically memorized the answers to the main
kinds of technical questions. After I started to hear the same sorts of
questions asked over and over, I knew the process was completely broken and I
would never ask stupid technical riddle questions to gauge someone's
competence on the job.

I think a brief conversation about software development and a longer
conversation to determine how smart the guy is is what matters. Even someone
who barely knows how to code can learn on the fly if he's smart/competent
enough.

------
util
How do you decide who you're going to court?

~~~
silverbax88
Apparently Jason lives in a world that's much like a giant cocktail party,
where everyone is just milling about in one room looking for someone to work
with.

------
kevinburke
Work sample tests are the best predictor (p of 0.54) of success on the job.
[http://bobsutton.typepad.com/my_weblog/2009/10/selecting-
tal...](http://bobsutton.typepad.com/my_weblog/2009/10/selecting-talent-the-
upshot-from-85-years-of-research.html)

~~~
tomjen3
Yes isn't p usually supposed to be less than 0.05 for the it to usefull?

~~~
Robin_Message
It (probably) refers to a measure of correlation being 0.54 on a scale of 0=no
correlation to 1=perfect linear relationship
([http://en.wikipedia.org/wiki/Pearson_product-
moment_correlat...](http://en.wikipedia.org/wiki/Pearson_product-
moment_correlation_coefficient)) rather than a confidence interval, which is
what p<0.05 refers to (<http://en.wikipedia.org/wiki/P-value>)

------
yangtheman
For anyone looking for a job immediately (as I am), I don't think s/he has
time to do multiple 3-months projects. I do believe current interview process
is broken, and interview results are not indicative of job performance. I
think that's why referrals work the best. Also, although much shorter, weekend
hackathons are good ways to gauge working chemistry.

------
mattacurtis
One issue that has been ignored is that of security / confidentiality. What
happens if the individual you are "courting" works for one of your
competitors? Any project that he would work on for you would likely require
access to sensitive, proprietary data. Sure, you could force him to sign an
NDA, but that puts him in a strange situation post-project.

How do you balance giving the candidate access to your data such that he can
work on a meaningful project (read: the results of which are actionable) vs.
having him toil away with some dummy data just to see how he thinks?

~~~
ericd
There are often projects that need to be done in a company that aren't related
to the core product, so it's not usually that difficult to give a contractor
something unrelated to work on.

------
nathanfp
I strongly agree that the try before you buy approach is incredibly important
for determining difficult to predict cultural and team fit. That being said
while it is essential for early-stage companies to get this right, as pointed
out in the comments, it can be difficult to scale and can be inefficient to
make-up projects for potential hires, give them access to code, etc.

Some companies have found ways to bake cultural fit into their standard
application process to great success. Twilio for example asks all applicants,
business or engineering, to build telephony apps using the Twilio API before
applying. This self-selects for people who are more willing to do research, be
creative, and in general improves the likelihood that they will get along well
with the team.

I am a bit biased here, but another path I would definitely advocate for is
using interns. This is similar to the contractor approach but with a number of
benefits in terms of price, and the fact that a hiring decision is not
implicit at the end of the term. We have seen many startups build a pipeline
of early hires by taking on multiple interns and seeing who is the best fit
over the course of a few months.

------
pkteison
I don't understand contract to hire. I get it from the employer perspective,
but why do employees agree to it?

If I was willing to work contracts (e.g. had a wife I could get health
insurance and maybe some income-in-case-of-layoff security from), why wouldn't
I just always only contract so that I could make more and get paid for my
overtime?

If I wasn't willing to work contracts, wouldn't requiring me to contract up
front take me out of the running?

Or does this whole thing assume people will just go without health care for a
while? I could understand that if there weren't other choices, but given that
other full time employment is readily available, who does this?

~~~
nostrademons
The article suggests that people who can't afford to take time off from their
regular job instead contract on a project that can be done nights and
weekends. He's not suggesting you quit your current job before the new job
makes you a firm offer.

He doesn't delve into the ambiguous IP ownership of the resulting contract at
work - I know that if I did that, my current employer would likely own the
rights to anything I did for the prospective employer. But I suppose the
prospective employer can get around that by simply throwing out anything the
contractor does for them. It's a pretty big cost, to eat 3 weeks or so of
wages of a highly-paid contractor, but it's probably cheaper than an IP
lawsuit or a bad hire.

~~~
kelnos
From the prospective employee's perspective, it also just doesn't scale. Most
people I know are talking to multiple companies at the same time, often being
in the middle of interviews at 4 or 5 companies. This just wouldn't work if
each one required a consulting period, especially if you still have your
primary job taking up your time.

------
ArchD
IMO, an interview is to working at a company what speed-dating is to a long-
term relationship. The interview process may get some measurements about
someone's technical fundamentals, but very little can be gleaned about rapport
with employees and in general how well an individual will gel with a company's
goals and other team members. The success of a team is not just about the
competence of each individual but how well the individuals work together as a
team, and individual competence is not totally correlated to working well with
a given team.

------
enjayhsu
It's not just a time issue for the interviewee; the interviewer then needs to
spend extra time checking the quality of the work, requiring extra technical
expertise.

Awesome idea; just not feasible in many situations.

------
jacques_chester
I know someone who could plausibly answer that interview question. She has a
degree in English literature, undergraduate and post-graduate degrees in law
(U. Queensland and Oxford, respectively), plus taking classics on the side
while learning Scots / Civil law at Edinburgh. She's scientifically literate
and active in various skeptical societies. She wrote an award-winning novel
and has another on the way about an alternate history Rome where Archimedes
was captured and not killed by the Romans.

People like this are out there. Perhaps such "Hail Mary Non-Sequiturs" are
actually worth including.

------
yumraj
IMHO the title of the blog is incorrect. What he is saying is that he never
hires anyone fulltime without at least working with the person on a project,
and not that he doesn't interview.

Since, think about it, how did he find the person who will do the project in
the first place, especially if there were multiple applicants. Unless of
course the blogger is also saying that he never posts a job and only works
through reference, which defeats the entire argument anyway.

------
desireco42
I don't suck at interviewing, people who interview are usually as clueless as
interviewee, so you can abuse this game from your side as well. I completely
agree that people you hire most of the time have very little with what you
need and want and that process is broken.

Suggestion in this post is how I would go about it as well, give people 2
weeks to try it out and see if we work for them as well as they work for us.

------
Tycho
Some interview questions I want to say 'do you realise how awkward that is to
answer?' like asking people how their former colleagues would describe them,
or what their weaknesses are. You couldn't do that in a normal conversation so
I'm not sure it's a good idea in interviews. But who knows. Maybe it's
effective or maybe it's simply the done thing.

------
raffi
Automattic followed this practice and I think it worked quite well. When I was
there, the company grew fast, but at least it was growing with known goods.
When I'm back with Raffi Inc one day, I expect that I'll follow this practice
too.

------
timedoctor
I totally agree with this article. You can get much better information from
actually working with people. You don't have to expose them to your system,
you can create a small project for them that is related and tests their
skills.

------
seanp2k
@JasonFreedman That's great if it works for you, but personally I feel that
you're just putting all the risk with the employee, which is shitty.

If someone is out of a job, the last thing they want is a 3-week gig. Yeah, I
get that "well if they're good, they'll probably be allowed to stay". I
wouldn't even consider working for a contract-to-hire position with a "few
weeks" of guaranteed work. As a business, the risk is on YOU to hire the right
person. As an employee, you're offering me what, maybe a month of rent while
preventing me from going on most other job interviews? That's not a risk I'm
willing to take. I'd rather do 3 interviews every day and have 5 jobs to pick
from at the end of a week.

Also, the projects...I'd guess that the projects you have people work on
aren't very beneficial to your company, or are so focused that they might not
take advantage of the talents of the employee.

On the flip side, I think working with someone is the best way to get to know
how they are / can be as an employee, and sometimes I feel that otherwise good
employees will stumble on inverviews, so giving them some time to really prove
themselves can be a good option....but I think that by limiting yourself to
people willing to risk a few weeks on a long interview are going to be the
most desperate of the desperate.

------
marksbirch
Actually, I kind of rock at interviews...

------
mtogo
| You should follow me on Twitter: @JasonFreedman.

I should? Really? Actually, i think i'm just fine without signing up to be
spammed by you. Thanks for asking so politely, though!

------
mpg33
I like the idea of a "mini-internship" instead of an interview...

An employer could take on a person for say 2 weeks non-paid and let that
person prove themselves.

~~~
eropple
Nobody who's any good would come work for you for free. Ever.

~~~
noarchy
I'd say that they don't even have to be good to turn their nose up at the idea
of working for free. If you're doing work with real value, do you really want
to do it for free, unless it is for charity?

