
Limitations and pitfalls of the job interview - galfarragem
https://fs.blog/2020/07/job-interviews/
======
simonw
If you want a really big competitive advantage, figure out how to hire great
people who don't interview well.

~~~
edoceo
I use a gig to hire process for this. It works very well. I've been meaning to
write more about it but I also get income for helping with hiring around this
model so im a bit conflicted. The whole answer is long and complex. Basically:
give candidate a proper task on your project and evaluate them based on that
work - the "interview" is less than an hour, the "work" is longer (many hours
(and you pay for it)) - provides a solid evaluation across their skill-set

~~~
fredophile
That will work for some people but not everyone. At my current job I can't
monetize work I do outside the company without approval. That means I can't go
through you application process unless I quit first. Even if I was allowed to
do this I don't think I'd want to. The time commitment of an extra project on
top of my normal workload is not something I want.

~~~
nawgszy
Well, then you can just work out a deal to not monetize it.

Your objection is of course valid - extra workload on top of a normal job
probably has its limits. But the fact one is willing to pay those who are
willing to be paid for such an interview doesn't suddenly make it a strict
requirement ha.

I must say though, I also view the interview standard I've faced - quick phone
screen, 1h tech screen, 4-6h onsite (probably online now) - as adding up to a
lot of time very quickly, especially as the "onsite" time is of course awkward
when it means you take PTO to go to an interview. In this sense, I'd probably
rather have an ~8-12h coding task than a 4-6h onsite. Maybe even a higher
factor

~~~
scarface74
It’s one day for the tech screen and one day for the on-site. Of course it’s
more of you have to travel.

------
stove
There's a growing rift in software between employers saying "there's a talent
shortage" and a rapidly growing population of devs who feel like they're
locked out due to the technical interview process.

Many of the engineers not being hired are recent bootcamp grads but there are
also tons of CS majors that can't seem to "crack" the interview process.

Part of my job is helping companies "fix" their hiring and one of the ideas
that I've been putting forth for years that's slowly gaining steam is
developing a "technical apprentice" role. This role would be responsible for
tasks that are frequently de-prioritized like documentation, testing, QA, bug
fixes, note taking, etc. and would be a foot in the door for entry-level
engineers. The role is designed to focus on communication and soft-skills
while also giving the person a chance to prove their "grit" on the technical
side. Even a few months in an apprenticeship role is generally enough for
companies to "take a chance" on someone as an entry-level engineer.

This has been a great way to shift interviews away from algorithms and more
towards finding people can add immense value to technical teams even without
having on-the-job programming experience.

I'm curious what the HN crowd thinks about that role as a way to bridge the
hiring gap.

~~~
mac01021
This sounds like what most internships are intended to be?

How much does it cost to employ one of these apprentices?

~~~
stove
Internships are generally thought of as something you do during college
(summers or otherwise). Most CS grads would scoff at getting an "internship"
after graduating. Internships are also very structured and generally involve
working on a specific project within a technical team (I know they're all
different).

The apprentice role, the way I've been pitching it at least, is different.
This is a role where you join an engineering org and learn the product by
QAing it, join technical discussions and help out by taking notes for the
team, show off your communication skills by documenting new features and, big
picture, you find ways to add value to the team in whatever ways they need.
Over time, the bugs fixed get bigger and the person can bite off small
features, etc.

The problem is that companies _want_ to hire new grads (even bootcamp grads)
but don't feel comfortable paying SWE rates for someone who hasn't worked as a
SWE (often rightfully so). The comp for this is equivalent to a QA eng but has
a clear path towards being an entry-level SWE (3-6 months maximum). If after
3-6 months it's not clear if the person can add value as an engineer then it's
clearly not a good fit.

------
Zaheer
Related message for students / new-grads: Find an internship. Internship
interviews tend to be much easier as the stakes are lower. If you perform well
during your internship (arguably easier / more accurate indicator of success),
the company will likely extend an offer. At big companies I've seen internship
to offer rates exceed 30%.

~~~
TrackerFF
Unfortunately, this has two sides.

I've noticed that in banking - and I'm not talking about investment banking,
or high-finance in general, but regular consumer banking - there's been a
trend to basically hire woefully overqualified people for the lowest positions
around - I'm talking about bank greeters, customer support, and what not, and
then train them from there.

This could very well be a local thing, but there are so many people today
qualifying for these jobs - lots of BBA and MBA candidates out there, willing
to do pretty much anything to get a foot inside.

When I was interning for a bank, even the greeters (basically the person that
just greets the clients, and forwards them to the right people within the bank
- a receptionist, really) had a Bachelors degree, many were working on their
Masters.

The more sought after positions had been re-labeled "graduate programs" or
"trainee programs", and were aimed at the top-shelf students. While the rest
pretty much had to get a foot inside by working their way up from the bottom.

So you suddenly have a ton of highly educated candidates applying for jobs
that only 15 years ago required a HS diploma, if that even.

Then when they're first inside, they tend to get moved around internally - as
a lot of positions only get posted internally.

It's almost the same way with internships. Internships are there to
practically train and select future company workers. If you do well, you get a
return offer - if not, well, at least you have some experience.

With the rising number of graduates, I can foresee a future where candidates
are being divided into the regulars and the elites. The regulars will, no
mater how qualified they are, will have to start at the rock bottom, proving
themselves for $8.5/hr, while the elites are trained for
leadership/management-track positions.

~~~
908B64B197
That should be a signal that's there's an oversupply for whatever Bachelors
degrees these employees were holding.

It's the same thing in law, the best advice to give to an aspiring lawyer is
to go have a chat with a practicing lawyer at a non-elite firm.

------
stepstop
Wow, that headline is a stick in the mud on a nuanced topic.

> What’s your biggest weakness? Where do you see yourself in five years? Why
> do you want this job? Why are you leaving your current job?

I do a lot of tech & business interviews, and I don't ask those questions
(unless they've recently left A LOT of jobs). I ask situational questions to
understand how they think, who they talk to, what research they did to
understand the problem, and the solution (was it simple?). If they tell me
they built a Rube Goldberg machine, I ask if they would have done anything
different with hindsight. If they don't realize they built a Rube Goldberg
machine, well, perhaps they won't be a good fit.

I look for people who can solve problems, do their own research, ask for help
when they get stuck, aren't afraid to attempt solutions (many that will
knowingly fail) and have the introspection to identify failures and admit it
are generally people you want to hire for senior positions.

Now I admit that it's a lot harder to hire this way for junior positions, when
they have less examples, less job history, etc. Educational projects are a
substitute, as well as working on personal projects.

~~~
dang
Ok, we'll replace it with some of the mud in the headline above.

------
grugagag
It is true, and not that I am trying to be dishonest myself but I am not the
same person while taking an interview. It's a sad that we can't be honest or
humble but in this system we have to sell ourselves, fake enthusiasm for the
hiring company is a must, do whatever it takes to pass the interview then
think later if we take the position or not. If not, somebody else with the
same capability or or less will snatch that job. Plus that interviews like
tests are gameable.

In an ideal world a trial period would ensure both the employees and employers
are a fit. But that could be abused as well if it becomes the norm.

~~~
Afton
Someone always suggests this. The issue is that if you did this, 95% of the
candidates that would agree to this kind of setup would be the kind you didn't
want.

If I'm sitting on 2 offers, one is a hire and one is a "Let's see how this
works out after 2 weeks of work", I'm going to take the first one. And that
says nothing for the necessary benefits question in the states, where changing
jobs often involves expensive (in money or time) changes to health care
insurance.

~~~
grugagag
We don't know for sure and it's hard to know how this works. What's obvious is
that the current system is broken and we need a replacement of some sort.
Being able to try a company and see whether they like me and whether I like
them and the type of projects I am supposed to work would be a major factor in
finding the right marriage. We take jobs for the salary tag and quite often we
do whatever we have to do to continue getting that nice paycheck but we're not
happy with the work we do.

~~~
Afton
For sure, it's an empirical question what %age of people who would accept this
kind of offer are "I have no other choice, so I accept your offer" or "I want
to make sure I will _actually_ want to work at your company".

I'm just saying that I would 100% not do this unless I had no other choice, or
was independently wealthy.

------
jaaron
I'm wondering if much of the discussion here is even about the article, which
advocates for things we long know work better:

* Structured interviews

* Blind auditions

* Competency-related evaluations

The title "Job interviews don't work" is rather bait-clicky when clearly they
advocate that _some_ form of job interviews work. Or are at least better.

As a technical hiring manager for over a decade, here's where I'm at:

\- The best interview is an internship. We can't always do that and often we
need senior talent _now_.

\- The next best interview would be a portfolio. I am _so envious_ of artists
with their public portfolios. If there's one thing I wish we as an industry
could figure out, it would be some way of stopping to test and retests
ourselves as if we have to constantly reprove what we've already done and
instead find a way to better showcase our work.

\- The next best technical interview would be a "homework" project, but I've
come around to the mentality that this just isn't fair to candidates. As a
hiring manager I love it, but most folks just don't have the time to do a
bunch of unpaid work. Even if you compensate them, it's unrealistic for many.

So we're mostly back to the suggestions in the article. They're good. A good
hiring process is _not easy_ but it's worth it.

And finally, a bit of anecdotal evidence: yes, there are folks out there you
probably shouldn't hire. They aren't a good fit for the role. You want to set
them and yourself up for success. That said, there are probably more people
who can excel than you realize. A major factor in their success is the
maturity of the team and leadership that's already in your company. Sometimes
you'll get lucky and hire some rare talent, but if all you're doing is looking
for "rare" talent, then you're likely poorly calibrated and relying too much
on outside talent to come in an fix the mess already on your hands.

~~~
RangerScience
> The next best interview would be a portfolio.

I tried using my (limited) open source hobby project portfolio as a substitute
for coding interviews. Companies either didn't take me up on it, or still also
required me to do their regular take-home. Twice now, I have had two companies
ask for the same take-home, although in the first case they asked me to re-do
the work in their preferred language.

~~~
chucky_z
FWIW, as a hiring manager, if someone has a portfolio I definitely judge them
on it, and if it's good it allows me to bypass huge swaths of technical/coding
interview stuff and dig much deeper into where/what I want. I always take it
as a positive, even if it's old stuff.

~~~
RangerScience
What do you find makes a portfolio better or worse for these purposes?

Not so much "more likely to get them the job", more... I felt like my projects
weren't actually suitable to take the place of coding interviews, largely
because I couldn't actually drop in and work on them in the way that coding
interviews show me actually doing work.

~~~
chucky_z
Literally anything. If I see someone with a lot of relevant forked repos, even
if they're old, I take that as interest and something I can bring up.

If I see a repo of rcfiles I know they care about working efficiently. If I
see abandoned stuff with more than 1 commit that's OK, that's something that
was cared about at one point. These are just two super generic examples.
Almost everything is a positive.

The _only_ thing I don't like to see is repos with 1 commit, and nothing other
than a README with the repo title in it. Not really negative, more of a 'cmon
gimme more.'

------
kanox
> What’s the best way to test if someone can do a particular job well? Get
> them to carry out tasks that are part of the job. See if they can do what
> they say they can do. It’s much harder for someone to lie and mislead an
> interviewer during actual work than during an interview. Using competency
> tests for a blinded interview process is also possible—interviewers could
> look at depersonalized test results to make unbiased judgments.

Why would a practical test be more effective than a discussion about prior
work experience? Short coding tests under pressure are extremely
unrepresentative of real work and I'm not sure homework-style interviews are
much better. I personally don't want to spend an entire weekend on your test
and I've dropped opportunities for that reason in the past.

All these "competency tests" are good for is catching blatant lying. Is this a
serious problem? In my view job interviews are mostly about finding a match
between skills/interests and project needs.

Communications skills are a very big advantage in interviewing and that feels
unfair but I'm not sure that true. Communication and persuasion skills are
extremely useful and important to all office jobs, even the most technical.

I'd bet that people who interview well also write beautiful code comments and
commit messages.

~~~
hellcow
> All these "competency tests" are good for is catching blatant lying. Is this
> a serious problem?

Of the people I interview that claim they have years of experience, mastery
over multiple languages, and expertise in various frameworks, a solid 80% or
more can't pass fizz-buzz in any language they choose.

~~~
smichel17
Claims made on resume or in person? On the seeking side, I've felt pressure to
put every technology I've used even in passing on my resume, to satisfy
"buzzword bingo" and get through the initial screen, but in an interview I
would give a (truthful) answer along the lines of "I don't have a _ton_ of
experience, but I know the basics; enough to be confident that I can quickly
pick up whatever else I need to learn on the job."

~~~
hellcow
> Claims made on resume or in person?

Both. And it's not "putting every technology I've used even in passing on the
resume" that bothers me. It's "I don't know how to write a loop or a function
in any programming language."

------
mundo
I interview a lot of people and found much to complain about in this essay.
This is the main thing:

> The key is to decide in advance on a list of questions, specifically
> designed to test job-specific skills, then ask them to all the candidates.
> In a structured interview, everyone gets the same questions with the same
> wording, and the interviewer doesn’t improvise.

This is good advice, _if and only if_ you're interviewing an undifferentiated
group of applicants, as in the cited examples (college entrance and army
recruits). If you're hiring a QA III and you have three different applicants,
it's terrible advice. You need to ask about the candidate's specific
experience, and ask follow-ups.

More generally, I don't think the stated goal of an interview according to
this essay (peering in to the candidate's soul to suss out traits like
"responsibility" or "sociability") is possible or reasonable. My goal is more
modest - I just want to figure out whether you were good at your last job or
not. If you say you're responsible, I can't prove you right or wrong in a one
hour conversation. But if you say you're a whiz at Selenium UI automation, and
you're lying, I will figure it out pretty easily.

------
MisterTea
My boss tasked me with hiring my assistant. HR filtered most of them and I was
left with three applicants. So I ran it the way I'd like to be hired which was
skip the useless small talk and other painful BS and just bring the applicants
around the shop and show them what I did.

The first applicant seemed like his mother dressed him and reminded him to
breath that morning. The second guy was pretty sharp but disinterested during
the walk and talk. The third applicant immediatly stood out. He was excited
and fascinated by our systems and kept asking technical questions - winner.
Excellent co-worker until he moved on to greener pastures.

All that wear your best suit and where do you see yourself in 5 years (best
answer: prison) nonsense sounds like it was lifted from one of those cheesy
1950's self help shorts the MST3K crew routinely riffed.

------
ironman1478
This is a great article. An undiscussed issue I've personally seen when
conducting interviews and being parts of roundups is a lack of self
understanding. lots of people who conduct interviews think they are way
smarter than they are (me included!). They hold up candidates to very high
standards and then dismiss them at the slightest mistake, however many of
these people also make many mistakes on the job. When I give interviews I
always just ask the easiest and clearest questions I possibly can that still
try to be relevant to the job to minimize this bias.

------
kube-system
> A job interview is meant to be a quick snapshot to tell a company how a
> candidate would be at a job.

> Unstructured interviews can make sense for certain roles. The ability to
> give a good first impression and be charming matters for a salesperson. But
> not all roles need charm, and just because you don’t want to hang out with
> someone after an interview doesn’t mean they won’t be an amazing software
> engineer.

If that's the attitude someone has towards interviews, then no wonder they
draw the conclusion that they don't work.

The real issue is that most teams either don't give much attention to
interviewing (because they have their primary job to attend to), or too much
of the process is delegated/outsourced elsewhere (where people only
tangentially understand the work area, and/or have no deep knowledge of the
job role).

Lean on interviews for measuring soft skills, and lean on demonstrations
(portfolios, code tests, pseudo-code, problem solving, etc) for measuring hard
skills. Every job requires _some_ balance of hard and soft skills. If you use
the wrong tool for the evaluation, or if the person using the tool doesn't
know how to use it, you get the wrong result. Interviews have their place, but
technical evaluation is not it.

------
xelxebar
This hits home too hard. My job search is going so poorly, that I have started
to doubt my technical value at all.

Confusingly, personal one-on-one interactions with companies or hiring
agencies almost invariably result in positive feedback and comments, "you are
exetremely hirable," "you have a strong technical background," etc. However,
none of these interactions has gone anywhere. Either they "move forward with
someone else" or simply evaporate into thin air. A few have even evaporated
after extensive interviews and claiming that they wish to hire.

Is positive-sounding feedback just a polite way of avoiding some "elephant in
the room" problem? Am I inadvertantly projecting an image of ineptitude or
hostility?

I have over 20 years of experience on Linux, tinker and program as a hobby,
and also lightly contribute to open source projects. I believe I have what it
takes, but geez, sometimes this job search is just soul crushing. I just want
to offer my skills and talents---to be a valuable member on a good team.

</vulnerable-rant>

~~~
putsjoe
Hang in there. I have a friend who's in a similar position, she's interviewed
and been ghosted a few times and it has really gotten her down. It can take
time but eventually a company will come to their senses and realise your
value.

------
eminence32
In the summer before my senior year at college, I did a 3 month internship at
a software development company. The interview for the internship was very
soft, partly because it was only an internship, and also because I knew
someone at the company who helped me get the position.

After I graduated, I applied for a full-time position there and have been
there for 10 years. The interview process for the full-time position was also
fairly soft and non-technical because I was hiring into a team that I worked
with during my internship. I like to describe it as a 3-month long interview
process. Not only did the company get to know me and what I was capable of,
but I got to know the company and its people (in order to make a decision
about if it was some place that I would like to work).

Surely this isn't scalable (internship are fairly rare, and generally are not
available to anyone except students or recent grads), but the whole internship
process worked out very well for me. It allowed me to bypass the traditional
tech interview, which is something I feel very fortunate about.

------
lasereyes136
Finding good people to work with is hard. Nothing you do will find 100% of the
good people (finding 50% of them is extremely good) and filter out 100% of the
bad people. Nothing you do will be effective for everyone. Trying to develop a
hiring strategy based on what you want interviews or hiring processes to be
like will be biased.

Accept that you will make mistakes. Accept that many of the good ones will get
away or be undiscovered. Accept that you will make hiring mistakes and have to
fix those.

If you are interviewing, do you really want to spend a few years of your life
at a place that does the bare minimum to vet you? They do that for everyone
and guess what kind of coworkers you are going to get. Sure taking PTO and
spending a day in an interview process is a lot of time. What is the
alternative? Not really being vetted and working with horrible people.

Finding the person or the right company to work with it hard. Take the time to
be comfortable that it can work for both sides.

------
exabrial
Silicon Valley interviews are worthless. The algorithm pop-quiz is just a way
for the interviewer to beat his chest about some obscure facts and demonstrate
his superior knowledge to the interviewee.

Has anyone found a better process than casual conversation? I've found it
effective as long as engineers and non-engineers get a chance to participate.
Usually I talk about what they want, talk about what you need, talk about
expectations from both parties, talk about needs, and talk about past
experiences both good and bad. Once expectations are set, there's literally no
opportunity for a "bad hire", because if they don't live up, it's a simple
conversation to refer back to the expectations that are set and help them
achieve , or worst-case, offer them severance.

~~~
the_jeremy
> because if they don't live up, it's a simple conversation to refer back to
> the expectations that are set and help them achieve , or worst-case, offer
> them severance.

That is a bad hire. You are describing a PIP and then firing someone that you
wasted time and resources recruiting, on-boarding, and training.

~~~
exabrial
That's not what I said... If a person doesn't feel they can meet the discussed
expectations, you wouldn't hire them in the first place.

------
bartread
This was potentially interesting but then:

> They are in no way the most effective means of deciding who to hire because
> they maximize the role of bias and minimize the role of evaluating
> competency.

I can believe that _badly_ planned and executed job interviews do the above
but I've overseen or been directly involved in the interviewing of hundreds of
candidates over the years, for dozens of roles, and the hit rate has been
pretty good. Two probation failures, and that's about it.

We're looking to assess skill and character in our interview process. We are
interested in whether you can do the job, and whether you're a reasonable
human being, and that's it. We have strong structures and guidelines in place
in terms of questions, answers, and evaluation. And inasmuch as it's possible
we strive to make our hiring process a pleasant experience, regardless of
whether a candidate is successful or not (obviously there's some level of
stress inherent in going through a selection process). We also give feedback
that we hope will help unsuccessful candidates in future (I realise this is
unusual and even frowned upon in some circles but our experience has been that
most people appreciate it enough that it's worthwhile to deal with the
headaches caused by the odd person who wants to argue about it).

I get it. There's a cohort of people on HN who don't like job interviews.
Honestly, I'm one of them. But done well, they work well.

Our process isn't perfect, and we're always looking for ways to improve it -
there was quite a lot of tweaking early on, for sure - but for us it's worked
well. We've spent a lot of time on it because - although my role is as a CTO
in a mid-sized firm, and this might not fit with everybody's expectations of
that role - literally my most important job has been and continues to be
hiring, building, and maintaining a strong, effective team.

And I am very happy with the people we've hired. Just as important, I'm also
happy with the decisions we've made about people we've chosen not to hire.

------
jokoon
The first time I ever heard the expression "social filter" was from Barrack
Obama.

There are many things that a democracy can give to its citizens. But
apparently, it seems civilization doesn't want to give up social filters.

I can understand social filters when it comes to friendships, sex and intimate
relationship, but for jobs, I will never understand why they exist.

~~~
antisthenes
> I can understand social filters when it comes to friendships, sex and
> intimate relationship, but for jobs, I will never understand why they exist.

Because jobs are just as social as intimate relationships, if not more so. Up
to about the 2010s, the majority of marriages came from getting to know
someone at work.

Even if nothing intimate does come from work, it's still people you have to
see 8 hours a day for years on end. If someone is repulsive/annoying/toxic, it
WILL make you miserable at work, and people are very wary of disrupting a
well-oiled collective.

~~~
jokoon
> Because jobs are just as social as intimate relationships, if not more so.

For very small or family companies, maybe. In small communities, maybe, but
small communities could also include people from any horizon.

But in other cases, I disagree. Work and production is the blood of human
civilization. Generally, the free market ideology says that if you're
competent, it's the only relevant parameter. Social filters are arbitrary,
unnecessary and backwards.

> If someone is repulsive/annoying/toxic

The nazis sent people who were not desired to death camps. Today, those same
people are being excluded from society through social filters. You cannot have
a healthy society if you keep segregating people like this, even if it's not
race, but other traits like education, politics or behavior. You're advocating
social darwinism through a detour.

~~~
the_jeremy
If you don't exclude socially toxic people from your company, talented people
who don't want to work with that type of person will leave. I know multiple
people who have left or transferred because of difficult coworkers, and more
who have left because of difficult managers. If one toxic person can't
outperform all the talented people who will leave, it is in the company's best
interests not to hire that person.

Yes, you can technically call this social darwinism. If you are an irritating
person that no one wants to be around, you will be socially excluded. I don't
know of any movement that is going to champion your cause. There is a
difference between preventing profiling / prejudice and avoiding manipulative,
mean people.

~~~
jokoon
> There is a difference between preventing profiling / prejudice and avoiding
> manipulative, mean people.

I see your point, and yes, there's a difference. I still believe those people
can be worked with, in some way or another. Being too selective at the
workplace isn't a healthy way to run society.

It's true that there's a difference between discriminating them and avoiding
them, but the result and the intention are the same, in my view. Avoiding them
is just politically correct and acceptable, but the truth is, it's the same
process.

The nazis lost the war, but they won the battle of social darwinism.

> If you are an irritating person that no one wants to be around, you will be
> socially excluded.

What's not really what I'm talking about. And that's a nature fallacy. The
role of civilization has always been to fix the problems of nature. There are
many ways to interpret some socials signs as being "irritating". They're often
normative or arbitrary.

> I don't know of any movement that is going to champion your cause.

Socialism, would it be democratic, or any form of progressivism, attempts to
aim at that cause. Europe is more progressive about this.

------
mathattack
This is very true. Sometimes it’s hard to know about hiring someone after a 10
Week internship. Crazy to think a 30 minute interview is better.

------
manfredo
A rather click-baity headline. The article doesn't claim that job interviews
don't work so much as it claims that subjective job interviews are more
subject to bias than structured job-interviews. I'd say that's true, but the
caveat is that structured job interviews are more subject to people studying
and honing skills specific to the job interview process. Grinding leetcode
definitely makes you a better at solving algorithms problems on a whiteboard
in 60 minutes but doesn't do much to improve working effectiveness.

I think there are 3 core trade-offs for the job interview process: logistical
feasibility, consistency, and resemblance to the actual work experience.

Doing 2 rounds of coding interviews, and one round of systems design from a
set list of questions with explicit rubrics is easy to implement and is very
consistent. But you can leetcode your way to knowing most coding question
archetypes. Systems design question archetypes are even smaller in problem
space. These questions have moderate to low resemblance to the actual work
experience. Sure, it can identify people who can't code or aren't experienced
in systems design. But does it show how well someone takes feedback, or
reviews other people's code? Not really.

One of my co-workers used to conduct 90 interviews that started with one
question: "how would you build a text editor?" He didn't specify whether this
text editor was WYSIWYG like Word, a web-based editor, a code editor, etc. It
expanded and touched on a whole variety of questions. It could be traditional
data structures, or UI design, or systems (e.g. implementing auto-saving text
fields on the web). This was low consistency, since the interview was mostly
unique to each candidate. It had moderate logistical feasibility since it was
hard to train interviewers on these open ended questions. But I think it had
better resemblance to the actual work experience, since it didn't just test
coding ability. It tested thinking through the problem and what the desired
end behavior for the user really was and navigating how those expectations
influence implementation.

An idea of an interview process that I have is to do it asynchronously through
github or another version control system. Give the candidate a task to open a
PR on a mock codebase. See how they implement the task and justify their
design decisions. How thoroughly the test. Respond to the PR with comments and
see how the candidate responds. And next, have the candidate review another
person's PR and see what they look for in a review. This potentially has even
better logistical feasibility since it's not dependent on the candidate and
employee being active at the same time. I think it would have the most direct
resemblance to actual work experience, since it's emulating the workflow most
developers actually use in their day to day work. Consistency may be difficult
to achieve, but if evaluation was broken up into multiple segments for
separate evaluation it may be able to be made consistent.

~~~
jaaron
Nice analysis.

In my hiring process, we use a number of filters to gather the _data_ we're
looking for to make a decision. That requires a bunch of different steps. By
the time we're done, we've spent at least 8-10 hours talking with this person.

From a technical perspective, we do the following:

\- A short screen to go over the resume and ensure we've got a rough fit to
the right role.

\- A _simple_ consistent coding exercise using coderpad. (surprisingly some
fail)

\- A series of consistent open-ended questions we ask everyone about their
tech background, such as, what was one of the most difficult bugs you ever
fixed?

\- A set of consistent design/architecture problems: "given this design, what
problems do you see? How would you fix them?"

\- Another consistent, more involved coding exercise with an existing code
base that is VERY much related to the work they'll be doing.

\- A Q&A session on a wide range of technical topics. Goal is NOT for someone
to know everything, it's to get a bit of map of their strengths and
weaknesses. We found we make assumptions of what someone knows based on our
background and their resume. We try to make this fun.

\- Another set of behavioral and situational questions with a shared scoring
rubric on values such as teamwork, collaboration, communication and
leadership.

And then we have to take all of that data and look at it holistically and
across a wide range of candidates. And even then we'll make mistakes, but we
keep trying to optimize it, reduce bias, and make it better for candidates and
us alike.

One last note: I like to finish my first interview with the question:

"Is there anything about yourself that you really want me to know that we
haven’t discussed?"

Because I know I've only had ~40 minutes to get to know this person. I have my
agenda of what I want to know, but I could easily miss a lot. So I want to
give them a chance to represent themselves in the broadest way possible.

------
k__
Never say no, but don't say yes unless you're sure about it.

You won't believe how often that will be good enough.

"Can you code in X?"

"I can program in many languages!"

Didn't answer the question, but was often enough for the interviewer.

~~~
commandlinefan
Yikes, I don't know about that. If somebody asked me if I could program in,
say, Perl, I'd say I knew what it was but that was about it. That's like
somebody asking if I can speak Chinese: I know what it sounds like, but no, I
can't.

------
29athrowaway
Does your interview process perform better than a coin flip? (50% chance of
making the correct decision). If the answer is yes, you have a useful process.

