
Tech sector job interviews assess anxiety, not software skills: study - sizzle
https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
======
dang
All: please don't miss that there are multiple pages of comments. The top few
subthreads have become so large that they fill out this first page entirely.
You have to click 'More' at the bottom to see the rest (there are over 700 at
this point).

[https://news.ycombinator.com/item?id=23848039&p=2](https://news.ycombinator.com/item?id=23848039&p=2)

[https://news.ycombinator.com/item?id=23848039&p=3](https://news.ycombinator.com/item?id=23848039&p=3)

We're working on performance improvements that will hopefully allow us to go
back to HN's original style of one big page per thread (not infinite scroll,
don't worry). In the meantime please look for those 'More' links when the
total number of comments is over 250 or so.

------
CobrastanJorji
I conducted a couple hundred interviews for my first FAANG employer, and I was
constantly amazed at the percentage of candidates with years of Microsoft or
Facebook experience on the resumes who apparently did not know how to program.
I always thought, 'huh, guess I know why they quit after 3 years, amazing that
they all lasted this long."

Then I interviewed for another company and utterly bombed. It became suddenly
clear to me that I had been an idiot. Of course nearly all of those candidates
were perfectly good programmers. They had had shit interviewing days, probably
mostly due to nerves, but they probably would have mostly been perfectly good
employees. How frickin' arrogant I had been for concluding that people who
couldn't solve incredibly high stakes algorithm riddles on a whiteboard in 45
minutes with enough speed and flair were somehow not qualified to be my
coworkers.

~~~
leafboi
You've been arrogant for years and you know what? So have I. I and many, many
people in this industry have been doing this for years.

What's different from you and many other people is that you admit and face
your bias rather than justify it.

I'm seriously curious what google interview board members have to say about
this study. People like Gayle Laakmaan have been saying things to justify the
whole process for years; but now that there's actual science, what do they
have to say in the face of science?

~~~
MattGaiser
From what I have read from Gayle Laakmaan, they know this throws away many
good engineers, but it is acceptable to them because it doesn’t let bad ones
through.

~~~
leafboi
But they're not aware that the filter is just anxiety.

Google is basically using anxiety to filter good candidates and eliminate
false positives which works in a sense but is still highly illogical.

Why not use a technical filter to filter for technical candidates? Anxiety
seems like a pointless filter.... how does that even eliminate false
positives?

~~~
mgkimsal
it would seem it might be 'letting in' a whole load of people who don't have
appropriate anxiety responses. There are some situations where anxiety is
perfectly normal, and perhaps even useful, but if you screen out people who
have normal anxiety, what impact does that have on your company operations and
culture?

~~~
UncleOxidant
I'm a pretty anxious person, so much so that I've pretty much retired instead
of doing anymore technical interviews. I had a discussion with a very anxious,
but brilliant software developer a few years back and we came to the
conclusion that as long as we don't let our anxiety get too far out of hand
it's actually kind of a superpower in the job setting. That's because that
niggling anxiety you get in the back of your head as you're coding generally
makes you more careful about how you're designing and testing your code.
You're more careful about security, safety and correctness. Whenever I get
anxious while coding I stop and ask myself if maybe it's a message that I need
to listen to - maybe it's trying to tell me to tread carefully because I'm
entering an area that's potentially problematic.

People without that niggling sense of anxiety always in the background, the
folks who are easily going to ace the programming interview because their
anxiety levels are so low, those folks, I theorize, are potentially not going
to be so careful. Now you could argue that that's a plus in many situations -
a startup that needs to get code out the door right away, for example. But it
really depends on the domain. For critical systems in applications like
healthcare, avionics or robotic control I think the anxious coder is the one
you want.

Companies that completely weed out the anxious programmers, as you imply, will
have a different culture with perhaps too much emphasis on risky behavior.

------
lrem
People bomb, including good people. That is sadly part of the system.

Once I had this candidate. Damn, I know my questions are deceptively simple on
purpose, but he literally couldn't do anything. Not even a trivial brute
force. Not even any related simple knowledge questions. Couldn't tell an
average from a median. The only good thing I could write in the feedback was
"seems to know some basic syntax".

It was quite a learning moment to read that everyone else was praising how
this was the most brilliant candidate they've seen in years. Well, mine was
the interview just before lunch, in a schedule that was atypically later than
usual. I guess starting an interview after most people start their lunch break
was asking for it. He also got hired - apparently the committee agreed that
"can't think when hungry" is not really that important a flaw.

~~~
koheripbal
One trend I've noticed that is markedly different from when I started
programming in 1998, is how dependent we've become to program via
Google/StackExchange searches.

I'm not sure I know how to write anything from scratch anymore, because I just
search/read/alter/test. The breadth of what I work on is 100x wider than it
used to be, and so I've become absolutely dependent on quickly reading docs,
copying code found online, and then deep diving into testing and rolling it
out the door.

I deliver a TON more, but I'm convinced I'd bomb literally every interview I
go to today. Today, for example, I updated our firewall rules, updated some
Java business logic, tweaked some javascipt and PHP on the site, reversed an
odd macro email-phishing virus we got, configured a new RDS server VM for
clients, and updated some network GPOs. 20 years ago, I would have never
worked on ALL of those things in the same week - let alone the same day.

Maybe we should interview with small take-home tasks to be submitted with some
write-ups to test how people research, reason, and write-up problems, rather
than writing code on the spot on a whiteboard. ...but maybe that's just me.

~~~
OliverJones
I've been doing this for a LOOONG time. Early on, the core skill for a
programmer was intensive in-depth knowledge of the language being used. We all
talked a good game about what we called "reusable software," but yeah. Talk.

Now things are TOTALLY different. Doing a good job requires extensive,
encyclopedic, knowledge of what's out there, how well it works, and how to
integrate it. npm, nuget, maven, anaconda, we all know the drill. In a day's
work it's much less necessary to implement an algorithm from memory.

What should programmers know from memory? Simple stuff like fizzbuzz. The
difference between TCP and UDP. What DNS is for. If SQL data's involved the
difference between JOIN and LEFT JOIN.

And, if they're experienced, they must be able to describe intelligently how
their own stuff fit into the bigger system they worked on. Because there's
always a bigger system.

If they're being hired to be a tech rep to an *sshole customer, possibly their
performance under pressure matters a lot. Otherwise, not.

This business of high stakes quiz questions serves just one purpose: feeding
the ego of the interviewer. "We have high standards!"

Yeah, so high that we wouldn't hire Jesus of Nazareth to be a storyteller.

~~~
bananaface
I honestly don't agree with this. The worst programs are written by people who
know how to plug a million and one things together, but can't drill down and
analyse the algorithmic implications of what they're doing. Electron runs like
shit and inhales RAM is because it was programmed by people who don't have
solid understanding of fundamentals. They understand a huge number of
horizontal abstractions but they have no concept of how it looks vertically.

Knowing how to maximally exploit a CPU is way more important than knowing
eight different Javascript frameworks if good software is your objective. And
frankly, learning Node is way easier than figuring out how to structure basic,
bare-bones Javascript so that it leverages your L1 cache.

And therein lies the problem. How many interviewers dock marks for iterating
over columns, instead of rows? Because that matters, a _huge_ amount. How many
interviewers would give credit for "how can you speed this up?" if the
interviewee said, "write it in C, and simplify the datastructures you want me
to use so we maximise sequential lookups over basic arrays, to maximise cache
usage." They'll look at you like you have three heads.

"Don't you know Big N complexity is the only thing that really matters if
you're looking for speed?" \- then you get Electron.

~~~
conradev
> Electron runs like shit and inhales RAM is because it was programmed by
> people who don't have solid understanding of fundamentals.

I feel like this is a huge oversimplification. I’d argue that for sufficiently
large projects (say, Chromium) every single programmer who works on it could
have a strong understanding of performance fundamentals but the product itself
could still have performance problems because there are necessary levels of
abstraction involved and no one person has the entire codebase in their head.
Performance problems could come from emergent behavior that could not have
been predicted by the original engineers.

But, as for implementing an app on top of Electron. Say I need to write a
cross-platform app with a consistent style and fluid animations. I know how to
write a NEON implementation for ChaCha20 for fun, or write a zero-copy binary
property list parser that’s faster than anything I’ve found. I have a decent
understanding of performance. That doesn’t help me write this cross-platform
app. I don’t have to time to write a novel native UI toolkit that allows me to
share code while maintaining platform consistency and native performance. I
don’t have time to fix the memory consumption of Chromium. I’m going to write
my Electron app in JavaScript, carefully, and hope the performance will be
acceptable (but not great). I don’t see how that can reflect on any individual
programmer poorly.

~~~
bananaface
I disagree. As evidence: games are _way_ more complex, have enormous teams,
and manage to simulate an entire game world rendering render hundreds of
thousands of polygons 60 times every second. Twitter crashes my browser.

~~~
conradev
But I don't think that it has _anything_ to do with the individual programmers
inherently understanding performance better.

There is likely a huge organizational focus on performance and more people
expressly dedicated to the task because it is a higher priority for Epic. It
is directly correlated to Epic's ability to deliver a successful product to
game companies, whereas for browsers it is slightly more orthogonal to their
success.

LLVM has a bunch of contributors who likely understand the intricacies of
machine code better than either of us, and:

> Each LLVM release is a few percent slower than the last.

> The larger problem is that LLVM simply does not track compile-time
> regressions. While LNT tracks run-time performance over time, the same is
> not being done for compile-time or memory usage. The end result is that
> patches introduce unintentional compile-time regressions that go unnoticed,
> and can no longer be easily identified by the time the next release rolls
> out.

[https://nikic.github.io/2020/05/10/Make-LLVM-fast-
again.html](https://nikic.github.io/2020/05/10/Make-LLVM-fast-again.html)

~~~
bananaface
Electron is a running joke because of how horrible the performance is. Again,
I don't buy it. I think it's much more a matter of "they can't" than "they
don't want to."

I think if those LLVM contributors understood _compilers_ better, they
wouldn't be introducing those issues. Look at Jon Blow's achievements on Jai.
LLVM is the bottleneck for his entire compiler at the moment, and he's
planning to replace it with what... Three people or something? But I also
think there are structural issues there. I don't think a top-down project from
a large company gets to use that excuse.

------
chpmrc
It seems pretty obvious to me that this kind of interviews are simply there to
assess how much you want to work for a certain company. As in: would you be
willing to study for months, go through mock interviews, read books, test your
skills etc. to have a shot at working for us? Even though the majority of that
stuff will likely have zero impact on your day to day work?

That's all there is, really. I've had no problems in the past implementing
complex algorithms, if I needed it to get the job done. Today I'd barely be
able to tell you their names. But to get to the point of knowing most of them
by heart and being able to implement them with no help whatsoever, would take
_a lot_ more work and dedication, knowing that you'll probably end up not
retaining most of it anyway.

The same system is used in competitive sports and academic tests. They mostly
measure how much you really want to be part of that league/team/school, rather
than whether you'll do well or not. If there was a perfect correlation between
pre and post entrance test performance there wouldn't be PIPs, performance
based layoffs etc.

~~~
benburleson
Amazon is the king of assessing how much you want to work for the company. The
recruiter will place a huge emphasis on nailing their Leadership Principles,
and coming up with stories from your work to demonstrate you embody the
Principles. They will even tell you that specific interviewers have specific
LPs they will probe on, but in the actual interview they ask very veiled
questions and you're supposed to know that they're asking about an LP. You're
expected to have a story all laid out in the format they expect; it felt very
much like a show of how much you're willing to conform to the interview
experience they expect, with no interest in actually getting to know you.

~~~
stbtrax
is there a centralized place with information like this but for other
companies as well?

~~~
fragmede
Not that I'm aware of, because it's a difficult problem. First off, reports
need to be anonymous to prevent employer retaliation, but if they're
anonymous, then how can the information be verified? Even if it's verified,
without extensive input, it's impossible to say if the details of someone's
experience is the norm for the company or just about that one part of the org.

Then finally, happy people generally don't go out an write a huge essay on why
things are fine. _un_ happy people will go on-and-on about every bad thing to
anyone who listens.

Thus, we're left with disorganized reports on various sites to glean culture.
Eg HN comments on the story about Amazon suing Brian Hall.
[https://news.ycombinator.com/item?id=23461326](https://news.ycombinator.com/item?id=23461326)

For those looking to move within the job market, it would be worth it to talk
to a tech recruiter for a few hours who's job it is, is to know company
cultures. (Traditionally, if you, the new employee that they managed to get
into a company with a bad fit, leaves before 90-days are up, they don't get
paid.)

~~~
rootusrootus
> it would be worth it to talk to a tech recruiter

IMO this is very good advice. Having been on both sides of the equation, I
think a good tech recruiter (emphasis on good) brings a huge value to the
process. As a hiring manager, I had one guy I knew I could rely on to bring me
really qualified candidates every time. His own filter was excellent, and he
only brought me people that were a good fit. I almost didn't need to interview
them. Other recruiters were terrible, one even sent me a candidate who had
faked their first phone screen so they weren't even remotely qualified for the
job.

That is the guy you want to know. He has earned his reputation. And it cuts
both ways, too -- his success isn't just when the company is happy with him,
but when candidates are too -- all the best ones are his clients, after all.

------
erentz
It was made the clearest to me when I interviewed a colleague who had left to
another company and two years later was interviewing to return. I knew he knew
everything and had the skills for the job because, well, I know him, I’d
worked with him before. He’d done this very job previously. Yet he still
performed terribly in the interview and (rightly or wrongly) I had to base my
evaluation on my knowledge of him rather than his performance in the
interview. Interviews suck so much for evaluating people. If you want a job
and good career progression the best advice has to be prioritize making
friends with lots of people.

~~~
decafninja
I feel in the tech industry, networking and knowing people only gets you so
far - typically just to the whiteboard interview. Then you have to solve the
leetcode problems just like everyone else. Especially so in larger companies.

FWIW, all of my friends who somehow managed to get into FAANG or other top
tech companies swear that were they to go through the interview again, odds
are they would fail. This despite all of them being great engineers.

~~~
goostavos
I'm actually a little scared to leave my current FAANG gig for that exact
reason tbh. I'm fairly certain I wouldn't make it back in the door without
more leetcode grinding + repeated loops than I'm willing to do at this point
in my career.

Plus, stack on top of that, I've seen how the sausage is made. I've done 90+
interviews on the other side. At the end of the day, it comes down to how
lucky you got in the loop. I've seen some people written off for some bullshit
reasons. Similarly, I'VE declined on people who're likely far, far, far more
skilled than I am because of the lowers/raises approach and the narrow slice
you're tasked with reviewing in the 50min you're given.

~~~
jsinai
What is the lowers/raises approach? Google isn’t up to the job (pun was not
initially intended, but here we are).

~~~
vishnugupta
Lowers/raises the bar. As part of interview feedback an interviewer is
supposed to rate the interviewee as raises/meets/lowers.

Typically, just one lowers in an a loop of 6 interviews is sufficient to
reject. On the other hand, a loop with all "meets" isn't a sufficient either.
It needs about 3-4 "raises" to overcome just one "lowers". With two "lowers"
the candidate won't even make it to the debrief or worse the loop will be cut-
short.

The bar-raiser (it's their job) are expected to specifically look for
candidates who raise the bar. Raising the bar == better than 50% of
"employees" (not job seekers) at their level. The rationale, when the bar-
raiser program was created, was that the collective competency in a
corporation should increase as more employees are inducted.

Source: I was a bar-raiser at Amazon; 400+ interviews, 300+ as bar-raiser,
conducted hiring boot camp, bar-raiser trainings, trained bar-raisers.

~~~
oa335
How does the organization gauge its success in bar-raising? Did you guys just
progressively hire less people over time?

~~~
vishnugupta
Bar-raiser program didn't have a feedback loop to track its effectiveness. Bar
raisers periodically met to discuss topics related to hiring and increasing
interview bandwidth but we didn't evaluate the process as such.

That said, the program (like any such program I guess) didn't scale well with
the immense pressure on increasing the head-count which in itself was a result
of delivering more. Hiring managers and recruiters (who are incentivised to
hire more) would rig up interview loops in a way to get their candidates
through.

The program itself came under increasing scrutiny (I guess for the right
reasons) as being too restrictive. I distinctly remember period between
2008-2012 they hired at a blistering pace, all over the world. In fact in
Seattle when they moved into SLU campus the buildings went from empty to
nearly 100% occupancy in a matter of year or so. You could even see it based
on the number of product launches from 2012 onwards (AWS, the hardware
products, new country launches etc.,).

And now Amazon is such a behemoth that it doesn't even make sense to speak
about it or analyse it as a single entity anymore. My prediction is in 5-8
years it'll be broken up into at least three companies under Amazon as a
holding corporation --- cloud, retail, and consumer devices.

~~~
foobarian
My judgment of Amazon is tainted badly by a bad apple we couldn't fire soon
enough - I had the unsavory role to collect evidence for failing the PIP that
would inevitably happen. Week after person was let go, he was at Amazon in a
pretty good division.

I keep wondering how they hired this person - never in a million years would
they have passed a regular hiring chain even with fizzbuzz type questions.
Wonder if they had someone stand in for the interviews.

~~~
decafninja
It's often been mentioned that interviewing skills is different from actual
work skills. Could be that he's an interviewing wiz but a poor day to day
engineer?

Netflix is well known for firing people that don't pull their weight with far
more ease than most companies. But a friend who works there tells me there's
still people there that perform poorly but somehow managed to get both hired
and stay on.

------
twicetwice
Interesting. I usually do pretty well in interviews and I've always attributed
that to my four years on the debate team in high school. This tells me it
might be even more true than I previously thought. Even when I'm nervous as
hell, when you put me in front of a judge (interviewer), I click into
"performance mode" where I project confidence even if I _know_ what I'm saying
is BS, let alone just a bit unsure. This is how you win parliamentary debates
when you've had five minutes to prepare for a topic you know little to nothing
about—and I guess it helps in interviews too!

That and debate gave me the ability to just start a sentence without knowing
where it will end and then talk my way to a conclusion that sounds like it
makes sense (useful when you have 2 minutes to prepare a 3-5 minute speech on
a topic you've never seen before). So when I get up in front of a whitebard, I
just start talking confidently, and even if by the end haven't solved the
problem quickly or efficiently or at all, it sounds like I've been thinking
sensibly about it.

My takeaway, I guess—parents, make your kids do debate!

~~~
Judgmentality
> That and debate gave me the ability to just start a sentence without knowing
> where it will end and then talk my way to a conclusion that sounds like it
> makes sense

You're effectively advocating for more bullshit. The problem is that this
works in the first place - we shouldn't be rewarding it.

~~~
twicetwice
You're right, unfortunately that's what it sounds like I'm saying. It's not
really what I meant. The idea I was trying to get across is closer to "being
able to communicate what I'm thinking about _as_ I'm thinking about it, and
verbalizing that as a coherent sentence rather than just fragments."
Ironically, I wish I had communicated that better :)

~~~
wallfacer
I’m a person who finds out what I’m thinking as I speak. Iterating towards an
idea with confidence in each turn. Even if you end up in the vicinity of where
you started, problem not actually solved, you demonstrate how you move through
problem space.

Similar to another comment I read comparing interviews to athletic tryouts and
competition, which tries to assess how a body moves through physical space.

------
rsweeney21
I'm a ex-FANG dev that started a recruiting company, so I am VERY familiar
with the arbitrary tech interview process. I'm a huge proponent of contract-
to-hire. Even started a company* to make it easier for companies to offer
contract-to-hire.

Now before you dismiss the idea because you are "too good" to do C2H hear me
out. Everyone knows that actually working with someone on the job is the only
way to get an accurate evaluation of an engineer. Everything we currently do
in technical interviews (whiteboard coding, pair programming, take home
assignments, etc) are just attempts to recreate a working environment. If you
are hiring a contractor first, you can do a light-weight interview process and
then just bring them on and start working together.

As the dev being brought on you can prove your value before you negotiate your
salary. You can also try out the company and manager before committing to an
offer.

It's crazy, I know, but I think it would drive dev salaries up if it weren't
so hard to change jobs because of the tech interview gauntlet we've created
for ourselves.

[*] [https://www.facet.net](https://www.facet.net)

~~~
rc-1140
Sorry, but C2H sounds like a great idea to you because you have the luxury of
having already earned FAANG money on your side. You'd have to pay me
considerable money for me to consider taking a C2H role so I could cover
health insurance across the usual suspects (general health, dental, vision,
specialists just in case). Like many people outside of this website (and
enough here), I don't have the luxury of FAANG money and I live in the United
States, so I _need_ health coverage either through my employer or on my own
provided I have enough money to offset it.

~~~
munk-a
This is one of the reasons I've always found it paradoxical that people in the
US seem to think that universal healthcare would depress the labour market.

Up here in Canada not all procedures are insured, Pharma, Vision & Dental are
the big three that most either pay out of pocket or purchase extended
insurance for - though there are generally pretty good provincial programs to
help with the costs of those. However, nobody is walking around in Canada
concerned that they may get fired and then be bankrupted by a heart attack.

Companies absolutely do still bear a significant per-employee cost to support
provincial healthcare and residents do need to pay a pretty light fee into a
pool (with options available for financial assistance).

The end result is that companies do end up bearing pretty heavy burdens on a
per-employee basis - but employees don't end up with as much of a burden
forcing them to remain in undesireable employment positions, though some
guaranteed income would really free up employees to have a more equal amount
of power in the job market.

~~~
hwillis
> However, nobody is walking around in Canada concerned that they may get
> fired and then be bankrupted by a heart attack.

Mate when I (in the US) took a month off because I got COVID, my insurance
tried to cancel. Fucking insane that we have to deal with this shit.

------
jcadam
Huh, the only successful interviews I have are the ones where I'm burned out
after a long, fruitless, frustrating job search. And so I walk in assuming the
interview will be a waste of time and just not caring anymore.

That's when I get an offer.

~~~
daenz
People can smell desperation, and rightly or wrongly, it pushes them away.

~~~
flattone
Haha. So the more driven and hungry i am the worse ill be viewed. The more
silver spoon and apathetic ill be cherished.

Sounds like dating ;)

~~~
rovolo
A different way to view this is that being desperate makes you behave badly.
For example, someone desperate for a job won't push back against bad decisions
and will become a "Yes Man". Someone desperate for a date won't engage with
the other person as a person.

~~~
flattone
That is a different way to view it. An incredibly negative seeming view of
humans. Ill lay my cards down though, i have limited years of really varied
experience (ups, ski resort, landscaper, metal grinder, fork lift driver,
house painter, door to door vacuum sales, google pm, msft Analyst, trying to
break into infosec.)

In my short list (ignoring years of time wasted travelling), i have only met 1
yes man. He came from money, and sales. Owned 4 large businesses, and would
over promise, under deliver and over receive ALWAYS.

So what are your experiences like and from where?

I think no backbone can be a personality trait, sure. But it is NOT a trait of
'need' and action, in my experience.

~~~
rovolo
Saying "Yes man" probably derailed my thesis since it's associated with the
higher rungs of the corporate ladder. I thought that this would be
uncontroversial.

I'd like to focus on Dating because I disagree with your offhand comment, even
though it was in jest and is probably more cynical than you are in person.

> The more … apathetic ill be cherished. Sounds like dating ;)

Feeling bad makes your behavior suffer. Being angry makes you less nice to
people. Being tired means you don't engage with people. Being scared or
anxious makes you hide your vulnerabilities and close yourself off. I'm not
saying that badly behaved people deserve to be ignored, I'm acknowledging that
it is uncomfortable to interact with people who are hurting.

Being "driven and hungry" while dating makes you behave worse. It makes you
over-eager to please the other person, and it flattens out your personality.
Instead of focusing on making the person like you, you should be exploring
whether you like each other. A date can be successful even if the two of you
don't end up dating because you two would not make a good couple.

------
jefftk
Here's the study:
[http://chrisparnin.me/pdf/stress_FSE_20.pdf](http://chrisparnin.me/pdf/stress_FSE_20.pdf)

They asked students to find the "Longest Substring Without Repeating
Characters".

For the formats: "We created two interview settings: public setting and
private setting (see Figure 1). In the public setting, wearing an eye-tracker,
participants solved the technical interview problem in the presence of an
experimenter. The participant was instructed to talk-aloud while solving the
task, which is a standard practice in technical interviews [28, 47]. The lab
space contained a whiteboard at the front of the room. An experimenter was
situated near the participant for the entire session without interrupting
participants’ thought process. If a participant asked for clarifica- tion, the
response was brief. For example, if a participant asked, “Does this look
correct”, the experimenter was instructed to reply, “Complete to the best of
your ability”.

In the private setting, participants solved the technical interview problem in
isolation. Participant were provided a private room with a whiteboard.
Participants were informed that the experimenter will step out the room and
will not return while they are solving the problem. Thus, they had to make
sure that there is no questions left for them. When the participant was ready
to begin the task, they wear the eye-trackers, the door was closed, and the
participant worked in privacy, until the task was completed within allotted
time. After the experiment, we clarified with participants if they had any
uncertainty about the problem they just solved."

They ran a randomized controlled trial on the effect of having another person
in the room, which is interesting. It does not, however, show that these
interviews fail to assess software skills.

This is also essentially the worst case for having another person in the room:
they are completely superfluous, just watching you, not helping, not
communicating, not collaborating. Not even saying things that might make you
more at ease.

~~~
PragmaticPulp
> They ran a randomized controlled trial on the effect of having another
> person in the room, which is interesting. It does not, however, show that
> these interviews fail to assess software skills.

The headline of the linked article is indeed heavily editorialized. There were
differences between the two groups, but the study didn't conclude that the
interview was only assessing anxiety.

> This is also essentially the worst case for having another person in the
> room: they are completely superfluous, just watching you, not helping, not
> communicating, not collaborating. Not even saying things that might make you
> more at ease.

That stood out to me, too. In an attempt to remove the influence of the
observer from the equation, they simply put a silent observer standing over
the person's shoulder. In the example photo, they have an observer standing
almost literally over the person's shoulder, silently, observing their every
move. It's not clear if that's how the experiment was conducted or if it's
just an example photo, but it's not how any real world whiteboard interviews
are performed.

Even the study participants noted how strange the situation was:

"P25 felt unnerved that someone was “watching me and they are not commenting
on my progress”

------
rmellow
The best technical interview I've had:

\- Take home: 2 days to read a newly released Deep Learning paper.

\- On site: explain the paper to the hiring manager.

\- Evaluation: (1) Did the candidate understand it? (2) Did the candidate go
farther, and look into the cited papers and understand that? (3) Can they
explain Deep Learning concepts and talk their way through related problems?

In 5 months of searching for a job, this felt like the most relaxed interview,
because it was really a conversation between people in the field. It was also
the most demanding and interesting job I encountered in these five months. I
got the job, but this happened especially because the manager actually did the
work of respecting my experience and talking to me, instead of going for the
traditional and "safe" BS.

An opposite experience:

Technical Interviewer asks me whiteboard SQL question. I answer it, only to be
corrected on a minor point. Turns out, when I tried it out on my own, I was
actually correct. This person was to be my manager.

I did get an offer from them, but this and other red flags made me decline.

Takeaway for managers who actually care about hiring well:

\- Don't force your candidate through one off tasks that don't represent their
day to day. Is your Data Scientist really just running SQL queries and
optimizing computational complexity of their algorithms? Isn't the main point
that they can think critically?

\- Hasn't their previous experience shown they can code? Their Github? Sure,
give them a (quick) take home project if you really want to make sure. In the
rare case they faked their way through the whole thing and you really couldn't
tell after talking to them for a few hours, you will surely notice in the
first few weeks.

~~~
proverbialbunny
That's a great idea! Was it for an MLE job?

~~~
rmellow
Data Scientist, but would be easy to adapt.

------
gregkerzhner
The interesting thing is that Leetcode style problems encourage your code to
go into the opposite direction that most production code should go. The game
of Leetcode is to write the most consise, clever function that solves the
problem in a tricky way. Thats exactly the kind of code I would flag in a
review - sure it might be super efficient and elegant, but good luck
maintaining it a few months later. I would way rather go for a solution thats
less efficient but easy to maintain - I can always improve it later if it
becomes a bottleneck (it usually doesn't).

I know the take home assignment is a divisive subject, but I think it's the
perfect way to evaluate a person's coding ability. Did they use dependency
injection or did they reference globals straight in the code? Did they break
down the code into small testable chunks or write one giant file? Would their
code pass code review? These things are way more important for most codebases
than shaving an order of magnitude of big 0 complexity.

The only issue with the take homes is that they take a ton of time. Most
companies say that it should only take a few hours but also ask for production
grade code. Thats impossible - clean code takes time. Companies should at
least start compensating for the time taken to do the assignment.

My favorite interview process involved a take home assignment which if you
pass takes you directly to the final onsite interview. There the live coding
sessions involved building more features onto the take home. As an experienced
programmer who sucks at Leetcode I really hope the industry moves in this
direction.

~~~
ngngngng
This is an accurate view of take home assignments. Most of the take home
assignments I've done were "supposed" to take 5 hours. Most of them too around
20.

The worst take home assignment I ever had was an extremely tricky brain
teaser, the answer to which could not be found on google, and had to be done
in the not very popular language this company used. So I not only had to write
useless brain teaser code, but I couldn't do it in a language I was already
comfortable with.

~~~
jcadam
Yea, I had one take home problem where the instructions stated "Please don't
take more than 1 hour to solve." So, just once, I decided to hold myself to
one hour, and barely managed to get to a solution that worked.

During the code review in the follow-up interview, I was raked over the coals
for not writing tests, failing to cover edge cases, etc. ,etc. Basically, they
expected production-ready code. Didn't get the job.

~~~
gjeck
I’ve shared the same experience! Coming out of college every time I’d get a
take home I would always blow way over the expected allotted time. They’d say
it should take 5 hours and I’d end up on it for 20 since I actually had the
time then.

Year’s later I did the same as you and decided to stick to the recommended
time, because I really don’t want to waste my nights/weekends on this junk
anymore. Working full time there’s barely any time for relaxing anyway. Sure
enough, the same thing happened in review. Got absolutely shredded on my
implementation. Didn’t get an offer and kind of wish they didn’t even call me
back.

------
mywacaday
It's ironic that people who want the job will be nervous and might interview
poorly but people that are only interviewing for experience or just don't care
will do better.

~~~
acomjean
My Mom makes this point about nerves. In the last century when she was
interviewing she got lost and showed up 45 minutes late to the interview.
Thinking she had no shot, she was relaxed and interviewed well and was hired.

When she left that job 10 years later, her boss noted, that "one of the things
she learned from her was that it was ok to hire someone who shows up 45
minutes late to their interview.."

~~~
drewmate
This is an interesting anecdote. All too often, interviewers seem to be
looking for any reason they can _not_ to hire someone rather than looking for
evidence that the person would be a good co-worker. I am guilty of this to
some extent, but a lot of it comes down to the interview culture/framework of
the company in question.

~~~
n00j
I think I'm the complete opposite. I want to hire you if you come in to
interview. Interviewing for an open position takes up so much time from so
many people during the process, why would I want to spend more time than
necessary. When I'm in the interview I try to do everything I can to make sure
you can succeed and looking for reasons to hire you, otherwise I have to spend
more time interviewing.

------
tomlagier
This is likely a source of gender disparity in technology. Men tend to be more
confident in math and engineering situations, even when they know as much or
less than their female peers (the much-reported "gender confidence
gap")[1][2][3]. We are disadvantaging equally capable women by using an
interview format they are less likely to do well on.

[1]
[https://onlinelibrary.wiley.com/doi/full/10.1111/j.1949-8594...](https://onlinelibrary.wiley.com/doi/full/10.1111/j.1949-8594.2012.00144.x?casa_token=twRjm26xk8YAAAAA%3AgoM49sKRPefxCR4nnnza8NU4DPTraQUiQb4opCN5SwEEJQ9ehW31zcF-0FSxqeAMG2NwoVa1zw7JDg)

[2] [https://eric.ed.gov/?id=ED542116](https://eric.ed.gov/?id=ED542116)

[3] [https://www.theatlantic.com/magazine/archive/2014/05/the-
con...](https://www.theatlantic.com/magazine/archive/2014/05/the-confidence-
gap/359815/)

------
jedberg
I've been interviewing SRE candidates for a long time. In theory, being able
to make technical decisions under high pressure is integral to the job.

But interview stress is very different than work stress. It doesn't seem to
translate well. Some people do fine during the interview and then crack under
real pressure, and some people it's the exact opposite.

I've changed my interviewing to involve very little coding. I might do
FizzBuzz on the phone screen just to weed out the folks who really can't code
anything on the fly, but that's about it.

After that, most of my interview is me explaining some real works tasks that
they would have to perform, and then asking them how they would try finding
the solution and if they have ever solved a similar problem in the past, and
if so, how. I also ask them about difficult problems they've solved, how they
solved it, how they found the problem, etc.

My goal is twofold -- try to figure out _how_ they research problems they
don't know the answer to, how they investigate issues; and give them lots of
examples of what will be expected so that they can bow themselves out of the
process if they don't think it's a good fit for them.

This obviously works much better with people who already have a job, and are
more likely to take themselves out of the process, but so far it's worked
pretty well.

~~~
draw_down
I've observed that candidates hamstring themselves in ways that seem to me
bizarre. Indeed the interview is an unfamiliar situation, so why would you do
something like use a language other than one you know well, or code in an
environment you don't feel comfortable in? I have had those happen in
interviews I conduct so many times! This is for the "front-door" interview,
not the on site.

Somewhere around halfway through, when the candidate is floundering and
failing on an objectively quite simple problem (I promise it really is that
simple), they'll say something like "sorry, I would usually have no trouble
doing this in Scala". At the outset I give the option to use their own setup
or Coderpad, either one works; they'll choose Coderpad and then halfway
through say something about wishing they had such-and-such from their
preferred IDE.

I think there is some psychology at play here, where candidates think if they
do it on hard mode, it's extra points or something. I really just need them to
solve a simple problem so we can be sure they know how to code. The ability to
code with both arms tied behind your back is not a signal we're interested in
measuring.

~~~
jedberg
Yeah I got that too. I would tell them, "do this in your favorite language"
and sometimes when they get stuck they say, "I don't really know Python that
well". Then why did you choose it?!

Although my favorite was, "I don't know how to write a loop, my IDE usually
does it for me".

------
dpc_pw
That experiment assumes that SWE job is exclusively about solving technical
problems, so we can just "lock the candidate in a private room and let them
solve it".

In reality human to human interactions, sometimes stressful, are part of the
job. Customer & PM requirements, technical debates, giving feedback,
explaining your technical solutions, etc. are common.

Maybe part of the problem is the education that does not prepare people to
reality of the job.

Note: I do realize that SWE interviews suck, and I'm not trying to defend
them. Just making comment on the study.

~~~
eckza
Hopping aboard your train of thought here... I've never been asked to
whiteboard anything truly heinous, and I absolutely despise interviews that
are set up as a gauntlet to be ran.

But if I'm interviewing you for a senior SQL engineer position, and you can't
talk me through whiteboarding a simple query with one join and one aggregate
function... you're not senior material. I'm sorry. We need people that can
communicate and explain themselves, and talk through their own line of
reasoning, where I work.

Again, I'm not defending SWE interviews... but of course people are better at
expressing their skillset when they're in a controlled environment. The
trouble is, the workplace isn't a controlled environment. Life isn't a
controlled environment.

I don't want to work with brilliant programmers that have zero social skills.
If devs can't muster up the courage to answer interview questions... maybe
that means that they should work on themselves a little bit?

~~~
mikelevins
Well, I can communicate and explain things reasonably well to a room full of
people. I ran a side business doing that for about seven or eight years.

And I can ship products. I've done that a bunch of times--in several cases
I've written most or all of the code in the shipping product.

I can't interview all that well, though, and I'm absolutely terrible at
whiteboard coding, or pretty much any kind of problem-solving with people
watching me. Except pair programming. Pair programming with a colleague that I
know well is no problem.

I have two obstacles. One is anxiety. I score really high on trait
neuroticism. It's one of the so-called "Big Five" personality traits. One of
the things it means is that I have a lot of self doubt regardless of cause or
circumstance, and a lot of anxiety to go with it. As for "working on myself a
little bit," the Big Five are extremely resistant to change. Basically, the
only things we know of that affect them much are psychedelic drugs and
catastrophic trauma. Working on myself a little isn't going to reduce my trait
neuroticism much.

I've been through quite a few tech interviews, including some familiar big
names. I've gotten offers. But that's never been because of my performance in
coding or solving brain teasers, at which my anxiety has been uniformly awful,
and so has my performance.

The second obstacle I have is that I cannot combine certain classes of
cognitive activity. For example, I cannot navigate and carry on a conversation
at the same time. I can drive just fine--safely and competently. I just can't
navigate if involved in a conversation, and will predictably get lost. It's
predictable enough that when my daughter was a teenager she exploited it for
laughs. "Let's see where we end up!"

Another thing I cannot do is solve logic and programming problems while being
watched or while carrying on conversations with strangers. The interaction
with the strangers forcibly occupies 100% of my attention.

I don't know if I'm representative of a vanishingly small fraction of the
population. Maybe so. But maybe there are others more or less like me.

Doesn't mean you need to change your hiring preferences. Probably does mean
you won't hire me. Oh well. I seem to be making do.

If there are others like me reading, and wondering what to do about it, one
thing that has worked well for me is to do good work for people and cultivate
good working relationships with them. I have more often than not been hired by
someone who already knew my work, or on the recommendation of someone who knew
my work.

~~~
dpc_pw
I'm sorry to hear that. Every situation is different. If I were in your shoes
I'd try to be open about it with recruiters and figure out / propose some
alternative that work better for you - whatever that might be. Also keeping
good portfolio of Open Source projects, blogposts etc. might help convince
potential employers that you're worth considering and that your 'condition'
(for a lack of a better word) is real and does not affect your work
performance. Keeping good reputation and connections (which you already
mentioned) is also very important.

From my experience companies are very flexible and open minded - mostly they
care about just not hiring someone that they will have to fire later.

~~~
mikelevins
No need for sorrow. Everyone is dealt some advantages and some disadvantages.
We work with what we have. I seem to have done all right. My resume is long
and I have good stories to tell. I can't complain.

I was responding to another poster who said that "If devs can't muster up the
courage to answer interview questions... maybe that means that they should
work on themselves a little bit?"

I mean, maybe. Maybe sometimes. Then again, maybe not.

Maybe "working on themselves a little" isn't necessarily the answer. There
isn't a known way to "work on myself" to affect my trait neuroticism in any
significant way, for example.

Maybe it's not necessarily about "courage". I mean, courage is doing something
that frightens you because it needs to be done. I know what that's like.
Technical interviews don't even rate. But that doesn't solve the brainteaser
on the whiteboard when a stranger's watching me.

~~~
dpc_pw
In general - working on yourself is usually somewhat possible. For many people
public speaking etc. is something they have to practice and they do eventually
learn. In some situations though it might not be. All individual cases are
different.

So I wouldn't be too hard on myself, but also I wouldn't give up. From what I
read, you might have already tried, so oh well, maybe you're just stuck with
it.

I like "We work with what we have". That's something I deeply agree with.

------
helen___keller
The methodology:

> For this study, researchers conducted technical interviews of 48 computer
> science undergraduates and graduate students. Half of the study participants
> were given a conventional technical interview, with an interviewer looking
> on. The other half of the participants were asked to solve their problem on
> a whiteboard in a private room. The private interviews did not require study
> participants to explain their solutions aloud, and had no interviewers
> looking over their shoulders.

> Researchers measured each study participant’s interview performance by
> assessing the accuracy and efficiency of each solution.

[...]

> “People who took the traditional interview performed half as well as people
> that were able to interview in private,”

~~~
codemonkey-zeta
I appreciate their effort to use science to support the narrative that tech
interviews suck, because I do think they suck and need to change, but isn't
N=48 a little small? Also, if it was only undergraduates or graduates, who's
to say their performance is indicative of the rest of the tech workforce?

~~~
tsumnia
The small N is addressed by the authors.

To acquire a larger N would mean recruiting more participants, which would
require either compensation/extra credit or rely on people volunteering ~1
hour of their time. Likewise, collecting that data would require time and
money. It is an time/resource issue with human research. You can require a
small N and get some results or require a large N and not have enough funding
to support it.

Also note, what is a "large N"? There is no set in stone amount and I've even
seen reviewers say a sample size of 10,000 is too small.

~~~
goliathDown
I'm curious as to what makes a good size of N. To me, N=10 would be do small,
N=48 seems a little under what is should be, and N=100 seems sufficient, but I
don't have a real basis for this (in fact, it's probably because of the
original count that had me settle on a min and max of 10 and 100). Maybe a
better question is: What are factors that studies consider, besides the
logistics you rightly pointed out, in determining a sufficient N?

~~~
learc83
>I'm curious as to what makes a good size of N.

That depends on the size of the population being sampled from, the margin of
error, and the confidence level.

For a huge effect like the one shown in the study, where one side performed 2x
as well as the other, a sample size of 48 is more than large enough to say
that the result is statistically significant. If there was as small effect,
that wouldn't be the case.

Put it this way. You want to find whether people from California prefer Taco
Bell or Pizza Hut, so you randomly sample 100 people. If all 100 people say
Taco Bell, then you can be reasonably confident that more people from
California prefer Taco Bell. Because if at least 51% of your population
preferred Pizza Hut, the odds of not getting one of those people in your
sample are minuscule (the odds of getting all Taco Bell people in your sample
if 49% of the population prefers Taco Bell is 0.49^100).

If 51 prefer Taco Bell and 49 prefer Pizza Hut, your confidence level is too
low to be useful--you need a larger sample size.

------
loktarogar
A company flew me out to San Francisco to interview once. On my way into the
building in the morning I was mugged (almost right outside). I told my
interviewer, they took me to the lunch room to sit down and compose myself.
"Take as long as you like", they said. 2 minutes later they came to collect me
for my interview.

I was a wreck. Worst interview of my life. Didn't get the job. I am pretty
sure I would have if I wasn't falling apart.

~~~
eckza
Oh my god. I'm so sorry... that had to have been very, very traumatizing.

It sounds like a net win that you didn't get that job, for multiple reasons. I
wouldn't want to work somewhere that I'd gotten mugged on the way in for an
interview, nor for a company that didn't give me as much time as I needed to
collect myself.

The closest I've gotten to that, was that I had an interview set up with a
company in Saginaw, MI, that I was really excited about. I did a quick
internet search to learn more about their company, and the first thing that
popped up was that a few months prior, an employee had been shot in a drive-by
in their parking lot.

I ended up cancelling the interview.

------
ipnon
>All of the women who took the public interview failed, while all of the women
who took the private interview passed.

Companies that are serious about avoiding gender discrimination in their
hiring practices need to carefully consider what they are really looking for
when certain candidates seem to have "the right stuff" after a whiteboard
interview. We need more research in this area.

~~~
jacinda
Given that women seem to be around twice as likely to suffer from anxiety as
men, this is a result I was not at all surprised to see.

\- [https://www.webmd.com/women/news/20160606/women-more-
prone-t...](https://www.webmd.com/women/news/20160606/women-more-prone-to-
anxiety-than-men-review-finds#1)

------
solinent
The only good way to assess a candidate is to hire them. It's sad, but true.
Some of the brightest and best will not even be able to code with someone else
even paying the remotest attention to them. Interviews used to take account of
this 15 years ago. Google came along and destroyed this.

I remember talking to a Google interviewer and I came up with a better
solution than they expected. Since my solution was non-standard for online
median finding-they wanted a unbalanced binary tree where I wanted a tree with
the values at the leaves and using a fixed height, in this manner you could
easily find successor and predecessor and I'd have actually found a better
solution than their expected solution. They wanted the standard binary tree
solution with predecessor and insertion, where I failed.

Everything will still jovial, until I mentioned that I consider the tools I
use and how to operate them as part of my skill-set. Apparently this is false.

This was me interviewing at Google for a potential job using computer graphics
or computer vision. I wouldn't even be working with problems anywhere close to
this.

I've found that everyone is just copying Google's process because of all of
Google's cash and success, but don't realize that most of Google's success is
not due to any investment but rather just one solid team doing search well.

Anyways, I'm glad I didn't get hired, I have a better job (pay and work). Both
of the projects I would have worked on have already failed or nearing failure
already, it's been like 3 months.

~~~
thelean12
I used to interview at Google and think you're misunderstanding why you failed
the interview.

If the solution works and is better, then the interviewer should recognize
that. I have this happen often when I use a new-to-me question (well, not
often better, but similar enough performance). Even if the interviewer failed
you for something they shouldn't have, it'd raise a flag in the hiring
committee if you passed the other 3-4 interview sessions and the committee
should look closer at the one failing interview to see what's going on.

And I have no idea what you mean by the "tools" and "how to operate them"
being false and failing.

~~~
solinent
Do you understand why I failed the interview? That's absurd. How can you
understand if I've not even given all the details? Your process is somehow
bullet-proof and guaranteed to be followed? Probably because that's the
assumption Google is predicated on. It couldn't have happened, so it didn't. I
mean, logically, that's true.

> And I have no idea what you mean by the "tools" and "how to operate them"
> being false and failing.

I mean I consider my skill-set (what they're buying from me) to be my
theoretical / intellectual ability, in addition to the practical activities I
take when using the computer to achieve my job (using a mouse, keyboard, git,
c++, testing). The interviewer mentioned they are looking for generalist
programmers who don't consider their tool-set (git, windows, whatever) to be
part of their skill-set. While I am quite general, I consider it a plus that I
know such a wide range of different technologies. It was my question which
prompted this, I asked about this very thing since I know it's a sore spot at
Google, knowing a few Google employees and some Google coops. I believe this
is the reason I failed, the interview completely changed his tone at this
point.

In any case, the interview didn't test anything I would have done on the job,
or my ability to do such things. It also didn't even test my intelligence
independently of my preparedness, since there's so much preparatory material
available, though it did eliminate people with lower intelligence who didn't
prepare.

I failed the interview because of a number of reasons, but I definitely solved
the problem. The typical process wasn't followed as this was right as COVID
hit. So maybe that had some effect as well, this was the second phone
interview with a new person (the first one I completed the problem, but there
was a glitch and the call dropped and we couldn't recover for some time), and
I've been ignoring the recruiter for years. I think I didn't really want to
work for Google in the first place, so that probably also came through. I'm a
big proponent of privacy and against Google's advertising business model.

I think based on the interview process I'm no longer considering Google
anymore, despite still having another interview in six months. I will probably
inform the Google recruiter in the future. It's just a waste of my time to
prepare for such a silly test. When the main book to get the test is a book
about how to cheat it and appear better than you are, it shows that the test
is completely broken. Maybe a company the size of Google should hire people
for short internships where they code throw-away prototypical problems and do
some paid work. Maybe on a problem that takes some time and planning. I'd do
that, even today. I think Google has lost most of the individuals who actually
have an impact in this industry beyond search.

Maybe that's why the process won't change, Google isn't selecting for some
traits which are commonly sought after.

------
Debugreality
As a hiring manager the test that I believe works best is a code reading
exercise. This helps reduce anxiety or the effects of anxiety (fear has a very
big negative effect on creative thinking)

I take a small fairly self contained function from the existing code base.
Give them 10 minutes to read the function.

You can also add a few actual bugs into the function if you like to see if
they are picked up.

I then generally ask them a couple of questions. First to step through what
the function is doing. Then what the larger context of the function might be
and finally if they have any thoughts on improving the function.

It's a quick test usually under 20 minutes you can even hand it over to non
technical interviewers if you describe what kind of answers they should be
looking for.

------
XCSme
Interviews are a joke and are as useless as randomly selecting people.

I did 14 interviews at Google (two applications, first one after being
contacted by them, for second one I contacted them to try again). As far as I
could tell, I passed most of them, and for some of them I even provided new,
unique solutions that the interviewers were amazed by. That being said, both
times I got reject in the last step before the offer without being given any
reason why. The only think they told me was that I should learn less from
programming books and have more experience actually building things. I have
never read a programming book, only learned by doing and had many projects to
showcase. Some interviewers were not even paying attention to what I was
saying, seemingly working on something else on their laptops. My recruiter was
both times so certain that I would get the job (based on the very positive
feedback she received from some of the interviewers) that she started telling
me about the relocation process.

Today, 4 years later, I am not mad about it as I got to work in a very cool
gaming startup for 2 years and now I run own company where I have created a
product used by thousands of webmasters. I am honestly just confused about the
process, as I know that I am a top-tier programmer and, based on what others
said, a nice person to work with. I guess that ultimately it was their loss,
but it's just a machine so no one really cares if X gets the job or not.

~~~
shados
Some states have GDPR-like laws that will absolutely allow you to request a
copy of your assessment. That can be a good way to know exactly what went
short (most companies don't expect this and will get VERY coy about what they
wrote, but the law's the law~)

Either way, more interesting is your first line:

> Interviews are a joke and are as useless as randomly selecting people.

I don't have a source, but I read a study once that showed most engineering
interview process is roughly as accurate in assessing success as flipping a
coin. I didn't look into the methodology, but based on experience I'm willing
to believe it.

Other fields don't have that issue because they don't use interviews as a
proxy for performance. Either they use portfolios (eg: designers) where you
can CLEARLY see the person's skills, either they use credentials
(certifications, or just resume). The interview is just to assess those data
points and to make sure expectations are clear.

In some fields (eg: trades like carpentry), people will look at the person's
history and hire them. Then based on actual performance over the first few
months, they may get let go or even demoted. Can you imagine a big tech
company DEMOTING a software engineer? Blasphemy! But really, that's the only
way to properly assess an employee.

Hire them based on their experience and claimed skills, have clear
expectations, adjust after a few months. Then you can give a shot to the
person with low experience who claims they "promise to work very very hard and
learn quickly". You can have apprenticeships too! And we can save everyone the
trouble of silly useless interviews.

------
cmrdporcupine
Multiple discussions about this on HN today, that's good. Maybe it will lead
to the pendulum swinging back a bit.

Interviewers need to do their due diligence. Coding tests can have their place
when interviewing for certain kinds of positions, especially for new grads and
entry level positions. In the end nothing can replace checking references and
looking into a person's work experience.

There is no algorithms and data structure whiteboarding test that can
differentiate between a junior and a senior developer. Being senior is a
product of acquired wisdom, not just skill. And in fact senior devs can easily
bomb out a whiteboarding test because we're further from university CS
experience.

~~~
glouwbug
Your last paragraph sums what why everyone on HN is so upset about the
interview process. That is, the process is tailored for new grads, and what is
being tested is not at all related to the job. 5 years of schooling, and 10
years of experience is not enough if you can't implement depth first search
using double linked lists, hashes - and ultimately graph theory - within 45
mins. Heck, new grads can't even do so.

~~~
cmrdporcupine
Well, yes, if you looked at my comments you'd see I'm part of that "everybody"
set.

But to moderate it a bit; in my interview training at Google it was clear: the
interviewee doesn't have to get the answer correct. The point is to see how
they think.

I mean, I think it's still insulting, but there is that.

------
robgibbons
Bad interviews make for bad hires. Bad hires can cost a company more than not
hiring at all.

Whenever I've conducted interviews in the past, I've gone out of my way to
help the candidates relax. Need to use Google? Go ahead, I do literally all
the time. Need some advice, something explained? Just ask, that's why we have
peers. Nobody writes code on a whiteboard, with no docs, with three people
watching, silently taking notes.

Interviews should reflect the environment within which the job is actually
done, and allow the candidate to demonstrate what they're good at, instead of
treating them like a circus animal jumping through hoops, giving them
performance anxiety.

For my most recent job, they circumvented my interview entirely and just did a
few days of contract work instead. That way the company gets usable work, the
candidate gets paid, and it filters out those who can interview really well
but may not be a good company fit.

~~~
northern-lights
>Bad interviews make for bad hires. Bad hires can cost a company more than not
hiring at all.

But is that really true? There are plenty of companies who take a Fire Better
approach (Amazon, Netflix to name a few) who seem to be doing just fine.

~~~
el_benhameen
Worth noting that, at least as far as I've heard, the pressure on employees in
at least some roles at the two companies you mentioned is incredibly
stressful. I would rather have an interview without undue stress than a
stressful one, but I'd rather have a stressful interview than continuous
stress on the job. (I am differentiating here between "working on difficult
problems" stress and "always wondering if you're going to get canned" stress.)

------
DrPhish
I hire for industrial IT, and being able to handle the anxiety of a high-
pressure situation with grace (e.g. mill down, losing tens of thousands of
dollars a minute) is an important factor. No one expects miracles, but you
need to be able to keep your cool and work the problem, not waste time
fighting with your own rising anxiety.

Seeing how they deal with a stressful situation is an integral part of the
interview in my opinion, and I imagine it is a feature for any IT role where
that is the case. Standard dev roles, or other roles where time-based stresses
aren't as prevalent, may be different.

Only yesterday I had a candidate bow out 10 minutes into the whiteboarding
exercise, which is probably best for both the individual and the company if
that kind of situation is untenable for them.

~~~
leafboi
Yeah, devs that work for me also work in a critical environment such as yours.

As soon as an issue is raised by our clients, they immediately kick off a 40
minute long timer and put a physical whiteboard in front of us asking us to
solve the whiteboard problem in front of them in under an hour. All the while
the client is physically behind us commenting on our code and watching our
every move as we paint the whiteboard with a solution. This is literally what
happens to me and my engineers on the Job. All of our code happens on the
whiteboard (we don't use editors or keyboard or mice!.)

The only thing we need to deliver to our client is a picture of our code on
the whiteboard along with some trivial complexity analysis and our client pays
us money for coding under pressure.

Our interviews just consists of me doing the exact same thing as what happens
on the job and therefore our interviews are 100% identical to our real life
coding problems.

When science tells someone that the interview process they've been conducting
for years is essentially wrong people have to sort of remake reality and their
own reasoning to justify the science rather than admit fault. Obviously, this
isn't what's happening with me or you in this case.... the interview process
at my company is 100% identical to real world scenarios so I'm good.

I've always been telling everyone I know since before this article that I
interview for anxiety not useless technical skills; anxiety is what determines
whether a developer is great or not.

~~~
redisman
I can’t tell if this is an actual white boarding purgatory you live in or a
joke ?

~~~
leafboi
Our company consults in a variety of software related things. For example in
our interviews we ask our candidates why are manhole covers round?

We ask them this because, literally on the job our employees are suppose to
program robots that make manhole covers. Daily problems we face include
deciding whether we should program the robots to make square manhole covers or
round manhole covers. Really hard stuff... and you really need to be able to
figure out why manhole covers are round or how many golf balls can fit in a
bus in order to do the job... If you can't figure this stuff out you're not
qualified to work for my company.

Not to mention, as I said before, none of our programming happens using
computers. We do it all on a whiteboard in under an hour while the client is
over our shoulder watching.

------
jconley
This largely applies to people that have already produced working software in
the real world, but...

Companies should trust the people applying based on their resume and ability
to talk about what they have done in the past or how they would approach real
world problems (organizational, interpersonal, technical). Maybe have them
write some small bit of real world code using libraries and frameworks they
are familiar with, while they are alone without anyone watching
[key|pen]strokes.

Maybe I'm lucky but after 20 years in the industry shipping products to
consumers I've never ran into a Software Developer or "Engineer" in the real
world that couldn't get things done in a normal working environment in private
industry (now in government, don't get me started). Some are slower, some are
faster, some can handle bigger problems, but I have no idea how to reliably
filter for that in an interview having tried many things.

The "OMG this senior person can't write FizzBuzz on a whiteboard or while
videoconferencing and knowing someone is watching your every keystroke so they
must be a horrible programmer" approach is just wrong.

I've only ever failed interviews where I had to solve problems or write code
with someone watching (virtually or in person). Let me be alone and give me a
product to build, though, and it'll get done. Customers will love it and it
will be great. See: resume and products in the wild. The Math-oriented
abstract tech interview is missing out on all the practitioners that care
about the product and can build reliable maintainable systems in a team
environment.

These days I'm primarily management, though, so I get to set the hiring
guidelines. Yay!

~~~
google234123
Could you imagine working with a senior software engineer that couldn't write
FizzBuzz? Anyway, I feel your approach may end up selecting for prestige and
be more venerable to implicit bias. A recent engineering graduate from Cornell
will probably have a much nicer looking resume than someone from some random
state university even if they both have strong technical skills.

~~~
jconley
No but I can imagine a senior software engineer that couldn't write FizzBuzz
during an interview. I would probably stumble on it myself. I prefer to look
at what people have done and not their credentials. I don't hire people that
haven't shipped something.

~~~
google234123
To clarify, by a senior engineer I don't mean a manager, I meant someone that
is going to contribute technically in a meaningful way to a project (by
writing code). I feel like it's a big red sign if the candidate doesn't know
how to write a for loop. People will lie about experiment when they are
desperate for work. Moreover, there are some people that are just great at
talking...

------
annoyingnoob
This is why I stopped writing code. After I bombed a bunch of interviews I
threw up my hands in disgust, it wasn't me it was the process.

One thing that stands out to me about the process is that everyone gets the
same interview, doesn't matter if you are fresh out of school or have a
lifetime of experience. Experience counts for nothing in these interviews. Can
you design twitter in 40 minutes, thinking through every possible failure mode
and issue along the way, or not? Apparently, I can't or not well enough. I
think these companies really want folks with great public speaking and
whiteboard skills - I'm kind of an introvert and that is a challenge for me
where the day-to-day work is not.

~~~
sosilkj
"This is why I stopped writing code."

would be interesting to hear about what career you transitioned to, if you're
willing to share.

~~~
annoyingnoob
I have a background in IT. I basically moved back to IT. I still write some
code but its not the majority of my work. The code I write now relates to
automation and integration and not so much building a product from the ground
up. I have to build for reliability but I don't need to build for Internet
scale.

Edit: I should say that the interviewing experience that put my over the edge
was actually not a whiteboard but a take home exercise. What I didn't know and
didn't appreciate was that it was designed with an adversarial review. There
was a long list of requirements and I thought I met them all. The response was
kind of rude and was a big turn off. After a second pass I was told that I was
missing some fundamental things, which were never requested or discussed. The
whole thing seemed designed to exclude rather than assess.

~~~
mav3rick
So you gave up on everything because of 1 experience ?

~~~
annoyingnoob
No that was the last in a long line of uncomfortable interviews. It wasn't
worth the effort to me to become a walking algorithm encyclopedia, most of
which I'm never going to use. Why would I continue with a process that appears
designed to exclude me?

Honestly, its their loss. I've done great work and continue to do so. For
better or worse, may name is on patents from more than one job and from
different industries. My work experience and history of accomplishments says
something about the kind of candidate I am. Hounding me at the whiteboard or
calling my take-home work crap seems unnecessary - if that is what the job is
like then I don't want it.

And I didn't 'give up everything'. I make the same money in IT without the
hamster wheel feel of always 'sprinting'. Marathon runners do not sprint the
entire race.

~~~
mav3rick
What is "IT" ? Do you mean sys admin ? Is that why you don't want to learn
algorithms ?

Here is the problem, many people have patents, many people have words. You
should be able to solve a simple BFS, DFS or DP problem. These algorithms ARE
used in good jobs. Seems like most people here write CRUD apps.

Also, lots of FAANG engineers have lots of patents as well. That doesn't mean
anything.

------
salamanderman
I certainly know my clinical anxiety has caused far too many interviews to
crash and burn spectacularly. It'll be like "wow, you're graduate educated, 16
years experience, you must be hot shit", and then I go up and just flounder.
For me it's not so much writing on a whiteboard, it's having someone standing
next to me, or sitting in a chair looking up at me, occasionally interrupting
me. I'll try to think out loud and all of that stuff, but that seems to invite
questions that sound judgmental to me. I'll start thinking "they don't believe
in me, they don't like me, they think I'm a fraud". Unfortunately, I've had
those fears confirmed some times too. I had an interview end with a guy
saying, as he handed me off to the next person, "You really are the lead of
the ... department?"

------
ericskiff
I experienced this personally as I bombed a whiteboard interview because I'd
been studying scala to work with them and programming a side project in C, and
literally blanked on simple ruby syntax and idioms when I started to panic.
When we started our own consultancy made 2 simple rules:

* Give people a take-home test that mimics the real work we do.

* Pay them to do it.

It's amazing how many candidates we find that are great at the actual work we
need to do, and happy to do it, and how many people it weeds out who wouldn't
be happy doing our normal work.

~~~
geogra4
Honestly this seems like the best way to do it

------
jwaterhouse
My 2c. This is purely anecdotal, and based only on hiring about a dozen
developers.

The best way to hire good developers is to take them out for coffee/lunch/beer
and talk shop for an hour or two. That's it. Make it more of a discussion than
an interview. Ask them about the their experience, what have they liked
working on, what have they hated, but also tell them about what you've been
working on and see what feedback/ideas/questions they have about it. I find
fielding their questions and comments just as valuable as asking them my own.
Joke about things that have gone spectacularly wrong, trade war stories, that
kind of thing.

I think it's a great way to assess whether a dev "knows their stuff" or not. A
couple of other notes: \- If you're not a dev yourself, get someone who is and
you trust to do it. Try and keep it "dev only" if possible, remember this is
supposed to assess their dev ability, but is also a good indicator of
communication and cultural fit within the dev team. \- Try to do this one-on-
one or at most 2-1, any more than that and it becomes a panel interview. \- If
someone is regurgitating theoretical or blog-post-scanned information without
much understanding, it becomes more apparent the longer you chat with them. So
an hour or two is a good length of time.

We usually also do a quick take home coding test too, but have found the
chat/discussion to be better indicator of long term success. As many have
already pointed out, I also find the coding tests (whether take home or in
person) don't really assess "real world" ability and can often be gamed.

------
axaxs
I can relate to this. There are two interviews that gave me really bad
anxiety, but I think it's because I went in with anxiety as both required
moving across the country in different directions. The first I was accepted,
but I think only because the hiring manager worked with me before. I felt
short of breath and like I wasn't listening to half the questions. The other
was Facebook, which was probably the hardest interview I'd ever done. One
particular question involved a lot of keeping things in your head and math
(network/node design), and I just froze and got frustrated and in the end gave
a crap answer with no thought. Given a half hour without someone staring at me
waiting on an answer, I probably could have came up with something decent, I
figure.

I'm not particularly upset at either case, I'm happily employed. But there was
a nagging feeling in the back of my head that I was disappointed that I didn't
get to convey 'the true me' to either company, whatever that means.

------
OnionBlender
I went through the interview gauntlet last year. When my extended family would
ask what it is like I would say:

Imagine you want to hire a comedy writer for a TV show. However, you conduct
your interview by having your candidate perform stand-up improv comedy like
"Who's Line is it Anyway". While you want to hire someone funny, you don't
need to hire a performer.

In my experience, programming interviews are basically an improv programming
performance. I didn't pass an interview until I had enough practice performing
while others watched.

~~~
decafninja
I agree with this. It really is putting on a show.

Ideally you get a problem that you are familiar with. Even more so if you know
the exact problem and optimal solution. If not exactly, then at least similar
to something you've solved before.

Then you put on a show acting as if this is the first time you're seeing the
problem. With practiced confidence, you roughly outline a naive solution, then
quickly and smoothly iterate your way to the ideal optimal solution. All the
while explaining your "thought processes" as if you serendipitously saw the
solution materialize before your eyes.

By practicing (say, grinding leetcode), you ensure that you have the
confidence to smoothly put on this show, and also not waste your precious
interview minutes meandering down suboptimal dead-end paths.

------
spideymans
>. For example, in our study, all of the women who took the public interview
failed, while all of the women who took the private interview passed

Talk about buying the lede. If there's any justification for abolishing this
practice, that's it right there.

~~~
borramakot
I agree that's a really striking and suggestive result, but keep in mind the
sub-sample size there is four passing on the private side and six failing on
the public side. My guess is there's a real and meaningful effect there, but
it may not be to the degree that sentence without context would suggest.

------
random3
Having performed hundreds of tech interviews, I realized that my developed
super-skill was to keep the candidates relaxed regardless of how well or bad
they did.

That meant carefully controlling my emotions when their answers were bad, and
having a smooth transition when the answer didn't come at all - a situation
that typically causes tension. You have to essentially hand out the answer or
transform the question into a trivial one so that the candidate can have a
small win and relax. If you're not doing that, chances are you're screwing up
the whole process by throwing the candidate off-rails after a simple non/bad
answer.

That's why the interviewer has more control over the outcome (besides the
scoring) as his / her bias will heavily reflect on the dynamic.

~~~
yalestar
That is indeed a super-skill! If you remove as much of the perceived power
imbalance between interviewer and interviewee (often difficult, to be sure),
and you can engage in a two-way conversation with them about the experiences
on their resume, then you'll get a far better picture of how they'll be as a
team member.

A candidate's ability to do performative whiteboarding while 1-5 interviewers
stare at them tells me very little. For a software engineering job, of course
technical ability is important. But equally (and often more) important is the
amount of friction they could potentially introduce to a well-functioning
team. What I want to know more than anything is: "Can I stand working with
this person?" "Is this person a dickhead code-reviewer?" "Will this person be
able to adapt and overcome when presented with a novel or urgent task?"

The way to divine these answers is by trying to put the candidate at ease as
much as possible. So many interviews, unfortunately, become a sort of gauntlet
or inquisition.

------
mathattack
This is why internships are so important. Sometimes it’s still hard to tell
after 10 weeks. It’s very naive to think we will know after a 30 minute
interview.

~~~
solidasparagus
I don't think people are naive, they just don't have a better solution. Most
established professionals are not interested in a 10 week trial period after
which they might be back on the job market.

~~~
_curious_
"Most established professionals are not interested in a 10 week trial period
after which they might be back on the job market."

Why not? If the position isn't a fit for either side after X time on-the-job,
that's OK right?

~~~
peruvian
Because maybe they support families with their job? Unless they had a ton of
savings, parents aren't going to switch jobs with a chance of being unemployed
again in 2.5 months. I wouldn't do it and I'm young and single.

~~~
_curious_
But isn't the chance implicitly always there?

------
astannard
One of the best hires I had made had a car crash interview. They could hardly
speak they were so nervous. I could see from their CV that they had some good
personal projects and thought lets give them a chance, that's what prohbation
periods are for. On the job they were amazing. It was great to see them grown
in confidence and they were a key figure at the company after a couple of
years.

------
non-entity
I wish I had an agreeable way out of this field. I'm not going to critize the
current interview process, it has its advantages and disadvantages and theres
a reason it's used, but if its shown me anything it's that I chose the wrong
career.

~~~
gouggoug
It's interesting to me that the interview process of our field makes you
question your choice of career. It seems like such a small thing compared to
everything else.

I absolutely despise the interview process, but there's no question to me that
I choose the right career.

~~~
non-entity
It's just part of it I suppose. My particular strengths and abilities aren't
what companies are really selecting for. This type of interview process just
kinda supports that. Of course things are more complicated than that, like I
said though I just mean it's a component.

------
BurningFrog
I want to push back on this a bit.

Writing working code well and fast is important.

But so is communicating with your coworkers!

Being able to talk about your code is not a weird thing you only do in
interviews. It's an essential part of your everyday work, and the programmers
who don't do this well are in a sense underperforming.

So just removing the talking part, as this study did, removes an important
part of the interviewing. When I interview people, I value their ability to
talk about the problem and how they think about it more than coming up with
great code quickly. I need people who can talk to _me_ , not just the
compiler.

That said, talking through your whiteboard programming task might not be the
ideal way to measure this skill either.

------
d3ntb3ev1l
Our industry has created this mess thanks to that stupid crackin “book” and
process designed by someone who never actually worked as a software engineer
or released and maintained an actual product.

When I “cracked” my Google interview and got the job, I joined a team I never
met. Of course we all could code , since we “cracked it”, but did I enjoy the
team dynamics or team culture? Nice people but if they were a 10 person
startup I wouldn’t choose them or would they have been successful.

None of the social dynamics which are important to teams was there or even
fostered, acknowledged.

You just did a transfer and kept fishing.

Why, b/c Google just believe coding skills are all that mattered.

Yet they matter, but so do other things.

------
qntty
One of the best interviews that I ever had was a phone interview where I just
so happened to have a good session of meditation beforehand. I never realized
how much how well I could do in an interview until I did one while feeling
relaxed.

~~~
HenryKissinger
Did you get the job?

~~~
qntty
I got an in-person interview, which didn't go as smoothly.

~~~
agustif
Did you meditate before coming into it?

~~~
qntty
Yes but my skill in meditation is no match for 4 1-hour coding interviews back
to back.

------
scott_paul
When I interview a candidate, I'm explicitly and intentionally looking at how
they respond to stress, how (if) they're able to say "I don't know", and how
they communicate complicated ideas. I open with soft-balls to put them at ease
and get them opened up and talking. Then dive deeper into technical things.
All the questions are technical, but the main answer I get is how they
communicate, how they relate to people, and how they deal with the unknown.

If you understand that a technical interview assesses anxiety, and you roll
that into the plan, if you know what the game is, then you're 1/2 way to a
win-win already.

~~~
vmception
yeah, but these anti-patterns are completely different everywhere. your
insight isn't a signal at all, when the next interviewer isn't saying what
they mean and evaluating only correct answers.

~~~
scott_paul
I'm not commenting on how anyone else does it, or saying that there aren't bad
patterns out there. I'm telling the room about a strategy I found that lets me
twist the bad into good.

~~~
vmception
I understood that, I want to highlight to others how there needs to be
standardization or a completely different approach, as this is all a crapshoot
and wasting countless work hours.

------
x87678r
Interviewing is a skill. I once had a good job where I was happy and didn't
move for 10 years. Big mistake - after that I bombed at interviews. After
doing 10 interviews I'm now up to speed. I kinda feel like I need to do some
interviews every year just to remind me how to behave and how to keep market
ready.

~~~
OnionBlender
I was in the same boat. Nearly 10 years experience at the same company that
hired me after university. My first phone interview had me write a linked list
and I froze. I had even practiced writing linked lists that week. The
interviewer probably thought I lied about my resume.

I spent 4 months doing leetcode problems and reading books before I started
doing interviews again. I would interview at places like Facebook that I had
no interest in working at but I needed the experience. 7 months after I
started I finally got an offer.

I don't want to go through that again but I agree that you probably need to do
interviews every year inorder to not completely lose that skill.

------
eagsalazar2
Traditional tech interviews (live coding challenges, algorithms, brain
teasers, etc) have so little value and cause so much suffering in candidates,
they are borderline sadistic IMO and in the end you are just as likely to make
a bad hire, then not be organizationally prepared to deal with that so you end
up with a broken team and bad outcomes. Why?

There are still challenges with contract to hire but overall I've had great
success with it. First of all, we get a realistic deep look at candidate
capabilities before we commit and they get a look at what it's like to work
with us. Then we're very prepared to part ways because that is the entire
point of the contract to hire. If feels 10x more successful in hiring great
people than any other approach I've personally seen and, very critically, it
is more humane and respectful.

There is a hypothetical concern that highly pursued candidates won't wait
around for that but we've found exactly the opposite is true. The best
developers are happy to see we have a more sound hiring process and are
psyched because we have a credible case that our teams are really high quality
and free from the random bad apple that spoils the work experience at most
other companies. Great devs aren't worried about being hired today or in two
weeks, they know they can get a job anytime they want. What makes great devs
great is usually their passion for great craft, process, culture - if we can
credibly promise that, they overwhelmingly are happy to do contract to hire.

------
blunte
I failed the third phase of the Toptal interview process (where you have to
solve two coding challenges, each within 20 minutes, while screen-sharing with
a human observer). The event was nerve wracking.

On the first challenge, I only completing coding and did a first run of my
solution with 90 seconds to spare. It didn't give the correct answer, and I
had only about one minute to debug and try again. In that case, I felt immense
anxiety and nearly gave up. But somehow I managed to keep it together long
enough to spot a misplaced array index [ ] reference and produce a correct
result.

On the second challenge, on my first run at about 17 minutes in I discovered
that my entire approach was not general enough to meet all the possible cases.
As I recall, while the rules of the problem are stated, you are not able to
see test cases until you run. With three minutes left and the understanding
that my approach was fundamentally flawed, I just gave up.

The experience was demoralizing. On the plus side, when I did live coding
challenges for job interviews after that, I found them all comparatively easy
and relaxed.

What would be better, in my opinion, is an overnight problem to solve,
followed by a detailed walk-through of the solution the next day. That
presented walk-through can prove to the interviewer that the candidate did not
simply copy a solution (without understanding it) from elsewhere on the
internet. Of course, if the solution was copied by understood, that should be
positively accepted in many cases. There's a reason we use trusted libraries
instead of writing our own code where possible.

~~~
dkarp
I had a very similar experience with Toptal and going down the wrong avenue
when solving the problem.

The interviewer muted themself and went non-responsive while I was working on
the problem. So I was asking questions and heard nothing back at all until the
time was coming to an end. At that point the interviewer came back to assess
what I had done.

I remember it being a really terrible experience

------
princetman
This can be said of all interviews, not just tech. It’s hard for great many
people to put themselves out there. I mean interview itself is a performance,
isn’t it?

Having said that communication and interpersonal skills are big part of most
jobs even in tech. We hardly develop software in isolation these days.

I don’t know about other industries, but in Tech there are some really
inexperienced interviewers out there, who’re impulsive and lacks discipline in
conducting objective and meaningful assessment.

------
tbirdny
I see lots of posts critical of companies and interviewers. The problem is
there are too many qualified candidates for some jobs and companies must weed
out candidates somehow. If they have 100 qualified candidates for 1 position,
how would you rather they pick? The real problem is something else. It's a
lack of jobs, inequality, the cost of living, and I don't know what else. Many
people are getting squeezed.

~~~
jlokier
It's not due to lack of jobs, at least not in a simple way.

Reportedly, there are _simultaneously_ more open positions than companies can
fill with qualified candidates, at the same time as many qualified candidates
struggling to find open positions they can fill.

I think that may be true, even though it sounds paradoxical.

The result is still inequality, cost of living peoples and people getting
squeezed.

But adding a bunch more of the same jobs won't fix it, if the problem isn't
caused by lack of jobs.

------
majormajor
I think the more accurate headline is likely to be "anxiety can negatively
affect performance in interviews."

I would love to see what a control group of non-tech interviews looks like.
Not sure what the equivalent "whiteboard while out of the room" is, but it's
hardly just software devs who get nervous and can panic in interview
situations.

------
throwawayiionqz
There is a lot of anecdotes in the comments about these interview processes.

The ones with the best data are the company themselves, especially those who
interview thousands of candidates a year. These companies (e.g., all FANG)
have enough data to assess which interviewer, which interview questions and
which interview process have positive correlation (and even causation) with
future employee performance. And they can optimize the pipeline to interview
most candidates with the interviewers that can actually assess future employee
performance for instance.

Either these companies have already done these optimizations and analyze the
trove of data they have, and the process is what it is for a reason. Or they
haven't yet and there is still much room to optimize the process, and
applicants have a chance to see those processes updated for the better
(although here better means better for the company, not necessarily better for
the applicant).

~~~
lapcatsoftware
"A new study from North Carolina State University and Microsoft finds that..."

Microsoft is studying this, because they realize that they don't in fact have
"the best data".

------
mementomori
Tech interviews are kind of like a lot of games in life: do something
difficult and boring in order to survive, and make it look easy and enjoyable
while you're at it.

------
aylmao
This is bad news for URMs. I am hispanic and was an international student—
looking back there's many "extra" things that added to my feeling anxious
before interviews. The imposter syndrome was real and I worried about wether
I'd fit in, for example. I luckily don't have much of a language barrier but
there were definitely a cultural barriers.

In the back of my mind I was always thinking I had to do more than "enough"
and be exceptional. You read studies about how two people with the same
qualifications might not get the same results after an interview due to
unconscious biases by the interviewer. I felt like I had to be good enough
that'd I'd overcome any kind of bias the interviewer might have. It's
stressful.

------
kevinskii
When I last did a serious job search a few years ago, I thought I was pretty
hot stuff. Nonetheless, I spent a few months reviewing algorithm text books
and practicing code interview problems.

Then my first phone interview came, and I totally bombed it. And the next one.
And the one after that. All in all, I think I bombed about seven code
interviews (didn't even make it to an onsite) before I started to get some
traction. One of the interviewers outright told me that I wasn't a very good
coder.

After that I managed to get a couple of decent offers, but the process was
quite humbling. If I interviewed again today I'd probably be in the same boat.
Whiteboard interviewing is definitely an acquired skill.

------
president
Which is why you get a lot of over-confident engineers who are good at
memorizing data structure/algo questions and talking BS but are terrible at
real day-to-day SE work. The modern tech interview passes over so many great
potential engineers.

------
4dahalibut
This is only true at FAANG, guys, stop working at FAANG its not doing the
world very much good and there are tons of companies around looking to hire
you that are. The future is starting now and FAANG engineers won't be a part
of it.

~~~
annoyingnoob
In my experience, every company wants to be a FAANG-like company and so
they've all picked up the same BS interviewing style.

------
some-guy
I have extreme performance anxiety when it comes to interviews. Recently I had
a very good experience interviewing with Netflix, where the take-home
assignment allowed me to show what I was capable of doing. All the technical
interviews after that were talking about the take-home at a high-level. The
experience felt more accurate to how actual work is (receive an issue ticket,
do work, then a code-review).

I was rejected in the end, and I believe I either a) wasn't a culture fit or
b) they found someone with more experience (I'm 6 years in at my first tech
job). What was nice about it though was that I was very calm and collected
throughout.

------
JoBrad
I feel like jkingsbery's comment[1] is relevant here:

> One of the interviewer asked me to recite the algorithm for constructing a
> Convex Hull. > ... > we struggled together for an hour, with him frustrated
> that I couldn't remember the details of an algorithm I last had seen 6 years
> earlier and couldn't recite in 60 minutes what took our professor 180
> minutes of lectures to cover, and me frustrated that would have taken me 30
> seconds to look up.

[1]
[https://news.ycombinator.com/item?id=23847814](https://news.ycombinator.com/item?id=23847814)

------
thehappypm
I just spent a huge amount of time interviewing for a new job. They put me
through an absolute ringer — something like 10 total screens and interviews,
over about a month.

At the end, I started feeling a really negative association with them. I would
never sleep well the night before the inevitable next interview and it just
got to become a really tough thing. And most of the interviews were totally
no-nonsense, no pleasantries, just tough.

I ended up getting the job but I turned it down. The interview process told me
everyone I needed to know about the company culture.

~~~
kasool
I had a similar experience. They wanted me to do 4 phone interviews before
even bringing me on-site, stretched over a month. Don't really understand the
rationale for dragging it out so long.

------
maerF0x0
> rather than whether the candidate is competent at coding

As a really quick reaction it's worth noting that coding is probably less than
25% of my job as an engineer.

In others' experience is this also the case for you?

------
deedubaya
I just went through an job hunt. I've been in tech for nearly two decades, and
tech hiring has gotten definitively worse. It was never good to begin with,
but it has become more painful from a candidate's perspective.

Employers are so focused on the well shared "the worst hire is a bad hire!"
that they implement obtuse and ineffective hoops candidates must jump through
in attempt to weed out "bad" candidates. This results in hiring those who
survive the process, not those that will do the best work.

------
lbriner
It seems like loads of comments consider "someone good at coding" as the only
benchmark of a good employee. Even the report implies that "people who took
the test in private passed". Passed what?

I am looking for people who can handle pressure, who can communicate clearly,
who can handle the unfamiliarness of the experience, who can handle time
pressure even though most of the time they wouldn't face that at work. If an
experienced engineer cannot split a string in less than a minute on a test
then they don't figure as a valuable emplyee to me. Why? Because anyone can
Google something or apply deep thought in a private room but good employees
imho can work with noise around them, with distractions, with changing
priorities etc. in other words the interview is a poor representation of a
real environment.

Sure, it means you might lose out on someone who just needed a few hours to
calm down but how are you supposed to know that? Maybe they needed another few
hours, days, weeks? Maybe the test you gave them was hard but if you asked
them about something else, they would have done better.

It's no different than someone turning up late. It very likely wasn't their
fault but you use it as an indicator anyway because you don't know them and
don't have much else to go on.

~~~
kelvin0
'but good employees imho can work with noise around them, with distractions,
with changing priorities etc'

Yes, then they burn out and the cycle continues ...

------
bcrescimanno
There are literally dozens of reasons that tech interviews suck. However, they
all mostly boil down to the same root cause: there's very little correlation
between the content of most interviews and the jobs people will perform. Some
real life situations that I've been on one side or the other of:

* Companies doing a "centralized" process that ignores the candidate. The questions are always overly generic and it really boils down to, "You need to want to work here enough to basically take any job we'll throw at you." Sorry, I don't want to work any company enough to just take any random job.

* Coders asked to complete academic problems that will NEVER come up for them in their real work.

* Coders not being allowed to leverage external resources like they will literally do EVERY DAY in their real jobs.

* Managers being asked to be coders in interviews.

* Questions that have binary "right" and "wrong" answers.

* Asking questions that have a million ways to do it and then honing in on one specific disagreement the interviewer and candidate had.

* Interviews that are more about the interviewer stroking their own ego and proving that they are smarter than the candidate.

* Completely ignoring that the absolute best indicator of future performance is past performance; not, "what quiz could this person solve in 45 minutes in an interview."

------
gridlockd
"Technical interviews are feared and hated in the industry, and it turns out
that these interview techniques may also be hurting the industry’s ability to
find and hire skilled software engineers."

It doesn't hurt "the industry", there's a competitive edge to hiring otherwise
superior engineers that are passed up by poor hiring practices at certain
companies.

Not everybody needs to work at FAANG, or worse, companies that act like
they're FAANG. Less money, less bullshit.

~~~
deedubaya
I just went through a job search and this is what I landed on.

Every CRUD webapp startup thinks they need FAANG-like hiring processes and
fucking suck at it.

I ended up taking a job that was mostly a no-interview process. Two calls to
talk about the fit.

They looked at my job history (gasp!) talked to some references (double gasp!)
verified my ability to code with my github contributes (omfg I can't breathe!)
and offered me a position.

Less money, but definitely less bullshit. It was a good choice for me.

~~~
tebura
Will love to know the company name !

------
TheOtherHobbes
The corollary may be that anyone with low social anxiety is likely to do
better.

[https://en.wikipedia.org/wiki/Dr._Fox_effect](https://en.wikipedia.org/wiki/Dr._Fox_effect)

I'm genuinely curious - do whiteboarding and code exercises mititgate this at
all?

Clearly there has be some minimal standard of ability. But given two similar
performers, do performance-interviews favour those who project confidence? Or
is there no influence on outcomes?

------
simonbarker87
I had my first coding test for an interview recently (second dev job but a
decade of other experience), over a video call, shared screen in a pre built
AWS in browser IDE and I assumed I tanked it because it didn’t go perfectly.
In hindsight it went like a typical 40 minutes of coding, I started down a
path, didn’t quite work, had to console log to see what was up, tried a new
tact and then it worked. They were clearly looking to make sure I had some
basics of Angular down but also that I could problem solve in the moment and
see where I went wrong.

I enjoy interviews (I like meeting people and chatting about their projects
and problems) and I think running a VC backed company for nearly a decade has
set me up well for detaching my performance in the interview to my self
worth/abilities (ie. I’ve grown a pretty thick skin).

Ultimately it comes down to “can I do the job?” and “is their process set up
to accurately reflect that?”. If the answer to the first is no then that’s
cool, I’d probably have a miserable time in the role and under perform even to
my actual level. If the answer to number two is no then I probably don’t want
to work for them anyway as they are unable to accurately measure and analyse
prospects/opportunities beyond just hiring.

------
cwkoss
I'm pretty proud of the interview loop we use at DefenseStorm:

\- First section is "pair programming". We try to replicate what coding on the
job is actually like: you can whatever IDE/REPL/language you want and can use
google to look up docs. Our problem is specifically designed to be a fiddly
problem which leads to edge-case handling with no possible 'beautiful
algorithm' answer. Once they get the first solution, they have to revise their
code to support a new fiddly use case. One interviewer actually provides help:
pointing out syntax errors and suggesting alternate approaches if they appear
to be heading towards a dead end. Most applicants "succeed" \- so we evaluate
on how well they can keep their code organized and readable, how they respond
to suggestions, and if they are easy to work with/good communicators.

\- Second section is code review. We give them several snippets of not-so-
great code and ask what they would suggest we change before putting it into
production. Does a great job of showing us where the candidate has depth of
knowledge, and is a good approximation of a portion of the duties we expect
them to perform. I was surprised to learn that very few companies do this - we
find it extremely valuable.

\- Third section is a conversation about tool familiarity (can you
linux/aws?), depth of knowledge in the various domains we use regularly and
designing an architecture (indicative of developer seniority). This is
probably the least remarkable section.

Every section is set up so there are no gotchas and there are always multiple
correct answers. I think we do better than most at minimizing the anxiety
inherent to the interview process.

~~~
learc83
That is very close to how I interview people. The worst thing about white
board coding is the adversarial relationship between you and the interviewer.
It’s so much less stressful when you can team up with the interviewer to work
on a problem together, and it’s much closer to being a real work sample test.

------
alexh
Coding interviews are not about coding skills, they are about communication
skills.

I thought I had discovered a clever hack when I was interviewing. You ask
enough closed questions about the problem that the interviewer tells you the
answer. Then you repeat the answer back to them, as code or aloud, and you
pass.

Now that I am interviewing, I am desperate to find anybody who will ask me
enough questions to have me reveal the answer, and then repeat it back to me.

~~~
hyeomans
Like the “guess who?” game?

------
kmclean
What are people doing as alternatives to these kinds of interviews? The
research is quite clear that technical interviews are worse than useless
(literally -- they will select for the wrong traits in your candidate pool),
yet they still seem to be the dominant hiring practice.

My current company does a few rounds of low-key chatting, some about technical
stuff, then a 1 to 2 month trial period with the understanding that there is
no guarantee of a job at the end, it only becomes permanent if it's a good fit
on both ends. Obviously this is time consuming and expensive, but it works
because we're a small and very stable company, which I realize is very much
_not_ the norm. This does work great, though. They've managed to build a team
of excellent, autonomous, highly skilled developers who stick around for the
long haul. Wow, that sounds a bit self-aggrandizing -- I'm really thinking of
my co-workers, I hardly count myself in that category. Maybe someday.

Anyway, I'm genuinely curious what other models you all use that work well. By
"work" I guess I mean result in high quality and long lasting additions to a
team.

~~~
musingsole
I like that model your company is using. Though there is a major drawback:
your sapping the momentum of your candidates in their job search. If they sign
on for the 2 month trial and it doesn't work out, contacts in their job search
will have moved on, and they'll effectively have to start over from the
beginning. Ultimately, a contract-to-hire strategy overwhelmingly favors the
employer's interest.

~~~
kmclean
Interesting, I hadn't thought about it that way. I always prefer a month or
two trial run before I commit to joining a company for the long term. I just
can't tell enough in a few hours of interviewing whether I'll enjoy working
with the people or not. And also you rarely get to meet all your future co-
workers in the interview. But then, this has always worked out well for me. I
could see it being worse and/or disruptive if it led to a situation of doing
one or two month contracts for a really long time.

------
staysaasy
Used to run interviews by putting candidates at a table with a laptop
(typically their own). I'd give them a coding question and 45 minutes to solve
it, and then spend 10 minutes walking through their solution together. I'd sit
across the table from them and do my own work. They could stop me and ask
questions all they wanted, just like we were teammates.

It worked great. It also gave me (the interviewer) some time back in the day.

------
rainyMammoth
The only thing that interviewing assess is how good at interviewing the
candidate is.

I swear that for 90% of the typical software interview loops, someone with
some minimal understanding of computer science that spent 2 month studying the
typical Leetcode problems would probably have been fine and do way better than
a software engineer coming out with 10 years of experience that didn't take
the time to review his leetcode problems.

------
sjburt
I think this is obvious, but look at the alternatives.

* Resumes can be exaggerated, falsified, or perhaps more often often leave out important details or relevant experience. Reviewers often also put unconscious emphasis on formatting or style (I had one colleague that I swear would accept anything in Computer Modern). * Take-home projects (if not carefully designed) can be a considerable time sink and reviewers may be biased toward particular coding styles or languages. * Prior employers are often not allowed to provide meaningful references, and even when able to, won't often have the time or motive to do so. * Referrals will be biased towards the friends of your current workforce, making it hard to improve the diversity of your workforce, and are a pool that will become exhausted as you scale or if your employees stick around. * Behavioral interview questions are biased towards candidates that know to prepare for an even more contrived corpus of questions with even less bearing on day-to-day job performance. * Candidates may not have prior code that their are able to share; certainly most employers make that difficult or impossible. Expecting side projects or open source contributions biases towards candidates that have means or desire to program outside of work.

I think it's ridiculous to think that a programmer should be hired without any
part of the process evaluating their programming ability. The key is to do
this while minimizing stress for the candidate. Explain the format of the
interviews up front. Tell them about the people they are interviewing with.
Let them have breaks. Help them through tricky parts if you have to. Avoid
things that aren't realistic (whiteboard coding) but don't shy away from
things that are (whiteboard a system design). If leetcode doesn't represent
what you do at work, don't make them do it.

~~~
ipnon
So the study seeks to investigate the role of an in person judge on the
whiteboard interview. The study found that a private whiteboard interview,
where the candidate gets the same problem but solves it without a judge in the
room, was better at evaluating software engineering skill than the same
question with a judge in the room. To your point, we can keep all the benefits
of algorithm questions with less of the disadvantages by modifying the typical
whiteboard interview to just remove the judge in the room.

------
thunkshift1
>”What’s more, the specific nature of the technical interview process means
that many job candidates try to spend weeks or months training specifically
for the technical interview, rather than for the actual job they’d be doing.”

Had to go through half the article before this showed up. You could be
learning something new and useful rather than spend so many cycles trying to
master trick problems.

------
bsk7
To me, the bigger question is, why do these interviews require every candidate
to prove their programming/design skills from scratch? Say I have worked at a
respected tech company for N years, my record there should provide a lot more
signal than a day’s worth of interviewing would. I think we need some sort of
standardized metrics for different dimensions of tech skills, and those
metrics update over years of experience. Then companies don’t need to assess
tech skills - they just need to look at your record, knowing that it is
objective. Certifications, open-source contributions are some examples. The
problem is the work you do closed-door in a company cannot be objectified or
standardized easily. If we can solve that, that would go a long way here.
Interviews can then focus on soft and behavioral skills solely. Not to say
that those interviews are any more useful or signal-deducing as they happen
today.

------
champagnepapi
Will anything change in the near term (~2 years), I would bet not. I sure hope
the way interviews are conducted changes. After all, most of us aren't trying
to spend many hours on sites like leetcode.com. I find those sort of problems
to be fun and enjoyable on my free time, but they can be VERY challenging
under pressure.

------
dgb23
Have you ever read Coders at Work?

One of the things that terrified me when I read it was Ken Thompson's
description of his interviewing process.

Paraphrasing: He would ask about a project the interviewee was proud of, then
drill them with critical questions about it.

I actually like the general idea. But the social pressure of this situation
seems to be quite intense.

------
Washuu
I have basically been doing a second job sending out applications and
interviewing hard core for about a year now. There is so much time I have
accumulated in take home projects. Anxiety is a big issue for me
interviews(Yes, I am a woman) and even in ones that go well I am still clearly
minorly affected by it. There was one back in November that I got to the stage
of being flown out to their office. At the end of the interview during small
chit chat I was nervously talking about the stickers on my personal laptop.(My
plane had been delayed to weather and I was basically interviewing with them a
couple hours after everyone would normally stop working.)

My count on applications is well past fifty and I have done so many phone
screens(HR and technical) that all of them seem like a blur to me. At this
point I wish I could throw a portfolio of my work at a company and get a job.
From a Reddit thread about "unrealistic car mods" shows my overall skill set
in my ability to get the job done.
[https://www.reddit.com/r/cars/comments/hdrgv1/car_people_of_...](https://www.reddit.com/r/cars/comments/hdrgv1/car_people_of_reddit_who_post_questions_about/fvn5iyt/)

My current job I got because I found a major vulnerability in the company's
authentication system and recovered the private key for the client to server
encryption.(I didn't make much money at the time and didn't want to pay the
subscription cost for the premium service.)

I have interviewed/hired nearly every person on my current team. One of them
was almost denied by a senior manager because the candidate wore a backwards
baseball hat during the interview.(Lack of professional attire, etcetera.) The
rest of the interviewers over ruled the manager saying that was a dumb reason.
Turns out this candidate was super conscious of a balding spot on his head and
he wore hats to hide it. He turned out to be an amazing coder and great at
solving problems that came up.

------
mnm1
For me, it's almost impossible to think in any kind of group situation. I do
ok sometimes on interviews. It really depends. If given time alone, even
during an interview, with my own tools and without distractions or having to
tell the idiot interviewer what I'm thinking (as if anyone can think and talk
about two different things, the problem and what the interviewer wants to hear
at the same time), I do amazingly well. With another person watching, in-
person or remote, I can't even think of how to start. I get caught in loops of
anxiety and I can't relax enough to think things through. I also am afraid of
trying wrong things so sometimes get paralyzed a bit. Mainly, I try to avoid
such interviews and go for ones that will let me solve the problems on my own,
with my own tools, without some idiot watching over me.

------
shadykiller
In one of my recent interviews at a mid size silicon valley company, I was
given 4 hours to make a tetris like game.

One of the most enjoyable interviews ever with no whiteboard and no one around
me to check on my progress. I was blank for first half an hour or so but made
great progress by the end. Eventually got the offer.

------
jonthepirate
20 years startup/internet engineering here. I have "failed" roughly 50% of my
interviews usually as the result of failing the coding rounds. Almost all of
fails were conducted by someone with nearly half my years of experience who
were interviewing for something from the CS academic curriculum. When I am
asked to conduct an interview, I don't do anything much more complicated than
a glorified fizzbuzz. I prefer doing architecture interviews.

My biggest problem with algo interviews is most of them are trying to
determine if you happen to be familiar with one particular algorithmic
solution to something - if you don't happen to know that one algorithm, you
fail. In the real world, you have to learn as you go. You never know the
solution to every problem in advance.

------
diehunde
The problem is we don't like the process, but not many people propose
reasonable solutions. One extreme is just hard LeetCode questions, and the
other is, "they should just talk to me about my experience and projects." We
need something in the middle. Some ideas I've been thinking of:

\- Pair programming with a problem that neither the interviewer nor the
candidate has seen. Doesn't matter if they can't find the optimal solution. It
would show how good of a communicator and how proactive the candidate is.

\- Make the candidate implement a solution to a problem that's already solved.
The interviewer shows the problem, explains the solution, and the candidate
needs to code it. It would reveal the coding fluidity, understanding of specs,
how they ask questions, etc.

Thoughts?

------
rootusrootus
This is why I really go out of my way to try and dissipate the anxiety as much
possible. And because I feel anxiety myself when I interview people, so I
remember that we may share that perspective and we'd both have a better
conversation if we could relax a bit.

My interviews therefore end up being conversational, and definitely non-
confrontational. I want to hire someone who the team will get along with, who
knows what they are doing, but in the end I have to remember that if we hire
them I'd prefer we didn't start off our relationship with an over-the-top
technical interview.

So far I've been successful at finding good candidates this way. Maybe it's
just luck. But I don't think leetcode interviewing is the only way to get
well-qualified employees.

------
collyw
As the interview process seems so crap and few people have any suggestions,
here is mine.

I got asked to bring in my laptop with some code of my own we could discuss. I
had a side project at the time so wasn't a problem. I got asked some
questions, asked to add a simple feature. It was pleasant. I knew the code, so
I was relaxed and didn't need to look up stuff. I think the interviewer
learned some stuff as well.

The only problem people might say is that you used someone else's code. Even
if that was the case, if someone else can talk confidently about code that
they didn't write and jump in and add a feature easily, you have probably
found someone good.

Can we try and push this as the new way of doing interviews. Low stress and
respects peoples own time.

~~~
mav3rick
Now you want someone else to go through your code ? Without any idea about the
mess you have written

------
nemonemo
There's one genetic factor known for the performance variance against the
stress, which is also called "warrior vs worrier":
[https://www.huffpost.com/entry/stress-
management_b_2671591](https://www.huffpost.com/entry/stress-
management_b_2671591)

IMO, there's Moneyball-like opportunity for companies who can utilize (or, in
another word, exploit) this aspect. A tech company may conduct tech interviews
with a private environment & a traditional environment, and if one's
performance is much better in the private setting, they might be able to offer
a job with a reduce salary than the candidate's capacity shown in the private
setting.

------
adamredwoods
I get massive anxiety interviewing for tech companies, and I've been doing it
for many years. The worst interviews are the "trivia quiz" questions that are
either correct (according to them) or incorrect. There's no discussion of how
things work and no frame of reference or anything. For example, when I
interviewed at Amazon coming from Microsoft, they asked me lots of Javascript
'trick' questions even though I told them I was using C# day-to-day. I knew
most of the questions, but got caught up on a couple of JS 'tricks' that I
just couldn't recall at the time.

Sadly, it's all a game, but the good thing is that interviewing is a skill and
can be improved with continued practice.

------
wellpast
First thing I'll say is that I'm not an advocate for how industry often
interviews today, whiteboard and puzzle problems and all, and their efficacy
is worthy of discussion. It's a hard, hard problem and there are major
deficiencies. But this academic "study" seems highly suspect.

48 subjects and we can make a generalization? People's stress reactions seem
intimately tied to _enormous_ number of contextual factors. Looking at the
abstract it seems the "goal" of this study is "to make problem-solving
assessment more equitable and inclusive". A noble and worthy goal but this
does not sound like academics or science. It sounds like its conclusion was
sought for.

------
sideband
The current "tech" job interview process is engineered to select for people
that are both competent and secretly insecure about their skills, and to do
this with little or no false positives. Competent employees are, well,
competent and secretly insecure employees tend to have a need to impress. Both
of these traits works to employers benefit.

A large amount of false negatives, like candidates that would otherwise be
great employees but maybe just suck at whiteboard coding, is considered
acceptable by most employers because research shows the cost of bad hires is
much much higher than missing out on many good hires. Employers know the
current process is miserable and leaves a lot of good people behind.

------
Communitivity
I can attest to this. A number of years ago, after fifteen years as an
engineer getting jobs through my reputation, I went into my first white board
coding interview, with one of the FAANGs. It was a Boggle like problem to find
possible words. While at the board my back was killing me, and I had a
horrible time thinking due to anxiety. I knew that I wanted a trie, but I
blanked on how to implement it properly. A couple hours later at home I
remembered all of it.

That said, it did filter out people who had trouble handling the new stress of
a whiteboarding interview and people who did not practice whiteboarding. If I
ever interview for a FAANG again I'll make sure to do a lot more preparation.

------
talmr
I think for senior level positions especially it is more important to see how
a candidate approaches a problem beyond just memorizing algorithms or LC
solutions. I wish there was way more emphasis on "case study" style approach
where you could discuss at high level how to build X. Even pulling the
candidate into a real technical discussion where you're trying to optimize
something and see if the candidate has any input. Building up on experience is
what we should be after.

To me, asking LC or the same algo and data structures questions (implement DFS
.. wow genius) is just gatekeeping via a measure that most people can pass by
just memorizing stuff for a month.

------
leafboi
If the results of this experiment are true part of winning the game may be
taking some anti-anxiety drugs before the interview... I heard high blood
pressure medicine works.

Anybody have any first hand accounts on using drugs to calm anxiety before the
interview?

------
sizzle
I think there are a lot of hidden biases in hiring that no one will overtly
acknowledge e.g. personality, gender, attribute of race (yeah I know, but
still there are people unaware of their racial biases), ability to take
orders, if they are too good and might threaten their job or make them work
harder because they will output more completed work, etc.

usually this line of thinking is summed up as the follow question: Can I see
myself working with this person for 40+ hours a week?

then they bring down the axe with the following: not a good 'culture' fit and
move on to the next candidate and the broken cycle continues.

This is all in addition to the broken whiteboarding challenges.

------
ralmidani
I'm susceptible to stuttering and/or rambling when I'm extremely anxious,
sometimes without even being fully aware of it. I made it to final interviews
for tech jobs twice, and I got the offer both times.

In retrospect, I was extremely anxious the first time, but the company was
about to enter a high-growth phase and didn't seem to have fully-developed
hiring practices (I got a tentative, very informal verbal offer on-the-spot,
and a phone call to confirm I had the job about 24 hours later).

The second time, I was interviewing with a big US bank so I assume they had a
better eye for details. Maybe I'm getting better at not showing my anxiety.

------
stakkur
The root problem isn't the interviews themselves, but the belief that--despite
a stunning lack of evidence--interviews are somehow useful measures of future
job performance.

And I believe this applies to _most_ job interviews, not just in tech.

------
habosa
At Google I'm convinced our interviewing system only works due to a
combination of luck, large numbers, and generally good intentions. None of the
individual steps is a particularly good way to tell if someone will succeed at
Google, but anyone who passes all of these filters is unlikely to be a
terrible engineer.

When you avoid terrible engineers and hire by the tens of thousands, you get
plenty of good engineers and a few great ones. You also miss a ton of
qualified candidates, but Google has more of those than it needs. It can
afford to miss them in a way that small companies can't.

------
mpweiher
Hmm...but if everybody is stressed out, then it equalizes and you're testing
for skills again, just skills under pressure.

And having skills retained under pressure is actually a valuable trait in this
industry. And at least at desirable companies, the process is designed to
minimize false positive, it doesn't care that much about false negatives. This
is by design.

My guess would be that they filter out orders of magnitude more people with
poor resume-writing skills or that didn't go to the right school.

Although I do agree that interviews that you can/must practice for are silly
and there's a lot oof that around.

~~~
redisman
In my normal work I don’t have someone staring at me and probably thinking I’m
an idiot for blanking so it’s not at all a simulation of work stress

------
MattGaiser
I have found that what best predicts interview success for me is the ability
to confidently give answers, even if they are nonsense. One of the problems is
that interviewers often aren't sure of the answer either.

------
kwyjibo1230
I work in an interesting tech career that does need to assess this: Pre-sales
Engineering.

At my current job, we have our candidates do a short take-home assignment to
assess coding skill, but we also have them do a live presentation to assess
their pre-sales skills and ability to present under pressure.

We try to separate the two tests to get a better read on the two skill sets
we're looking for. So far we've hired strong candidates. We prioritize
avoiding false positives over false negatives; in other words, we prioritize
turning candidates away over hiring someone we're unsure about.

------
afpx
I leave tech interviews feeling like I had just been through a hazing ritual.

------
ian-g
I don't think it helps that what works for some people doesn't work for
others. I coded very very little before I graduated college but got my first
tech-ish job because I knew a guy and I could explain my thought process well
despite having no idea what grep was.

A friend of mine would have fallen on their face during that sort of
interview, but they're more book smart than me and write quite well.

If we were the two candidates there, I would have got the job, but they would
have done it better. I don't know what the solution is, but it isn't more of
the same

------
harryf
Engineers tend to over-focus on correctness. I’ve run hiring process in the
past where over a year we interviewed around 50 engineers. Feedback from our
own engineers involved in the process would often be like “Seems like a nice
guy. But his solution to the challenge - one he’d never seen before - had a
bug in it! And the solution was just O(2^n) and failed to notice the O(n^2)
solution. Even though I glared at him a lot and kept distracting him with
hints” ... I’m exaggerating for effect of course but some form of that would
happen often

------
m3kw9
The best interviewers can feel you from certain job experience questions that
you’ve done in the past. It’s looking for a series of related answers that a
candidate just cannot make up on the spot.

------
LockAndLol
So what are we going to do about it? Nothing. Absolutely nothing, that's what.

This happens all the time. People complain and complain, something comes out
proving them right (or at least confirming their opinion) and all that happens
is people relate. Politics, jobs, healthcare, interviewing practices, work
conditions...

If programmers actually joined a union and organized themselves to make a db
of companies with shitty interviewing techniques, bad business practices, etc.
maybe something would change, but hey, complaining is easier, right?

------
stx
At one company where I was the interviewer the company used a few simple java
questions for each screening (not great I know). It was basically just a way
to make sure they did have experience with Java at at least a junior level.
This became an issue when we started getting candidates from recruiters. One
candidate bombed on the first question. Then when we asked the second question
they answered the 3rd. Clearly the recruiter was trying to coach the
candidates based on past questions.

------
hackermailman
The strangest thing about jumping through big tech corp take home projects and
multiple interviews is they then place you in their internal training boot
camp and you realize why didn't you just stick me in here in the first place
then rank my results when we finished training instead of wasting money for a
week shopping me around departments so highly paid senior employees could all
stop what they were doing just to have a go at me with the whiteboard and
multiple on site lunches.

------
almost_usual
This makes sense if you’re at a FAANG organization you shouldn’t need to make
breakneck code changes with people breathing down your neck.

At a lean startup or smaller team though this isn’t unheard of. I’ve had to
debug and code up multiple things while everything was on fire around me.

Mistakes are guaranteed to happen no matter how many precautions are in place.
You can reduce the probability of a mistake happening but when it does whoever
is fixing it needs to understand how to stay calm and work well under
pressure.

------
TinkersW
Most interviews I've done in my software career were done on more than 24
hours without sleep.

Not by choice, but because I cannot sleep the night before an interview. I
have difficulty sleeping in general, and the anxiety of an interview drives it
through the roof.

Whiteboarding is also terrible, pretty sure it is used intentionally to skew
hiring results toward certain types.

If someone is looking over my shoulder while coding I can't think, I think
best when there are no external inputs(no noise/eyes closed).

------
cmonnow
When you're under time-pressure, you actually spend more time thinking about
how much time you have left, rather than thinking about how to solve the
problem.

It's like that parable.

2 students approached the master, wanting to become swordsmen.

'I want to reach your level in 10 days', said a student. Master replied 'It
will take you 10 months'. The other student said 'I don't care if it takes 10
years, Master'. The Master said 'and you will learn in 10 days'.

------
mleonhard
Idea:

Assign a short coding challenge and rank candidates on their solutions.

Then, one at a time, hire each candidate as a contractor for 20 hours of work.
Let them fix bugs and do code reviews. See how effectively they communicate
through code and in code reviews. Invite them participate in the weekly
standup meeting.

Finally, have team members who worked with the person write confidential
feedback with a hire/no-hire rating and justification. Hire the first
candidate who gets all 'hire' ratings.

------
kube-system
I find it interesting that academia often points out that skills interviews
have a soft-skills bias, but rarely do I see them consider that this may be a
feature and not a bug.

> “Our study suggests that a lot of well-qualified job candidates are being
> eliminated because they’re not used to working on a whiteboard in front of
> an audience.”

This is a common daily task at any company I've ever written software at.
People who can communicate their ideas effectively are better at their job.

~~~
derivagral
I've had the opposite experience. In >10 years of software, I've never written
code on a whiteboard outside of an interview. Systems layout, designs, code
pathing, sure; but never real, compilable code.

I agree with the last point, but I don't think either of us can tie our
anecdotes to it.

~~~
kube-system
There is something to be said for the soft-skills evaluation being relative to
the expectations of the job position, sure.

I'm saying: I find it odd that I rarely see these types of studies discussing
the merits of that nuance. Especially when the idea of differentiating between
soft and hard skills is not something that would be foreign to the average
interviewer.

"Interviews are 100% for soft skills and technical evaluations are 100% for
hard skills" seems to be a common assumption which I'd argue is false. Few
tasks a developer does is purely a hard or a soft skill, there's always some
overlap.

------
austincheney
This problem is perfectly clear to people who have had senior level careers in
any other professional industry. Software does not know how to assess people
because it lacks ethics and standards.

It’s a problem that most people seem to not want fixed until they are
themselves negatively impacted. Since the very concept of ethics is so utterly
foreign its need and application cannot be effectively communicated without
resulting in some manner of embarrassment or hostility.

------
mangatmodi
As suffering from anxiety disorder, It totally hits home. I have been a top
notch employee at every place I worked, but I totally bombed most of my
interviews.

------
klodolph
> In short, the findings suggest that companies are missing out on really good
> programmers because those programmers aren’t good at writing on a whiteboard
> and explaining their work out loud while coding.

In most of my programming jobs, explaining your work was _most of the job_ and
writing code was less important.

The whiteboard environment is a bit unnatural but I don’t want to hire someone
who gets the right answer if they can’t explain it to me.

~~~
RivieraKid
I would definitely not want to work for you, I think there's something deeply
ineffective when most of the job is explaining your code.

I say that as someone who cares a lot about writing easy to read code and has
no problem explaining it. Also, I'm very bad at solving a problem while being
required to think out loud.

~~~
danans
The skill of explaining your work is a very important one in practically any
creative problem solving field. This only increases as the technical
complexity of the problems faced increases.

Sometimes this explaining happens in textual format, like code reviews and
design specifications, but often it happens verbally. Absent an explanation,
it's hard to build consensus among the parties involved (co-workers, users,
management) that your solution (as manifested by the resulting code) is the
right one, and for anything non-trivial, you surely can't expect others to
just assume that it is the right one.

~~~
gmtx725
explanation is important but its unreasonable to expect someone to do it on
the spot in a time-pressured environment. Often you have to go away and think
about it alone in order to come up with a good way of explaining things to
others.

------
bakedbeanz
I sincerely appreciated that the technical portion of the interview for my
current employer consisted of pair programming with their developers to
improve on the pre-screening "take home" assignment I completed. It actually
gave me a chance to explain my thought process and work through problems in a
similar way to how it happens on the job. It was infinite preferable to white
board algorithm questions.

------
johnwheeler
Nothing will change.

I think the problem often lies with the interviewers’s ego that results in
this strange paradox where they’re trying to demonstrate their competence to
the interviewee by way of technically “outwitting” them.

That way, in the event the interviewee is personable enough to get hired, the
working dynamic is that much closer to being formed between the two, and
chances are reduced that the interviewer is hiring a potential rival.

------
musingsole
I shared this article with my company. It was amazing how fast our leadership
(particularly those involved in interviewing) came back with critiques about
sample size or if they properly accounted for student performance in the
study. Perhaps fair critiques, but it's not like this effect is unknown by
software engineers (unless your head is really stuck in the head). Ego defense
at its finest.

------
mastayoda
How many years took them to realize this?... As someone who spend like 6
months job hunting and doing more than 20 interviews(including Google,
Microsoft, etc), this was pretty evident. Yeah, I forgot to code that weird
problem which implies the use of dynamic programming in a white board, but
hey... what about all those years of experience and evident code skills. It is
ridiculous.

------
shrimpx
A lot of it relates to how much the interviewer can create rapport and
comfort. The interviewer having the upper hand in the power balance, they
should go the extra mile to set the stage correctly. Many good engineers are
terrible at interviewing. In the wake of the interview they boldly set forth
their assessment while being largely ignorant of their own negative impact on
what just happened.

------
laurieg
Occasionally I work as a speaking examiner, assessing people's foreign
language level.

The test is rigorous and well designed. Few minutes of questions are designed
to be so easy any can answer them. "How do you spell your name?" etc. This is
by design as people can be extremely nervous when taking a foreign language
exam.

If you are interviewing people, do what you can to put them at ease at first.

------
m0zg
Very true. I suffer from this myself, and I have interviewed for Google twice.
When I interviewed the second time I thought I bombed, but I was hired. With
the benefit of hindsight, I can rationally tell myself that me getting into
some pretty high flying places is not a coincidence, but I'm 1/3rd of a
programmer in an interview compared to when I'm not under stress.

------
ponker
Totally true that people with no self-doubt/impostor syndrome overperform
their ability level in whiteboard interviews compared to people with more
self-doubt, who are often the better engineers. I am personally a beneficiary
of this. It also contributes to all of the terrible dick-swinging culture in a
lot of tech companies since these are the people who pass the interview.

------
ironman1478
Its really funny. I was interviewing at a large company recently and the way
they do interviews is weird. Each team that is interested in you does their
own on-site. The first one I bombed b/c I was so anxious but after I
experienced it once, I aced all the other teams I interviewed with. Nerves
definitely play a role, its cool to see that it is being quantified.

~~~
nosefrog
Sounds like Apple. Same thing happened to me :)

------
jkubrynski
I always care about explaining to the candidate how the interview will look
like, and what are the most important elements I care about. When you skip
this part, the candidate has no idea if for example during live coding, you
care about the results, the code quality, IDE shortcuts fluency, etc. The
feedback after the interviews proves it is a game-changer.

------
dominotw
I used to be super anxious and get brain freeze.

This doesnt' happen to me after I grinded leetcode for a couple of months.

you are anxious if you feel unprepared.

------
mox111
I do think this is true - but I also think that the ability to control anxiety
(an increased tolerance for uncertainty) is a characteristic that is learned
with experience and can be a marker of an experienced engineer. Someone who
panics in an interview might also be inclined to panic when a manager
frantically reports a bug in master, for example.

------
pjdemers
I've posted this before on HN: Tech interviews are designed to find people who
are happy in a role where their level of responsibility begins and ends at:
"Here's your coding assignment. Go do it". The interviews are designed to
eliminate people looking to get in the door, then quickly move into or up to a
non-coding position.

------
aloukissas
This should hit home with everyone. What I would add is one thing that adds to
the anxiety is whether the interviewer is looking for reasons to _reject_ a
candidate (i.e. don't let the bad ones through) vs reasons to _hire_ a
candidate (i.e. let them shine on aspects that are hard to communicate on a
resume or when checking references.

------
DabbyDabberson
Anxiety management is absolutely foremost in these interviews. I did not
successfully get through one of these interviews until I started experimenting
with taking anxiolytics beforehand.

I eventually found that a moderate does of phenibut 5 hours prior was an ideal
balance of social inhibition vs the types of impairments that come with these
drugs.

------
emerged
I remember interviewing at Google and being completely unable to think or
relax. Instead of making any effort to learn about me and my skills, the
interviewers became more aggressive and nitpicky.

Looking back, my life is dramatically better working remotely for the same
level of income with friendly helpful coworkers in a non toxic environment.

------
mountainboot
As someone with bad anxiety and who gets panic attacks in roughly one third of
panels, it does feel fairly discriminatory.

The best interview I had was when they asked me a whiteboarding question, gave
me a pencil and paper, and left the room. I solved it, they came back, and we
talked about my solution. They still got to hear my thought process.

------
dev_on_earth
Last year, I went to both Amazon, FB on-site interview at Seattle and
Vancouver office, and completely bombed it (not quite, I solved 50% of the
programming questions). I felt like a fraud when try to answer the LP
questions. Anyway, is there any companies other than FAANG that can support
relocation (to US or Canada)

------
DeonPenny
But did they see if being neurotic is a net positive or negative for doing the
job. Skill isn't the only variable in being good at a job.

If you skilled but you neuroticism makes you less productive even compared to
someone less skilled who cares about skill level.

You could end up with a skilled team that folder under low amounts of stress.

------
cleandreams
I have totally blown interviews and also done very well. To me, the interview
is a distinct skill set. You have to treat it as it's own thing. What matters
is practice, practice. Not a little bit. Say 6 weeks. Then you are ready to
slay. When I haven't done that I've been rattled and done poorly.

------
csense
Suppose you're BigTechCo, and your interview process passes up this huge pool
of talented developers who don't interview well. Then your competitor
ScrappyStartup, who gives some of these folks a chance, ought to be eating
your lunch, right?

Something to think about, especially if you're involved in a startup...

------
shadowgovt
To a first approximation, that conflation is fine; software engineering is
stressful work where the tasks to be solved often involve known and unknown
unknowns, and it's important to observe how a candidate performs under
pressure because the job will include pressure.

------
alkibiades
Looking at the article, it seems all they looked at was whether the code
worked or not.

interviews also assess communication and thought process. and working code
isn’t necessarily good code. also wonder if the people in the private room
used their phones to research answers

------
m3kw9
To be fair, even these interviews are mostly garbage, they don’t have a better
way to gauge you with just 2-3 meetings. Which really sucks. What they should
do is to check your pass job experience, verify from the people you work with.
That’s not easy.

------
mementomori
I feel like sometimes big tech companies give technical interviews when
they're not actually hiring, that it's a sort of formality. Can't really
confirm it, but thinking this way makes me feel better about not passing to
the next stage.

~~~
angf
Correct. In many of these companies, part of your evaluation is how many
interviews did you give. This means: the game is to do as many as these
interviews, typically once a week, such that you are not penalized for doing
too few interviews.

I was once penalized for giving too few interviews but there weren't enough
candidates at that time! While I cared about giving every candidate a good
chance for success, I can say confidently that not many did care. They only
cared about getting credit for doing the interview.

------
blondie9x
I didn't really want to speak up on this but yeah this has been a major
problem for me. I have a disability and diagnoses with anxiety disorder. These
intensify the symptoms even more and sometimes I blank out.

------
varispeed
I failed every single interview when I had to do programming task on the spot,
however people who hired me based on my resume and how I talk about solving
problems, had great business with me for many years.

------
tropdrop
This was interesting:

 _For example, in our study, all of the women who took the public interview
failed, while all of the women who took the private interview passed._

That is an amazing statistic! From 100% failure to 100% success!

~~~
ipnon
Yes, I don't think a company can continue with whiteboard interviews anymore
and consider themselves to be treating women fairly in the hiring process.

------
irrational
I haven’t interviewed for a job in nearly 19 years. I think job interviewing
has changed a lot in the intervening decades and I’m a bit worried about how
I’d do if I ever need to interview again.

------
ken47
Technical interviews do not resemble any kind of professional software
engineering environment that exists in the real world. That is their biggest
problem. Everything else is secondary.

------
papito
Get a whiteboard at home - even a small one. Try to write an algo on it. You
will be amazed at how stressful it is, WITHOUT anyone looking over your
shoulder.

But with practice, that stress goes away.

------
soamv
Direct link to the paper:
[http://chrisparnin.me/pdf/stress_FSE_20.pdf](http://chrisparnin.me/pdf/stress_FSE_20.pdf)

------
smsali97
Agreed! Cut throat interviews in which you are expected to perform under heavy
scrutiny can heavily affect performance. I prefer the case study approach
instead!

------
zelly
I fear there's a movement to get rid of leetcode interviews and I'm against
it. What's the alternative? We go back to "you have to look the part",
nepotism, credentialism, and other such BS in other industries. A data
structures and algorithms test is fair and objective. You can objectively get
all the challenges correct and then there's little reason to not hire you.
Compare that to interviewing for, say, an investment banking position where
you're judged on stuff like what frat you were a member of and how charismatic
you are.

~~~
AnimalMuppet
There are more alternatives than "look the part" and "leetcode". If I'm hiring
a software engineer, I'm not hiring for how they look - the compiler doesn't
care. And I'm (probably) not hiring for the ability to solve leetcode
problems, either - of the work I would assign to someone I'm hiring, almost
zero of it looks like leetcode problems.

What I need is a way that tells whether the person I'm interviewing can
actually do the work that I'm hiring them to do. Maybe I should ask them about
that. Maybe I should give them some problems similar to the actual work.

~~~
balfirevic
> What I need is a way that tells whether the person I'm interviewing can
> actually do the work that I'm hiring them to do. Maybe I should ask them
> about that. Maybe I should give them some problems similar to the actual
> work.

Would you mind sharing with us how to draw the rest of the owl?

Look, I don't even want to defend leetcode here, but it _is_ a struggle to
find questions that are:

1) Correlated with the actual work performance

2) Neat enough to be explained and solved in the course of an interview

3) Have objective enough answers so that it's possible to overcome likability
bias in interviewers

4) Have objective enough answers to be able to meaningfully compare different
candidates

5) Demonstrate that candidates can actually go from hearing a description of
the problem to producing code that solves the problem

~~~
AnimalMuppet
The best I've ever seen is the way my current boss does it. We do a two-hour
interview (they've already passed a phone interview before we bring them in).
We spend the first half hour just talking to them personally.

Then we give them a piece of code on paper. It's fairly simple - half a page.
The function name is foo(). It's written in C#, but we don't expect them to
actually know any C# details. The function takes an array of Strings as input,
and returns an int. The questions are, what values does it expect for the
Strings, what value does it return, what would they name the function, and
what can go wrong (what exceptions can it throw - but we don't expect them to
know the details of C# exceptions, so what might cause exception-like
circumstances). This shouldn't take more than half an hour.

Also, it's the same code for every candidate, and we've seen a few dozen over
the years, so we are fairly calibrated on how a quality candidate should do.

Then we ask them to write some code on the whiteboard. We give them a simple
problem (again, same problem for every candidate). They can use any language
they want. We don't expect them to be a compiler - we don't care if they get
the parameters a bit wrong on a function call, as long as the code would be
sound if the system accepted the syntax they expect. Again, this should take
half an hour.

Then we give them a simple design problem. How would you design software to do
this? It's a problem that is not rigidly specified, so they can have to flesh
out the requirements (hey, welcome to the real world). It's got enough corner
cases that we can ask them about one that they skipped, and watch them try to
modify the design to handle it. We don't expect perfection - there is no
perfect answer for this question. But we want to see them have some kind of
reasonable taste or sanity in their design, and some ability to iterate the
design as more requirements emerge. Once again, this takes half an hour.
They're not done with the problem at the end of half an hour, but we've seen
enough to know how they did.

Other than allowing them to ask any questions they want, that's it.

~~~
balfirevic
Thanks for the answer.

> Then we ask them to write some code on the whiteboard. We give them a simple
> problem

Can you share what kind of problem?

This actually sounds pretty close to what we did at my previous company,
except that candidates were given a computer (nothing had to compile, it's
just that we don't expect anyone to be able to write code on whiteboard) and
two coding problems.

First one was pretty simple, such as printing a folder structure with nice
indentation (given some predefined input data structure). The second one was
harder, although we hired plenty who did not manage to completely solve that
one, but if they did it was a plus.

Also had some design questions after that. Some people still complained that
the interview was abstract algorithmic nonsense without relevance to real
life.

~~~
AnimalMuppet
It has some string splitting and manipulation, but nothing that couldn't be
done in C. (It would be a lot more tedious, though...)

I don't want to give the exact problem, just in case we interview an HN
reader.

------
baron816
I think we need to come up with effective alternative to whiteboard
interviews. Who’d be interested in forming a working group to come up with
something?

------
DenisM
Ok, so what can I as an interviewer do to reduce stress for the candidate?

Let's make good use of this discussion and compile a list of practical
suggestions.

~~~
DenisM
I'll start:

\- Offering candidates their choice of whiteboard vs take-home. I've found
that different people have different preferences, and they liven up when they
are told they have a choice in the matter.

------
ashconnor
Can I pose a question to people here.

Would you have moved companies earlier or moved more often if a coding
interview was not standard practice in hiring?

------
username90
Managing your own emotions is a part of EQ. Why should we hire people with low
EQ to a job as collaborative as software engineering?

~~~
cheschire
Control is not a part of EQ. Awareness is. Being able to point to the source
of your emotions is key in being able to identify the source of others' as
well.

Fear responses are deeper than conscious thought. You can't expect a person to
respond calmly to a life changing event like an interview anymore than you can
expect them to stay completely calm in a locked coffin.

------
sa123
Unfortunately corps understand this and assessing anxiety is the purpose.
Experience is more indicative of technical capability.

------
mraza007
Just curious does anyone knows or has created a list of companies that gives
you take home tests I would love to find out

------
mraza007
Fizz buzz/leetcode != Real software engineering

I think we should come up with new software engineer interviews techniques

------
sfpoet
I always had the feeling this was the case, so my strategy has been to
gaslight the person interviewing me.

------
m3kw9
I love how so many are passionate about this subject, always comments into the
hundreds.

------
ChrisMarshallNY
I've written about this before.

I've become pretty much convinced that many of the leetcode tests are "young-
pass filters."

I recently saw an ad for a two-month, intensive course on how to pass
interview questions.

The heck with that. I want to learn _relevant_ stuff.

If people don't want me working for them, then I'm not particularly interested
in working for them. Too bad. We could have made beautiful music together. I'm
no slouch.

------
tobyhinloopen
I usually just imagine visiting some friends that I haven’t seen in a while.

------
jahaja
Well they are auditions rather than interviews. So it seem to be accurate.

------
leesec
Well that sounds good to me. Sofware work can be pretty anxiety inducing.

------
tazedsoul
The tech industry, if it can manage to do it, would benefit immensely from
enacting interviewing methodologies that aren’t ultimately rooted in an
underlying need to haze others. It’s as if the CS nerds forgot that when the
frats hazed others we were disgusted by it.

------
heldrida
What causes anxiety is wasting numerous weeks of your life that you could have
used to pay your rent or spend with your friends, family or do something
meaningful with your time. That's how much this people think you're worth!

Tinder generation, swipe next.

------
pts_
Social engineering like it's 1000 AD pre technology era.

------
tomerbd
No matter how difficult a coding interview was, the job afterwards was always
harder and more challenging - taking that into account I would rather for even
harder job interviews to reflect the actual difficulty of jobs.

------
bishalpaudel
I stopped applying to FAANG because of the exact reason.

------
StavrosK
The comments in these threads must give tptacek PTSD.

------
mister_hn
again, this demonstrates that interviews in tech sector are all wrong.

You don't ask plumbers to show how they do the job before hiring them.

------
master_yoda_1
I think this is known for years. Is't it?

~~~
leafboi
It's been known, but there are a lot of naysayers as shown by people who still
conduct this type of interview.

Science serves as definitive evidence which side is right or wrong.

------
bpodgursky
Uh, maybe at times, but I assure you -- I've interviewed some VERY confident,
VERY incompetent software dev candidates.

~~~
weego
You're conflating the personality traits of the interviewee with criticism of
the generally accepted 'best practice' of interviewers.

On my experience far too many companies are using processes that select for
entirely the wrong criteria.

~~~
pantaloony
I think it’s pretty funny that HN constantly promotes articles about how
shitty tech hiring is, and the threads are always a mix of workers agreeing
and then managers often _also_ agreeing but 100% confident that their hiring
process works well and weeds out enormous numbers of faking bozos. It’s like,
ok but almost none of you have useful _actual data_ to back up your process
and you can’t all be anywhere near as good as you think if hiring is, in fact,
broken.

------
borroka
Tech interviews are an evergreen source of delight /s. I have been the
interviewer and interviewee plenty. A few reflections.

\- I try hard, and I believe with a certain success, to follow a sound
approach to interviewing. I read the resume, I see where their contribution to
the team I am hiring for can come from. Then, false positives and negatives
never go to zero, and such is life (although some people are much better than
others at observing and judging). I mean, people get married and divorced in 6
months, and it follows that there are inevitably bad assessments also in tech.
But I try to ask questions that can allow me to gauge the skills of the person
in front of me for the job that they are hired for. A conversation, more than
a test. Things that can get googled are of no interest to me, for example. Or
theoretical and never-to-be-used-IRL knowledge. Can you imagine do a try-out
for a hoops team, you are a guard, and they try you out as a play? Or you do a
try out for the Patriots as a quarterback and they test you on punting? Which
is what happens in tech more times than expected by chance.

\- Among the most nightmarish experiences I had as an interviewee, I remember:
a hiring manager yawning for the whole interview; another time, I successfully
(brilliantly?) completed a coding test in a few minutes and the interviewers
said it was all great, I have no other questions. But then the recruiter told
me that no, the interviewer felt that coding was so and so (what?). Another
time I was dismissed after the first of five planned onsite interviews. No
idea why.

\- What about homework, you may ask? I took a few (2 or 3) homework for some
of the most popular companies in SV. Well, one time I made this model and I
don't know, maybe I will die an Achilles death for my hubris, but I believe I
can fit a tree-based classification model after 20 years of work. The
recruiter said that both graders like my model, but I did not do
hyperparameter optimisation. I told them that the optimisation was both in the
documentation of the homework and the associated Scala code. They said they
were asking the graders again (it was just one, the other copied what the
other wrote) and that they would have been back with an answer in a week.
After a gentle nudge, they said that no, the graders did not like it anyway
(they never took a second look and I wasted my time: for what, exactly?). I am
not gonna take any other homework from any company, if not for reasons of
desperation or destitution and I hope it won’t ever happen.

\- Most companies rarely if ever do any business or technical analyses of
their interviews. Google pollinated the tech world with their interview
strategy and the other companies said: if Google interviews this way, why
should we do any different? I work for a big company now, there is no post-
mortem, and neither pre- nor middle-. We hired more than 100 people this way
for a new team, it could have been better, worse, who cares? Nobody checks
anyway. Because, and this is the main insight I’d like to present, especially
for big companies (for start-ups it is different, an exceptional IC can “make”
the company), it does not matter. There is a big area of 20+-D skill-space
that is “flat” with respect to the employer’s contribution to the company. You
don’t hire that uber-incompetent and you are doing fine because the market,
overall economy, luck, people skill of the C-suite are much more important
than ICs, managers, or directors’ skills (beyond a certain threshold, of
course). Google prefers to hire talented people and then it does not allow
them to work on anything crucial because the most important thing is not
letting them work for other companies? It does not matter. And if the question
is: but if the interview process is inefficient, are we not wasting thousands
and thousands of employee-hours? I have got the same answer for you: it does
not matter, because there is often no need (in big companies) for more than
20% of the time of the employee, people get hired for all sort of reasons, but
rarely for (true) technical or business reasons.

------
sandGorgon
does it mean that take home interviews as the best form of interviews ?

------
Steven_Vellon
One observation of mine is that interviewing is often more about finding the
most feasible and consistent process rather than the most effective one.
Whiteboard interviews make for consistent evaluation and it's easy to train up
new people on a set of questions. Scoring based on how far the interview got,
and the level of optimization of the approach used is relatively unambiguous.
So it's logistically easy and less subject to bias - that's why it's popular
even though it has little resemblance to day to day work.

A more effective interview would involve tasks that more closely resemble day
to day work. However, the examples of this that I've seen so far make for much
more ambiguous evaluation or are much more difficult to schedule:

One example interview process I've seen is having people submit PRs to a
codebase to fix bugs or implement features, and then review a PR submitted by
the interviewer. This more closely resembles day to day work, but has the
disadvantage of spreading the interview process across multiple days.
Additionally, you might tell the candidate that they shouldn't take more than
2 hours to implement the PR but who knows if they spend 10+ hours on it.

Another alternative interview format is a 2-hour long one that starts with the
interviewer asking "how would you build a text editor?" There's no right
response. If the candidate gives responses like ropes and gap buffers then the
interview might go in a direction focused on data structures and systems. Some
candidates ask if it's a WYSIWYG editor like word, or an ASCII/unicode editor
like Vim. In that case the interview might test the candidate's abilities to
think through how to build an interface to decouple the UI and underlying data
structures. The candidate might start off with a simple array, and work
through why it becomes infeasible at larger lengths and think through ways to
mitigate that. This interview was flexible to test technical knowledge and
reasoning for candidates at any level, and could go in a variety of
directions. But on the other hand that makes consistent evaluation difficult
and training people up on this interview similarly difficult. This was at a
small company with maybe 30-40 engineers, where each team basically had total
latitude on how it carried out its own hiring. That's not how a lot of larger
companies' interviews work, which often emphasize consistent evaluation and
multiple interviewers.

The only interview which does more closely resemble real world tasks, and I
suggest that more companies employ is debugging interviews. It requires the
candidate bring a laptop and that the company build a couple repository
templates, but once that's done it's a very easy interview to conduct. Just
observe the candidate debug and make note of how many bugs are fixed and
whether the fixes do bad things like breach layers of abstraction.

------
draw_down
> the findings suggest that companies are missing out on really good
> programmers because those programmers aren’t good at writing on a whiteboard
> and explaining their work out loud while coding

Sure, no love for whiteboards from me. However, this does suggest an arbitrage
opportunity for an employer who is willing to look differently at these
candidates, and then profit handsomely, Moneyball-style.

There's a bunch of money just being left on the table and nobody is picking it
up, really?

My two cents is that interviewing candidates can give a whole lot more insight
into how to perform in an interview. Interview performance is a skill that is
separate from the actual job, and gauging that performance is sub-optimal. But
everything in this space is sub-optimal, it's all pick your poison.

~~~
SpicyLemonZest
Many smaller companies do pick up the money, which is one of the major factors
allowing them to succeed in a world where the big software companies toss
around ridiculous amounts of compensation. The Googles and Facebooks of the
world are structurally unable to do the arbitrage because many (probably most)
of their interviewers don't particularly care about interviewing.

------
cheesecracker
Maybe some people (how many?) fail because of anxiety, but on the other hand,
anybody who doesn't fail certainly has proven some minimal coding skills?

I usually took it as a positive if companies did some basic coding skill test,
showing they at least cared that people can code.

Also, the people who fail because of anxiety, how will they fare in the normal
job situation when they are supposed to discuss stuff on the whiteboard with
colleagues?

Maybe the results could be taken as "give people who fail the coding interview
another chance", but I wouldn't take it as "don't do coding interviews".

------
starpilot
Current SWE interviews seem extraordinarily effective. They've produced
corporations that dominate our lives, with systems that have unimaginable
capabilities. Their profits currently dwarf other companies such that the
S&P500 is basically a tech company index. More companies will adopt hiring
practices like these as the applicants become more competitive. SpaceX has
grueling all-day multiround tech-like interviews like this, whereas Boeing
just does 1 hour with 2 managers. The effect on employee quality is clear.

~~~
rsynnott
There's a phenomenon where an entity which is successful for reason X is
unaware that it's hurting itself for reason Y, and may even attribute its
success to reason Y. Which is all very well until X shrinks or Y grows.

The point is that you can't assume that just because something is doing well,
everything it's doing is perfect. In fact, success is likely to hide all sorts
of bad practices, because the bad practices just don't hurt enough to offset
the success coming from another source.

I kind of suspect that this is why declining tech companies often crash and
burn so dramatically; they tend to have a lot of myths that they believe about
themselves, and when the one thing keeping them going declines, they can't
address all the things that are pulling them down, because they don't really
believe in them in the first place.

