
Ask HN: What is a better approach to interviewing? - awalsh128
I want to make the interview experience better for applicants but have no control over the format or time. There is a lot of criticism of the algo centric approach and I&#x27;d like to change that within the confines given to me.
======
gkoberger
I ask everyone to bring their own project to work on!

It could be adding a feature to a side project, starting something new, or
contributing to open source. (Or, we have some stock ideas if people need
inspiration.) Rather than asking them to solve a problem they had never heard
of before or use a codebase they're not familiar with, I get so much more out
of watching them work in their own environment.

I can ask them questions about why they're doing something, and they tend to
have much more detailed answers because they've been thinking about it for
weeks. They're solving a problem they care about in a codebase they know,
which mimics how working with them will be a few months in.

We can talk about tradeoffs and design decision, and I get a real sense for
how they think. Plus, I've found most people are so much more comfortable and
excited.

(If you're interested, I wrote a blog post about all the ways I've designed
how we interview, to give everyone the chance to show off the best version of
themselves: [https://blog.readme.com/designing-a-candidate-focused-
interv...](https://blog.readme.com/designing-a-candidate-focused-interview-
process/) And of course, I'm hiring!)

~~~
greggman3
Plenty of great programmers have no side projects. They have family or hobbies
or something else that occupies their time outside of work.

~~~
gkoberger
Yup! We have projects and ideas ready to go if they don’t have one :)

For people like this, though, I’ve seen many use it as an opportunity to work
on something they’ve been wanting to for a while and never had the time! A
Hello World in a new language, merging in PRs on an old repo, writing a script
to automate something or even a take home coding exam for another company.

~~~
stennie
In the example interview schedule in your blog post there's only an hour
allocated for BYO project discussion. Is that representative of your typical
interview schedule? The scope of some of your examples in the blog post may be
challenging to cover in much detail in the first hour working with brand new
collaborators, although perhaps is enough for calibration. Are you able to
share any examples of the fun projects/ideas you have as suggestions for
candidates that don't have a project of their own?

~~~
reificator
I was surprised it was that long, I can talk about a project for much longer
than an hour but the average interviewer won't give a shit past 15-30 minutes.

~~~
gkoberger
It's not just talking, you'd also be adding a feature or fixing a bug or
something like that :)

------
akdas
I've been writing about this topic for the last 7 months. The absolute biggest
change you can make within the confines you presented is to switch from
"looking for weaknesses" to "looking for strengths". I wrote about this here
during the beginning of the year[0].

You can find all my past articles[1], and I'm happy to find you specific ones.
I think, given that you can't switch away from algorithmic interviews, at
least make the interviews more collaborative, define your expectations up
front, and generally find ways to help the candidate show their strengths.

[0] [https://hiringfor.tech/2020/02/10/false-positives-and-
false-...](https://hiringfor.tech/2020/02/10/false-positives-and-false-
negatives.html)

[1] [https://hiringfor.tech/archive.html](https://hiringfor.tech/archive.html)

~~~
awalsh128
This is a great starting point. Would love some examples sent to
awalsh128@gmail.com

------
dcolkitt
I think the harsh truth is that interviewing will never be a pleasant
experience. The cost of a single bad hire is so high, that companies have a
strong incentive towards false negatives. It’s better to pass over 25 good
candidates than let one bad one through the door.

That means that regardless of the format, interviews are always going to be
inherently unfair. Even the slightest hint of incompetence will nuke your
chances. Assessing a person you’ve never met is just an inherently noisy
process, and there’s always going to be false negatives. And almost every
company in the world would much rather be safe than fair. You probably are
having a bad day or misunderstood the question, but employers can’t afford to
take that risk.

For the interviewees part, the process is just plain soul draining. You’re
literally subjecting your life’s work to the harsh, quick and unfair judgement
of people you’ve never met. You’re putting yourself in a position of extreme
vulnerability, and it’s easy to feel like a piece of meat.

I don’t really see how twiddling with the format will change the fundamental
dynamics. The only thing I can say is that both sides should have more
understanding and sympathy for the other.

Candidates, don’t take any single rejection too personally. Remember the
interviewer is a person, who also can be having a shitty day or make mistakes.
Interviewers be kind and positive and give constructive feedback, even for the
clear no’s.

If you really hate interviewing, then spend more time networking. Hiring
personal contacts and recommendations is much less noisy/risky, and therefore
an overall less adversarial process.

~~~
ErrantX
This.

Another variant of this challenge is; when you have multiple interviewers and
some are positive some negative about the candidate. Generally companies will
always side toward rejection unless there is a specific reason not to (for
example, you need someone super technical but they struggled on the soft-
skills side and you have the right team to help them grow in).

~~~
qznc
I can derive two very different strategies from this. Either "generalize and
avoid showing any reason to reject you" or alternatively "specialize and
provide a reason too good to reject you".

Surely, it depends on the situation of the specific employer. So the trick for
a candidate would be to figure out if there exists a single argument (e.g. a
certain skill) which makes it impossible to not hire you.

Coming back to the original question: You can be honest to the candidate;
figure out what your organization is desperate for and tell them. Of course,
this gives up some negotiation strength.

------
cepp
I'll share what we've transitioned to recently: a testing focused interview!

We provide a candidate a piece of production code with some slight tweaks made
that will break it in (common) edge cases. Further, we give a test harness
that already supplies some basic tests for which it will work. What we're
looking for is to see if the candidate can grok the provided code (< 30 lines
for a 2hr interview) and reason through ways it could break. Writing the tests
insures that the candidate can _actually_ write the code and shows any areas
they might be less focused on or miss.

Compared to our old procedure of writing the (provided) code snippet this has
been a far more successful approach. In 2 hours I learn more about an
applicant than through the entirety of the previous process. An added benefit:
this workflow is representative of real work - you'll need to track down code
you likely didn't write and implement new functionality, or fix something
that's broken. Doing this in an interview is a near direct translation to the
job (minus understanding the whole codebase).

I'd strongly recommend this unit testing based process and it can definitely
be tweaked to fit your own scenario!

~~~
lambdatronics
>representative of real work - you'll need to track down code you likely
didn't write and implement new functionality, or fix something that's broken

This. The stereotypical algo interview assumes a greenfield project, but
there's almost always "legacy code" you'll need to grok first.

“Indeed, the ratio of time spent reading versus writing is well over 10 to 1.”
― Robert C. Martin

~~~
disgruntledphd2
And yet so many people refuse to read working code, and agitate for re-writes.

It's just bizarre to me.

~~~
lambdatronics
When all you have is a hammer.... Think about it: coursework is all about
writing new code. I can't recall _any_ resources for learning "software
archeology."

Part of the deal is the 'humiliation' of being stuck following someone else's
stylistic/design/naming-scheme choices. Another aspect is the curse of
knowledge, leading to useless documentation. Plus, it's easy to underestimate
the amount of effort required to reinvent the wheel: all the necessary
complexity looks like incidental complexity at first glance. "Why should I be
forced to learn all this gobblety-guck when I should just be able to re-
implement from scratch in an afternoon? [... three weeks later] Oh, now I see
why."

------
andrewflnr
Idea, based on my extensive qualifications of reading HN hiring threads:

Provide candidates a small sample "project" of 50-100 lines, a Flask webapp
perhaps. Work with them to get it running on whatever interview machine
they're using. Have them extend it with some fairly trivial feature, around
FizzBuzz complexity but with a bit of a design component. These questions
should be carefully designed and standardized across interviews.

Key point: completely strip out identifying information from their solution
and pass it to someone else for grading. The person "giving" the question is a
proctor, not a judge, and they're officially on the candidate's side. Possibly
add a side dish of talky conceptual, design, and behavioral questions, with as
much blinding in the assessment of the answers as possible, though that's
harder if it's a dialogue... Hmm.

A good interview is fun. You get a nice bite-sized problem to solve with
someone to bounce ideas off of, and maybe learn something in the process. I'd
suggest almost a game level-design approach to coding problems. Make
interviewing fun again.

~~~
wry_discontent
This is the kind of interview I try to give.

My favorite interview question I ever had was to design an elevator system.
There's no right answer. It wasn't even mostly code, it was mostly talking.
But every so often, they'd say "write that as a method" and let me imagine
whatever api I wanted. I had so much fun doing that.

~~~
gmanis
I don’t know if you’re aware of this but I had a lot of fun and maybe this can
be used for some interview as an initial ice-breaker problem?

[https://play.elevatorsaga.com/](https://play.elevatorsaga.com/)

~~~
wry_discontent
I love elevatorsaga!

It would be an interesting idea just to have a candidate go to the site and
screenshare to watch them do some coding.

------
laboo
Create an exam, broadcast an invite to take it, hire randomly from all who
pass.

All excuses for why this won't work can be answered with: then make a better
test.

As a species, across all cultures, we tell ourselves that the human we're
after -- the student, the employee, the promotee -- can't possibly be selected
by test alone.

When in fact, if we can't write a test to select the qualified humans, then
either we're too lazy to write one, or, more likely, we actually _want_ to
leave plenty of room for human bias to do the actual selecting.

And this is ok because we have a special power: we can judge the value of
every human, and its future likelihood for success, with a single
conversation. If we weren't in a tech company, we could make a very good
living reading palms.

We never hire people who _don 't_ pass our wonderfully fuzzy exams, so we have
no evidence that we're selecting the best people or not. No worries, though,
our palm reading is very, very accurate.

The way we look at it is like this. We make a test no one can pass. We always
have one more question or one more "level" that there's not enough time for.
Because when all candidates fail, we have to fall back on our palm reading,
which is just how we like it.

Power and privilege are precious resources to us. We give them out to those
most likely to reciprocate. That's what we're poking for with our "culture
fit".

In the future, students of our culture will look back on our hiring practices
and say, "It was illegal to hire based on race, age, sex, and a million other
things, but not beauty??? They didn't start with beauty? And they never
realized that beauty needed to be in the mix? I don't understand."

But we understand. It makes perfect sense.

~~~
marcinzm
That's the whiteboard interview in a nutshell. Standard problems given to
everyone and asked to answer them in a standard way. Done in person to lower
the chance of cheating.

The same reasons whiteboard interviews are problematic will apply to any test
done at scale.

~~~
laboo
Not standard questions. Companies switch away from a problem as soon a it
becomes known they use it. They want a problem the applicant has never seen
before. And I'm saying, it's not because they are testing for intelligence --
they will help you through it if they like you -- it's because they want you
to struggle with it. Which gives them the right to use their gut to decide.

~~~
marcinzm
That's not been my experience at all. Companies keep using the same problems
that are on leetcode or some minor variant of it. More to the point, in many
places companies don't assign problems, the interviewer has full leeway in
doing that. They will keep using the same problem since they have a rubric for
how people perform on it.

~~~
laboo
I have never worked at a FAANG, but people I know who do, and give interviews,
tell me that questions get banned.

~~~
nullsense
If that's the case eventually every leetcode question will be banned and the
problem will have resolved itself.

------
angarg12
I work in a large tech company and I also hate leetcode-style interviews, but
I need to abide by the rules.

My ideal interview questions are as follows:

* No trick question. Reaching a solution shouldn't involve obscure knowledge or an "aha!" moment. * Only basic data structures and algorithms, the kind that you will realistically use in most day jobs. * Start with a very basic problem, and ask follow up questions with increasing complexity. * Extend the problem in different dimension to probe the candidate in different axis. For example, ask the candidate to design an API for the code, or to run it at massive scale.

It's hard to come up with problems that cover all these points, but you can
get close. I find these kind of questions allow me to assess candidate without
making the process unfair or overly stressful for them.

~~~
dnautics
I give a problem where it is quite literally "implement this spec". It's
indeed a tricky one with corner cases and design that needs to be thought
about. Unfortunately some very good programmers expect there to be an
algorithmic trick and make it more complicated than it needs to be, even if I
tell them that's not the case.

------
mrweasel
Is this mainly a US problem? Every job I’ve ever interviewed for has mostly
been to see if I’d fit in. If my resume says that I know how to do something,
that has always been accepted as good enough.

Technical talks are never about coding, but more a conversation about previous
project and perhaps a little side track on how to tackle large projects.

The style of interviews discused here on HN show an alarmingly low level of
trust in the people you want to hire. That’s a bad way to start a
relationship.

~~~
snypox
Some people - to say at least - exaggerates in their CV. You can’t trust
someone automatically.

~~~
onion2k
Outside of development there are very few jobs that have a competency test as
part of the hiring process. Some industries have "continuous testing" through
certification, and some have tests of things like physical fitness, but _in
general_ it's assumed that if you're applying for a role then you're capable
of doing it. The economy isn't collapsing under the weight of corporations
hiring the wrong people all the time.

I suspect there are technical tests in developer interviews _because there can
be_ rather than because they're actually a useful way to appraise someone.

~~~
dcolkitt
There’s very few jobs as specialized and desirable as software engineering
without any formal credential requirements.

The incentive to bullshit your way in the door is _way_ higher than most other
fields. Imagine if you could get hired as a doctor, without having to graduate
high school.

------
axihack
Just a technical conversation.

I have hired dozens of engineers and have never failed do find great hires.

What I mostly do is just a conversation, which I call “geek conversation”,
just like when you meet people at meet-ups or tech conventions.

With a simple talk I can first make the person relax, this allows me to have a
great impression about her/his personality, passions, interests, personal
roadmap, attitude towards problem solving and etc. Whenever we ponder on a
more technical subject I throw some questions that might help me measure their
technical level a bit, but what I care to measure is mostly awareness on
things, nowadays software engineers need to be, above all, great at
researching, for most roles they must have a broader overall knowledge and be
able to creatively join the dots with a ton of pragmatism. I don’t care if
they are able to implementing Dijkstra's or to modify a Trie in front of me,
although having awareness of what problem these things solve is a good
indication of some seniority.

These interviews take a lot more from me, I have to properly learn about the
candidate and the role before I enter the interview and that requires people
skills and real interest in the product, and of course I am not able to
interview 40 candidates to find just “the best one” (which is ridiculous and
something I actually don’t believe actually exists).

Yes there are roles which require the extreme algorithm coding skill, but
those are a minority and it is unfortunate if your company is treating every
candidate as if they were going to do that.

------
lampe3
Job interviews in the IT world are a reason why I will go freelancing...

Writing some algorithm No why should I? If I need an algorithm that must be
fast I will look up a peace of code that is used and tested by other people.

No I will not live google in front of you watching me. How is this a real
world scenario? How came up with this?

Like stupid questions like: How many log entries should a service have each
hour? Yes this was a real question in a job interview.

IT Job interviews are broken in so many ways.

I refuse to live code or have this you now have X minutes to solve a problem
stuff. Again this is not how I work and also I have never seen people work
like that in the real world. Only in shitty teams with shitty organization and
where they were thinking that they were agile because the sprint ends in 5
minutes...

Here is the only way of interview I tolerate these days: 1) (optional) Have a
call with HR

2) Talk with people from the team in an open way. Talk about real challenges
and how the applicant can help in the challenges of the team. Check if the
applicant will fit your team! Good team chemistry eats technical knowledge by
1000x. If you only take people because they know some algorithm you will most
likely end up with something in Germany they call "fachidioten". They will
solve the problem on a technical level but will bring with them 1000 other
social, product and long term problems with them.

3) If you still not convinced that this person is good enough for your team
then. Give that person a small challenge he/she can do at at any time she/he
wants.

4) Let the applicant send you the solution and if you think it is okay then
talk about the solution of the applicant and see how he/she reacts to
criticism. Again a team that has good social skills and likes each other will
eat every team were you have rock stars that think they are the best.

~~~
shoo
Possible failure modes with this approach: quite a few candidates who are not
strong at actually doing the work will be able to somewhat convincingly "talk
the talk" well enough to pass stage 2. If stage 3 is done async then there's a
minority of candidates who will cheat and get someone else to solve the
challenge for them. If the org has set up a hiring pipeline rather that first
tries to identify strong hires before figuring out exactly which team they
would join rather hiring to fill a specific role then step 2 might not make
sense either, as there's no specific team to interview.

I agree that the in person time pressured performative problem solving &
coding engineering interview process is often a poor approximation of the
work.

~~~
crehn
Some companies (e.g. Amazon) have a separate verification interview to ensure
the candidate is the author of the deliverable.

~~~
heavenlyblue
I think I failed one of the interviews this way actually: after having done
the async part of it I did about 10 other test interviews and literally forgot
about what I did for this one.

I had quite a few “aha” moments where I was “ah wait, I totally didn’t do
that, have I?”. Didn’t get the job, however neither did I like them.

------
nithinpb
I remind myself before every interview that we are here to know what the
candidate knows and not what the candidate does not know. This advice from my
lead a few years ago has completely changed the way I conduct myself, probe
their background and ask questions. By letting the candidate speak about a
topic, you will learn an awful lot about their depth in the subject. This will
help you make your assessment about their ability to handle the current
challenges.

~~~
visarga
How do you handle bullshit? A candidate (most) might be great at talking like
he has experience, but in reality have some pretty huge lapses in basic
knowledge. So many under qualified people apply 'just to see if they hire me'.

~~~
grumple
What do you consider a huge lapse in basic knowledge?

Knowledge is usually the easiest problem to fix. It’s a google search away.
All the bad hires I’ve seen have had masters or phds and the problem has been
poor work ethic, focus, and practical application of knowledge towards solving
business problems. Interviews, in my opinion, can easily test for basic dev
knowledge but should focus more on the way the candidate will conduct themself
as an employee.

------
ALittleLight
One thing about interviewing that surprises and confuses me is the number of
people who seemingly can't program despite working as programmers or having a
master's in CS.

My "FizzBuzz" coding question that I ask on phone screens is "Write a function
that takes a string and returns a list containing the most frequent
character." It has a few clarifications that are needed and I expect the
person to ask about (What if there is no most frequent character like in an
empty string (return an empty list), multiple characters equally frequent
(each equally frequent character should be in the returned list once), etc)
and has some basic logic requirements.

I am continuously astounded when people can't do this. I encourage them to use
the language they're most comfortable with. I give them time to think about
the problem, encourage them to ask questions etc. If they seem stuck I'll ask
them to walk me through what they're thinking about, etc.

I've had multiple people with master's degrees in CS or with years of
professional programming experience get stumped by this problem and it just
makes zero sense to me. I took undergraduate CS courses and this would've been
an easy problem in those classes. I was a TA in an undergraduate intro to
programming course and I would've expected all of my students to be able to
solve this problem in similar circumstances.

It seems to me like someone with a Master's degree in math has come in and I'm
asking them to "solve for x" in the kind of equation you might give to an
eighth grader and they're unable to do it.

~~~
noodlesUK
I was often shocked by this as well, until I did an interview where I was
asked a similarly trivial question, and I _became_ that person. I completely
flubbed the interview, locked up, and, whilst I could write code that did
things, I just couldn’t wrap my head around the question, and no amount of
effort would chill me out. Obviously I didn’t land the thing. Some things like
fizz buzz are gonna be so trivially easy that anyone should be able to do it,
but try to understand where your candidate’s headspace is.

(I was also a TA in a course that did these sorts of trivial challenges, and
going through that experience really made me take a hard look at how I
perceived those 1st year students)

~~~
ALittleLight
I proposed to my manager (although this proposal was rejected) that we try
something like putting our high performers through our interview loops. This
would let us calibrate our questions and answers. If you're right, that
intermittently otherwise good candidates flub simple questions, I feel like
that would come up in the test interviews - and maybe there could be a good
compromise that somehow takes this into account.

~~~
noodlesUK
Unfortunately, I think that whilst it's a good idea to run a few mock
interviews with "known good" people who already work well within the company,
I also think that you can't replicate the stress that a potential candidate
might feel where the stakes are non-existent, and a rapport already exists
between interviewer and interviewee.

~~~
mattm
A good rule of thumb I've heard is that for an hour interview, you should
expect people that work for you to solve and complete it in about 20 minutes.
If it takes them longer than that, the question will be too difficult for
candidates once stress is factored in.

------
jpgvm
It depends on what level you are hiring at.

For senior engineers I would say it's all about design/architectural problems
and talking generally about concepts and taste to gauge fit.

Mid-level engineers I think the take-home is king. A scoped problem, 1-2 hours
at most that can be followed up with an interview talking about an extending
their solution is pretty bulletproof. You get to understand what tools that
are comfortable with, practices around source-control, testing etc that you
would expect from an experienced but not overly senior engineer.

Finally for entry level candidates I don't really know. Personally I hire
entry level people on gut feel. Algorithms and datastructures just tell me if
they a) went to university and b) paid attention. Seeing as I did neither of
those things and still write IMO decent software that seems like a
hypocritical measure. So I stick with how I feel about them, their level of
enthusiasm, how well they researched the position, what stuff they have played
with etc.

~~~
RandoHolmes
I once did an interview where I described some software I had designed and
built that was deployed around the world.

At the end of it the biggest burning question they had was "where are the unit
tests". I was like ... you see this circle here with this label inside it?
This was critical to the entire system and therefore had unit tests
surrounding it.

didn't get the job, they seemed kind of annoyed that I didn't get into great
detail about things like unit tests and instead talked mostly about the
architectural problems I solved.

I wish companies actually valued that level of knowledge.

~~~
hackinthebochs
The fact is, people tend to over-emphasize their area of competence and de-
emphasize areas outside of it. It was probably the case that the architectural
problems you solved were above their paygrade, so they ignored that and over-
emphasized what they knew. The fact that you didn't share their enthusiasm for
unit tests just shows you weren't a top notch developer (which always happens
to be a mirror image of the interviewer).

~~~
RandoHolmes
My interpretation was similar, but slightly different.

The company itself wasn't actually solving complex problems and therefore the
developers involved couldn't actually understand the level of complexity I was
able to deal with.

I get the feeling a lot of developers believe dealing with state in a react
app is a complex problem, but I've been doing this stuff for 25+ years and
left that behind as complex long ago. I know that's probably going to come
across as arrogant, but it's how I feel about the entire thing. I see people
all the time squabbling online over very specific source code related things
as if they're important.

~~~
potamic
Out of curiosity, what was the software which you built?

~~~
RandoHolmes
Nothing glorious :)

It was a VPS automated deployment system that could deploy both windows and
linux to various datacenters around the world while also installing some 50+
pieces of software (as configured).

------
angel_j
Hiring managers in tech need to realize they are working with high level
learners. An expert in JavaScript can easily learn React, TypeScript, etc.
Somebody who knows the ins and outs of Linux can easily learn Docker and
Kubernetes.

Hire them, give them a week to catch up, and their off; or wait months to find
a perfect match? Any developer with a few years experience knows how to pick
up the next piece of technology, the same way everybody else does: read docs,
search blogs and forums, ask StackOverflow, etc.

~~~
brianwawok
Depends on supply though. Say you found two perfect people for the job, can't
chose one from the other. One has 6 years of JS, one has 6 years of Perl. You
are hiring for a JS company. Which do you pick?

Now if all you find is Perl guy, that is another problem.

------
neilwilson
When it boils down to it, hiring is about whether you can make a correct
character judgement or not.

Some people can. Some people can't.

The secret to a good hire is probably to exclude people from doing the
interviewing who are essentially poor judges of character.

In other words let's stop trying to auto filter the candidates and instead
let's filter the interviewers. Pick good interviewers.

Let those firms that play sunk cost Frat games get the sort of people who will
work all hours for next to no pay. Those people always burn out quickly
anyway. Instead treat the interviewees' time with great respect and you will
get better long term people anyway.

I suspect saying that interviews are an hour's chat and then a decision will
get you a set of outstanding interviewees who are just too busy to be bothered
with anything else. And they will stand out from the crowd even on paper.

------
lordnacho
I've interviewed loads of people over the years in different ways. I'd say the
real dud hires were not that different from other hires. The duds were people
who for one reason or other did not want to do what was assigned, yet took
some time to say so. Personal issues for sure.

When hiring what works for me is looking at past experience. If someone says
they are a cpp performance guy, that creates an expectation of what they
should be able to chat about. If you're making it up, you will run out of
stuff to say, and I'm patient enough to let people keep talking. Hopefully
I'll learn something too, it happens often with top pros. But basically if
you're experienced in a field it should be hard to come up with anything where
you draw a total blank. If that's the case you're pretty good in my book, the
main differentiator after that is whether you've done exactly what the firm
needs or just something similar.

With entry level people it's hard. That's why they typically make less money,
it's an Akerlof lemon cut due to the being a lot of people who just aren't
going to be good at it. This isn't even necessarily a coding thing, but
general life maturity. We had a guy who couldn't wake up on time to get to
work, and he couldn't phone in when he was late either. I reckon there's no
way to know about this kind of attitude problem until you hire them. This is
also why you can expect a relatively high jump in pay after your first few
years, it's proven that you aren't one of the lemons. The distribution is a
large mass of hard to differentiate professionals, plus some duds of near zero
value.

So for entry level, your best bet is simply finding someone you think is
smart. You'll end up falling back to the same thing as everyone else who is
hiring: the prestige of the uni they went to, and some pot luck questions to
make sure they picked up something. However that's really not enough to know
what you want to know, which is whether they are able and willing to learn
what you need.

~~~
RandoHolmes
> The duds were people who for one reason or other did not want to do what was
> assigned, yet took some time to say so. Personal issues for sure.

If there's one thing I don't want to do, it's work by assignment. I'm not a
school kid, I'm a professional software developer. I'll tell you what needs to
be done. You can inform me of the goals you have for the company, and I'll
make sure that happens.

What I'm saying is that despite all my successes, you would consider me a
'dud'.

~~~
lordnacho
I'm perfectly happy to let people explore, that's why it takes a bit of time
to determine that someone isn't going to work out. These guys I'm thinking of
basically didn't want to do the job they were described. At least one of them
changed careers, preferring something more to do with people than code. Fair
enough, but for the hiring manager there's not that much warning. They still
have the skills so they'll pass the interview, and the money on offer is
enough to temporarily summon up some motivation.

------
sg47
45 minutes - write a simple API matching the spec in one of the 3 languages on
your/our laptop. Use Google if you have to. Someone will be in the room to
help answer questions. API should be functional. Returning fake data is fine
(or provide data in a sqlite db)

45 minutes - do code review of this extremely buggy code like you'd review a
coworker's code

45 minutes - presentation on a technical topic to one or more team members

45 minutes - behavioral. Let's talk about failures, conflicts in your career.
Who are your role models? What do they do well? Leadership abilities, career
progression

45 minutes - troubleshooting. Here's a docker container. Make the service
inside it work

~~~
Supermancho
That's 4+ hours if you count initial setup, meet, breaks, etc.

~~~
richardknop
A lot more since you have to prepare for the interview / interviews. So
there's several hours to prepare at least.

~~~
sg47
Which part. Except the presentation, there's no prep required. It's all about
what you know and what you have done in your career. No Leetcode, no
algorithms, no bs questions.

~~~
richardknop
Presentation sounds really vague to me, it could mean anything from 1-2 hours
of prep or a day or two of intense preparation to make a good presentation.

Also number 1 (writing simple API matching a spec) - this doesn't require prep
if it's using language that I am using currently / have been using in my daily
job for a long time.

But I have found often been required to write code in languages that are on my
CV but I haven't used them in 5+ years.

Also talking about past failures, career history. I can barely remember in
detail 2-3 years so I would need to spend a little time to just be prepared
about a random very specific question about a project I worked on 5 years ago
that might come up during interview because it would silly if I didn't
remember (which I wouldn't if you ask me randomly about something I did 5
years ago).

Small things like these add up, obviously the more you want to get a certain
job the more time you would spend preparing, it's only natural. If you are
only looking around you might just wing it since you don't care much.

I am not criticising your system, it seems pretty sensible actually btw.

------
FpUser
All my interviews are very simple: tell me what projects you did, how and why
and give me some references so I can check. I might ask how do you solve some
particular problem on conceptual level or how they would approach the task
they'll be working on. After that it is either hired or not. I've long stopped
asking those quiz type questions: knowledge of particular algos, how do you
write this in language X/Y/Z etc. pass some ingenuity test etc. As I also do
side consulting I expect the same from my potential clients. If they are
starting to bore me with what does this construct in that language mean I
apologize for wasting their time and go look somewhere else. I design and
develop whole products and it takes particular kind of mind. I have it myself
and I expect the same from the people I hire as it pays back big time. I do
not care about deep language knowledge as the type of people I just described
will do reasonable design in any. There are of course exception: if there is a
need to squeeze last tick of performance for some low level codec, transform
some insanely complex math problem into some meaningful algorithm then I would
look for domain experts of course.

------
KevinAiken
I really liked one place I interviewed at that gave an option for the
technical portion of the interview. You could choose a 4 hour homework,
whiteboarding, or talking someone through a significant non-proprietary
project of yours.

------
quadrifoliate
This sounds a bit glib, but hear me out. Ask the question:

If you were a team lead tomorrow, how would you go about hiring your
coworkers?

Honestly, that should tell you everything you need to know about:

a) What kind of technical skills they value (i.e. will work to improve in
themselves if necessary)

b) Their interaction style with coworkers.

Sure, this interview can probably be gamed, but IMO a lot less than your
typical "reverse a linked list" style interview.

~~~
dropit_sphere
It does sound glib, but I agree, I really think this is the way to go.

------
gagan2020
I had to take 300-400 interviews in short period of time to hire few
candidates. I created excellent team by asking basic concepts. If they could
answer those then they could build a product or figure the product out.

That works best for programmers with entry to mid-level. In Senior level you
need to focus more on problem solving.

~~~
danielzh
What are some examples of the basic concepts?

~~~
jasonpeacock
Data structures, complexity, algorithms, abstraction, encapsulation, ...

You want someone that can easily talk about the pros & cons of using different
data structures or algorithms to solve problems. Who can move up & down layers
of abstraction without getting confused & lost. Who can hide complexity to
make clean, readable, maintainable code.

Many people, especially junior/less experienced engineers, have only memorized
solutions to problems. They don't have a strong toolbox of solutions and the
experience to know how to apply them (and evaluate the options).

A good problem solver is methodical - they don't just guess at random data
structures, they identify what the requirements are of the problem and then
choose the appropriate data structure (or compose one) which meets those
requirements. They'll have a process, which may be different than your
process, that they use to break down problems into smaller, solvable pieces.

~~~
lampe3
Why do we focus so much on data structures and algorithms?

It sounds to me like we are forgetting all the other important things.

Like how easy it will be to replace that data structure if maybe the
requirements changes? Do we really need to optimize this thing fully now? Or
maybe for the next 5 years to simple solution will be more then enough? And so
many other questions.

A lot people need to think longer then 1 minute to come up with a good
solution or maybe you need to try or maybe you need to learn something?

Isn't that what we actually do? How often did we think that framework X is the
best but after a while it was not that great? Don't we need to play around
with an API to understand it better? Or let me read about that and tomorrow we
can talk about because last time I was doing something with binary trees was
in university because in the real world I don't need it that often because
yeah we have other important things to do.

~~~
jasonpeacock
This is about fundamentals - e.g. when to use an array vs vector vs linked
list, and I'm not talking about optimizing performance. I'm talking about
functional requirements and picking the data structure with the correct
behaviors to support those requirements.

There's like 5 basic data structures. Choosing the right one based on desired
functionality and explaining why you picked it, that's just table stakes.

I'm not paying you six figures to write software professionally if you can't
tell me when to choose a linked list over an array, and apply that knowledge
to solve a problem.

~~~
lampe3
If you programming language has this concepts and even then this will highly
depend on the programming language you are using.

The right data structure does not only depend on performance. I can write you
the most performant code in the world if you want to but it will take 5 years.
Oh we only have 1 month? then sorry for now an Array will do.

In most cases you pay someone for the skill to learn new things fast and stay
motivated. Why? Because in good companies you want to hold on to the people
for a long time and this people need to educate themselves. Why? Because even
the basics depend on the context they are in and languages, frameworks and
runtime and other stuff will change over time. 20 Years ago memory space was a
problem? Do we even think about memory now? Not really unless your working on
some embedded systems.

------
athoscouto
I don't think there is a silver bullet here. I've been working on our
engineering hiring process for the last couple years for a company with ~200
engineers - reading everything I can on the subject during this while.

Most articles complain about the status quo but don't offer any practical
advice. When they do, it is some approach that may not be an option on most
cases - e.g. 40 minute interview followed by a 90 day probation period.

Live coding interview can be pretty tough for candidates, but we try to make a
lit bit less difficult by:

\- Giving them time and resources to prepare.

\- Letting them use their preferred language and IDE/editor.

But since you have no control over format or time, maybe try to:

\- Use reasonable problems - to assess skills people will actually need on the
job. In our case it usually boils down to using comodity structures like a
dictionary or map to do simple tasks efficiently.

\- Use problems that start simple but can be extended. This is good because
when candidates finish the first part, they get a boost in confidence. It is
also nice because if they freeze, you can help them finish the current "level"
and still have material to assess other skills.

\- Set them up for success during interview:

    
    
      - Reserve the first 5~10 minutes for introductions, ice-breakers or questions about the interview.
    
      - Reserve the last 5~10 minutes for the candidate to ask any question about the company or the hiring process.
    
      - Make clear that is OK to ask any question.
    
      - Help them on small blockers.
    
      - Be friendly.
    

Making these little tweaks helped us to diminish the pain of this kind of
interview.

Some links/references:

[1]: [https://lawler.io/scrivings/erics-guide-to-hiring-
software-d...](https://lawler.io/scrivings/erics-guide-to-hiring-software-
developers/)

[2]: [https://medium.com/@alexallain/what-ive-learned-
interviewing...](https://medium.com/@alexallain/what-ive-learned-
interviewing-500-people-the-interviewer-skills-ladder-for-high-growth-
software-37778d2aae85)

[3]: [https://www.holloway.com/g/technical-recruiting-
hiring/about](https://www.holloway.com/g/technical-recruiting-hiring/about)

------
necovek
"Format" is very broad, so in a way you are saying that you can't change
anything.

Rather than saying how you are constrained, what freedom do you have?

~~~
awalsh128
I have the freedom in the question I decide to ask or how much time I can
spend before the question. I have to ask a code question though.

------
jasonpeacock
Identify what data you need to collect to show that the candidate has skills &
knowledge to perform the role, then ask questions to collect that data.

Of course, this requires knowing what skills & knowledge are needed for the
role - you may need to review the job description or push on the hiring
manager to quantify them.

When asking problem solving or coding questions, work with the candidate as if
you were two teammates solving the problem together - don't be adversarial,
don't play "gotcha!", don't try to show off or prove yourself smarter.

If you have an awesome candidate, they will be smarter/more skilled/more
experienced/more knowledgeable than you in some technical areas. That's a good
thing!

------
AlexTWithBeard
Almost every software job has multiple skills involved. An example picked from
one of my recent headcounts is:

\- algorithms

\- experience with a specific technology

\- getting things done

\- communication in "not being a dick" sense

\- generally "being smart"

I want a person to score "B" in all of them and "A" in one.

It's important to realize that you don't need all A's: it's good if you manage
to find an all-round A-student, but it's rarely possible and takes huge amount
of time.

One "C" may be allowed, but then I'd probably want more than one "A".

So I keep my questions simple. For algorithms, for example, I ask to write a
bubble sort. If a person can do it - it's a firm "B".

------
6keZbCECT2uB
I am new to interviewing, but I have two working hypotheses:

1\. Hire based on the resume, gut check on the interview. Is it plausible that
they played the role that they said they did? If you were a major contributor
to an RPC framework in C, but you can't walk a linked list... Maybe your
resume is too inflated. This means that the questions are basic because their
purpose is not to rank candidates but to ensure the validity of the resume.

2\. Focus on reading over writing. I would rather a candidate who can read
code someone else wrote, articulate what it does, and maybe plan for how to
extend it than someone who can pretend to invent something new.

~~~
tucif
#1 is good, but I’d suggest always using the same set of questions to have a
good baseline against which you can compare candidates.

#2 I have mixed feelings about. it may make sense if you’re after an expert on
that language but may evaluate wrongly candidates that would be great reading
code once they get just a bit more familiar with and get the overall
architecture. In many companies, you’d be forbidden from sharing actual code
with non employees, so the excerpt shown in the interview would have to be
purposely crafted and loose value. I propose a more interesting exercise could
be to examine a random piece of opensource that Both the interviewer and
interviewee are unfamiliar with and trying to get a good discussion from it,
but it’s a risky move leaving the interviewer exposed.

------
alexbanks
In my experience, being an advocate for the candidate (during the interview)
is the best way to make interviewing more pleasant. My goals (during the
interview) are to learn as much as I possibly can about the candidate without
making them feel under pressure. I ask a lot of questions about what's on
their resume, but with the goal of finding out what gets that candidate
excited about tech (since I pretty much only interview developers).

I generally take a look at their resume and then do some research about the
tools they've used in advance. Then, during the interview, I ask about what
they like/don't like/find interesting about those tools. The goal is
absolutely not to gotcha them, but instead to find out what they're interested
in in that space. If I ask a question that it becomes clear they've
lied/fabricated about on their resume, I say something to the effect of "No
worries" and change the subject.

Depending on the role, you need more info than just what languages/frameworks
they've used. For more senior roles, or roles that involve architecture/cloud
functionality, I'd ask about how they've built systems in the past. If they
call out AWS, I ask about what resources they've used, how, and why. If you've
written down DynamoDB but cannot speak intelligently about access patterns or
secondary indexing, it's kinda clear that you just used a system someone else
defined. Whether or not that's a problem depends on what role they're applying
for. If they can speak intelligently about how they got to a specific DynamoDB
structure, they probably are being honest enough about their experience. Note,
it needs to be clear that the candidate is not speaking in the abstract, but
about things they've actually done. Googling stuff is easy, finding the weird
parts of tech in practice is hard.

Ultimately I want them to feel comfortable enough to get chatty about
development. Usually I find out enough about their skills while they're
chatting - I think most would be surprised to find how clearly you can
understand a person's abilities without directly asking about them. You just
kinda have to spend some time up front learning pros/cons/common pitfalls of
the tech on their resume.

------
juped
How do I get interviewed by any of the people posting about their great
candidate-friendly interview processes in this thread?

There's always a ton of posts, but I have never had a pleasant or unbiased
interview experience in my life.

------
yakubin
Casey Muratori showcased a new model of a programming interview here:
[https://www.youtube.com/watch?v=cfyWvJdsDRI](https://www.youtube.com/watch?v=cfyWvJdsDRI)

It shows to the interviewer how a given person works in a programming job, how
they researched solutions to the problems they were faced with in the past
etc.; as opposed to the usual exercises that have nothing to do with the job
and won't really show you how well they will perform.

It's also less confrontational which should put the interviewee at more ease.

------
TudorBirlea
I'm a behavioural scientist and I work with startups in their first hires,
maybe my experience helps: while it is relatively easy to qualify technical
capabilities, most interviews fail to qualify the personality traits of the
candidates. This means that only after 6 months or so you realise that you
hired a Divider while you wanted a Multiplier. That your new CTO has little
patience with the rest of the team, and so on.

We work on fixing that, so we implement a process: founders pick some
personality traits they find essential for a great hire, we prepare an
interview playbook, and they can be certain that the new guy is actually what
they are hoping for.

Interviewing for personality (cultural fit) is hard because we (humans) are
hardwired to survive so we give answers that we think are in our favour - and
not necessarily truthful. Key to our approach is that we use falsifiable
questions, meaning that you can't guess what I believe is a right answer.

If this helps any fellow founders here, am happy to lend a hand.

------
songzme
I don't think interviewing as a concept should even exist. In a perfect world,
everyone should be able to freely learn new things and help contribute their
life experience in a positive way. In tech, there are plenty of opportunities
to do that in the way of open source.

So what I started was a local library program (pre-covid) that I would attend
before / after work to be around people who are doing career switch to coding.

Then when students learn enough and have strong foundations that I look for, I
recruit them to work on open source side projects that benefit new students
who are learning.

For the students who have shown the maturity to build and maintain production
features, I refer them to the company I work at and they interview and
(hopefully get in).

In the past 2 years, I've referred 5 students successfully and they're all
doing pretty really well (I secretly stalk their internal code contributions).

I hope that one day, I could have enough reputation at my company to hire
engineers based on their open source contributions. I also hope that the stock
market doesn't crash next year because I plan to use my RSUs to start a fund
to hire junior engineers students so they don't worry about medical benefits
while they contribute to open source projects and have a hard time finding a
job.

If you mentor a student as they are learning, you can instill good engineering
practices: like taking the time to communicate and write good documentation,
write code and comments to make life easy for the next engineer to take over,
how to handle tradeoffs between time and quality, etc. These are critical
engineering skills that are very hard to test in 1 hours but are very
effective if mentored early on.

------
austincheney
I recommend stepping back from code and focusing on abstract organization and
modeling.

I find it astonishing how many developers absolutely cannot think through a
tree model. I deal with tree models absolutely every day in programming,
management, and personal concerns. For example: the DOM, file system,
company/employee organization, outlines, security models, business/customer
needs, and so forth. All these things are tree graphs and I encounter many of
these EVERYDAY.

Many developers simply cannot explain walking from one point on the tree to
another point. I am not even talking about code. Even in casual conversation I
am astonished at how many times I have encountered people like lawyers, QA,
bosses, and others who can engage this sort of logic while some of my
developer peers are just lost on the subject.

That is absolutely something I tested candidates for as a filter and will
continue to do so. If a person even talking through such a real world common
form of abstract logic how can I expect them to write code? It’s like they
expect some magic wand to simply provide the logic for them and just sprinkle
on a little bit of syntax.

------
codingdave
Ask questions that are open-ended. Don't ask them something with one right
answer - give them a scenario and ask them to show you how they'd solve it.
That is more realistic - if they know an algorithm that helps, they'll use it.
If they don't, they'll try something else. Either way, you are hearing their
answer, not trying to lead them to "the" answer.

------
contingencies
I never stick with formal lists, but often ask something like these...

1\. Tell me briefly about your professional development, including challenges
you have faced, mistakes you have made, how you overcame them and what you
have learned. Have there been any critical mentors or resources? What
motivated you in each position and how did that lead to your current
application.

 _Opens a floodgate of possibilities for discussion, generally gives a sense
of career pattern or path, potentially personality_.

2\. What are you learning right now and why?

 _Screens for curiosity and /or goal orientation_.

3\. Choose any recent technical or business idea you have had and explain why
you considered it to be interesting.

 _Possible insight in to how prolific their idea generation may be, and
whether it is circumscribed by professional domain or not._

With a bit of two-way questioning on these, you should have a basic sense of
their capacity as a communicator as well as personality.

Important: Predominantly co-workers and direct managers should interview, not
random HR departments, external contractors or disconnected tiers of
management.

------
qznc
What exactly are you supposed to figure out in the interview? The skill level
of the interviewee? A list of skills they have? Suitability of their skills?
Cultural fit? Intellect? Knowledge? Compress it to 3-5 points.

Now pick some arbitrary rating system (5 stars, school grades, 1-10 points,
whatever). It does not matter that much which one you pick. The hard part is
to stay consistent about it over time.

Now, here is the way how to make it more pleasant for your candidates. After
3/4 of the time, write down your rating and discuss it with the candidate.
E.g. "I gave you four stars for cultural fit. I get good vibes with you but
you would be the only PhD in our startup, so you are probably more academic
than everybody else." Give them a chance to correct your rating. "Oh, I forgot
to mention I'm involved in my brothers construction startup. I feel fine in
that non-academic world."

Giving such honest and clear feedback to candidates is a better experience
than most companies.

------
tboyd47
Very easy process to hire for all experience levels and techs:

Use multiple (2-4) recruiters paid on contingency. Receive about 5-20 resumes
a week from them (total, not per recruiter!)

Filter these resumes to about 3 per week. Interview these three yourself for
30 mins each. Just ask them for more details about technical experience on
their resumes to detect liars. If your recruiters are any good they will have
filtered out people based on the soft skills / career questions.

Reject the liars and accept just one of the honest people (if there are any).
Push the one forward to your team. Let your team loose on the candidate with
any bizarre (but legal) questions they like. Give them the final say on the
hire. Repeat until position is filled.

------
solias99
I honestly think a basic data structures and algorithm round is great.
HOWEVER, the interviewer has to make sure the algorithm isn't obscure enough.
That means no questions like Median of Sorted Arrays, Kadane's etc, where it's
all about previous rote memorization or some form of pattern recognition.

A lot of interviewers get lost with these obscure algorithms, and forget what
they're actually trying to test: the candidate's thought process.

A counterpoint on why I wouldn't ask anyone to implement something (such as a
web server) using a framework: They're ever-changing. Furthermore, if I were
to try and see if candidates understand building an API well, I'd rather ask
them a mix of networking and open-ended, system design style questions.

~~~
snypox
> A counterpoint on why I wouldn't ask anyone to implement something (such as
> a web server) using a framework: They're ever-changing.

Can you elaborate as to why this is a contradiction? Yes, they’re ever
changing but doesn’t mean it’s not an interesting problem to solve.

------
tester756
I do like things like:

How would you design/write xyz

What's wrong with this code [code] (bug, security issue, bad code)

What do you think about

------
jimnotgym
How about discussing some projects they have worked on, and ask them questions
about their role in the process? Have a tick list in front of you of the
skills you need and see if they bring them up. If not you can always ask
specifically about them at the end

------
iamAy0
Disclaimer: I have only interviewed for junior positions.

I never focus on algorithms or particular technologies. When I interview
someone, I'm looking for two things:

\- First and most important: we both are in the same wave. I want to make sure
the applicant understands what's the goal of the position. For example, once a
guy with over 10+ of experience applied for a junior position but demanding
responsibilities and salary of a senior.

\- Second, expose a general problem and work with the applicant to get an idea
about what can we do. I don't expect a solution in 5 minutes, I'm looking for
ideas of how we can tackle this problem and how we would start working on it.

------
tacomplain
My approach is to provide 3 topics of conversation (no right answers just
talking): Let's say the person is applying to be a dev in product P. 1 how
would you build an mvp for P? Which stack, what are the main issues, how would
you deploy etc (usually I ask if it were a side project , so I can see which
technologies are in the intersection of fun and reliable for this person)

2 what would be a moonshot feature for P? (If you had a perfect AI lib or
unlimited processing power or devices spread all over the world)

3 if you had to find someone to take over your job (for you to manage the new
one), what would you look for?

------
janbernhart
Anything you do will raise criticism. This doesn't mean you shouldn't listen
to it, but don't make avoiding criticism the main goal of your approach.

------
dennis_jeeves
Some tips on what NOT to do.

-Don't ask questions designed specifically to gauge how the candidate 'thinks'. If you do pose such a question have another interview with the candidate some days later so that the candidate can think though various solutions.

\- Don't ask questions which are cliched like 1) Tell me about yourself 2)
Tell me your strengths and weaknesses

-Don't continue interviewing a candidate if you have decided not to hire him/her.

What you can do:

\- Ask them to look up an answers using google/search

\- Paid assignments

------
furstenheim
For the later interviews, once you're left with a couple of candidates to
decide from, I like product driven interviews. You choose a complex business
requirement that is already implemented.

You discuss different architecture options, feature trade offs...

It's easier for both parts to connect, it will give you a glimpse of how the
candidate faces requirements from other teams and you give him a taste of the
inner workings.

------
tootie
It completely depends on what your company does and how it does it. My past
jobs never did algo interviews because we don't do algo work. It's certainly
relevant to ask CS topics for a job that will require you to apply CS topics.
Short answer, ask things that are relevant to the job function. Both in terms
of core competencies, but also personality, teamwork and communication.

------
natch
I haven’t interviewed at Tesla, but I like the question they ask on their
recruiting web site. Might not be an exact quote but they ask “what have you
done?” and they make it clear they are looking for evidence of something
outstanding, whether it be effective effort, creativity, or some impressive
application of useful skills/drive/etc.

------
mrwww
The company i ultimately chose to work for had a take-home assignment setup.

It was an easy assignment that covered a lot of concepts, and it was followed
by a 2 hour techinical discussion on it and all things related and unrelated
with my future colleagues.

Outside of that the interview stuff was just around personality and culture
fit.

------
sloaken
Ask them to describe / explain the work they have done, how they did it, and
what their participation was.

The algo interviews are partially to test can you do the stuff. If you have
done similar and can describe it in sufficient detail, then you know it.

------
ochronus
First: I'm so happy to see you're trying to change things for the better!
Second: could you share more context around "you have no control over the
format or time"? Could you describe the current process/format?

------
doh
We have very standardized process at Pex ([https://pex.com](https://pex.com)).
It's not perfect and we are always trying to improve it, but it works quite
well at this point.

First couple of rules:

\- quick turnaround. We want to give an answer to a candidate within a week
they engage us. If we make an offer, we give them plenty time to sign the
offer, even shop around, before they make the decision

\- no step can be repeated or returned to. If a candidate progressed from step
2 to 3, our hiring managers are not allowed to go back to the previous step

\- we disclose as much information as possible early so there are no surprises

Here is the process:

1) Initial 30 min call with one of our recruiters. We cover mostly the
company, the compensation package for the role (usually range based on the
level they fit in) and our expectations. We also cover the culture and share
as much as we can so the candidate can make informed decision if this is a
right fit. We ask some high level questions, but usually focus on the company
itself.

2) Take home challenge. We have one for every type of position, from
engineering to marketing. They are usually simple and fairly arbitrary
problems that illustrate candidate's thinking about problem, how they analyze
it, what tools they choose to solve it, etc. When we design the challenges, we
are aiming for it to take around 30 minutes. We are not trying to get the
person to any significant work for us, rather just figure out how would they
solve problem if it was given to them at their position.

3) The candidate meets 4 people:

\- their manager where they cover the role, the expectations, the growth at
the company and anything related to their job

\- random colleague. Essentially someone they will not work directly with. We
use this person to help us to focus on the candidate themselves rather than
their skills

\- our COO, who covers with them more high level topics like what they
did/didn't like about previous employments, what they care about in working
environment and such

\- our CEO (me). I usually focus on their individualities, especially passions
and hobbies. I want to see that they care about something, anything. I also
answer any unanswered questions from vision, company's cash position,
customers, how we generally work, etc

As I mentioned, we try to get an offer to candidate's hands within a week.
With this process we get around 95% offer acceptance rate.

------
capkutay
Instead of treating interviews like an exam where you measure someone based on
number of correct answers, use it to gauge their problem solving ability. Can
the candidate articulate their thought process? Is there some evidence of
strong fundamental knowledge/theory being applied naturally in that process?

As an interviewer, I don't really care if a candidate happened to know the
answer to an algorithmic puzzle or has a deep knowledge of academic minutia. I
want to know three things: can this person reliably and meaningfully solve
technical problems, can they communicate their findings to the team, and
finally – do they strive to learn the details or do they just
skim/superficially solve problems? Obviously expertise in a job-relevant
domain is great too - but I'd consider it a bonus; the difference between a
senior and a junior/mid-level hire.

~~~
starky
I like your description of the 3 things you want to evaluate. It is pretty
similar to what I try to evaluate.

The first two are part of a quiz we give candidates, it has a couple basic
questions on it that simply require them to logically think through a problem
they likely haven't seen before and provide feedback. I don't even care if the
answers are correct, but they should be able to explain their thought process
and reasoning clearly in a couple followup questions. Bonus points to
candidates that ask for feedback on their answer.

For the last point, during the interview I get the candidate to describe a
project at a recent workplace, once they have given the description, I'll ask
questions to see if they evaluate and learn from their previous work.

I am constantly shocked at how many candidates I interview that fail one or
more parts of this spectacularly (especially senior level candidates), to the
point where I question whether evaluating candidates these ways is unfair, but
also knowing that if someone doesn't do this they likely are going to fail
when given a problem without an obvious answer.

------
techslave
casual culture talk with 2 people, then a 2-5 day contract. only works for out
of work candidates of course.

~~~
jasonpeacock
Don't do "culture talk", thats where unconscious bias lays and you will end up
hiring people that are like you.

There are many awesome people you can hire that you wouldn't hang out after
work, and that's OK - you don't have to be friends with all your coworkers,
you just have to be able to work together.

Instead, identify qualities & behaviors that your company values or are
exhibited by successful employees and ask for examples of the candidate
demonstrating them.

~~~
autarch
"Culture" is a really broad word. I agree that "culture fit" is often cover
for "let's hire people who are just like us because I'm more comfortable that
way".

Instead, I think we should attempt to really dig down into elements of
"culture" and identify the ones that are relevant to the job vs those that are
not. For example, some relevant elements might be:

    
    
        * Preferred development process.
        * Pace of delivery and willingness to ship bugs.
        * How disputes are settled.
        * Using libraries vs build it in house.
        * Process for adopting new technology.
        * Preferred communication methods. Email vs chat vs video calls vs in-person (back in the pre-plague times).
    

These are all important topics to address in order to ensure that the
potential hire will be able to work effectively in the position, and that they
won't be miserable.

And there are plenty of things that are _not_ important:

    
    
        * Hobbies and how people spend their time outside of work.
        * Favorite films, music, books, food, drinks, etc.
        * Whether you work on side and/or FOSS projects (unless there's a legal issue with side projects).
    

All of these things, on both lists, could fall under the nebulous "culture"
umbrella. So let's stop using that word and start talking about specifics.

I'd also note that for some of these things I expect the candidate to ask _me_
about them if that's something they care about. In fact, those sorts of
questions are a very positive signal to me. It shows that the candidate has
some insight into their own work and what makes them happy and productive. I'm
very happy to find out that this isn't a good fit _before_ they've started
working with me!

~~~
returningfory2
I think it's so tricky though. Take:

> * How disputes are settled.

Suppose you pose to the candidate a situation in which they are in the right,
and ask them how they would proceed. They respond by giving the impression
that they wouldn't easily compromise on their correct point of view. Is this
positive or negative?

For many interviewers the answer will unconsciously depend on the candidate's
gender. If the candidate is a man it's positive because they are "assertive"
and "stick to their guns", whereas if they're a woman it's negative because
they're "pushy" and "unwilling to listen to other points of view".

~~~
autarch
> Suppose you pose to the candidate a situation in which they are in the
> right, and ask them how they would proceed. They respond by giving the
> impression that they wouldn't easily compromise on their correct point of
> view. Is this positive or negative?

This should, ideally, depend on your team and your organization, right?
There's no one right answer here.

I agree that there is an opportunity for bias here, but that goes for pretty
much any question you could ask. You could ask them about programming
languages they've used and find out they've used a lot of different languages.
And the bias response would be to think that for men this means they're
inquisitive and curious, but for women this means they're flighty and unable
to stick with anything.

One thing that I think can help is to talk with the hiring team in advance
about what you hope to get out of each question. For example, in asking about
how you'd handle a technical dispute, you could focus on whether they would go
directly to their teammates or manager before going to the next level up in
the management chain. And on the flip side, you may want to make sure that
their answer isn't "if I'm right I will never let it go because it's important
to be right." By focusing on a few key points you're looking for, it may (I
hope) limit the potential for bias.

But it's legitimate to ask about non-technical soft skills in an interview.
Those are a _huge_ part of doing any job that involves interacting with other
people. And conversely, there are people who are hugely destructive to a team
or organization because of the lack of such skills. I don't think the answer
to bias is to simply avoid any topic where bias could come into play.

------
cool-RR
Ask them to implement a common Linux utility like cat or ls.

------
gravypod
There's no "correct" interview because each company is filling a different
role for a different culture. In the past I've done the following and it's
worked, and is working, pretty well for the types of roles and cultures I was
filling for.

1a. Optional coding test at home (no time limit)

1b. Optional coding test in front of me (1hr time limit)

2\. Very simple fizzbuzz-level problem. Essentially can you break down a
problem with an if statement and while loop. This has _the least_ bearing on
the process.

3\. System architecture/design "whiteboard". We ask a question that's
essentially an entirely different problem space from out company but who's end
system design, and lifecycle, looks almost exactly as the design our company
took. Essentially if you `s/A/B/g` you'll have a 1:1 idea of what our
company's infrastructure looks like by the end of the problem. There are
absolutely no wrong answers, just answer how you would if you were given this
ticket ("design X") and I'll ask from time to time "how many QPS can this
handle", "any bottlenecks you can identify", "how much bandwidth/cpu/memory
will this take", etc to see if they understand the implications of their
decisions.

Surprising facts:

1\. Most people can do the take home coding test "enough" but do horribly on
the on the phone coding test. This has been shocking to me because I'm not
someone who struggles with this aspect of the interview process (I'm horrible
at explaining myself).

2\. When we get into the system arch/design session 99% of people have no idea
how to proceed. They've never sketched out the design for something in front
of someone else and have often never thought about latency, memory, CPU, disk,
etc. We had one engineer take it who couldn't even come up with a way to
estimate how many QPS a service might receive in the worst case. Another
engineer settled on a design that was going to cost >$1mill/month in AWS DB
costs and could not imagine any way to reduce this bill and thought this was
reasonable (just needed to put a cache node at an edge location).

3\. People use buzz words. They use them a _lot_. If I had a working feature
for every time I've heard someone just say "autoscale it" whenever they hit a
bottleneck during the arch/design section I wouldn't need to hire an
engineers. It shows a core lack of understanding of where the bottlenecks in
our interview question are coming from.

4\. We see some (small) percentage give up on doing the coding test. I'd say
it's less than 10% of inbound leads. Obviously a significant portion of people
don't want to code in front of someone and don't want to code at home.
Unfortunately I can't advocate for hiring someone until I see code that
they've written. We've had horrible experiences hiring "senior" engineers who
actually don't know how to program.

5\. I've had people freak out, and start cursing at me, when we get to an
interview question they get stuck on. It's something I'd never have imagined
happening in a professional setting but, at least, they've been filtered and
won't be working with me in the future!

6\. Resumes and pedigree have the worst correlation to quality of the engineer
I've ever seen. I've seen undergrads working part time in the summer output
10x value to full time engineers with 5+ years of experience. "10 years of
experience or 1 year 10 times" is the most accurate pattern to describe this.

~~~
awalsh128
I like the design question idea for engineers with some YOE or seniors. Hear
you on Pedigree. I interviewed a PhD from Princeton and they couldn't analyze
a basic design question.

~~~
gravypod
> PhD from Princeton and they couldn't analyze a basic design question

Yea, it's a very different mentality to get yourself into. I find the system
design question to be the most fun. Anyone can learn to code, anyone can learn
your style guide/convention, etc. Very few people can be:

1\. Given a problem ("{A client,we} want...")

2\. Come up with a solution ("Ok, we'll be able to...")

3\. Implement that solution with no oversight or assistance while only asking
for pointers on where existing things are defined ("I'm trying to X, Y does X,
where does that code/config live?")

I mostly work at small startups that need to improve their engineering quality
so most of the roles I am attempting to fill are engineers who can be a
massive value add and lead projects on their own without much hand holding.

If I was instead working in a company where 99% of the work was maintenance,
phone calls, and busy work I'd probably forgo this screen-er. I'd _likely not_
work for that kind of company but it's just something to keep in mind.

------
RandoHolmes
Give the person 3-5 presentation topics they can choose from and have them
come in and present to your company (non-technical & technical). Afterwards
hold a Q&A where the technical and non-technical identify themselves as such
and then start asking questions.

It allows your company to learn, to get a feel for the person's ability to
communicate, and to stay away from stupid things like tests.

It's also easily the best interviewing process I've ever experienced.

\---

edit: Since it appears that multiple people have taken this to represent an
hour+ long presentation, let me clarify here.

A better way to think about this would be a long-form discussion about a
topic. The initial "talk" would be 5-10 minutes, with a Q&A afterwards to ask
for clarifying questions.

I've been working in this field for over 25 years. I spent roughly 30 minutes
of prep for the interview and it was easily the best interview process I've
ever experienced.

~~~
hackinthebochs
Unless the job is mostly giving presentations, it seems like its optimizing
for the wrong thing. For all the talk about software development being a
collaborative field, for most people it really is still mostly isolated mental
work.

~~~
mlthoughts2018
All modern software jobs are mostly about explaining your solution to others
(stakeholders, reviewers, senior/staff engineers, managers, executives).

If you can’t boil your work down into accessible presentations and adapt them
for technical or non-technical audiences, you won’t get anywhere.

Nobody will agree to help you, allow your team to grow, give you budget for
tools you need, make compromises with you around integration requirements or
shared design patterns, unless you can frequently convert your software work
into highly professional presentations.

I’d argue that competent presenting and writing skills are of equal, or even
greater, importance than software skills or expert knowledge, whether for a
research software job, software contractor job, SWE within tech / ecommerce /
banking etc industries - across the board there are pretty much zero types of
software engineering jobs in which software skills are actually the most
important thing. Presenting and writing are frequently much more important.

~~~
username90
> All modern software jobs are mostly about explaining your solution to others

This isn't true in my experience, there are plenty of well paid jobs where you
don't have to talk to non-technical people. For example, lets say you are
developing infrastructure at Google, how often do you think you talk to non-
engineers? Never, everyone in your management chain will be engineers, your
product managers are ex-engineers, your users are engineers etc. This is true
for directors managing lots of people as well as well.

And even when you develop services much of it is just getting the technical
parts right so you can work there as well at Google with very little time
spent talking to non-technical people, so senior positions there are mostly
about technical stuff.

You could argue that Google and other big companies are a special case,
however if we exclude low paid jobs where you just earn half of what you'd do
at Google or Facebook then pure technical work like this is a very significant
fraction of the market. I'm not sure why I'd work on my presentations skills
so I could get hired by a company paying me less.

~~~
mlthoughts2018
> “ For example, lets say you are developing infrastructure at Google, how
> often do you think you talk to non-engineers?”

I can speak from experience since I help run the site reliability incident
response team in my (large ecommerce) company.

How often do rank and file SRE ICs need to speak with non-technical staff
about projects? All the time.

They give tech talks and council presentations, they contribute work to
quarterly roadmap project pitches, and they give business scorecard
presentations to walk higher level managers through the breakdown of costs and
benefits, why we should pay money for a new cloud tool, what time & effort
some org-wide planned migration or upgrade is going to require.

Let alone basic stuff like writing effective tickets / issues / RFCs / PR
descriptions, postmortems, etc., and walking teammates through work artifacts
daily.

For every two hours of software work, I’d expect 1 hour of “wider audience
communication” work - and that is for deeply technical ICs in roles like SRE.
The closer you get to product engineering, the more the ratio shifts towards
communication.

I can tell you this translates specifically to Google & Facebook - because
most of the SRE colleagues in my company came from Google & Facebook, and
attest to that being the case there too.

~~~
username90
I worked as a SWE at Google for several years on two different teams, I never
talked about work with a non engineer, and one person I worked with talked to
non engineers. SWE's outnumber SRE's by something like 10 to 1 at Google so my
experience should be more common than the people you talked to.

~~~
mlthoughts2018
Your described experience is so wildly different and long tail uncommon from
the hundreds of other ex-Google SWE colleagues I’ve had over the years that we
unfortunately just have to exclude it as extremely unrepresentative.

