
Ask HN: What should an ideal developer interview process look like? - rvivek
We have always complained about the developer interview process being broken. According to you, what does an ideal process look like?
======
Puer
My most enjoyable interview was for an internship in college. I had a take
home coding challenge where I had to write some simple code to fetch
information from an API using whatever language I liked. I was given a week to
do it, but it only took me an hour or so to meet all of their explicit
requirements. I liked that there was no time pressure in that regard.

After the week was up I went into the onsite and in the "technical" portion of
my interview two engineers went over the code I had written for the assignment
and asked me about design choices I had made and what I would do if constraint
X was added or feature Y. It was all very open ended and much more of a true
discussion than an interview which I really appreciated.

I think this kind of format is ideal for interviews. The assignment
requirements were simple enough that you could fulfill them easily and in your
most comfortable language without any time pressure, but you could also go
above and beyond and show that you really knew your stuff. For example, in the
instructions they didn't explicitly ask for error handling on the input, but
both of the engineers I interviewed with really liked that I had included it.
You weren't penalized if you did just the bare minimum because you had the
opportunity extend and build on the assignment during the onsite. I felt
enabled to showcase my knowledge and justify my design decisions and that that
effort was rewarded.

~~~
ldite
With this approach you'll miss good senior people; I have dependants and
_very_ little free time outside work. Last time I was looking for a job I
skipped everyone who wanted me to do prep/homework. If the other interviews
hadn't worked out I'd have gone back to them, but I didn't need to.

~~~
chrisseaton
> With this approach you'll miss good senior people

Aren't you already spending many hours researching a job as the part of the
process? Aren't you already taking a full day off work for the interviews? Why
is that time all fine to spend, but not fine to spend a couple of hours on
doing an assignment?

~~~
ldite
Not really. I've never needed to research a job beyond a bit of
Linkedin/glassdoor stalking and reading of corporate guff.

I've never had a full day interview for a company I'd actually want to work
for (based on what I've learned while grinding through the interviews) so I'm
beginning to take requests for those (along with requests for advance coding
exercises) as a recruiting smell.

~~~
scarejunba
Interesting. Would you mind sharing in which country you work (or if America,
which state), what your field of interest is, and how long you've been in the
business?

My team has full-day interviews and I'm curious to see who has been self-
selecting out of this.

~~~
sandos
Wow. I live in Sweden, and I have the same expectations: I would not really
want to spend much time on an interview process, and I have never, ever heard
of day-long interviews either!

We have had a perpetual shortage of developers though, it sounds much harder
to land a job in the US.

~~~
scarejunba
To clarify, full day means 4 sessions of an hour each bridged by lunch (1 hr).
The hope is to give the candidate sufficient information about the team while
the team gathers info about the candidate.

~~~
ghaff
That sounds pretty typical. In practice, it's not unusual to do one or more of
the interviews by video because of travel schedules but four "in person"
interviews is probably the norm.

~~~
scarejunba
Yeah, that’s how I understood it, but there’s often opposition on HN to that
process.

~~~
ghaff
I guess if you're hiring junior people to stick in a standardized box, it's
mostly just a numbers game. Way back when I got a job offer out of school
strictly based on my resume (took another job).

But even when the interview process was mostly a formality because I knew many
of the people and they knew my work, I still had some on-site interviews.

I'm pretty sure on-site interviews are the norm for the vast majority of
professional jobs. Including, or perhaps especially, for the most senior
positions.

------
weliketocode
Many people here will disagree with what I'm about to say, but if you're going
to give a coding challenge...

I _like_ hackerrank/codility style coding challenges.

 _gasp_

They're timed. They're run against a standardized test suite. And they have
support for many languages.

It's really unfortunate the number of companies that give custom take home
coding challenges that run the gamut. The problems with which are manifold:

\- Inconsistent technology and language choices by candidates

\- Time spent varies wildly and is easily underestimated by companies - and
4-5 hours for an unpaid coding challenge in an early round with a company is a
significant investment

\- Good candidates will de-prioritize your company against others that move
faster

I know that I only answered a small subset of the question, but it's very
difficult to lump all developers together and come up with a single,
wholistic, and ideal interview process. From the nature of the role
(IC/Lead/Manager) to the technologies/specialty to the company's mission, the
process will need to be customized to some degree.

However, do not think that your custom coding challenge will somehow be the
perfect key to assessing developers.

~~~
azangru
When we were looking for a developer for our team, we created a set of small
coding challenges testing the candidate's knowledge of the most relevant
aspects of the language. We wrote automated tests for these challenges, and
the main bulk of our interview was having the candidate solve them.

The advantages to that approach, as I see it, are:

\- the developer is tested for knowledge that is most relevant for their
current job (as opposed to writing general-purpose algorithms)

\- the developer is showing their familiarity with a testing environment

\- writing working code in a code editor seems to me more practical than
writing (pseudo)code on paper or on a whiteboard

There are, of course, disadvantages to this approach as well. The developer
may have hard time working on an unfamiliar machine and not in their favorite
code editor (though what we had was quite standard). This might be a tad
stressful (although I don't know how this compares with solving a general
coding problem on a whiteboard, or with a task asking you to think not just of
a way to solve a problem, but of a performant way too). And it made us focus
on the candidate's coding skills too much (which some hiring managers might
find objectionable).

~~~
mcv
The advantage of using something like codility, or making it a take-home
thing, is that they can do it in their own time, free from the stressful
environment of an interview.

~~~
pytester
I _hate_ doing take home projects.

* It's _way_ too easy to set a project that takes too long. I was given one the other day that would have taken about two to three full days. That simply doesn't happen if you do the test sitting in front of them - the time spent (and potentially wasted) is _always_ capped because the test setter doesn't want to waste too much of their own time.

* Take home projects inevitably have issues ("bugs", if you will). You can't just "resolve" those problems if the person who set it isn't sitting in front of you.

* Take home projects take employer skin out of the game. It's _way_ too easy to give out twenty tests and ignore ten of them. I was actually half way through a homework test the other day when I was told that _surprise_ they had an open offer and the candidate had just taken the job. I doubt they'd have wasted their own time testing me IRL knowing what they knew.

From the employer's perspective:

* In the last case I also realized after googling that the answers were online and I could just have copied someone else's.

* You lose a valuable source of feedback which allows you to iteratively improve the test if you're not watching the candidate while they do the test. I suspect this is partly why the take home tests I've been given are generally of a lower quality than the in person.

* It's valuable to talk through the solution with the candidate while they do it (describing trade offs they made, pitfalls they're cognizant may crop up, potential downsides to their approach).

~~~
mcv

      > It's way too easy to set a project that takes too long. 
    

They do often take longer than they claim. That's true.

    
    
      > Take home projects inevitably have issues ("bugs", if you will). You can't just "resolve" those problems if the person who set it isn't sitting in front of you.
    

In my experience, you're usually assigned someone you can call or mail with
questions. Asking questions is good. Though I usually figure it out on my own.

    
    
      > Take home projects take employer skin out of the game. It's way too easy to give out twenty tests and ignore ten of them
    

Only if they do it wrong. The right way is to give this assignment only to
candidates you will definitely want to hire, unless they produce terrible
code. The candidate doesn't mail in their code, but presents it to a group of
developers who ask questions about it.

This gives the employer skin in the game: they're not going to keep their
developers off their work for dozens of candidates who are probably not going
to make it; they only bring them out to assess a very likely hire.

I guess some employers use these coding assignments badly. You're totally
right to avoid those.

    
    
      > I could just have copied someone else's.
    

But could you have been able to explain the choices of that someone else?

    
    
      > You lose a valuable source of feedback which allows you to iteratively improve the test if you're not watching the candidate while they do the test.
    

Maybe, but to many people, coding on the spot is more stressful than it is at
home. Although I've also done some really enjoyable pair programming on what
was probably my only long interview day (which had an interview, pair
programming, and a game to get to know each other).

    
    
      > It's valuable to talk through the solution with the candidate while they do it
    

But it's also valuable to talk through the solution after they've done it.
That's when they have made those trade-offs.

------
lkdjjdjjjdskjd
Respectful of my time. Think of dating - if after 3 dates, you still are not
sure if you like somebody, maybe it is better to call it quits. I had
companies call me in for more than 4 interviews, distributed over several
days. Not respectful.

Coding: I actually like coding tests in interviews. Then I know my potential
future colleagues will also have had a coding test. The company cares at least
a little that my colleagues can code.

Just don't take the coding test too seriously. Don't expect me to write
correct syntax on a whiteboard, for example. It should be more about having
the right ideas, or even be able to discuss possible approaches if a solution
isn't obvious immediately.

Also, make it a little challenging - no computing of Fibonacci numbers or
FizzBuzz.

I liked one where I was asked to implement Quicksort (not that original, but
OK), but they also gave me a definition of Quicksort along with it. That
seemed fair.

~~~
JustSomeNobody
> Just don't take the coding test too seriously. Don't expect me to write
> correct syntax on a whiteboard, for example. It should be more about having
> the right ideas, or even be able to discuss possible approaches if a
> solution isn't obvious immediately.

This is important. I'm nervous. I don't know you people. And modern
IDEs/Editors fill in most of the boilerplate for functions/classes, etc as
well as help me recall library function names.

Be respectful of my time and don't expect me to practice whiteboard coding for
a month just to interview with you.

~~~
sandoze
I’m fully convinced that 70-80% of my energy and brain function during an
interview is spent on listening, formulating responses, breathing and in
general trying to interact in a social situation with people I don’t know.
That leaves very little to solve problems that I’m not prepared to speak
about, usually a riddle or coding puzzle that has no bearing on my day to day.

As developers we are a bit introverted so being in a room with 2-3 people
knowing that after this there will be another 2-3 people for round 2, is very
difficult. As an interviewer I’ve watched candidates open up talking about
what they know and why they enjoy being a developer and completely fall apart
(sweat, shake, refuse to get out of their chair) when you hand them a dry
erase marker and show them the white board.

------
mrfusion
Check out my open source code. Call some of my references. Verify my past
employment and make me an offer.

It’s really that simple.

If 3+ professionals are willing to vouch for me and I have 20 verified years
working at major tech companies, do you really think I’m a guy faking I know
how to code?

~~~
arnvald
As a hiring manager: your coding skills are just one of many things I look at
when I decide whether I'll make an offer or not. I need to think how you'll
interact with the team, whether our company culture is something that you'll
enjoy or not, whether my team will become a better team when you join us.
Maybe you're defensive about your code and do not like having it reviewed? In
that case I feel you won't fit well a culture with a heavy emphasis on code
review. Maybe you enjoy pair programming and working in a place where it's not
a common practice won't let you do your best?

I wish it was as simple as checking your code and references, but I think it
is not.

~~~
azangru
Let's see the questions you listed:

\- how you'll interact with the team

\- whether our company culture is something that you'll enjoy

\- whether my team will become a better team when you join us

\- maybe you're defensive about your code and do not like having it reviewed

\- maybe you enjoy pair programming

As a hiring manager, how do you find answers to these questions?

~~~
arnvald
These were just example questions, for example pair programming might be
irrelevant, if it's not a significant part of team culture. I admit some
questions here are tricky. It's hard to predict how candidate will interact
with the whole team: maybe they're very respectful to me, but they will look
down on junior developers?

Here are a few things I do that help me better predict whether candidate will
fit the team:

\- ask candidate to do coding task at home and later discuss the solution
during face-to-face interview. While discussing the solution and suggesting
improvements I can see how candidate reacts to feedback, whether they can
explain why they came up wit this particular implementation, what do they say
when I suggest an idea for refactoring etc.

\- I like having another team member with me during interview. That person is
an observer and they help me review candidate's performance (e.g. it happened
that I had an impression that candidate did quite well, but my observer told
me that I was often helping candidate with their task, which I had not
noticed)

\- lunch with candidate might be helpful. I take 2 team members with me and 4
of us grab some food out of the office. This helps candidate meet potential
teammates and talk in a more relaxing atmosphere, so both sides can see how
they feel about each other.

\- one more thing I do is during initial phone call when I tell candidate
about company, at first I give a very brief introduction, then before I say
more, I ask whether candidate has any questions. This gives me a lot of
insight: some candidates ask advanced questions, some ask very basic ("so what
does your company do?" happened a few times!), some focus on product, some ask
whether they'll be able to learn a particular skill. Some candidates don't
have any questions, which is usually a red flag. These questions themselves
are not enough to decide that candidate won't fit, but they're helpful.

\- I ask candidates what they would like in their new job to be different than
in the previous one. One candidate tells me they want to focus on one
particular JS framework. If in my company we chose libraries and languages on
per-project basis, she might not like it. Counter example: candidate that
wants to learn multiple new frameworks might not feel well in a place that is
conservative when it comes to introducing new libraries

------
morcutt
\- Clear timelines on the interviewing process. Some companies show no respect
for your time. Telling me to pop by the office to "chat" at a specific time
and then telling me your agenda for a 4 round technical interview on the spot
is not going to inspire any sort of trust.

\- Ask me questions that actually apply to the job. If I'm building an iPhone
app, please don't ask me to run through a gauntlet of white-boarding/coding
challenges that don't apply to the job. Have me walk through something I've
built or talk about how I would build a theoretical iPhone app. Most of us
aren't building software for self driving cars so please quit acting like we
are.

\- If I'm writing code or solving problems, give me an ideal scenario to do my
best. Let me use my own computer (not one that you just handed me setup to
your preferences.) Potentially let me do it at home (where I am comfortable
and not in an unfamiliar place.)

\- Gauge confidence on the technical stack

\- Don't say we'll be in touch shortly and then ghost. You can say goodbye to
someone without false promises. I'm likely to tell fellow developers about how
the process went.

\- Have people with decent social skills interview

\- If I ask questions like, "What should I be prepared for?" please give me a
basic agenda or guidance. It'll go much better for both of us.

------
cjCamel
I find the move to take home tests extremely unfair. Anyone with dependents
will typically need to block out most of a day of a weekend, or risk doing it
in intervals through their week.

Got a decent CV? Good luck trying to juggle applying for more than 2
interesting opportunities at once.

We mostly hire full stack web devs. IMO It's impossible to really test the
abilities of each candidate across the changing landscape of front end, back
end, DB & devops tech within an interview process that doesn't use a vast
amount of time for everyone.

Instead, we don't whiteboard or code at all in our process at the moment, and
try and get it all done in a face to face hour or two by:

* Taking their experience at face value. Examples: If they have been coding for a couple of years, don't waste everyone's time with fizzbuzz. Assume they will be able to adapt to our source control system, if they have been using a different one.

* Insisting on real world examples when asking competency questions.

* Asking generic questions about code, such as "What is clean code?", "what should you take into account for password security for a web app?", and looking for their ability to communicate as much as their actual answer.

* Looking for areas of strength and weakness to compare across candidates, rather than trying to catch them out.

* Scoring highly for enthusiasm, flexibility and a willingness to learn over pure technical knowledge.

I appreciate this approach wouldn't work for all organisations but we've done
really well out of it.

~~~
drakenot
I’m never dropping the Fizzbuzz style problem. It has absolutely blown me away
the number of people who market themselves as senior people who can’t perform
the most simple or basic coding tasks.

No fancy algorithms. No obscure data structures. Just simple loops and
conditionals in any language.

It has been the most effective (and most depressing tool) I’ve seen in
eliminating the myriad of fakers and unqualified people.

~~~
mrfusion
I think it’s more that people get really nervous in interviews. And they’re
trying to use the social part of their brain and the coding part at the same
time. It’s not normal.

------
mbesto
The ideal process is having one that actually exists.

Most startups basically try to wing this without any definition of how they
want the interview process to work. IMHO this is about 50% of the problem. On
the flip side, large companies appear to be overburdened by process (anecdote
example - I've heard from several people that getting into Google takes 6
months on average, along with the notorious b-tree whiteboard process).

That being said, I think Aline Lerner[0] and Triplebyte[1] have some good
ideas on the topic:

[0] - [http://blog.interviewing.io/](http://blog.interviewing.io/)

[1] - [https://triplebyte.com/blog](https://triplebyte.com/blog)

I don't think implementing either one of their services is a silver bullet,
but are likely n% better than what most companies are doing.

~~~
badlucklottery
>that getting into Google takes 6 months on average

I can only speak to my relatively recent experience, but it was <5 weeks from
application to offer for me. No idea if that's typical.

------
james_s_tayler
An interview is designed to answer three questions.

1) can you write software?

2) can we tolerate writing software with you?

3) can you tolerate writing our software?

~~~
Etheryte
This is a great brief summary. Currently, most interviews put very heavy
emphasis on 1, slightly touch 2 and often forget 3 altogether.

~~~
conbandit
I think the idea about skipping 3 is that the applicant is choosing to apply
to a particular company. That's implicitly saying that they're willing to work
on that company's code. I don't know how valid this is nowadays with the
common practice of throwing applications at the wall and seeing what sticks.

~~~
james_s_tayler
Well 3 is the job of the applicant to figure out. That's about having the
right questions on your end.

------
mikekchar
I have a slightly related question. Imagine that you were being head hunted by
a company and that they've decided that they _definitely_ want to hire you,
even before the interview. Let's say they can afford to pay you whatever you
consider a "reasonable" salary (not high, but not low either).

How would you like the interview to go so that you can decide if you want to
work there?

Interestingly, _I_ would like to see a variety of people that I might
potentially work with. I'd like to pair program with them. I mean, _really_
pair program -- with each pair writing some code. I'd like to write some code
and see how they react to it. I'd like to see them write some code to see how
they approach it. I'd like to work though some simple conflicts to see how
they react.

I'd like to see some code (and I'm willing to sign an NDA). I'd like to work
through it with some of the people who work on it and ask questions. I'd like
to see how firmly they hold on to the existing code and how open they are to
new ideas.

I'd like to talk to a few people in management. I'd like to see a few plans
(again under NDA). I'd like to see what kinds of statistics they collect and
how they think those statistics help them.

Finally, I'd like to talk to management about how they do reviews and see some
statistics about attrition rate, typical pay raises per year, how many stocks
_actually_ vest before people leave (yeah, yeah, NDA).

That would be awesome.

~~~
EliRivers
I'd want to see the kitchen, or wherever people prepare / consume beverages
and food.

~~~
sixstringtheory
Funny you bring up kitchens, because I always bring up the _stage_ concept
from my time cooking, where you actually go in to work a real shift to see
how, as I call it, the “dance” goes.

My favorite interview process is where I’ve done a screen interview or two and
then do real work to see the process as it exists. Everything else is
advertisement and I’ve grown to distrust the evangelizing people do about how
they work.

------
klenwell
I notice a lot of these answers focus on the technical or coding challenge. As
a hiring manager, that's critical, but technical competency only accounts for
about a third of the qualities my team is evaluating in a candidate.

I've given this question a lot of thought over the last couple years as I've
lead teams that have needed to get organized and expand quickly. Here's my
summary taken from a Google Slides presentation I put together titled "Hiring
in a Time of Cargo Culturalism".

It starts with Principles and Guidelines:

\- Hiring cycles will be structured and as short as possible.

\- When we start a hiring cycle, we will finish it by hiring the most
qualified applicant who accepts our offer.

\- Every applicant will receive a response within 48 hours and be updated on
the status of their application at each step asap.

\- The hiring process will be as transparent as possible.

\- Objective and fair-minded measures will displace biased and bigoted ones.

\- Every applicant will appreciate their experience, even the rejected ones.

\- The process will be agile and adapt over time to improve and meet the
specific needs of the organization.

\- Onboarding will begin with hiring.

Then an outline of my team's current Methodology:

\- A thoughtful and literate job posting will accurately describe the job and
foreshadow the company culture.

\- Simple challenges and honeypots will filter serious candidates from the
applicant bots.

\- At the end of every step, we will inform the applicant what comes next.
Courteous templated responses will be promptly delivered.

\- Two interviews. No more than three. The coding challenge will represent a
genuine work sample. It will be no more than one or two hours.

\- Candidates will be evaluated using a simple quantitative assessment of core
competencies (see Ch. 21 of Kahneman’s Thinking Fast and Slow).

\- Final decision will be a collective decision of the hiring team.

\- After hiring cycle is complete, hiring team will hold a retrospective.

I've hired over a dozen developers this year. They haven't all been homeruns
but no strikeouts either. A few singles or walks. A lot of solid doubles. And
that's mostly what my company needs.

~~~
vtesucks
Your answer shows you are experienced . How do you test for inter prrsonal
skills?

~~~
klenwell
In the final interview, the candidate meets with 3 to 4 of us on the team and
we talk for about an hour. Our team comes in with a set of prepared questions
which address past work experience, company culture, team practices and
relationships. We make sure it's the same group of interviewers for a given
position and we don't robotically stick to the script but try to cover the
same ground with each candidate so we're making fair comparative evaluations.

Also, I suspect our job description and pre-interview process (requesting a
cover letter, asking a couple short-answer questions in a personal message as
soon as we receive an application) succeed in filtering out candidates who
aren't thinking about a job as a social inter-personal commitment.

------
mannykannot
One thing that I took away from my flight instructor is that she had me say
aloud everything I was thinking and doing. If I said, for example, "those
trees might cause a downdraft on the approach", then, if I did not handle it
well, at least she knew the issue was with my execution and not in recognizing
the possibility.

I don't often interview developers, but when I do, I try to get into a
situation where the candidate leads us through solving a programming problem,
and the issues that come up. The most satisfactory case was where the
candidate and I jointly investigated an issue that neither of us had the
answer to.

Unfortunately, this is not an approach that many people have much experience
with, and it seems unfair to penalize people who find it disconcerting, but
when it works, I think it is most satisfactory approach for all concerned. It
is not easy to automate either the practicing for or execution of this style
of interview, which may or may not be a disadvantage.

------
stickfigure
After many years of winging it, I developed what I think is a pretty robust
interview process, inspired by Pivotal's RPI. If the candidate makes it
through a quick 15-30m remote screening, I call them in to the office for
pairing session.

* The session takes place at a pairing station - two keyboards/mice, two monitors (mirrored).

* We work through a fake problem that is relevant to real-world problems that we actually work on. No brain teasers, no complex algorithms. Basic software engineering using some of the tools the company uses.

* The problem is exactly the same for every candidate. This is essential; you need to be able to compare apples to apples. You can't use "real work" because real work is different every week.

* The stuff I look for is pretty mundane - can they name things sensibly? Do I have to push them or do they naturally drive out behavior with tests?

It takes 1-2 hours. It's partly a way of evaluating candidates, and partly a
way of communicating "this is how we develop software at this organization".
Some folks really respond well to it. Some don't. That's ok.

At the end of the pairing session. I know 100% if I want to extend an offer.
I've gotten some really great hires out of this. My bad hires ended up bad for
reasons unrelated to technical ability (like, their life was a trainwreck).

~~~
sethammons
Part of the value of a real pair programming session is brainstorming and
discovering together. That part can't happen in a known problem for the
interviewer. How does that part play in on the interview? I like how you use a
reproducible problem for comparison, a lot of advocates ignore that.

~~~
swish_bob
When I do this, I'm pretty upfront that it's not going to be "real" pair
programming, but you can come up with a pretty close approximation. As you do
it more, with the same problem, you start to understand what questions to use
to keep things from stalling, and when to make suggestions. It's not easy, but
none of interviewing is easy, really. It's another skill you have to learn.

------
oceanghost
Counterintuitively I think the problem with hiring is actually a problem with
firing. Hear me out.

Everyone is so focused on finding this mythical perfect candidate, instead of
giving people a chance. Which is really another way of saying "We're afraid to
fire people."

Culturally, we should hire and fire more freely.

I once got a call from a company that said I was an absolute perfect fit for
what they were doing-- would I consider a 6-month contract-to-hire? I had been
in my job _10_ years and told them no. They were so flabbergasted they called
back and asked if I wouldn't consider it, that the CTH was simply to see if I
was a "culture fit." I told them in plain language that if I was such a
perfect candidate, they could hire me straight out. That I'd been in a job 10
years, was obviously happy, why would I jump ship so they could dangle
employment as a carrot in front of me?

I can't tell you how many jobs I've not gotten because I didn't get some
gotchya question-- or in some cases understood more than the person
interviewing me.

The abusive hours need to end, as does the concept of "culture fit" which is
just a proxy for age, race, religion, binger drinker status, etc.

We need to stop judging people and trying to feel better about ourselves by
dismissing people who can't answer questions we just googled ourselves. Almost
every company I've ever talked to claims they only hire the best candidates.
That simply, cannot be true.

How is anyone supposed to grow if you can only get a job you're an expert in?
And when what you're supposed to be an expert in changes every two years?

The answer is, culturally, you're not.

~~~
chillacy
So instead of giving you a 6 month trial run followed by employment they
should give you a permanently temporary gig? Because a job where they hire and
fire freely sounds a lot like how companies treat contractors.

~~~
thaumasiotes
That's already the way almost all jobs are. Being "hired" isn't a protection
for your job. It just means you're getting automatic tax withholding.

I was a formal full-time employee at NCC Group until just a month ago, when
they fired me with no warning, no explanation, and no severance.

~~~
jinglebells
You can't fire people for no reason in the UK and probably the rest of Europe
as well or you'll end up being taken to court for unfair dismissal. Roles can
be made redundant, but it's a lengthy process.

~~~
gezxfPajuc9x
I think, it depends on your agreement / contract.

------
spectre256
My favorite coding interview ever is still from many years ago when I
interviewed at Pivotal labs.

It was an in person pair programming session, with the interviewer driving.

Most importantly, it was collaborative, whereas many interviews often feel
adversarial.

It also tested how you communicate, since you weren't the one typing.

However, it still tested your coding skills, but because the interviewer was
"on your side" with respect to the desired output code, it wasn't a
dealbreaker or stressful situation if you forgot the signature of an important
method or function.

But it also touched on actual coding skills, while not requiring you

------
inertiatic
After getting burned a few times too many, wasting days of work building
entire applications (at worst) or even simply a few hours (at best), I won't
be doing anything longer than a 20-30 minute fizzbuzz take home exercise. I
will also never waste a day of leave on a company, unless it's the opportunity
of a lifetime and I feel I have an almost guaranteed job offer if I jump
through that hoop.

My ideal interview proccess thus is:

1) Talk to me for 30 minutes (not your HR, an engineer that is trained for
this task). Fizzbuzz me if you have to, but preferably at this point in my
career, don't. 2) One or two onsites that last a maximum of 2 hours and are
early or late in the day, so that I don't have to waste a day of leave. Don't
fizzbuzz me or ask me edge cases on your language of choice in this. Don't
come with a checkbox of things you want to hear. Have a discussion with me and
actually pay attention to the things I have to say that aren't strictly part
of the answer to your question.

~~~
username3
What’s your rate?

~~~
inertiatic
Are you trying to find out how that sort of attitude correlates with pay?

I make what Glassdoor tells me is an average salary for my area and seniority,
but I believe I could do better if I could muster up the courage to interview
heavily and change jobs.

------
wonderwonder
In my current role I had what I consider the best interview I have had to
date. I had a brief non technical video interview with the software director.
Then I was assigned a programming task, essentially to create a site focused
on a specific task using a given set of technologies. Once done, I sent the
interviewer the url to the site and a link to the code on git. The next day I
had a code review and video interview with the original director and several
Sr. engineers to review my code and answer relatively technical questions on
my design decisions and code.

I thought this was perfect, they did not waste much time on the original
interview, essentially just determined I was not insane and would be an ok fit
for the team. Then via the assignment they were able to see me build something
from start to deployment including database development and server deployment.
One of the required technologies was a javascript framework I had never used
before, they knew that but I spent 8 hours (I really wanted the job) reading a
book on it before starting the project and they appreciated it. I had a couple
of options on server side stack and they were able to observe and question my
decisions. In the code review they were able to see how I handled criticism
and reacted to stress.

Whole process from scheduling the first interview to the job offer took 4
days. No white boarding, no obscure technical jargon. They met me (virtually),
learned if I could code to their standards and made a decision. In my mind it
was great. Via the same process they interviewed someone else I recommend and
very professionally and politely turned them down, someone who in my mind was
a pretty solid engineer but he just lacked the specific skill set they wanted.
While creating the requested application, he knew he was not going to get the
offer, which is great as you as a developer are then able to determine for
yourself if you are a good fit for the role. No mess no fuss.

~~~
eli
I've considered this approach, but I worry that it consumes too much of the
candidate's time. That it ends up favoring people who have the time and the
financial freedom to do a full day of unpaid work.

~~~
camjohnson26
I have been interviewing recently and started turning down any coding
assignment that will take more than a couple of hours. To get a reasonable
number of offers to enable you to negotiate requires going through the
interview process for several companies since some percentage of them will
turn you down. After a phone screen I don’t have enough information about the
role to be able to potentially waste 10-20 hours on a toy problem.

A few companies gave a simple api project or a timed hacker rank and those
feel reasonable. An entire web app with database, rest api, front end and CI
pipeline following production code best practices does not.

~~~
wonderwonder
Negotiating a salary is a situation I have never been in as I have always told
them up front before interviewing how much I want to get paid after finding
out the salary range. Whenever a recruiter calls, I ask the salary first and
tell them where I want to be on that range before even sending a recent
resume. Not that negotiating for more if I have several offers would not be
great, just not something I have experienced. I interview, they already know
my number as I have already asked the hiring salary range and told them what I
want. If I pass the interview they offer me my original request and we move
forward. I could see how if one was interviewing with multiple companies and
had multiple offers how one could seek to play them against one another for a
potentially higher amount. Just hasn't happened to me.

~~~
pjc50
First you have to find out the salary range, and a horribly large number of
companies are secretive about this.

~~~
scarface74
That’s the easy part - don’t apply blindly.

I’ve always gone through local recruiters that have relationships with the
hiring manager. They know the salary range, the interview process, and where
other candidates fail because they get feedback from both the candidate and
the hiring manager.

------
steventhedev
As the person being hired: the process should help me reject workplaces that I
would not enjoy, succeed, or grow at. I should get accepted into any place
that satisfies the above criteria.

As someone doing the hiring: the process should help order applicants by the
value they will provide to the company, accounting for future growth and
current abilities.

In both cases it should do so as quickly as possible, to avoid wasting my
time. Sadly, neither of these are really realistic, but starting from these
first principles, you can see a few simple things that can improve the
process: Early rejection in both directions, settings expectations, and that
the interview is a two-way street and while some interview strategies may work
in one direction, they will alienate the other side so quickly they are
ineffective on the whole.

I have some ideas regarding the rest, but nothing that hasn't been mentioned
elsewhere.

------
l0b0
I would give them opportunities to show their skills at real-world tasks:

\- Version control. Given a terminal (or Explorer with TortoiseSomething) and
an existing project, make some simple changes and commit.

\- Testing. Given a simple piece of code, its tests and either a bug report or
feature request, explain at least in high level terms how to move forward.

\- Code review. Give constructive feedback on how to improve any aspect of a
diff.

These tasks indicate an understanding of code and facility with communication
beyond the very basics, and you'll never finish learning them, so I believe
they are appropriate for any non-entry level position. They are also
sufficiently open ended that the candidate has to prioritize getting at least
something done on all of them.

After that, unless they are completely hopeless I'd arrange coffee or lunch
(on company money of course) with at least part of the team they'd be working
with, so they can tell if they are at all compatible.

Needless to say, give the candidate plenty of options for a suitable time, let
them know exactly how long it'll take, coordinate the time with the team, show
up on time with a computer—which the admin team has wiped and you've just had
to copy a few files onto—and find somewhere quiet and comfortable for them to
work.

~~~
maccard
> Version control. Given a terminal (or Explorer with TortoiseSomething) and
> an existing project, make some simple changes and commit.

Does this really tell you anything about a candidate? If you sat me down in
front of a terminal and said "here's a git shell, make a change and commit it"
I'd fail the test immediately. I've used perforce for the last 9 years, and
the terms are not the same, and the commands are _not_ discoverable. So you're
eliminating anyone who doesn't know git basics (which can be taught in 5
minutes, or with a GUI of their choice), or you're giving a test for some
criteria that you won't judge people on, so it's a waste of time.

> \- Code review. Give constructive feedback on how to improve any aspect of a
> diff.

I like this idea actually. Have the person I'm interviewing review a piece of
code (maybe a diff? I'm not sure about different diff formats, maybe it just
needs to be some text files, or maybe they just have full internet access and
the ability to download their own tools), and judge based on that.

~~~
jgforbes
I also balked at it, but for the opposite reason. Everyone should be able to
sit down to a terminal, make a change to a repository, and commit it.

I mean even if you don't know the command (or they are using a different
version control system then you are used to) you can always check the help or
man page. It's still a trivial task.

~~~
maccard
> Everyone should be able to sit down to a terminal, make a change to a
> repository, and commit it

Disagree - You're testing can someone use the basics of git in a terminal.

> you can always check the help or man page.

Assuming you know how to do that, and what you're looking at. Say I'm sat down
in front of a terminal, and I'm told "here's a terminal, make a change and
commit it"

I type in "git", and I get: "usage: git" \- If you're used to using shell
tools, then sure you can make sense of it, If you're not, then you're done.

You said "commit" and the description for that in the "git" command is: >
grow, mark and tweak your common history > commit Record changes to the
repository

Right, I don't know what that means, but lets try this `git commit`

> fatal: not a git repository (or any of the parent directories): .git

Ok, No idea here. I somehow figure out that git init will give me a .git
folder. Now that I've got that out of the way, I try git commit , I get:

> nothing added to commit but untracked files present

Ok, how do I track files? "git help" doesn't mention tracking files. I'll try
"git commit hello.txt", which gives me:

> error: pathspec 'Hello.txt' did not match any file(s) known to git

I give up at this point really.

(by the way, I got this far by doing this walkthrough this morning, and
googling "how to use git" \- which told me the answer in 3 seconds).

Not knowhing how to use a terminal vcs, or knowing the commands to perform
even the basics doesn't mean you can't use version control, it just means you
can't use a terminal for git. Is my 9 years of C++ and perforce experience
because I didn't know that commit was analogous to submit, or because I've
used a graphical interface for all that time?

~~~
flukus
> I type in "git", and I get: "usage: git" \- If you're used to using shell
> tools, then sure you can make sense of it, If you're not, then you're done.

That's part of the test, to see if they're familiar with the command line and
if they know how to open the man page. It's trying to weed out the people that
can only work in the confines of an IDE and gui tools. That said I wouldn't
expect anyone to know git from the man page, I would however expect anyone for
a senior role to be familiar with what is a de-facto industry standard.

And source control in general is a great topic for interviews on both sides.
Many devs (and companies) don't know what a branch is or what you'd use one
for. Many companies make it hard/impossible to create feature branches, either
by policy or crazy mono-repo stuff. Even their choice of SCM says a great deal
about them, I'd avoid anyone that uses clear case or TFS.

~~~
maccard
> That's part of the test, to see if they're familiar with the command line
> and if they know how to open the man page.

That doesn't tell you anything about how good a programmer they are. I don't
need to use a command line or man pages for 99.999% of my work, so I'm not
going to waste time learning to use more tools.

> It's trying to weed out the people that can only work in the confines of an
> IDE and gui tools

Ah, so anyone who uses a terminal is superior to someone who uses an IDE or a
GUI?

> I would however expect anyone for a senior role to be familiar with what is
> a de-facto industry standard.

In _your_ industry. As I've mentioned before, I use Perforce (which is
standard in my industry).

> And source control in general is a great topic for interviews on both sides

Agreed, but asking somone to rattle off `git init git add . git commit -m"I
can remember three lines"` doesn't tell you anything about how much they know
about source control. Talk to them about branching/workflows to find out how
much they know about source control, or let them use the tools they're
comfortable with, but plonking someone in front of a terminal to rattle off
some commands is the equivalent of looking for a "culture fit"

~~~
flukus
> That doesn't tell you anything about how good a programmer they are. I don't
> need to use a command line or man pages for 99.999% of my work, so I'm not
> going to waste time learning to use more tools.

I don't want to offend but you sound like exactly the type of developer I try
to filter out. IME there is a large correlation between at least rudimentary
command line proficiency and being a good developer. Aside from that I want
someone to know about the world outside of the IDE and what options are
available, because a unix like environment offers far more power than an IDE.

I expect developers should be able to automate common tasks, work with build
tools, grep through logs, remote into servers, debug on servers without an IDE
installed and a million other things that are very common.

Maybe this doesn't apply to your specific industry but it does to everyone
I've worked in to varying degrees. Not memorizing git I could forgive if you
could explain branching workflows, but living life in the IDE I wouldn't.

------
austincheney
The interviewer should ask the candidate, in the candidate's own words, to
walk through solving a problem proposed by the interviewer. No whiteboard.
Just a simple conversation. The candidate must be provided the opportunity to
organize their thoughts with notes on paper and have a short pause of time to
think through an answer.

Problem solving is a fairly universal thought experience not limited to
writing code. The goal is to ascertain whether the candidate can break down
the complexity of a problem into simple steps, organize their thoughts into a
clear flow, communicate clearly, and finally recommend a valid solution.

You don't need to test whether the candidate can actually write code, because
this is built into the nature of the exercise as qualified by the feasibility
of the proposed solution. This exercise also implicitly tests confidence,
creativity, experience, and approach style.

Most importantly though, it separates the competent from the incompetent. No
amount of framework foolishness and dependency baggage will communicate a
solution for you.

~~~
sethammons
I've interviewed many folks over the years who talk the talk and just simply
ace problem solving, but they can't write a unit test in their preferred
language.

------
issa
I think the most useful interviews are ones with three parts:

1) Casual talk. Does this person have basic communication skills and are they
someone you want to deal with regularly for a long time?

2) Give them an actual coding project to do. Pay them for it. Pretend it's
just like real work (it should be).

3) Review their code with them. (both to make sure they are the ones who wrote
it and to explore their thinking process)

~~~
umvi
I really like the idea of #2. I've never actually seen this done before but it
seems like micro-internships (1 month) would be invaluable. Bring the
candidate onboard for 1 month and start paying them their real salary. At the
end of 1 month it should be clear whether they are a good fit for your team or
not.

~~~
andrewingram
What you're proposing is basically the same as a notice period (at least in
the UK), and neglects that it's still high risk for the candidate. To be able
to come in and work for a month they have to leave their existing job. I'd
have to be pretty damn sure I'd have no difficulty passing my probation in
order to risk being out of work.

~~~
umvi
It's high risk for currently employed candidates, yes. I hadn't considered
that mainly because my company's main source of candidates is unemployed
college graduates.

------
new299
Most interviews have asymmetric costs. There’s a potential payoff for both
parties, but costs are higher for the interviewee (time off, prep, work done).

I recommend trying to make the cost/benefit more symmetric. One way of doing
this, is offering compensation for real work. Essentially, hire interviewees
to do a small amount of useful (a day?) and evaluate them based on that.

This isn’t going to work for all interviewees, but where possible it feels
like a better way of doing things to me.

~~~
gnarcoregrizz
Not going to happen - I think the supply of software developers has started to
outweigh the demand, so there's no way that companies are going to offer
compensation to interviewees. If anything, the difficulty bar is just going to
continue to go up. If there were a shortage there wouldn't be so many of these
silly hoops to jump through.

It's not really worth it to interview nowadays. Practicing leetcode for
months, flying, doing all day onsites, when all the company has to do is
interview you for a few hours.

We've been issuing take-homes lately, and we ask that candidates spend no more
than 3 hours on them. Well, it turns out almost no one spends only 4 hours on
them... The most recent candidate we interviewed spent about 3 days on theirs
judging from the commit logs. It was a fully fleshed out application with unit
tests and everything. After all that they still have to pass the all day on-
site, lol. Anyway I spend all of 2 minutes looking at each submission

I really wish there were a better signal.

~~~
marcell
Your company might be inadvertently attracting poor candidates. I imagine a
lot of good candidates (like myself) would just pass on a 4 hours of homework
for a random programming job. And good candidates almost certainly won’t spend
3 days on it.

Remember Joel Splosky’s classic observation: all the good programmers already
have good jobs. The ones on the job market who apply to jobs are usually the
bad ones.

~~~
stult
It's more like, given a good job market and low barriers to entry, the
percentage of candidates who are not qualified will greatly outnumber the good
candidates. Good programmers might be looking for work for any number of
legitimate reasons, even in a hot market. Doubly so if you are offering a
premium on compensation or otherwise trying to make yourself more attractive
to applicants (which four hours of take home is the opposite of). The signal
to noise ratio is just lower the less attractive you make applying to your
company.

------
aaronbrethorst
I like asking candidates to give me a code review on a pull request. Because
of IP concerns, I normally do this with an OSS project I’m involved with.

Sure, they don’t know the codebase, but they won’t know the codebase that
they’ll be working on in a month if hired either. I want to see how they think
about creating software and whether they notice potential defects or tricky
areas.

~~~
odonnellryan
How long do you expect them to spend on this?

~~~
pydeveloper22
Hello,

My apologies in advance that this may not be the response you were looking
forward to on forum question.

But I discovered an older post online on the Hacker News site some time ago
for a Remote Junior Programmer/Assistant at Luma. I noticed you were looking
to make your first hire and find someone that has some knowledge in
technologies such as Python and Linux.

Well, I had completely forgot about viewing it and so I was checking out
Hacker News job board today and saw it was for a remote Part/Full-time opening
available at the Luma in the NYC area.

Well I'm someone who enjoys coding and learning with Python as far as learning
purposes/hobbies go. Plus, I'm someone who has been going the self taught
route trying to break into the IT field/Python Development world. I also have
interests in some of the things mentioned in the post such as finance/trading
and business.

But, more than anything.. I'd looking to further learn and grow in my skills
as far as Python development goes. I don't have much experience with Django
Web development but feel I can learn and pick up on it as well as with any
other technology requirements quickly.

After quickly reading your post on Hacker News I was encouraged and interested
to reach out and contact you for more info.

So with that said, here I am..and I wanted to inquire to find out if this
opening is available? Or has it been filled? Also would you allow for training
of junior developers to get up to speed? I had to ask because but I wasn't
sure if you meant junior or for more seasoned developers.

Also, you have a contact number to learn more about this position or
information on the things you require in regards to the nature of the job to
increase one's chances to be a part of your startup team? Any help in this
matter will be greatly appreciated. Thank you

Feel free to contact me at: pydeveloper22@gmail.com

\--K

------
pjc50
The interview should not focus on assessing basic competence, it should look
at relevant experience and understanding of the business in question as well
as communication and teamwork style.

Of course in order to _get_ there would require a working certification
system. Requiring candidates to write fizzbuzz in person over and over is a
waste of your time and theirs, but it exists because nobody's managed to crack
a "fizzbuzz certified" system that employers actually trust for this. The
fraud pressure is quite high and nobody wants to pay for it if there is a
possibility that someone might benefit - 1000 candidates applying to 100
employers results in a huge amount of waste, but requiring the candidates to
pay for certification cuts out a lot of good candidates, and paying on their
behalf _and allowing someone else to use that result_ is seen as unacceptable.

~~~
stult
You're right. We ask fizzbuzz simply because it eliminates a shockingly high
percentage of applicants. We then ask more challenging and open ended
questions to evaluate actual competence if they get that question. Fizzbuzz is
there as a shibboleth of sorts. But I don't see it as a waste of time even for
qualified candidates. For many qualified applicants, getting them to relax and
speak naturally is the greatest challenge. Starting with a softball question
helps eliminate some of the natural interview anxiety and lets us see how they
really communicate. Then once in a while, it highlights the really arrogant
pricks who think it's beneath them, which is another category of candidate we
want to filter out.

~~~
hvidgaard
It's amazing how many junior developers fall flat on that, and I've even had
candidates claiming several years of experience to unable to make a for loop
with an if statement. And I honestly don't think we lose any good candidates
to it.

I don't ask FizzBuzz because many expect it, but I ask equivalently simple
questions. I have the following criteria for an interview code question

1\. Have an obvious KISS implementation. 2\. Ability to modify requirement so
the KISS no longer works.

I expect 2. to enable us to gauge their thinking. I don't expect a working
solution to it, but they should be able to identify why it does not work
anymore and outline ideas to solve it. This part is a conversation where we
can lead them if they are not good at controlling the conversation.

------
gwbas1c
In the end, I'm always trying to have a "discussion about code" with the
candidate. I ask questions that really are easy, but require a certain amount
of listening. This way I understand how the candidate behaves in more abstract
discussions.

Some examples are things like:

\- Walking the candidate through an outdated API (that the candidate isn't
familiar with, but should be able to understand given the nature of the job)

\- Walk the candidate through code that converts a database query into objects
without an ORM. (Candidates who can't do this are incompetent. Really.)

\- Discuss commonly-known details of exceptions / error handling in the
language that the job is for

\- Discuss commonly-known details of memory management in the language that
the job is for

\- Discuss API choice tradeoffs in an API that the candidate should be
familiar with. (I like to pick serialization APIs built into the library in
the language we will use.)

Also:

\- I try to emphasize how I would do with my interview at the candidate's
level of experience

\- I have 2nd chances, and will usually stay on the phone for the scheduled
interview out of respect. (There are still certain points where I will cut an
interview short.)

\- Make a decision to hire quickly.

Usually works well.

Signs to reject someone:

Figuring out how to use the teleconference software is an unofficial part of
my interview. (Most of my interviews are conducted via teleconference because
I'm on a global team.) There are candidates who take 15 minutes to figure out
that they can type their phone number into the teleconference web site and it
will call them back.

Candidates who want to answer a different question, or keep asking, "why can't
I use XYZ" technique usually aren't hired. Again, the purpose is to have a
discussion about code and demonstrate understanding of the discussion. I don't
want to hire someone who can't adapt when the correct solution to the problem
is some tool / design pattern / API / ect that isn't the candidates first
choice.

------
abhirama
I have had good success with an assignment based interview process. I have
written about it in length here - [https://abhyrama.com/2018/11/17/startup-
hiring/](https://abhyrama.com/2018/11/17/startup-hiring/).

------
vinceguidry
Interviewing is an adversarial process, as such, there can be no ideal way to
conduct it. It's like asking for the best way to have a divorce. Any imposed
solution means at least one person is leaving something on the table, and so
it won't be ideal.

~~~
fsloth
Why is it adversarial? I figure it's about the candidate trying to find a
position that is suitable for him or her, and the employer is trying to find
the correct person to fullfill a specific role.

If the role does not fit the candidate, certainly it's advantageous to both
that the candidate does not get hired.

~~~
sigstoat
> Why is it adversarial? I figure it's about the candidate trying to find a
> position that is suitable for him or her, and the employer is trying to find
> the correct person to fulfill a specific role.

that's ideal, but in practice i think employers and candidates are conditioned
(perhaps by prior less professional employment? or the market as a whole?) to
think that it is the candidate's job to trick their way into the company, and
the employer's job to guard against the unqualified sneaking in.

------
luord
The best interview process I've read about is the one detailed here:
[http://www.nomachetejuggling.com/2011/05/27/a-different-
kind...](http://www.nomachetejuggling.com/2011/05/27/a-different-kind-of-
technical-interview/)

And sadly I haven't once seen it in action. I really wish I could go through
such process, or implementing it in whatever company I work at (or create).

~~~
pytester
I've set tests more or less like this in the past and I took one just the
other day.

It's actually kind of interesting doing this from the hiring side because it
became evident that there are a lot of relative "bargains" out there -
excellent developers who do poorly in whiteboarding and chatty interviews so
have lowered salary expectations but who are nonetheless very good at their
_actual_ job.

------
thisisit
The best one I had was where we had more of a discussion about things rather
than go into a question and answer session. It was an enjoyable experience to
talk to people who are working in a similar field and had seen similar things.

On the other hand, I dislike interviews where interviewers come up with a
question-answer sequence. So, if you misspoke even a single word they outright
reject you. And mostly, I find this in places like TCS, Wipro, Infosys etc.

------
mattbee
When I was at Bytemark, we decided it was important that you knew up-front
exactly what the demands on your time would be, what challenges there would
be, and we don't deviate from that:

[http://careers.bytemark.co.uk/full-
process](http://careers.bytemark.co.uk/full-process)

It's also anonymous until the 3rd stage, so you (by and large) interview
remotely and without anyone knowing your name / gender.

(this dates from 2015)

------
kwang88
A process that I've seen have some success (years ago):

Start with a very, very simple initial phone screen or take-home test,
intended to basically verify whether the candidate can write any code, at all.
Max 1 hour, weeds out more people than you'd think.

For the first in-house interview, ask the candidate to code up a problem that
is representative of your company's work and requires coding a significant
amount, ideally 100+ LOC. The problem should not require any major leaps of
intuition, dynamic programming, or recursion – all of these are areas where
people do way worse when they're nervous, and this is an engineering interview
not special forces training. Let them bring their own laptop, give them the
prompt, and have them code, although they can ask the interviewer questions at
any time. When they're done, go over the question in detail with the
expectation that their code compiles and runs, discuss extensions, etc. Max 1
hour. This interview should answer the binary question "can this person
promptly produce meaningful working code and discuss it intelligently?"

For the next in-house interview, do a deep dive into a technical project that
the candidate worked on that they're proud of. They describe it and you ask
questions. Keep asking questions, especially getting at the "why" behind
different decisions, for as long as you can – you're trying to get to the
borders of their knowledge and intelligence. Look for mastery of the area,
thoughtful decisions, and communication skills. Max 1 hour. This interview
should answer the question "is this person a thoughtful, effective, smart
contributor on a project?" A good answer should make you think "damn, that's
really smart, I wonder if I would have thought of that?" at least once.

End with a final behavioral interview, intended to sell the candidate. This is
also a last gut check on whether they're insane, dangerous to themselves or
others, extremely arrogant, etc. Also use this time to ask the candidate
questions about what really matters to them to improve your closing rate. 30
minutes, and can be combined with the step above.

I've liked this system, YMMV. It's a relatively efficient process, doesn't
have weird tricks, and based upon a longterm analysis of candidate outcomes
was quite effective (this included an analysis of candidates whom we rejected
and who rejected us).

~~~
kod
Are you saying max 1hr to understand a problem and write 100 loc? Seems
unreasonable unless you are working in languages with a bad signal to noise
ratio.

------
aplummer
I wanted to add my 2c that in the mix with whatever else you have, I love a
take home.

It’s actually building something in my element, IDE, docs, just like real
life. You get a chance to show you care with the details.

A whiteboard interview is not like RL as your entire career is on the line so
shouldn’t be the main KPI, your brain is not working as usual. I don’t even
hate presenting to groups or anything, although I do hate presenting
unprepared.

------
rococode
I've often thought about this, and based on my own experiences this is the
process I would like:

1\. Test language-specific knowledge - it's not a bad thing to not know all
the small quirks or details of a language since they're generally not too
useful, but when someone does know them it tends to be a good sign that they
really enjoy coding and learning. And of course there is a certain minimum
amount of knowledge that should be required - listing Java as "proficient" on
your resume while not knowing the difference between abstract classes and
interfaces might indicate a problem.

2\. Design question - just a simple toy problem like "how would you design the
book management system for a library?". It's easy to use such a problem to
probe into some different aspects of programming like concurrency and
databases, just to see how much the person knows.

3\. A not-crazy-hard algorithms questions - people often say algo questions
are irrelevant to actual work, and I agree with that. But I think algorithms
are really a core part of the CS curriculum at every school, so getting
completely stuck on a medium-difficulty algorithm question should raise some
flags.

4\. Resume-specific things - it's always nice for an interviewer to show that
they've actually read your resume, and it can be a good way to convey some
strengths that aren't otherwise evident.

I guess my philosophy is to interview in a way that can test the depth of a
candidate's knowledge while not being obnoxiously tedious or memorization-
focused. i.e. someone who has written a lot of production code should do
better in an interview than someone who's just memorized every problem in
Cracking the Coding Interview. So, ideally with the screening round clearing
the first part (language-specific knowledge), and then 2 or 3 subsequent
interviews that cover design and algos.

~~~
mav3rick
Language quirks are the worst thing to ask in an interview. Please don't do
it, it's barely a step above memorization.

~~~
lawnchair_larry
I think it depends how you do it. When I interview, part of what I do is try
to find quirks that they know about, but not count the ones they don’t know
about against them.

I think you can quickly get a sense of depth, potential, curiosity, and
passion if you can get them talking about quirks and their opinions on those
quirks.

For example, if someone says that they once attached a debugger to the JVM to
confirm a pathological issue arising from benign looking Java code and found
that the JIT compiler falls apart when a certain construct is used, I’m
probably going to hire that person, even if we have no Java code in the
company - assuming I can also confirm that they are productive and don’t just
waste time going down deep rabbit holes.

~~~
chillacy
How has that been working out for you? I used to think the same and I'd ask a
"programming passion question" right before a simple coding question, and then
I'd run into these amazing bullshitters who could talk shop but couldn't write
fizzbuzz. But they'd come up with a bunch of excuses as they were writing the
code as to why ("oh my company uses this other framework so I forgot how to
write a for loop"). So sadly I stopped paying attention to that part. I only
ask it to make the candidate comfortable before we dive into code.

~~~
lawnchair_larry
Nobody who can go into that level of detail doesn’t know what a for loop is.

~~~
chillacy
It seems you're doubting my story. Maybe you think my threshold for "that
level of detail" isn't sufficient, or whatever candidate sourcing we use
doesn't filter these people out.

------
sandoze
Don’t wait for the candidate to show up and then start a conversation about an
algorithm (or technology or mathematics) that they may not be familiar with
haven’t used in some time. Offer of some topics for discussion before the
interview (e.g. Quadtrees, tries, infinity) that they can study, or read a
white paper about and have some increasingly in-depth questions, applications
and conversations about one or more of these topics.

No one is ever expected to solve an unfamiliar problem in a 60 minute meeting.
Generally a story/feature with technological unknowns are pointed for several
days and are in the developers current wheel house, yet we expect a candidate
to use most of their energy and facilities in a social situation with people
they’re unfamiliar with and solve whatever random algorithm, riddle or puzzle
that you Googled the answer to before the interview.

------
thatswrong0
Speed and no trivia / bullshit interview question. Interviews should be
assessing me on the skills I’ll need to get the job done, not recanting some
data structure or algorithm to solve a problem that’s unlike anything I’ll
ever actually deal with.

Ask me about my past projects and decisions I made and mistakes I made and
what I’d do differently. Sit down with me and actually code with me and get a
feel for what that’s like. Ask me to how I’d design some actually realistic
system, and drill into the details of each.

I shouldn’t have to train to pass an interview. Prepare, yes. Practice a bunch
of problems unrelated to the job at hand, no thank you.

------
CamTin
Describe what the job entails, ask me if I believe I can do it, and then hire
me if I agree. Fire me later if I was wrong.

~~~
fernandopj
That "hire fast, fire faster" mentality doesn't work in all countries,
industries and legislations.

~~~
CamTin
The OP was asking about the ideal process. This should be a starting point and
any other demands that practicality and regulation cause an employer to want
should be compensated. Can't hire me right away? OK, then just pay me my
eventual salary while you interview me. If they can't afford this, then they
can't afford to hire, probably because the complexity of their business is too
high, and they should be broken up or nationalized.

------
pytester
1) A test that tries to mimic what the developer would be doing in the actual
job as closely as possible. This is what most interviewers fail at - they seek
out stupid proxies like whiteboarding rather than simply trying to mimic real
life.

2) > 45 min < 3 hours long pairing test or test otherwise done in the presence
of the interviewer.

3) Minimal setting up development environments, frameworks or test
environments. No boilerplate should be required during the test.

4) The test is improved iteratively. Catch bugs during each interview process
and fix them in subsequent interviews.

~~~
ArchTypical
> Exercises that try to mimic what the developer would be doing in the actual
> job as closely as possible. This one is critical.

(the following is just inspired by your bullet)

The job of a developer consists of problem solving something they have never
encountered before (under some pressure), communicating to other developers,
and demonstrating knowledge in various combinations and amounts over time.
Startup developers are the hardest spots to fill and most delicate, so I tend
to always think as if I'm hiring for a startup.

Talking through a few whiteboard problems and talking about tools they have
used (why and how) are sufficient. 30-45 minutes. I don't subscribe to adding
more elements (environments, tools, test frameworks, etc). 3 people, 1 from
the destination team, 1 from another team (if possible) and the immediate
manager.

Startups to multinational corporations, I'm still convinced this is the best
way over the last 20 years.

Interviews with higher management is just a redundant filter and another
reason for the management to justify their existence. It's a job smell that
every company has and is obviously prejudicial in every regard (trying to
tease personal details or redundant workflow experience and opinions). It's a
bad practice.

~~~
pytester
>Talking through a few whiteboard problems and talking about tools they have
used (why and how) are sufficient.

I've done interviews with a whiteboarding "tell me about stuff you've worked
on" component followed by a test.

It thought it gave a weak signal about the skills of the candidate. I canned
it eventually because the test gave a very strong signal and it wasn't telling
me anything the test didn't.

>Interviews with higher management is just a redundant filter and another
reason for the management to justify their existence.

I agree. I had suspicions in a previous company that this upper management
"team fit" interview was adding a race/nationality filter that ended up with
good candidates getting dropped.

~~~
ArchTypical
> [https://medium.com/@alexgolec/google-interview-questions-
> dec...](https://medium.com/@alexgolec/google-interview-questions-
> deconstructed-the-knights-dialer-impossibly-fast-edition-c288da1685b8)

I'm really frustrated when these kind of problems are presented as
"whiteboard" problems. It's not constructive and discouraging to potential
employees. When something as simple as "implement X without using Y" or "fix
this program with pseudocode" are practically useful experiments and more
indicative of what developers are doing to be doing ALONE every day. If
there's inefficiency, code reviews and planning and integration all reveal
that and you can pull in specialized resources if you really need it.

------
linsomniac
I can tell you was a non-ideal interview process looks like from the "coding
challenge" POV. I had one company pose a "dev challenge" of taking a multi-
million row public sample data set and the goal was to generate a report from
it. This was for an operations job "but everyone has to take the dev
challenge".

Part of the problem I had with this was that I didn't feel like it was even a
dev challenge. I've hand coded reports before and that has always led to a
world of pain. I also felt like it was a data sciences challenge, not a dev
challenge, and my data science is really rusty.

I spent most of the 1-4 hours they said most people complete it in, just
thinking about the problem. "If I had to solve this problem in my company,
first thing I'd do is look at Crystal Reports. The last thing I'd do is open a
file and type "import sqlalchemy"."

I set up a repo that did all the operations stuff (remember, it was a Ops job
I was applying for), and put together deployment parts to set up a test system
and load the data into the database, configure everything, etc...

A few weeks later I finally got the parts all to work and was able to solve
the problem in an 80x25 screen worth of SQL. I suspect I was the only
applicant that solved it in SQL.

------
asdfokd8
For front-end positions, if I were interviewing myself, a 1 hour interview, as
follows:

1\. "Imagine you are building a simple Gmail clone (search header, sidebar,
action navbar, main content area). Walk me through how you would approach and
implement this." (20 minutes: What architecture choices do I make? What
tradeoffs am I comfortable making with this? Do I ask questions about the
audience? How do I handle state management? How do I approach tooling for
this? How do I approach testing for this? Lots of specific questions along the
way..)

2\. "Tell me about an interesting article you read recently on a tech topic?
Which resources do you use to stay current in the front-end space?" (10
minutes; Am I committed to learning and staying abreast on an ever-changing
landscape. Can I impress me with quality resources I'm in tune to? Also, can I
effectively convey ideas at a high level; can I critique it and walk around
the topic from various vantage points.)

3\. "Could you pull up your github and walk me through your last few public
commits." (15 minutes: Gives me a chance to see my code, talk about the
process of writing it? Are there tests? etc)

4\. "Would you mind code reviewing this [FizzBuzz-like] code and tests? Then,
what would your next iteration on it be?" (15 minutes; Am I a good team
player? Can I communicate effectively? Can I spot areas for improvement?)

5\. "Finally, could you provide me with a list of past/current co-workers" (0
minutes; With this I will be able to assess what my peers thought of me? Work
ethic? Pleasure to work with? Ego? Best qualities? Shortcoming?)

I suppose if the first question is switched, this could be used for any
position.

------
kampsduac
After the initial phone screening for a full stack engineer, we send a take
home questionnaire that doesn't require any coding - but rather presents a set
of questions that span across different domains (development process,
databases, product, infrastructure).

The candidate selects 3 questions to answer from the set of questions. We
expect them to take ~15 minutes to answer each question using the English
language.

------
horatiocain
Our Android dev task: write a login/signup dialog. Shows that you have layout
skills, networking skills, general Java skills, etc. Candidates either bomb it
or nail it, no mis-hires yet after about two years. You get a real good sense
of experience level, confidence, and "I like clean code" attitude.

~~~
RandomNewGuy
Interesting approach, away from all the CTCI bs. are you guys hiring?

------
gpm
Speed.

Obviously not a complete answer, but I think it's a component that isn't being
mentioned here. Disclaiming that I don't have much experience (still in
university).

One of my internships went from "applying -> interviewing -> accepting" in
under 24 hours. Impressed the hell out of me. It was really the reason why I
ended up accepting their job. 24 hours is obviously abnormally fast, but a
week or two doesn't need to be.

One of my friends ended up rejecting an offer both from Google and Apple
because the interview process took too long with no responses. They got a good
offer in the mean time, and after waiting over a month for a response they
decided they had to go with it. (They followed through with the rest of the
Google/Apple interview process anyways for the experience... which was
basically just host matching and getting an offer).

~~~
emmanuel_1234
That can only work if you have a large number of available positions
(typically, low-paid interns). In other cases, you have one position and try
to hire the best candidate for it. Interviewing enough candidates takes some
time, if only because not everybody is available at the same time.

------
monksy
It should be clear about what they're looking for, should be collaborative (I
want to see if I can work with them), it should be respectful (ending it early
for not a good fit is not a good reason), they should allow for you to talk to
your strengths and not let it be a we are pushing you to failure.

------
justfor1comment
A friend of mine is a UI designer and I really like how she gets interviewed.
Her first interview during an onsite is to give a presentation about herself
and her prior work. The audience are the 3-4 designers who will be
interviewing her that day. This helps her set the tone of the future
interviews and also gives an opportunity to show case her best work. We never
get this chance during software interviews. I feel software interviews are
like QA testing. The interviewers will each enter edge cases and see if you
break under any one of them. Your resume might as well be shredded by the
recruiter, cause it never gets read by the interviewers. You never get to
paint a fair picture of yourself and that is quite frustrating.

------
mrdependable
Ideal, though unrealistic, would look like this.

Phone call from interested employer to ask a few questions back and forth. If
things seem good set up an interview.

At interview, show potential employee where they will be working, what they
would work on, then sit at a table for the interview. Potential hire knows
their ability level and is honest with the employer about it. Employer knows
exactly what they need in the hire and is honest with them about it. They come
to a mutual decision about whether they are a good fit for employment there.

Interview done, and both parties know what to expect from each other
afterwards.

------
erkanerol
I read many of the comments. There are lots of good suggestions. I believe
these skills should be checked as well.

\- to be able to explain something (whatever the candidate knows) by drawing
boxes+arrows on white board

\- to be able to write short and clear messages especially emails in
corporates

\- to be able to learn something new in a limited time

\- to be able to document something clearly

\- to be able to hack arhitecture/framework/existing mechanisms when necessary

\- to be able to design something so that it will require minimum hack in the
future (similar to O of SOLID)

------
cosmosa
I've seen many different formats used.

1\. Focus on open ended questions with very little programming

2\. Give very difficult algorithm questions 3\. Take home assignment

4\. Knowledge based questions

5\. Brain teasers

By far I think the most useful is take home assignment as it is the most fun,
relaxed, and realistic way to measure someone's ability to code.

I don't think algorithm questions are that realistic because usually they are
unrealistically difficult.

Mixture of take home and knowledge based questions with maybe some algorithm
questions of realistic difficulty is best.

------
werber
The current role I'm in did a great job. There was a technical phone
interview, but the in person was with the entire team and they gave me a ton
of time to interview them and just have a conversation. It was fun and
informal and I started the role with a pretty realistic idea of the people I'd
be working with. The only thing I'd change is adding in time to actually
shadow someone in a similar role and doing some pair programming.

------
bsvalley
[https://news.ycombinator.com/item?id=18514637](https://news.ycombinator.com/item?id=18514637)

------
jondubois
I will answer short and complex technical questions but I will not do
technical tests anymore. Managers who are too lazy or not technical enough to
know my skill level based on a conversation with me are not the kinds of
people I want to work for. I've worked for a lot of companies and the most
successful ones didn't make me do any tests. There's definitely a correlation.

------
scarface74
1\. Test for basic competence to make sure they know the language - what’s
OOP, the difference between a class and an object, etc.

2\. Whiteboard database design of a problem. I’ve found that it weeds out
people who can’t model a solution. I also ask them to write queries that can
answer simple questions.

3\. A sample skeleton of a project with failing unit tests. They have to make
the unit tests pass.

------
simonsaidit
Ideal process for me is being placed in front of other engineers and
discussing what I did in the past and how I understand their challenges.
Meeting the team I’m working with is a plus to see if we are a match. I don’t
do tests and stop the process before it gets to that. I also consider talking
to HR a waste of time. The company should sell itself to me.

------
wheaties
I like the following points touched:

1\. Can you review code?

2\. Can you work with product on requirements for X?

3\. Can you design a system?

4\. Walk me through the process to debug situation Y.

That's it.

------
gipp
The top two comments are currently the most _complained-about_ strategies
(whiteboarding and take-home assignments). I think the best conclusion here is
that for all the issues with the process, it's not like there are a bunch of
way better solutions just waiting in the wings.

------
lbj
I'll tell you what really improved our successrate. We implemented a small IQ
test and a small relevant coding challenge. Failure in one of these got you
disqualified immediately. We started weeding out sub-optimal candidates real
quick and were able to focus on the star talent.

~~~
merlincorey
I believe your process is on treacherous legal grounds in the United States,
at least if you literally conduct an IQ test with a score and a cut-off value.

See this article for some amount of information:
[https://www.hiresuccess.com/resources/guide-to-employment-
te...](https://www.hiresuccess.com/resources/guide-to-employment-testing/is-
employment-testing-legal)

~~~
lbj
We're based in the Nordics and there's no conflict here. But if there was I
would just change the test enough to scrape by. Its wildly beneficial and the
best predictor of future success known to man.

------
zoba
I documented my ideal process here: [https://tech.residebrokerage.com/how-we-
hire-software-engine...](https://tech.residebrokerage.com/how-we-hire-
software-engineers-4cb5034ad09a)

------
lawnchair_larry
No whiteboard code, no algorithms questions. If you want them to code, let
them do it on their own time in a comfortable environment. The industry needs
to stop the bullshit leetcode meme that millenials are propagating.

~~~
passivepinetree
I’m a little confused — can you explain how this is a generation’s fault?

It certainly seems that millennials are subjugated to this type of process,
and it’s possible that millennials are the ones giving these types of
interviews more often than not, but as far as I can tell these interviews
propagated because large, well-regarded, highly successful companies (Google
etc.) started performing them and then smaller companies just copied their
formula. I’m not quite sure where the generational connection is.

~~~
didlydoo
Well said.

------
issa
Am I the only one who feels strongly that people should be paid for their time
to do coding for you in interviews? I feel like every company has some small
tasks they can have a potential employee take on.

------
jdlyga
Take home coding challenge, timed if possible. The on-site should be standard
whiteboard style questions, but focusing on problems that are more related to
the job.

------
sunstone
It would be just like a Vulcan mind meld and when it's done the interviewer
would say, "Yeah, you're good."

------
_fool
I wish I'd had "a real plan" in jobs past as an individual interviewer in a
series, instead of a vague goal like "talk about his technical chops" or "see
if she can work across teams". I turned out not to be very good or consistent
at either freestyling questions completely OR reporting reliably whether a
candidate was "successful" in my area without planning and consistency.

I'm a former developer, turned hiring manager for a very technical support
team and employ several things I feel are fair and useful and I believe would
also have been in that past life.

0\. repeatable interview process. When comparing candidates, the comparison
should be apples:apples. "check for culture fit" isn't; "ask questions 1,2,3
of each candidate, with goals of finding out x,y,z and help to N degree if
requested" is. Leave room for the candidate to demonstrate personality and
style of course, as well as (short) tangential conversations that come up
during the Q&A process. The goal isn't to make it robotic/automatable.

1\. There is a take home exercise. It has a clear scope:

\- accomplish goal X (create a website)

\- with tools like Y (your choice of static site generator)

\- in Z fashion (public github repo)

\- in less than 4 hours.

Encourage questions before candidates begin the exercise, so that people work
on the right problem rather than guessing wrong. Don't force a timeline ("you
have until the job closes which won't be sooner than a week from now"), and
use an objective grading rubric for each section of the task.

Here's an example of the objective rubric: 1 point for grammar, 1 point for
each of two distinct points of content, and 1 point for providing the context
as to why those are important points. Few of the questions have only one
correct answer and it is not always obvious to candidates what we're looking
for in an answer; but it is known in advance to the grading committee what is
being sought, e.g. "perspective over plot".

2\. interviews are in slack. We're a remote team who communicate 95% via
writing, and it doesn't matter if you stutter or what you look like or how you
present. Get comfortable in your PJ's with a cup of tea, or use your standing
desk if that helps you feel sharper. It only matters how you communicate in
writing.

Once again, the interviews are structured - "interviewer 2 will ask these 5
questions" and the criteria for success aren't "this woman was well spoken"
but instead "she showed ability to think on the fly about something
unexpected, demonstrated empathy in communication, and made a compelling
argument for her proposed solution". Not quite as objective as the take home
test, but still "gradeable". And, in case of a hard choice between two
candidates, other stakeholders in the hiring process can review the transcript
of the chat, providing their grade considering the pre-specified criteria, to
help decide.

It's not ideal, but it is practical, allows for cooperative interviewing
(anyone can help a candidate start the well-documented interview process;
anyone can do any stage of the interview and know that we cover all the
questions we want asked before the end), and encourages consistent judgment.

It's worked well for us so far and people have appreciated it; it also lets us
give concrete answers to "what could I have done better?" to candidates we
reject who ask for a follow-up. I feel pretty strongly that you spent your
time working with us (particularly on the take home exercise) so I can spend
time explaining how you could improve (often it is useful to other
applications for other jobs: "your code should have comments to explain
confusing things or highlight clever things", "consider using git for revision
control rather than only committing your completed project", "you'd benefit
from using a grammar checker"). We've had many repeat applicants who answer
better with each iteration.

------
bsvalley
\- For someone fresh out of college: Don't change anything, the current
process is perfectly designed for college grads. So Leetcode it is and
whatever the google's, facebook's, etc, are currently using to hire "the best
of the bests".

\- For someone with 2 to 5 years of experience: A candidate should have a
choice between a full day onsite, working with the hiring team on a mini
feature or a bug fix. Or, a 1 week project assignment with a shared/public
repo between the hiring team and the candidate. In this case the candidate
could work remotely on the evenings/weekend. Code should be reviewed and
pushed into the repo at the end of the assignment period. Collaboration along
the way is highly recommended for the candidate. The hiring team should be at
least available to answer the candidate during the process. The onsite would
ONLY be a lunch with the team to get to know each other. If the candidate
chose the 1 week assignment, he or she would have to stop by for the last step
if everything went well, which would also be a lunch with the team to get to
know each other.

\- For someone with 5 to 10 years of experience: Same as 2 to 5 years of
experience, but would involve more technical choices from the candidate. This
could also be a feature or a product optimization task, etc. Requirements have
to be very high level and the candidate has to make design choices and define
a scope as well as delivering a working prototype. If the candidate is closer
to 10 years of experience, he or she should assign a coding task to at least
one member of the hiring team and make sure to help and review the work. This
process could also work for +10 years of experience as a Dev, or even a "Tech
Lead".

\- For an engineering manager: No Leetcode please! Stop now! :) The candidate
should take over the current sprint from the hiring team, or part of it. This
could also be a sprint exclusively designed for the interview process. Stories
could be made up or could be real stories from the team backlog. This could be
a 1 week project assignment. As a Hiring manager you should be able to keep
track of stories and to host standup meetings remotely with the team (5-10 min
conference calls all week for example). The team would simulate blocking
issues and conflicts between peers and the descriptions of these problems
would be sent to the candidate for review. You would have to show up onsite
and host 1-1's with the related folks in order to go over these problems to
try to fix them. This would also include a team meeting to share the status of
the project with everyone. During this day onsite, they might also simulate a
mini hiring process for a new Developer. Team members would be the actual
candidates and you would be the interviewer (a quick 15 min interview for each
candidate). Other hiring managers would be involved in this process as well
where you would discuss about the candidates and make a final decision.

That's it. As you can tell, this would require companies to do a lot the work
in order to set that up. Unfortunately, not enough energy is allocated to
hiring new people. Companies rely on the existing and broken process started
by Google in the late 90's... this was the only company asking academic and
weird puzzles to candidates. Also, I cut a lot of corners because my post is
already way too long, but you get the picture...

------
bra-ket
no whiteboard

------
SamReidHughes
That depends on the needs of the company doing the hiring.

------
yjhoney
I've spent alot of time thinking about this, and I've concluded with this:

Hire local people minimum wage to learn and teach each other coding structured
around your company's codebase. Once they get to know the basics, have them
work on your company's open source projects. Identify the ones who actively
help others and convert them as a full time software engineer.

It takes an average person about 1 year to learn enough basics to contribute
to a codebase, and paying someone minimum wage only costs about 30k / year so
it works out financially on both sides.

~~~
didlydoo
For reference, this only works in places where the minimum wage is also a
reasonable amount.

In the UK, the minimum wage is laughably poor and in Switzerland it's
significantly below the amount needed to live.

A better threshold would be to look at the cost of living _in the area_ and
pay a reasonable starting wage that actually lets people live there
comfortably.

