
How We Designed Our Interview Process - gkoberger
https://blog.readme.io/how-we-designed-our-interview-process/
======
jcbeard
"we want the interviewee to succeed"

This should always freaking be the case. You, the interviewer, are there to
decide if you want to work with this person in the future. You're not going to
learn that by making an adversary. You're not going to learn that by making it
super easy either. If you're worth your salt (as an interviewer, and you
should be given that you're now part of the hiring process), you should be
able to go through an open ended problem (it's what I do). This open ended
problem should have increasingly difficult sub-tasks and of course be related
to what you intend the person your interviewing to do. One of my worst
experiences interviewing was with a company that clearly didn't give a rats
rear end (at one location). Interviewers were constantly late, didn't know
what I'd applied for, or even at times my name. Same company, different
location...totally different story. Your most valuable resource as a company
is your people, treat them as such from start to finish. Needless to say, I
chose where I'm at because of the people. I'm glad to see others sharing this
attitude as well.

~~~
newscracker
I've been on both sides in worse situations. The resumé is printed out and
handed to the interviewers at the last minute or someone is asked to interview
a candidate who has showed up and the resumé is handed at that moment. It
makes up for a poor experience where the first question is "describe your
experience and what you've worked on", most of which is usually present in the
detailed (and long) resumé.

~~~
jcbeard
Me to, in which case I explain the situation...I apologize to the candidate,
and we move on. Last minute scheduling accidents happen, it's how you recover
from them that counts. Openness seems to improve the impression of the
company. Trying to hide the truth from the candidate signals an issue, if
you're a candidate...run away.

------
wwalser
I think about hiring a lot and I see a lot of companies doing things that I
think are completely opposed to their own goals. Specifically around technical
talent where, given the current market, the goal is almost always to out-
compete other companies by finding good candidates that others would have
turned away because of cargo-cult awful interview processes and standards.

This looks great from what's printed on the page.

The question that jumps out at me from reading the interview prep screenshot
is: How often does a candidate fail the "work on your own project" part simply
because the people watching them have no interest in the problem space or
worse, no idea what's going on? I just don't see how that type of interview
could be scored equitably across a range of candidates.

Is that considered a failure on culture grounds? I'm highly suspect of
basically all culture related hiring standards.

------
caffed
It's really good to hear that some tech companies are being mature about the
process.

I've been in almost all positions of a hiring team: phone screen, white
boarding, Q&A, etc.; and in the end I want to hear what this person can bring
to the company and see if they're a good fit or vice versa.

I feel like so many places put emphasis on abstract knowledge and technical
ability and leave out so many other traits that are good. Also, in Techlandia,
I feel there is too much of a male machismo that permeates through and there
are many people that treat the interview as a chance to demonstrate ability to
the interviewee and/or trick them - to the point of intimidation.

~~~
flukus
> I feel like so many places put emphasis on abstract knowledge and technical
> ability and leave out so many other traits that are good.

I don't disagree, but personally I would rather work with a competent (or
brilliant) asshole than an incompetent nice person.

~~~
Noseshine
Sample-of-size-1 statistics time (from which nothing can be learned, just for
fun)?

I (freelancer) once had a software-making client that employed me for a month
do contribute a special component to their big project. When I arrived the big
boss himself received me and showed me around - and when it came to showing me
where and with whom I actually worked he apologized profusely before he even
started.

Apparently I had to work with the one (and only) guy who sat alone in his
office. But he was the guy who had written the specs that I had to write the
code for, and he had room since all offices were designed for at least 2
people. The apologies were because that was the most difficult person in the
entire company to work with, by a margin, but he was too good so they just
kept him in his cell...

Well, I wasn't daunted at all actually. Turned out that yes, he was difficult
socially. For example, there was a woman who came in to ask him something (it
was rare anyone talked to him while I was there), and in an audibly annoyed
tone he pointed out that he had already sent her that exact information 5 days
earlier in document XYZ on page N, headline "3.2 foobar" (yes, to that detail,
without looking anything up).

He also was really, rally good. The specs he had written, several hundred
pages in three documents, were near perfect. I didn't have to ask a single
question at any time, he had thought of _every detail_.

I myself also had no trouble talking to him, I had fun with the other people
that I visited in their offices (who repeatedly told me pitied me for having
to share an office with that guy), but also with my room mate. Because of the
surrounding highly unusual circumstances this still is one of my most
memorable (and completely positive) freelancer jobs.

~~~
emilburzo
Loved the story.

It sounds like that guy had a high IQ and low EQ, which is highly efficient
for work scenarios, but it usually sucks for any kind of social interaction.

------
jacques_chester
I kinda like this one. The "what to expect" is really well done.

I also like the do-a-real-thing step. We do something similar at Pivotal. Once
you pass the basic tech screen, we invite you to work with us for a day.

We pair program, so this is easy to do: take the candidate, drop them
(usually) into a real project with a real engineer working on a real problem.
Then they work together on the real problem. That's it.

Assuming nothing goes drastically wrong: One project in the morning, then
lunch, then a different project in the afternoon.

If we agree that you know what you're doing and that we want to work with you,
you're hired. Every engineer at Pivotal has passed the test of "do you want to
work with this person?"

In the quest to work out "can this person do the job?", I personally continue
to be amazed at the gymnastics companies will engage in to avoid just ...
trying folk on the job.

~~~
praneshp
I hope I was the only person to have had a bad experience with Pivotal ever.
Their recruiter straight up lied to me (told me on the phone that they'd
schedule a phone screen, went on radio silence, and claimed they had emailed
me about moving on). From my personal experience, I don't think Pivotal
belongs in a thread discussing great hiring practices.

~~~
jacques_chester
I'm sorry.

We had a period where we had many candidates and not enough recruiters (hiring
good recruiters is actually really hard). Some of the recruiters would get
stressed out and quit. Some would, as you say, simply drop candidates on the
floor without telling them. We also had a _lot_ of trouble with our previous
candidate management software. We switched to Greenhouse 2 months ago, it
seems (touch wood) to be doing much better.

If you like, you can contact me by email and I'll follow up for you. This
offer extends to anyone else.

------
BinaryIdiot
Love the idea of giving the person an idea as to what to expect though this is
a pretty general outline; I've gotten the same (although I'll admit far less
pretty) when I've interviewed at Amazon, Apple, etc.

The idea of letting someone actually work on a problem is far better than
generic algorithm or data structure questions. Two things though...

> As soon as we schedule the interview, applicants receive a link taking them
> to this welcome page.

Love the page but the content is all static. This should be in an email. If
it's in an email, especially in invite form, I have access to the content at
all times on my phone and can even accept an invite. Putting it in the browser
means I have to find the email and open the browser (alternatively bookmark or
keep the site open). But the content is static so it should be in an email.

In my opinion anyway.

> After that is the main part of the technical interview: we ask interviewees
> to work on their own project, using whatever resources they deem necessary
> (and yes, that includes Google). We want resourceful developers who can
> problem solve, not people with an encyclopedic memory. We use this as an
> opportunity to learn what our candidates are passionate about. This helps us
> understand whether ReadMe is a place where they can pursue their long-term
> goals.

Hmm. So many projects I want to work on, especially for an hour or two
session, may not necessarily reflect my long term goals but be more oriented
towards completing something that either myself or I perceive my interviewers
being interested in. Yeah I get the whole "work on what you want" thing but in
such a short timeframe to show off what I can do is absolutely _not_
indicative of my long term goals.

~~~
jghn
On the last bit, perhaps these people are just better developers than me or
perhaps it's an artifact of the sort of things they work on vs what I work on
but I can't think of a single thing I could complete in a couple of hours
which I'd be proud enough of to show as an accomplishment.

I see a lot of these interview discussions talking about having candidates
bang out real features in a couple of hours and I just don't understand how
that's possible.

------
clifanatic
That strikes me as being highly optimized for people who are one or two years
out of college.

~~~
pmiller2
In what way?

~~~
sidlls
I don't have a GitHub (bitbucket, etc.) account, can't remember the last time
I contributed to any open source project, and have very few hobby programs I
work on outside of my job.

Yet somehow I'm consistently rated very highly by my peers, to the point where
I have significant influence on the technical direction of projects I work on.

The reason I don't have hobby projects or dev social media accounts is this:
by the time I get home, I have a spouse and two children who I have a life
with outside of programming. I frankly don't have time to work on hobby
programming outside of work to the level of sophistication that rises to the
sort I'd feel comfortable showing in an interview. And picking some random toy
project that does would quickly reveal how little I care about it.

Edit for clarity: that said, I think this level of transparency is fantastic.
I have nothing negative to say about it.

~~~
mikekchar
On the other hand, it is not necessary for a company to hire every good
developer that walks through their door. What they need to do is to hire
enough people and try to optimise their good to not-so-good ratio. Even if
their process would not select you (a good developer), it does not mean that
it is not a good process.

The original question asked whether this was slanted towards younger
developers. Although I don't interview people this way, I would expect that
good senior developers could pull an interesting kata out of their back pocket
as long as they had enough warning to set up their development environment
ahead of time. Hell, I'm quite sure I could fill up a couple of hours with
FizzBuzz variations and still come away giving a good impression.

Basically, when interviewing for junior developers, I want to see if they can
code and if they can take direction. So it's cool to write any stupid code and
then help them explore it. I'm trying to find out if they can listen to
criticism and if they can utilise that criticism to improve their code.

For senior developers I want a lot more. I want some evidence that they have
spent their time thinking about software. I want to see their opinions, but
more than that, I want them to be able to show why their opinions are worth
listening to. When a senior dev comes on my team, they are not there just to
write code. They are there to inspire others to greater heights.

If you quickly revealed to me how little you care about some random toy
project, I would try to help you see the point. It's not about the toy
project. That project is only a medium for communication. If you still
couldn't see the point, then I would probably conclude that you were not right
for our team. That might be my loss, but as long as my filter selects enough
good people it's not really a big problem for me.

~~~
vonmoltke
> On the other hand, it is not necessary for a company to hire every good
> developer that walks through their door. What they need to do is to hire
> enough people and try to optimise their good to not-so-good ratio. Even if
> their process would not select you (a good developer), it does not mean that
> it is not a good process.

I am led to believe, by the people on this site and others, that unless you
are AmaGoogBookSoft or a Valley darling good developers are so hard to come by
that they can't afford to miss good ones.

~~~
mikekchar
Sampling bias? ;-) I've worked in famous companies. I've worked in obscure
companies. I've met amazing programmers in both. I've met horrible programmers
in both. Most are somewhere in between.

There are good programmers everywhere. Grabbing a couple (ideally including a
senior dev) of really excellent people will go a long way improving your team.
But you need to do more than advertise and interview to get these people. You
need to go out into your community, participate, and create a reputation. You
need to work on your internal processes and be interesting enough for an
excellent dev to be interested in you. My advice for a small up and coming
team is to spend the cash to hire a well known senior consultant with a good
track record. From there you can get your culture set up and you can make
contacts with other good devs that the consultant knows. After that it is
really just hard work to hone everything that you do so that good people think
that it will be a worthwhile career move to spend time with you.

If your management strategy is to throw money at talent and hope that it gels,
I think you will have a lot of trouble. Once you have talent, it's important
that they are set up to succeed. This is more than just turning them loose on
a project. IMHO, good teams have many dimensions to them. You don't
necessarily need (or want) to have a superstar in every position. It can be
good, but it requires a corresponding level of proficiency from management to
get them all working together well.

If you are worried about the one that got away in your interview process, then
I think it is a sign that you have very much bigger problems that you need to
deal with. If you tell me that this is common in the Valley, it will not be
something that surprises me ;-)

~~~
vonmoltke
That's not really what I am talking about. I'm talking about the oft-parroted
"talent shortage" wherein companies claim they can't find enough good
developers. If true, companies cannot afford to let good ones get away. I
personally think "we have a talent shortage" is code for "we don't know how to
hire".

------
seanwilson
Sounds interesting but what if the candidate doesn't have a personal project
or one they're willing to show you the code for as they're working on it?
Also, seems like you could game this by rehearsing what you'd be developing
before the interview (same way you can game questions about binary trees etc).

I think someone having a personal project is a strong indicator that they're
passionate about the trade though which is a great thing.

~~~
gkoberger
That hasn't been a problem so far. A lot of candidates start new projects, and
that's fine too. If you can't think of _anything_ you would want to work on,
that's a pretty bad sign for us.

~~~
seanwilson
What happened in the worst case scenarios? Did anyone turn up with no project
or fumbled at adding a feature to their project?

------
EliRivers
The candidate has to bring their own laptop to work on during the interview,
at their place of business?

I know lots of people have a laptop already, but I'm sure I'm not the only
person who doesn't (I just can't work on one; I get frustrated). Or is that
part of the selection process; they don't want the kind of person who doesn't
own a laptop, or doesn't bring it with them to interviews.

~~~
ddorian43
Or maybe you're more comfortable on your laptop compared to a random pc ?

~~~
EliRivers
On a personal note, I'm far more comfortable with a proper keyboard and a big
monitor (or two), and frankly, so long as the OS has a couple of standard
editors and the standard build tools, surely everyone can start working pretty
much instantly? How long does it take to find the terminal and start typing?
If a candidate's laptop is some kind of security blanket, or they can't do
anything without exactly their particular micro-choice of environment, that's
not a good thing.

Provide a desktop for the candidate to use. Make sure it's got vim, emacs,
gedit, and whatever the cool kids use now (sublime?). Make sure it's got a
recent set of tools for the technology at hand. GCC and build tools or the
equivalent standard. Job done.

~~~
jghn
It's a little more complicated than that. Maybe they're a Windows user and
your machine is OSX, or vice versa. Maybe your keybindings are different than
what they use. In my normal workday you're right, that wouldn't be a big deal.
But in the high pressure situation of a timed interview environment it can
freak people out.

We tell candidates that we'll provide a laptop for them if they don't have one
but also tell them to feel free to bring in their own.

Also to turn your statement around a bit, perhaps one could claim that your
need to have a big monitor (or two) is your own security blanket, and that you
should be able to work without one. In my office some people have oodles of
monitors. Some people just use their laptops. Some are a bit of both. Everyone
has their preferences.

~~~
EliRivers
_Also to turn your statement around a bit, perhaps one could claim that your
need to have a big monitor (or two) is your own security blanket, and that you
should be able to work without one._

The day a job interviewer tells me to bring in my desktop and monitor
selection with me, I'll be the first to complain.

It was "just a personal note". My point is that telling people to bring their
own laptop in for a job interview is odd enough to object to.

------
alphabravodelta
Great post and it's awesome to see many companies try to make interview
process better but my happiness didn't last long.

They require[1]: 'Your GitHub is full of everything from embarrassing hacks to
impressive Open Source contributions'

I understand some reasons behind seeing Github profile of candidate but
requiring it is too much. But this is still better then company i saw the
other day which was requiring Github project with X000 stargazers.

I think we have to choose between bad and worse.

[1] [https://readme.io/careers/#job-full-stack-nodejs-
developer](https://readme.io/careers/#job-full-stack-nodejs-developer)

~~~
mintplant
Plus, what about contributions to projects that aren't on GitHub? I have
commits in Firefox but you wouldn't know it from my GH profile.

~~~
rtpg
Well you can tell them this, right? The essential part is having some code out
there in the open. I imagine if you host your projects on bit bucket it's OK
too.

Though I agree that a lot of skilled people don't have the will/feel the need
to build a public profile of their work.

~~~
genofon
even more, some companies might claim the rights on their work, depending on
the country and company. In my experience (UK) there are a few like this.

------
pzaich
The presence of an itinerary for the interviewee should be a requirement.
Sadly it's not. I've taken two interviews in the past week where I was in the
dark as to what my visit would look like.

------
Ntrails
I don't own a laptop, so I guess that's the defining part of the article for
me. Does not meet minimum requirements. ^_^

~~~
gkoberger
That page is dynamic, so everyone sees something different. By the time a
candidate gets that page, they've already talked to 1-2 members of the team.
Of course there's no "own a laptop" requirement.

------
codecamper
Great attitude. readme.io might consider getting into the tech hiring
business. Spread these ideas around & make some money too.

------
rer
> First impressions are everything.

They're not.

~~~
netule
Yes, they absolutely are. The one thing I look for the most in a potential new
role is cultural fit; pretty much if I think the team will gel with me. Much
of this becomes apparent through the interview process: the type of questions
asked, the level of exhaustion shown, and how interested the interviewers seem
in their own jobs. These things are as important to me as the type of company
and the offer they extend since this is the place where I will spend a great
deal of my time and effort. I'd rather not spend a significant part of my day
with people that I don't like in a place that I can't stand.

~~~
cema
Very true. And, of course, it is important to remember that interviews are in
many ways two-way streets: the candidate interviews the company just as the
company interviews the candidate.

------
JustSomeNobody
How are different engineering disciplines interviewed? For instance, take a
mechanical engineer, are they going to sit down with a stack of tooth picks
and be asked to design a bridge with a Pratt truss? Are they asked to show
their designs on Github?

I seriously think there's an underlying pissing contest that has permeated the
software development culture. If other engineering disciplines can be hired by
talking intelligently and answering questions related to their field, why
can't software engineers?

~~~
DanBC
> If other engineering disciplines can be hired by talking intelligently and
> answering questions related to their field, why can't software engineers?

Other engineers have professional registration and thus some form of
regulation. In some countries you're not allowed to cal yourself an engineer
unless you have some particular qualification and the registration.

~~~
JustSomeNobody
Then maybe we need to push for that. I'm just tired of the way we do things
currently.

------
runcougar
have you gotten any feedback from new hires on this process?

~~~
gkoberger
It's a self-selecting group, of course... but we're all happy with it! We've
also gotten a lot of good feedback from people we've interviewed, since it's
not often interviews feel encouraging rather than adversarial.

------
mwfunk
Lots of people here are describing interviews (from the perspective of the
interviewee) as being an adversarial process, where people have bad
experiences because the interviewer is more interested in being intimidating
or showing up the interviewee, but I wonder how much of that is just the
process itself being intimidating.

For me, job interviews were always incredibly nerve wracking. It feels like
you're being judged, but of course you are being judged- the whole point of
the interview is to judge your suitability for a position. It's also nerve
wracking because I was always interviewing for something I wanted really
badly, that was a step up from whatever I was doing at the time. So, I was
always interviewing from a position of weakness, and it made me feel even more
self-conscious.

At this point in my career I've probably been the interviewer >= 20x as often
as I've been the interviewee. Some were phone screens, the rest were in
person, and about half the time I was one of two interviewers (some companies
do one-on-one interviews, some do two-on-one, and supposedly some do many-on-
one interviews but I've never experienced that (but it sounds terrible)).

In all of that, I don't think I've ever been malicious or tried to show off or
make someone feel bad, and I don't think I've ever witnessed or even heard of
a colleague doing it either. Maybe I'm just lucky, and got all my jobs at
companies that had a less horrible interview process, but I wonder.

Job interviews at companies I really want to work for have been intimidating
because it _is_ intimidating. I wanted to move up in the world, take the next
step, and so it always felt like the people interviewing me were implicitly
people I wanted to emulate. That's so awkward- it feels like approaching
pseudorandom strangers and asking, "I want to be like you, am I cool enough
for you to give me a job here so I can be like you?" You're being measured
against a yardstick, and it feels like the yardstick is the person
interviewing you, because they've already got a job at the company you want to
work at.

So I would have to fight against being intimidated by the people who were
interviewing me, but what I didn't realize was that although I felt
intimidated, they (probably) weren't being intimidating. I was just projecting
what I was feeling onto them, they were just being their normal selves and
sucking at interviewing because everybody sucks at interviewing.

There's no reason for anyone to be a jerk to an interviewee, the interviewers
want to have successful interviews just as much as the candidate. Nobody wants
to drive away a potentially good hire just to act superior for a few minutes
in front of a job seeker that they will never see again for the rest of their
life.

So I would like to think that all of the horror stories about terrible
interviewers come from interviewees projecting their (totally natural) stress
onto other people, but who knows. People can be irrational jerks too
unfortunately. :( I'm also curious if the bad interviewer behavior was
exclusively in one-on-one interviews. Those seem more likely to go off the
rails, since there's no other interviewer to counteract someone who is an
especially bad interviewer to begin with.

~~~
thaumasiotes
> I'm also curious if the bad interviewer behavior was exclusively in one-on-
> one interviews.

I can hardly speak to this in general, but Google always schedules one of your
interviews with two interviewers, one of which is supposed to be training the
other to do a good interview.

I felt that those interviews were much nastier than the one-on-ones (2 out of
2 times).

------
suyash
JavaScript exercises, might as well give them an exam? It's not different than
puzzles at other companies.

------
scandox
Owlbert? "It twit looks to like whoo you're twit writing a to letter whoo..."

------
simplehuman
I am quiet impressed that there is even a niche market for people who want to
host docs. It really just takes 15 minutes to setup a gulp/grunt script to
build and push to cloudfront. After all, we need those scripts for local
development anyways.

~~~
gkoberger
There's a huge market for good developer experiences :) More and more people
are becoming "developers" every day: computer-literate people who want to
build something.

It's easy to push __something __. It 's a lot harder to build something good.
Our goal is to get away from static docs, and show each developer exactly what
they need.

~~~
deckeraga
Documentation is great and all but I think the real huge market here is in
merchandising that mascot of yours.

~~~
gkoberger
Email me (greg@readme.io), and I can send you some Owlbert swag :)

~~~
jtfairbank
Bro, send me some too. :D Email is in my profile.

