
Ask HN: How do you assess teamwork skills during interview? - koopuluri
The standard technical interview process: 4 technical interviews that cover data structures, algorithms, system design, etc., with perhaps a conversation over food, is not enough to assess how well a candidate would fit into a team.<p>Do you disagree with this premise? If so, why?<p>How do you gauge team skills during your interview process? What do you look for? How has it worked for you?
======
HeyLaughingBoy
We used to have a two-person team provide a standardized programming exercise,
clearly telling the candidate that they could choose how to do it: preferably
discussing their approach with us, get help/feedback as they progressed, -or-
completely silently if they were more comfortable, or anywhere in between.
Generally we'd spend about 5 minutes just explaining how it could work,
assuaging their nervousness, etc.

Most people opted to do the exercises while thinking out loud. If they paused
for a few minutes or seemed lost, we'd offer hints or ask questions, etc. Let
them take the discussion where ever they wanted.

All in all worked very well. Candidates seemed more comfortable than just
doing whiteboard exercises and we got a really good feel for how they worked
with others. We didn't really care much about the code; we wanted to see _how_
they approached problem solving, how well they responded to feedback or
questions about potential optimizations or offers of help. Or what happened if
they were shown that their approach was a dead-end, but they still doggedly
kept pursuing it.

It revealed personality issues very quickly. Nervousness was normal. Some
people were hostile when bugs were pointed out, some simply couldn't get
started even with extensive help, and others went heads down for half an hour
and then came back up with the answer.

It wasn't perfect, but of all the interview techniques we used over the years,
it was the one tool that we never got rid of because it proved to be so
useful.

~~~
dasmoth
_others went heads down for half an hour and then came back up with the
answer_

How did people who took this approach fare in your scheme? Thumbs up for
solving the problem or penalised for “not a team player?”

~~~
HeyLaughingBoy
Thumbs up of course.

We're hiring them to be developers first and foremost, so if that's how they
work best, so be it. Then we move on to discussing their solution and if they
thought it could be improved in any way. The rationale for this phase of
interviewing was mainly to accomplish the following:

1) Can you take a problem statement and show progress in writing code in the
language of your choice to solve it? You don't have to complete the entire
exercise, but you do need to show us that given enough time, you'd probably
get the right answer and that you can explain your logic.

2) Attempt to weed out the personalities who can't take negative feedback or
refuse to accept guidance when they're wrong.

------
kostarelo
A bit out of topic and not really an answer to your question.

Once in an interview for a startup company(~10 people), when I got into the
office they had an uncompleted coat hunger from IKEA and there was a person
trying to assemble it. They intentionally let me wait for a bit next to him
and he started to look confused with the manual so naturally, I offered to
help him and got on my knees and started to read the manual.

They told me much later that that was a test and I had successfully passed it.

I am not sure about the validity of such a test and maybe someone else can
verify it.

~~~
tebugst
I love to help other people to fix there coding problems. But in this
situation I may withdraw unless I really feel person needs help. As per my
experience, sometimes offering help hurts ego ( particularly senior brogrammer
)

~~~
kostarelo
Sensitive egos isn't quite a proof of healthy teamwork players, right?

------
core-questions
For me, this kind of assessment is based on a number of factors:

1\. Personality and language skills - are they sufficiently outgoing and
conversant to be able to interface well with others? Introversion is common in
our profession but in order to work on a team you've got to be able to talk to
people in person. Panel interviews are good for this; anyone who can keep up
decently, seems personable, not stressed by the situation, and manages to
leave an imprint is likely a good candidate by this measure.

2\. Written communication skills - while arranging the interview, you're going
to be emailing back and forth a few times. Ask some questions and see how they
respond.

3\. Asking about previous teams and their views on team composition, dynamics,
and work distribution; handling large workloads, dealing with absence due to
vacation/sickness/etc. This is a good kind of question because unlike asking
about what their previous teams _did_, which someone can embellish, asking
about the actual process of dealing with different scenarios shows their own
understanding. Present a few scenarios and ask the candidate what they think a
team lead / manager should do in that situation.

It's going to be hard to get a quantitative metric for this, you have to rely
on your intuition.

~~~
koopuluri
> ...anyone who can keep up decently, seems personable, not stressed by the
> situation, and manages to leave an imprint is likely a good candidate by
> this measure.

I'm not sure high stress environment test is a good measure. I've worked with
some amazing teammates before that might have been nervous if in such a
situation.

>Present a few scenarios and ask the candidate what they think a team lead /
manager should do in that situation.

I like this approach.

Regarding the question examples you gave, do you look mostly for how they
reason through those situations, and their ability to put themselves in their
teammates' / managers' shoes, or do you also have a spectrum of [right -->
wrong] answers for those questions.

~~~
core-questions
> I'm not sure high stress environment test is a good measure.

Talking with a few of your future coworkers in a room where nothing is on fire
shouldn't be "high stress" \- interviews are always a bit of a stress inducing
thing, but ultimately if you're hiring someone who may have to deal with tough
situations from time to time on a team, this sort of situation should be easy
enough for them to handle.

If they're not in so much of an ops role and won't be dealing with things that
are on fire, you can always reduce this stress by starting 1 on 1, and then
adding people as the interview goes along in order to give them a more
comfortable base.

> Regarding the question examples you gave, do you look mostly for how they
> reason through those situations, and their ability to put themselves in
> their teammates' / managers' shoes, or do you also have a spectrum of [right
> --> wrong] answers for those questions.

For me, it's more of a matter of determining whether they've just worked on a
team as a member, or if they've actually thought about the dynamics of teams
and have their own opinions and ideas about what makes for a high performance
team vs. a dysfunctional one. Getting a feeling for what sorts of business
processes they like, how they like to communicate, what level of tracking
artifacts they require/feel comfortable with, how they think work should be
broken down, etc. is a good way to understand if they're going to be someone
who works well with the way your team is already composed, and if they're
going to be someone who is going to help evolve the team in a better
direction.

If they say something that makes you bristle, pay attention to your intuition.
If you think, "gee, person X on my team isn't going to like that", consider
it, because you want person X to be happy and productive and introducing a new
team member who is going to potentially have conflict with them may not be
worth it. Every aspect of your evaluation forms a heuristic in your mind that
you can use to decide if they're the right blend of worker for your particular
team.

I really like trusting a lot of this sort of thing to my hindbrain. There's a
lot of magic happening back there that combines things in a big-data sort of
way that we don't consciously understand, but when it comes to interactions
with other humans, we evolved to be good at this kind of judgement - since
it's life or death, in the state of nature - and we should trust our
instincts, provided we've tuned them to not be irrationally prejudiced.

------
im_down_w_otp
Why not simultaneously ditch the weak proxy for capability and ditch the weak
proxy for what working with them would be like all at the same time?

Instead, just work with them. Have them pair with 1 or 2 people for a few
hours. Maybe even engage them in a small team meeting if one comes up
naturally. There's no assessment as good as actually doing the things they'd
be doing and working with the people they'd actually be working with.

It's a better, more accurate assessment in both directions. For them to want
to join your team and for you to want to have them on it. Its also a ton
easier to train your team how to execute this method effectively. The standard
alternatives are spurious generally, and what little value can be derived from
them requires well-prepared, expertly-trained assessors.

Why contrive a weird edifice?

Disclaimer: am currently a person responsible for reforming and building a
high-performing team in a very demanding field across hardware and software
(both internal and external) disciplines.

~~~
koopuluri
I agree that working with the candidate is the greatest signal for competence
and team fit.

My concern is that oftentimes, there's a lot of context to absorb before being
able to contribute, and a few hours wouldn't be enough. Don't want to extend
interviews to multiple days --> biases against candidates that have packed
interview schedules.

I'm thinking we could structure a problem to take a few hours, and pair
program with them, with the interviewers swapping out every hour and half
(which would also help test how well the candidate brings someone who doesn't
have context, up to speed).

Or perhaps, have multiple candidates work together to solve a problem, and
observe how they work together.

~~~
deepaksurti
>> My concern is that oftentimes, there's a lot of context to absorb before
being able to contribute,

And that exactly is the problem with interviews where we can't measure what we
should really measuring, so despite these marathons and what not, ending up
with misfits. I can't explain it better than below: source [0]:

Your scenarios are different, the normal scenario inside a company is: a) Your
boss tells the new problem to the team at Monday morning. b) You have your
first approach in your mind to the problem. c) The team discuss the problem
for 50 minutes. You still don't have a clear idea about the problem and for
sure you don't have any idea about how to solve it. But who cares? Is pancake
day ! d) Lunch, you think about the problem by your own, for first time in a
"serious" way. e) One day later you write an email to clear some doubts about
the problem. f) You talk with another member of the team. He notices some
potential problems with your approach but also you realize that, after all,
your approach is better than his. You walk in the supermarket alley with your
wife thinking about how to explain why your approach is better using a
metaphor and trying to foresee objections from other team members. g) Thursday
you finish those TDD pending upgrades, do the pull request and then you think
an hour about the problem meanwhile you are still in the "zone". You have
those beautiful "I got it! " moments. h) Friday you "play" with some library
that you find googling and you read some code from github from a guy who did
something similar. You then notice that there is a part or a step that you
didn't consider before, that could change your design to solve the problem in
a deep or shallow way. i) After redditing for an hour, yo do a minimal code
that solve the functionality. At this time the problem has been processed in
your mind for a week from different angles and using different tools. j)
Monday at lunch time you take the marker and you write in the whiteboard
trying to explain to the team your solution. The team decides to follow your
approach. The other scenario is: a) The interviewer tells you an isolated and
descontextualized problem. b) Ten seconds later you have a marker in your hand
in front of a empty whiteboard. My problem is: how the real cognitive process
can be replicated in an interview? Notice that in the real scenario is not the
most smart developer but the best "architect" who gets the best solution after
many trails and error experiments and dead-ends paths. Of course companies
must have a selection process, but the whiteboard kind is just producing a lot
of false negatives.

[0]:
[https://www.reddit.com/r/programming/comments/4h15a1/why_is_...](https://www.reddit.com/r/programming/comments/4h15a1/why_is_hiring_broken_it_starts_at_the_whiteboard/)

------
goldenbeet
So we just went through the process of interviewing candidates for summer
internships. Our interview process is far less technical than most and is
almost all about determining how the candidate will fit into the team.

I believe it's called Behavior Based Interviews. But basically the
interviewing team looks at a list of qualities and picks a few (~3 per person)
that they think are valuable for someone joining the team. For example, I
chose to look for: Priority Setting, Compassion, and Self Development. Once we
have some qualities chosen, we develop questions that are specifically meant
to test for those qualities.

Overall I really enjoy this kind of interview process. I feel like these
qualities are more important than technical ability, so it's nice to put more
emphasis on them during the interview process (We still do assess technical
ability of course). And for us, it results in new team members that I'm
excited to work with!

~~~
koopuluri
Nice!

> I chose to look for: Priority Setting, Compassion, and Self Development.
> Once we have some qualities chosen, we develop questions that are
> specifically meant to test for those qualities.

What are some questions that you used?

~~~
goldenbeet
So all of the questions had some kind of context/primer that I'd talk about
before asking them, but the actual questions were:

Priority Setting - Have you ever felt overwhelmed by the work you've been
given? How do you manage the work so that it gets done on time?

Self Development - How do go about seeking feedback from those around you to
facilitate your personal growth? What is something that you've learned about
yourself recently that you found helpful?

Compassion - Tell me about a time where you worked with someone who was
struggling, for whatever reason. What were some of the challenges? How did you
go about helping them deal with their struggle?

------
w4tson
I spoke to guy once who was researching what makes a high functioning team for
some large (unarmed) company. His insight over the various studies he’d looked
at was as follows.

1\. High functioning teams are in the order of 5–10 people 2\. Everyone in the
team is respected for what they bring to the group regardless of experience.

If you’ve 20 years experience you’re valued for your knowledge of pitfalls
etc. If you’ve 2 years you might be valued for your enthusiasm. Or if your a
fresh graduate you could be valued on your ability to question conventions
that others do not.

My question is: given this, how would an interview ever hope to sift out a
potential candidate. Especially when the qualities of the final aren’t (could
never?) be properly specified relative to the team they were due to go into.

~~~
koopuluri
> 2\. Everyone in the team is respected for what they bring to the group
> regardless of experience.

I agree with this.

I think there are ways to notice this during interviews. I'm a fan of a junior
engineer interviewing a candidate for a senior position - as a way to see how
well they interact. Perhaps the question could be more open ended (e.g.
explain x,y,z concepts - maybe oven ones that the interviewer doesn't know but
wants to learn). If the junior feels like they had a good interaction, were
able to learn, and felt respected through the process, great. If the candidate
seemed offended by the lack of experience of the interviewer, then not great.

A highly subjective signal though.

~~~
w4tson
That might be a start. But there’s even more subtle things going on.

What about the person who can resolve conflict. The person who always takes on
a task to the bitter end regardless how bad it smells. The person who thinks
ahead. The developer who loves process and documentation. The developer who
can code you out of a tricky situation. The person who’s always trying new
things.

Personally I think it’s a lottery. Like putting together a football team but
we don’t even know the positions yet. We just know they can play football

------
bjourne
Should the candidate fit into the team, or should the team fit into the
candidate? Given that most companies want to have a as diverse workforce as
possible, it seem reasonable to argue that the _least fitting_ person should
get the job. :)

~~~
koopuluri
I think you're talking about culture fit. Even when striving for diversity, I
want to ensure competency in the role they'll perform at the company, which I
feel depends on a combination of technical experience, understanding, ability
as well as communication skills, and ability to function well in a team.

I'm trying to think through how to assess team skills well that would also fit
into the constraints of an interview.

------
sidcool
Code pairing rounds usually give a good picture. How they respond to
disagreement from someone lesser in experience than them? How they deal with
conflicting ideas etc.

