Hacker News new | comments | show | ask | jobs | submit login
Ask HN: How do you assess teamwork skills during interview?
20 points by koopuluri 8 months ago | hide | past | web | favorite | 25 comments
The standard technical interview process: 4 technical interviews that cover data structures, algorithms, system design, etc., with perhaps a conversation over food, is not enough to assess how well a candidate would fit into a team.

Do you disagree with this premise? If so, why?

How do you gauge team skills during your interview process? What do you look for? How has it worked for you?




We used to have a two-person team provide a standardized programming exercise, clearly telling the candidate that they could choose how to do it: preferably discussing their approach with us, get help/feedback as they progressed, -or- completely silently if they were more comfortable, or anywhere in between. Generally we'd spend about 5 minutes just explaining how it could work, assuaging their nervousness, etc.

Most people opted to do the exercises while thinking out loud. If they paused for a few minutes or seemed lost, we'd offer hints or ask questions, etc. Let them take the discussion where ever they wanted.

All in all worked very well. Candidates seemed more comfortable than just doing whiteboard exercises and we got a really good feel for how they worked with others. We didn't really care much about the code; we wanted to see how they approached problem solving, how well they responded to feedback or questions about potential optimizations or offers of help. Or what happened if they were shown that their approach was a dead-end, but they still doggedly kept pursuing it.

It revealed personality issues very quickly. Nervousness was normal. Some people were hostile when bugs were pointed out, some simply couldn't get started even with extensive help, and others went heads down for half an hour and then came back up with the answer.

It wasn't perfect, but of all the interview techniques we used over the years, it was the one tool that we never got rid of because it proved to be so useful.


others went heads down for half an hour and then came back up with the answer

How did people who took this approach fare in your scheme? Thumbs up for solving the problem or penalised for “not a team player?”


Thumbs up of course.

We're hiring them to be developers first and foremost, so if that's how they work best, so be it. Then we move on to discussing their solution and if they thought it could be improved in any way. The rationale for this phase of interviewing was mainly to accomplish the following:

1) Can you take a problem statement and show progress in writing code in the language of your choice to solve it? You don't have to complete the entire exercise, but you do need to show us that given enough time, you'd probably get the right answer and that you can explain your logic.

2) Attempt to weed out the personalities who can't take negative feedback or refuse to accept guidance when they're wrong.


A bit out of topic and not really an answer to your question.

Once in an interview for a startup company(~10 people), when I got into the office they had an uncompleted coat hunger from IKEA and there was a person trying to assemble it. They intentionally let me wait for a bit next to him and he started to look confused with the manual so naturally, I offered to help him and got on my knees and started to read the manual.

They told me much later that that was a test and I had successfully passed it.

I am not sure about the validity of such a test and maybe someone else can verify it.


Wow... that's out there. Intuitively it makes sense: I would want to work with someone that chooses to help others around them when presented the opportunity. But then again, I wouldn't feel comfortable taking away points from a candidate if they didn't help - maybe they were stressing over the interview they were about to have, etc. and didn't want to distract themselves.

> maybe someone else can verify it

I'm really curious to see if anyone has implemented such a test over and over again.. The biggest challenge right now with changing up the interview process is validating the change is tough. You need to use the same challenge in a lot of interviews, and track how well the candidates did over time at their jobs, if hired, in order to be able to verify. Tough to do for a startup.


>> they had an uncompleted coat hunger from IKEA

http://worknearyou.net/wp-content/uploads/2013/08/ikea_job_i...


Assembling furniture as part of an interview process for a white-collar job is a little silly, and I can easily see how that might introduce bias if it's given much weight - that's sort of a stereotypically male task, down to the usage of spatial orientation skills to understand and compare the instructions versus the assembled object.


I love to help other people to fix there coding problems. But in this situation I may withdraw unless I really feel person needs help. As per my experience, sometimes offering help hurts ego ( particularly senior brogrammer )


Sensitive egos isn't quite a proof of healthy teamwork players, right?


For me, this kind of assessment is based on a number of factors:

1. Personality and language skills - are they sufficiently outgoing and conversant to be able to interface well with others? Introversion is common in our profession but in order to work on a team you've got to be able to talk to people in person. Panel interviews are good for this; anyone who can keep up decently, seems personable, not stressed by the situation, and manages to leave an imprint is likely a good candidate by this measure.

2. Written communication skills - while arranging the interview, you're going to be emailing back and forth a few times. Ask some questions and see how they respond.

3. Asking about previous teams and their views on team composition, dynamics, and work distribution; handling large workloads, dealing with absence due to vacation/sickness/etc. This is a good kind of question because unlike asking about what their previous teams _did_, which someone can embellish, asking about the actual process of dealing with different scenarios shows their own understanding. Present a few scenarios and ask the candidate what they think a team lead / manager should do in that situation.

It's going to be hard to get a quantitative metric for this, you have to rely on your intuition.


> ...anyone who can keep up decently, seems personable, not stressed by the situation, and manages to leave an imprint is likely a good candidate by this measure.

I'm not sure high stress environment test is a good measure. I've worked with some amazing teammates before that might have been nervous if in such a situation.

>Present a few scenarios and ask the candidate what they think a team lead / manager should do in that situation.

I like this approach.

Regarding the question examples you gave, do you look mostly for how they reason through those situations, and their ability to put themselves in their teammates' / managers' shoes, or do you also have a spectrum of [right --> wrong] answers for those questions.


> I'm not sure high stress environment test is a good measure.

Talking with a few of your future coworkers in a room where nothing is on fire shouldn't be "high stress" - interviews are always a bit of a stress inducing thing, but ultimately if you're hiring someone who may have to deal with tough situations from time to time on a team, this sort of situation should be easy enough for them to handle.

If they're not in so much of an ops role and won't be dealing with things that are on fire, you can always reduce this stress by starting 1 on 1, and then adding people as the interview goes along in order to give them a more comfortable base.

> Regarding the question examples you gave, do you look mostly for how they reason through those situations, and their ability to put themselves in their teammates' / managers' shoes, or do you also have a spectrum of [right --> wrong] answers for those questions.

For me, it's more of a matter of determining whether they've just worked on a team as a member, or if they've actually thought about the dynamics of teams and have their own opinions and ideas about what makes for a high performance team vs. a dysfunctional one. Getting a feeling for what sorts of business processes they like, how they like to communicate, what level of tracking artifacts they require/feel comfortable with, how they think work should be broken down, etc. is a good way to understand if they're going to be someone who works well with the way your team is already composed, and if they're going to be someone who is going to help evolve the team in a better direction.

If they say something that makes you bristle, pay attention to your intuition. If you think, "gee, person X on my team isn't going to like that", consider it, because you want person X to be happy and productive and introducing a new team member who is going to potentially have conflict with them may not be worth it. Every aspect of your evaluation forms a heuristic in your mind that you can use to decide if they're the right blend of worker for your particular team.

I really like trusting a lot of this sort of thing to my hindbrain. There's a lot of magic happening back there that combines things in a big-data sort of way that we don't consciously understand, but when it comes to interactions with other humans, we evolved to be good at this kind of judgement - since it's life or death, in the state of nature - and we should trust our instincts, provided we've tuned them to not be irrationally prejudiced.


Why not simultaneously ditch the weak proxy for capability and ditch the weak proxy for what working with them would be like all at the same time?

Instead, just work with them. Have them pair with 1 or 2 people for a few hours. Maybe even engage them in a small team meeting if one comes up naturally. There's no assessment as good as actually doing the things they'd be doing and working with the people they'd actually be working with.

It's a better, more accurate assessment in both directions. For them to want to join your team and for you to want to have them on it. Its also a ton easier to train your team how to execute this method effectively. The standard alternatives are spurious generally, and what little value can be derived from them requires well-prepared, expertly-trained assessors.

Why contrive a weird edifice?

Disclaimer: am currently a person responsible for reforming and building a high-performing team in a very demanding field across hardware and software (both internal and external) disciplines.


I agree that working with the candidate is the greatest signal for competence and team fit.

My concern is that oftentimes, there's a lot of context to absorb before being able to contribute, and a few hours wouldn't be enough. Don't want to extend interviews to multiple days --> biases against candidates that have packed interview schedules.

I'm thinking we could structure a problem to take a few hours, and pair program with them, with the interviewers swapping out every hour and half (which would also help test how well the candidate brings someone who doesn't have context, up to speed).

Or perhaps, have multiple candidates work together to solve a problem, and observe how they work together.


>> My concern is that oftentimes, there's a lot of context to absorb before being able to contribute,

And that exactly is the problem with interviews where we can't measure what we should really measuring, so despite these marathons and what not, ending up with misfits. I can't explain it better than below: source [0]:

Your scenarios are different, the normal scenario inside a company is: a) Your boss tells the new problem to the team at Monday morning. b) You have your first approach in your mind to the problem. c) The team discuss the problem for 50 minutes. You still don't have a clear idea about the problem and for sure you don't have any idea about how to solve it. But who cares? Is pancake day ! d) Lunch, you think about the problem by your own, for first time in a "serious" way. e) One day later you write an email to clear some doubts about the problem. f) You talk with another member of the team. He notices some potential problems with your approach but also you realize that, after all, your approach is better than his. You walk in the supermarket alley with your wife thinking about how to explain why your approach is better using a metaphor and trying to foresee objections from other team members. g) Thursday you finish those TDD pending upgrades, do the pull request and then you think an hour about the problem meanwhile you are still in the "zone". You have those beautiful "I got it! " moments. h) Friday you "play" with some library that you find googling and you read some code from github from a guy who did something similar. You then notice that there is a part or a step that you didn't consider before, that could change your design to solve the problem in a deep or shallow way. i) After redditing for an hour, yo do a minimal code that solve the functionality. At this time the problem has been processed in your mind for a week from different angles and using different tools. j) Monday at lunch time you take the marker and you write in the whiteboard trying to explain to the team your solution. The team decides to follow your approach. The other scenario is: a) The interviewer tells you an isolated and descontextualized problem. b) Ten seconds later you have a marker in your hand in front of a empty whiteboard. My problem is: how the real cognitive process can be replicated in an interview? Notice that in the real scenario is not the most smart developer but the best "architect" who gets the best solution after many trails and error experiments and dead-ends paths. Of course companies must have a selection process, but the whiteboard kind is just producing a lot of false negatives.

[0]: https://www.reddit.com/r/programming/comments/4h15a1/why_is_...


I don't see how any of those other options you mentioned ameliorate the concern of requiring contextualizing the work for the candidate. You're still going to have to contextualize the fake work. It also continues to favor weird gauntlets and puzzles without any basis for what kinds of gauntlets or puzzles are meaningful to create, nor whether or not you'd know how to interpret the results even if one could squeeze useful meaning out of them.

I also assume you know well ahead of time what kind of work you're hiring for and what kind of work (in general shape) your team will be doing, so that you can schedule the interview or the team's work to appropriately coincide?

I have to hire people who either have an esoteric skill-set or people who will have to rapidly learn one (formal methods, vehicle hacking, control theory, microkernel architecture, etc.), and I've yet to find a decent substitute for just planning ahead and setting up the environment to get the existing team and the candidate directly engaged on something immediate and real. I've seen a ton of bad hires made trying to contrive scenarios and meaning from more traditional methods.

:shrug:


So we just went through the process of interviewing candidates for summer internships. Our interview process is far less technical than most and is almost all about determining how the candidate will fit into the team.

I believe it's called Behavior Based Interviews. But basically the interviewing team looks at a list of qualities and picks a few (~3 per person) that they think are valuable for someone joining the team. For example, I chose to look for: Priority Setting, Compassion, and Self Development. Once we have some qualities chosen, we develop questions that are specifically meant to test for those qualities.

Overall I really enjoy this kind of interview process. I feel like these qualities are more important than technical ability, so it's nice to put more emphasis on them during the interview process (We still do assess technical ability of course). And for us, it results in new team members that I'm excited to work with!


Nice!

> I chose to look for: Priority Setting, Compassion, and Self Development. Once we have some qualities chosen, we develop questions that are specifically meant to test for those qualities.

What are some questions that you used?


So all of the questions had some kind of context/primer that I'd talk about before asking them, but the actual questions were:

Priority Setting - Have you ever felt overwhelmed by the work you've been given? How do you manage the work so that it gets done on time?

Self Development - How do go about seeking feedback from those around you to facilitate your personal growth? What is something that you've learned about yourself recently that you found helpful?

Compassion - Tell me about a time where you worked with someone who was struggling, for whatever reason. What were some of the challenges? How did you go about helping them deal with their struggle?


I spoke to guy once who was researching what makes a high functioning team for some large (unarmed) company. His insight over the various studies he’d looked at was as follows.

1. High functioning teams are in the order of 5–10 people 2. Everyone in the team is respected for what they bring to the group regardless of experience.

If you’ve 20 years experience you’re valued for your knowledge of pitfalls etc. If you’ve 2 years you might be valued for your enthusiasm. Or if your a fresh graduate you could be valued on your ability to question conventions that others do not.

My question is: given this, how would an interview ever hope to sift out a potential candidate. Especially when the qualities of the final aren’t (could never?) be properly specified relative to the team they were due to go into.


> 2. Everyone in the team is respected for what they bring to the group regardless of experience.

I agree with this.

I think there are ways to notice this during interviews. I'm a fan of a junior engineer interviewing a candidate for a senior position - as a way to see how well they interact. Perhaps the question could be more open ended (e.g. explain x,y,z concepts - maybe oven ones that the interviewer doesn't know but wants to learn). If the junior feels like they had a good interaction, were able to learn, and felt respected through the process, great. If the candidate seemed offended by the lack of experience of the interviewer, then not great.

A highly subjective signal though.


That might be a start. But there’s even more subtle things going on.

What about the person who can resolve conflict. The person who always takes on a task to the bitter end regardless how bad it smells. The person who thinks ahead. The developer who loves process and documentation. The developer who can code you out of a tricky situation. The person who’s always trying new things.

Personally I think it’s a lottery. Like putting together a football team but we don’t even know the positions yet. We just know they can play football


Should the candidate fit into the team, or should the team fit into the candidate? Given that most companies want to have a as diverse workforce as possible, it seem reasonable to argue that the least fitting person should get the job. :)


I think you're talking about culture fit. Even when striving for diversity, I want to ensure competency in the role they'll perform at the company, which I feel depends on a combination of technical experience, understanding, ability as well as communication skills, and ability to function well in a team.

I'm trying to think through how to assess team skills well that would also fit into the constraints of an interview.


Code pairing rounds usually give a good picture. How they respond to disagreement from someone lesser in experience than them? How they deal with conflicting ideas etc.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: