Hacker News new | past | comments | ask | show | jobs | submit login
Hiring Without Whiteboards (github.com)
481 points by kipply 17 days ago | hide | past | favorite | 450 comments

We use a 3 step process, sans whiteboard: 1. First interview, the candidate interviews us. What market we serve, what our development processes are, what our technology stack is. If they express interest by being prepared and asking good questions, we send them home with

2. a programming task. Choose 1 of 5 tasks. The tasks are not abstract problems or puzzles, but come out of the designs we've implemented. We ask for 100 lines of code and not more than 2 hours. They take as much calendar time in days as they need, most have full time jobs, and let us know when you're done. The candidate then comes back, typically in 2 to 7 days and hosts a code review in front of our team. I give strict instructions before hand: The purpose of the review is to learn how they thought through the problem and how they solved it, not to criticize their style and approach.

3. If we decide we like them to this point, the third interview is with managers from other functions. How well does the candidate communicate, come across to non-developers, express interest in the company and role, etc. It's a check point to look for concerning weaknesses, as well as get buy in from the broader organization.

We like this approach because it allows the candidate to code in a much more natural environment. They can take the time they need and comfortably solve. No one spends their professional career coding on a white board. This approach so far has yielded excellent results. We ask developers how much time they took in the programming task and why they chose the one they solved. The programming task is telling, not in the quality of their code and how long they took, but how much did they get into it? We have had the range from candidates who did not complete it at all and opted out, to candidates who stumped 40 year veterans with elegant code. In every case, we learn how well they can express their thoughts, and importantly, their level of love for the discipline. This is as important to me as any other attribute.

A lot of people in other comments below are criticizing take-home tasks, but honestly, this sounds nice and balanced. I'd much rather have a couple days to think about a problem and a few hours to implement it over having to take time off work, pay to go all the way to your company at a time that works for both of us, and deal with a white board interview. It's a way bigger investment. So long as it's a small, confined, practical problem and not a full app, this sounds good.

Plus, if your company gives someone what they might think are ridiculous questions or they realize it's just not what they're good at, they can easily reject proceeding any farther with the interview. If someone walks into a crappy whiteboard interview, they feel kind of stuck. Seems to be better for both sides. Less pressure.

Despite having a family not much spare time, I also like the take-home approach.

But any time take-home tasks come up on HN, there are always people here complaining about it and even demanding to be paid for their time.

I get that it doesn't suit everyone, but nothing does. Coming up with an approach that works for the company and candidates is really hard - as evidenced by the many, many threads on this subject here on HN.

Perhaps the best compromise would perhaps be to permit the candidate to perform the task there and then or take it home, but this may require more of a commitment from the interviewer, if they then need to additionally devise tasks to be completed in a shorter time.

I oppose take-home tasks as an interviewer. I don't think employees or prospective employees should be taking work home with them, ever. Instead, collaborating on a problem with the interviewee, discussing the problem and potential solutions, observing how the interviewee recognizes when they go off-track, how they respond to hints and suggestions... these are all better signals than the culmination of some homework task, and easily captured in a 30-45 minute interview.

> collaborating on a problem with the interviewee

That's not how developers spend the vast majority of their time. It's a poor signal in my experience. What it selects for is candidates who are immediately comfortable collaborating in real time with someone they have never met. That doesn't mean you aren't getting good candidates. What is means is that I spend far less than you finding good candidates because I also hire people who are great developers and who would fail at your interview.

If you're working on hard problems, 30-45 minutes isn't even enough time to fully context shift and get into flow, let alone in an interview where you are a) stressed, b) trying to impress, c) unfamiliar with the specifics of the organization, d) potentially unfamiliar with the industry entirely. This simply isn't enough time to get a sense for how someone thinks or approaches things that are going to be impactful to a large organization except at the most superficial levels. However, if they have a couple hours at their leisure to approach a problem, it's very easy for them to go over how they approached it and where they got stuck in just a few minutes.

The compromise here is to talk about the employee's past projects. The downside is now the interviewer needs to get up to speed quickly, and won't have all the necessary context of the organization and (potentially) industry. Not to mention the fact that candidates lie a lot about how much of a role they actually had in projects.

We have both elements. The code they wrote is subsequently reviewed live with 3 or more of our engineers and all of these aspects are explored.

I think what gives our approach balance is that we are also committing to making a time investment. Reviews typically last an hour, which means that the 2 hours we ask from the candidate is matched with 3-5 hours of our engineers' time to review and discuss with them.

>1. First interview, the candidate interviews us. What market we serve, what our development processes are, what our technology stack is.

I somehow feel this rewards opinionated developers.

Also right now I'm looking at a couple places because their recruiters called me up and said they were willing to meet my requirements which are basically pay - I don't care about most of these things, if they want me to do stuff and are willing to pay my rate and if they, after looking over my cv believe I can handle their stack, that's it. I don't really have any questions for them.

So I guess you don't have people looking for people for you - you only have people coming in that think your company is special enough that they really really want to be there and are invested about finding out everything just to confirm that commitment on their part is good?

on edit: this of course supposes that 90% of the companies out there are very similar and do not have any problems that are so interesting that I should feel prompted to ask. If I ever applied to the Internet Archive I'm sure I would have things I would like to ask.

Aren't opinions something an employer should want in developers? And not just opinions on tech stack but all the way up to requirements etc.

If I hire a developer "to build X," I would rather work with the developer who explains that X is the wrong thing and we should actually build Y, than the one who shrugs, asks no more questions, and builds a really excellent X - that later turns out to be the wrong thing to have built.

At the interview stage, the candidate has been looking at your use case for at most a couple of hours with very little context. Assuming you've done any work on X and that you are competent, X is probably a good choice or at least there's good reason to believe it is a good choice. It might be nice if the candidate asks why you chose X over Y, but they shouldn't seriously propose that you should switch over to Y. Best case scenario they're assuming their little glimpse gives them a deeper understanding of the problem than you, which suggests they're cocky, worst case they may be deliberately selling you on a potentially worse solution that they just happen to be more comfortable with, ie they're inflexible. A good developer should wait until they know what they are talking about before they form an opinion on what would be the right tool for the job.

There's a difference between opinionated and having an opinion. I think Rust is much more pleasant to work with (an opinion), but I'm not going to let it stop me from using other things (not opinionated)

Something seems badly off to me with the phrase "candidates who stumped 40 year veterans with elegant code".

I don't want code that stumps anyone. I want code that delights 40-year veterans, for sure. But the signal of genuine elegance is "oh, that's a lovely way to do it" much more often than it's "how the hell does this work?".

It was more of 'I didn't know the C language could do that'. The elegance was that what it did was obvious.

How do you avoid awarding candidates who violate your request of no more than two hours and spend considerable time coming up with something amazing? They may still produce 100 lines of code — but those 100 lines may end up being unusually elegant, beautiful code that stumps 40 year veterans. How do you avoid punishing someone who took your requested time constraint seriously (or otherwise didn’t have enough free time to violate it anyway)?

That’s not to say I don’t think this process has promise, but I’d need to be convinced that the above two issues can be fairly dealt with first.

I'm of the opinion that this is fair enough as long as the pay isn't crazy high. The truth is, no interview is ever truly fair, and all interviews can be gamed. "Grinding leetcode" is much the same problem you are describing wrt spending 40 hours on a 2 hour project. People game non-technical interviews too. And resumes.

It sucks, but you only need to be better perceived than the next best competitor. Most folks don't have 40 hours to spend on a 2 hour max coding challenge. Hopefully the ones that do are already employed and not currently competing with you, or are so inexperienced that 40 hours produces horrific over-engineered abominations instead of clean efficient readable code.

I reject entire the premise of this question. Anyone spending huge amounts of time is either going to produce way too much or they just took too long. There is nothing magical about programming where taking 8 hours to solve a 2 hour problem is going to stump 40 year veterans or be unusually beautiful. That's just unrealistic.

You are taking it to the other extreme. Sure, in 8 hours I will not write something "unusually beautiful". But if I spend two hours solving a problem and hacking together a solution, I come to better understand both problem and solution. Hereafter, I then can rewrite my solution in another couple of hours and improve it significantly with respect to maintainability, readability, naming, messy choices, &c.

If I want this job, I would spend this extra time to submit this better solution because obviously that will improve my chances of getting the job. But that means that these take home assignments might not be such a good indicator of efficient proficiency. You might be overvaluing people spending more time on these tasks and undervaluing people spending the allotted time on these tasks. For example, a better developer who did write a solid solution in under 2 hours might get passed over because her solution seems not as good as mine where I did spend another hour to rewrite my initial messy solution.

That's not my experience at all. To a limited extent, you can get something more beautiful by just putting more work in it.

Actually, if they spend more than two hours, the question is why. We are happy to see them get interested in the task and dive deeper, but we just don't want to ask for more, that would be unreasonable in our view. It's fine if the code is broken, or has a couple of bugs. (It's best if it compiles though!)

The goal is as much to learn how they think, solve problems and communicate as it is whether they know how to write software.

I'll give one example: One task is to figure out how long it takes to grab the current time from the operating system. We use a rather obscure OS. A candidate chose this and solved it simply (though it's got a couple of points to mind), but then he got curious and wanted to know whether a virtual machine had a different timing and then what the performance on a different operating system was. He told us that he spent about 8 hours altogether. I felt a bit bad on one hand, but on the other, it's exactly what we like to see: Solid programmers who love programming technology for its own sake.

Something i've seen a couple of times is giving a take-home exercise, then working on it further in the on-site interview. The first stage filters out people who can't get things done even given as much time and help as they want, the second stage filters out people who can't get things done at a reasonable pace without a helping hand.

I think it's common for an on-site interview to include a reasonable chunk of programming, and using a continuation of the take-home to do that is good because it means the candidate isn't starting from scratch. Coming in warmed up should minimise noise from nerves, unfamiliarity with tools or context, etc.

This. I just can't see this not penalizing conscientious candidates.

Damn, can I do one just for fun and then let's code review together? :p

I recently had a humiliating experience. Just was approached on LinkedIn and at first they sent me a take home. I learned the front-end tech which was required for it, and then coded it. They just wanted me to fill out a doc but I put it all up on stackblitz. Another portion was back-end which I also put up on another online env so they could see not only the code but how everything was laid out.

Get to the over the screen share interview - just nervous and both the interviewers were impatient, and just the atmosphere was so nerve-wracking - they did nothing to make it more relaxed. Just seemed like the senior guy was stressed out and I was wasting his time. My brain kind of shut off and the seemingly simple problem at that time just seemed insurmountable because I felt myself second guessing my every thought.

Been developing for a number of years for a few companies, and I guess I haven't prepped for the tech interview - I thought my existing skills are what they need and should be enough to evaluate but boy was I wrong.

Needless to say they didn't go with me. It's not like I needed the job, or probably wouldn't have taken it anyway since what their maximum range was lower than what I make, but the startup seemed quite interesting. Just such a bad experience.

That's annoying, but sounds like you avoided a bad team environment anyway?

If they're not placating you at all than I doubt that'd be a great environment to work in. Or they'd already selected someone.

A key point in your process is that you have a conversation before the homework phase. The last time I was job hunting I would be given homework without ever having a chance to talk to someone. I did that once, sent my work into an automated system and never heard anything back besides an automated thank you.

I think if I was interviewing using your process I would submit code because I would guaranteed a chance to talk about it with other developers.

Yes, if you aren't planing to invest in me. I won't do your homework. In this way you will only get desperate candidates.

Why not time the programming task? Since it's supposed to take about two hours you could ask them to choose a two hour slot, email them the task at that time and have them return it two hours later. This wouldn't penalize people who choose to abide by the time guideline vs more dishonest candidates.

When I have limited amount of time even if it's more than enough I get stressed. When I'm on a job I have time to think about the problem, come up a rough solution in my head then start coding. If I have a hard limit I don't think that much because what if I don't have enough time to write it down? In the sort thinking phase I came up with a subpar architecture, but even if I have an extra half hour in the endI can't really change it or worst You cant finish it because I didn't think of an edge case.

If your slot starts, and then something comes up - your elderly grandmother phones, your internet goes down, your roof springs a leak - that would be pretty annoying.

What about visiting a coworking space and blocking your grandmother's number?

Temporarily only

> The tasks <...> come out of the designs we've implemented.

I guess the tasks are something like "take this scaffolding and implement thing x". What would you think if the candidate responded saying the scaffolding does not allow to implement X properly without sacrificing things y and z, and instead of turning in code for review would show up for a discussion about the compromises and deficiencies in the scaffolding?

> if the candidate responded saying

Responded how, when? (At what point in the process)

> At what point in the process

> > instead of turning in code for review

I.e. something like instead of turning in code example they turn in a motivated analysis of why the scaffolding provided cannot support "proper" (and then analysis of so called proper) implementation of the thing asked, provide analysis for changes required in the scaffolding.

Basically, the candidate would turn "provide and review code" process into "let's talk about architecture, implementation and tradeoffs". Would you discard such a candidate for not following the process because you expect seniority or would higher level thinking be a plus?

How do you feel about paying candidates for the time they spend on the assignment?

I once applied at a company that started with a programming test before any human communication. I think I did a pretty good job on the test. After that I was rejected when I assume they saw my resume. They gave me $50 store credit on their website (custom sticker printing). I assume it cost them a lot less than $50 but it was still pretty cool.

This past week one of the companies on "who's hiring" sent me three challenges that "would only take 2 hours" before I ever got to talk to a human being or even got a human being responding with an email. This was a fully automated system apparently. "Take this programming test, we'll let you know if you're good enough to talk with a human."

Whilst the challenges were not particularly complex, you also had to ensure that the person who was going to run your code could stand up your environment, so part of the second challenge was "thoroughly document how to run your code, the environment needed, etc, etc."

Life is too short to deal with this kind of stonewalling.

We (the whole developer community) really need to stop responding to such companies.

I did one recently after the initial HR person chat. I spent a morning on it as they suggested 2 or 3 hours. But then I was rejected for the most pedantic reasons (2 white space PEP violations), not setting up mocks for testing (that would have doubled the scope of the project). Providing an HTML response rather than JSON when the spec asked for a "page, don't bother about styling". And various other trivialities.

I tried passing on my complaints, but the HR girls didn't like that. Pretty disrespectful after giving up a morning of my free time.

Sennder is the company, as I feel that we should be naming and shaming.

I just completed this exact challenge (Yes, for Sennder) and I think their challenge is fine. It is not really difficult or takes much time at all. They didn't seem too strict in their review and I was able to discuss some of the "mistakes" and choices I made in the tech interview. Including some PEP8 violations. Despite these mistakes I passed the tech interview just fine and I just have the CTO interview left.

I don't see how mocks would have doubled the scope of the project. I think I have mocks set up in a few lines of code or so. If your project is so difficult to test or if it is difficult to set up mocks for it maybe there is a serious flaw in your project structure?

> Providing an HTML response rather than JSON when the spec asked for a "page, don't bother about styling".

I don't understand what you mean with this. Did you provide a HTML response or did you provide JSON? I did a super simple HTML response with no styling and that was fine. If you did JSON I could see how they would mark it as a mistake when they clearly asked for a HTML page.

They did say that they were planning on changing it so maybe you got a different one. I gave them HTML, and they claimed that "JSON would have been better" despite the specification asking for a "page" with "no need for styling".

It sounds like your submission was unrefined, maybe amateur. That's what tests like this are for. Referring to "the HR girls" isn't winning you any points either, and makes me wonder what kind of attitude you displayed to the people involved.

Not as amateur as the spec.

I turned down an offer from a direct competitor company because they tried to funnel me through an automated programming "test". I told them I have years of experience at their competing company whose hiring bar is notoriously high, and they still demanded I take their weird test.

The company was Goldman Sachs. Name and shame.

You need to try to see it from the hirer's perspective too.

I don't interview much these days, but I spent several years of my career sifting through CVs and interviewing candidates.

The number of candidates with seemingly impressive CVs and apparent several years of experience, yet whom basically couldn't code, was shockingly high. And when I say "couldn't code", I'm talking "couldn't even fizz-buzz" levels of incompetence.

The fact is that there are many devs out there who coast through jobs, while all the time it's their colleagues who are doing all the work.

I wasted, many, many hours of my life before realising that it was pointless interviewing candidates that couldn't code - asking them to perform a sort of gatekeeper test improved things immensely.

So it might seem insulting to have to perform these entry tests, but unless you already have code publicly available (e.g. on GitHub), you shouldn't take it personally.

I had similar experience with Facebook, not directly competing as they are just expanding into my area of expertise, but they had been looking to poach. The recruiter told me that the first step will be a test to determine if I can program so I told them I am not interested without even trying to negotiate out of it.

> a test to determine if I can program so I told them I am not interested

It seems you don't enjoy coding?

No, not writing code per se.

I created a similar task at my company as the parent commenter, building a small frontend to an existing API that can be completed in a couple of hours. There's no time limit for turning it in (we suggest two weeks but don't enforce it) and we specifically tell people that a good, concise solution to the problem at hand is perfectly fine, and even might be better than a sloppier solution that tries to add bells and whistles.

When I set it up the goal was to avoid a lot of the issues I've personally had or heard about with coding challenges: they take way too long, they're too abstract, or you feel like the company might just be using you to get a couple of days of free work on their actual codebase. For this reason I feel fine about not paying people, as it's just a small part of the application process and not free work for us.

I think it's worked pretty well so far. It happens after the initial phone screen and before the first technical interview. We send people the challenge, they return it whenever they want, and if we like it we set up a technical interview. Since the challenge uses our tech stack and is similar to the work we actually do, a large part of the first technical interview is discussing their solution. Why they chose certain patterns, why they added a certain library, how they'd consider testing it, why a certain function might be slow with 10k entities, and so on.

I would feel bad about it.

You don't normally pay candidates for the time they spend doing the whiteboard interview, do you? You can view the take-home task as a replacement of the whiteboarding session, equally with no pecuniary obligations for the candidate.

Another thing, what if it seems some people just want the money, and didn't try seriously

Then what? Would it feel good to pay

Do you have a few examples of tasks you assigned? I struggle to come up with tasks that I think have the right level of complexity, but which can also be completed in a reasonable amount of time.

Much better thought out than any I have performed in on either side. Thank you for sharing, this would be a good template for others who need one to follow (which many do). You also hiring?

Do you have some kind of a screening process before the first interview?

Typically an introductory phone call: ‘is this the kind of job you’d be interested in.’ Maybe that’s step 0.

This sounds like a pretty reasonable process, are you guys hiring?

Digital Dynamics, a small company in Scotts Valley, Ca. We make specialty industrial control computers.

Great approach. How do you shortlist candidates for the first step?

Resumes & phone calls if they look interesting.

I interviewed with one of the companies on that list a few years back (noredink). They gave a timed hackerrank style coding question as round one (so I guess technically not a whiteboard). I passed that, then I had an interview with an actual person.

He asked me vague question, like what is architecture, I started to reply with what design tradeoffs I made on the app I was working on. He literally laughed at me, and said that is not architecture. I was shocked that someone would laugh at a candidate but said ok, what does architecture mean to you? He responded, I ask the questions here not you. I immediately ended the interview. Worst interview I have ever had. Every time I see that company brought up, I think back to that experience.

I've had one such interview go really bad. I am normally a very trusting and confident person, but that interview destroyed me for days. This was for an internship on my first year of uni when I just started learning to code C and C++. The position was for a Junior programmer (which I was).

At some point the interviewer asks which I would choose in a project, C or C++ and reasons why. Basically answered that it depends on the project, but that both languages had their pros and cons, and it all depends. I told him I did not that many projects under my belt at the time.

Interviewer got somewhat pissed off and kept asking me about specific languages constructs which I was not yet familiar with. In a way felt like he was trying to ambush me and make me feel incompetent. I mentioned to him that I was not yet experienced enough to answer these questions, but was eager to learn. Well he did NOT like that either and ended the interview promptly.

It seems like the guy really did not like me (which is fine), but also wanted to turn this interview into some type of power trip ....

Had this been today, I would have politely declined and left. But I was a young and eager cub. Live and learn!

I rarely see it talked about just how much ego plays a part in interviewing. In fact, I'd wager it plays one of the biggest parts of the interview process. If you get even one interviewer that is bitter, unhappy, on a power trip, or otherwise a sociopath, you're not getting hired. Many folks hate interviewing candidates; it can put them in a bad mood. They're going to take it out on you because it's safe.

Ego issues sometimes apply to the interviewee as well. I once had to interview someone that clearly thought the interview process was beneath them. This person answered every question as if it was a stupid endeavor and we should just hire them since it was just obvious how great they were.

What would have been a way to find out s/he had that mindset, before having the interview?

A crystal ball?

All jokes aside he was not foaming at the mouth or unpleasant when we first started the meeting, and I'm not the quickest one to pick up on subtle body language cues. So your guess is as good as mine.

interesting that weren't any "signs" before

What type of signs do you have in mind? Maybe some examples of your own experience will help me understand missed clues.

> Basically answered that it depends on the project ... it all depends

Sounds like a good judgement thing to reply :)

Thank you so much for actually naming the company.

Too many times we read someone's experience, which even if only half true, would be horrifying, yet no indication of who the company was.

Worst interview you ever had, but ending it was one of your best moves. Imagine having to work with someone like that!

Yeah, some interviewers enjoy the interviewee/interviewer power imbalance and like to play with it, as in this case in a sadistic way. I think that he wanted you to guess what he had in mind but when you put him on the spot to give you his definition of architecture he didn't have a solid answer or was afraid you could easily scrutinize him.

What does architecture mean to you? He responded, I ask the questions here not you. I immediately ended the interview.

Again, thanks so much for naming the guilty party (NoRedInk) in this case. So that we can firmly add them to our list of companies not to bother engaging with, if this is the kind of behavior we can expect from their interviewing team.

The above anecdotes speaks volumes about the mentality that drives companies like these -- and the "culture fit" they're apparently looking for.

I had the same experience on some vague Postgres question for a Rails job when I was junior. I asked a similar question along the lines of: how often do you use this day to day here?

Similar response.

For many engineering candidates, we ask them to design the data schema for a calendar. I’m sure my employer and the question are not a secret. We have various follow-ons, like handling the wild number of ways meetings can be periodic, exceptions, and so on.

Nobody has ever asked me how often we need to build calendaring software, because I explain right up front that while this is a “toy” question, our core product functionality schedules people, and nearly every feature from a calendar app has some analogue to things we either do, or are asked to do but haven’t prioritized yet.

I think it’s ok to ask “toy” questions, but I also think that there should be a ready answer to the question “Does this have anything at all to do with the job?”

p.s. We don’t ask a question directly about scheduling, for a simple reason: Almost everybody understands the basic idea of a calendar, so it’s a more “level playing field” for candidates to think about calendars than schedules.

Schema is something you work with everyday so that is fair. Even if the particular schema is for a made up problem it's fairly close to reality.

It is still a whiteboard question according to the definition on the above mentioned github page - which is correct.

Name and shame the company. It's the best way to get them to change their behaviour.

They did name the company (NoRedInk)

Didn't see that initially.

This is a great list. Thanks for sharing it.

I actually had the pleasure of interviewing at one of these companies before. They had a take-home project, which was to choose and implement and couple enhancements to a toy app. In the subsequent conversations, we discussed how I approached the problem, details of my design, technical tradeoffs, etc -- all the sorts of things you would expect a professional sw engineer to be able to do on the job. It was challenging but fair, and I was struck by how reasonable the process was. It left a wonderful impression on me.

Take-home projects are not a perfect solution to the interview problem. A major issue is that they require candidates to have free time outside of work to complete them. Personally, though, I'd much rather spend a few hours per application working on projects than countless hours prepping for stressful whiteboarding puzzles. As a developer who prefers to digest problems slowly, I look forward to the demise of on-the-spot whiteboarding interviews.

If you work at a company that subjects candidates to such high-stress algorithmic riddles, and you have the ability to make your process more sane, please consider following the lead of cos on this list.

Take-home tests are fine in isolation, but if every company does that you'd pretty much be spending all your time doing take-home projects and nothing else.

Also many companies will insist on a take home project in addition to everything else, including a whiteboard interview. So you can pour time and effort into this project only to be eliminated further down the line (or just ghosted).

It's like they assume applicants are applying to one job at a time and have no other work or life commitments.

I dunno, it feels like the tech industry has forgotten how to interview candidates, as in "just chat with someone like a normal human being to see if they are a good fit for what you need", and like many other aspects of the profession have fallen for dumb processes, fads, cargo-culting, outsourcing to third parties, and dubious technical solutions in lieu of experience and social interaction. I learned this early in my career, and found I can pretty much size up someone's technical chops - assuming they're in my wheelhouse - and get a good idea of their professionalism - with a short, half hour phone/Zoom interview. Yes, I can see how this might lend itself to bias and prejudice, but it seems the solution there is to train interviewers to overcome biases and have more diversity (in all senses of the word) in your interviewing team.

Just look at their GitHub and if it looks good, dive in at the interview to make sure they actually wrote the code. Isolated take-home tests seem ridiculous to me if the candidate has public code available. You're filtering out anyone who likes to avoid duplicated effort.

I don't think you should require GitHub, but if a candidate came to me with projects on GitHub I'd skip all technical aspects of the interview, since they have already demonstrated their technical ability.

Requiring github as a CV is worse from the POV of not forcing people to spend a lot of time outside work programming.

Not everyone has the inclination or time to code outside of work. Many of these people are still very competent developers.

If a person has some interesting Github activity (that they can back up) I would certainly discuss it with them. If I'm hiring for a Django developer and the candidate has contributed to the Django core or a related library, that would be a strong signal - but it should not be a requirement. A strong candidate is typically a mixture of good work history, side projects, hobbies and interests, personality and other things that indicate they would be a good fit for your team. How much and in what way depends on the job and the person. Maybe the startup company needs a person who does lots of side projects, builds stuff on their own etc because you need self-starters that don't require lots of hand holding and are ready to take the initiative to learn. On the other hand your 1000-strong engineering subdivision at big corp needs more heads-down 9-5ers who just produce reliable output. Maybe the things you ask a junior dev out of college are different to what you would ask a mid-career developer. There's no one size fits all approach that people seem to be looking for.

Sure, that's why I said "if" - if they don't have a public presence then give them a test.

But frankly I don't really buy that excuse. Those people have time for multiple coding tests but they don't have time for a (small) public project? Ehhh. Not convinced. If you can get away with no portfolio to begin with then you're not going to be inclined to accept coding assignments.

I hate take-home projects more than I hate whiteboard interviews.

I think the best approach would be giving the candidate a broken app and have they debug it live or in a take-home debug-and-fix or debug-and-improve exercise. Could also have some refactor assignment.

Has any company tried something like this before?

I don't understand how take home interviews are being held up as a better solution. They have two major flaws:

They lower the cost for the company to zero. So they can widen their pool, and take a chance on more candidates, with no extra cost incurred for them. If you take this to the extreme – every company asking every candidate to complete a take home – then the result is far worse for candidates.

There is no feedback mechanism for candidates during the process. I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable. On occasions where we had a system for capturing progress, I found that most candidates far exceeded the time recommended to them. Some were spending up to six hours. The frustrating thing was the extra time never made a difference to the outcome. I could tell from their initial hour whether they were going to pass.

As a hiring manager I've used them, but work hard to be considerate to the candidate. I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.

My company does take-home interviews (or rather, we give candidates the option to do them; nearly all candidates choose take-home instead of a live coding session) and I don't think I agree with the "lowers the cost to zero" point. First of all, they are never the starting point; we only give a take-home problem to a candidate after we've had an initial discussion with them.

When we get a take-home submission, we essentially run it through our code review process, and as one of the engineers who does reviews, it usually takes me nearly as long to thoroughly examine a homework solution and write up my feedback for internal discussion as it would to do an in-person interview.

Being able to review a candidate's code on my own schedule rather than being yanked from the middle of some knotty problem to go do an interview at a fixed time is a big quality-of-life win for me. But the rate limit isn't much different than it would be with regular interviews.

All that said, it's totally possible our process is super atypical.

At Caltech, most exams were take home with a time limit of 2 hours. Some were "infinite time". The students hated the latter, because it means there was no end to the time you needed to spend on the exam, meanwhile there were other exams, etc., to take.

With a 2 hour exam, there's an end to it, and you know how much time it should take to do the problems. Much better.

It's even worse if you're being graded on a curve, as you would be in a job interview. You're competing against others who will spend infinite time on it.

Like you said, "there were other exams, etc., to take.". Presumably other competitors have other exams as well? You'll thus be competing against people with different time management skills.

> I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable

> I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.

Why don't you just let candidates decide for themselves how much time they want to spend on the task? You can set the expectations, saying that you expect that the task of a given complexity will to take between 1 and 2 hours; but candidates should be free to spend as little or as much time on a task as they like. Some candidates will value their time and spend no more than is advised; others will be sufficiently obsessed by the task to spend more time. Why should the latter group feel guilty about exceeding the recommended time limit?

You're trying to evaluate a candidate's ability to accomplish something with finite resources. If you allow someone to put additional time into it, you're biasing the results towards the more desperate and less in-demand individual.

- If the more desperate and less in-demand individual writes bad code, or makes poor judgements, you will be able to see it in their submission, won't you?

- Whereas if such an individual, because of their desperation or lack of demand, has enough time and motivation to elegantly solve the problem, then what's not to like?

I think a greater concern would be that with take-home assignments it's impossible to be sure you are actually evaluating the right individual. How can you be sure they didn't receive help from someone else?

The programming test is just one aspect of a candidate, in my estimate about a third of the total "score". Take-home exams bias towards evaluating programming ability but bias against other desirable candidate qualities such as experience.

Their desperation to get the job doesn't necessarily carry over to their day-to-day programming. If they take a month to do a week-long task, I would want to know.

In terms of cheating, that can be easily resolved by asking detailed questions about the candidate's submission. It is also helpful to show that you took the time to review it. (There's nothing as frustrating as taking a day for a take home test and getting no response).

> but bias against other desirable candidate qualities such as experience

Won't you see, in the code submission, at least some evidence of experience? The way the candidate organises their code; the overall style; the third-party libraries they choose to use (or not to use); the tests they write (or don't write) — doesn't it all, combined, speak of experience or of lack thereof?

There are still other stages of the interview, where you can assess candidate's experience more directly, by asking about what they worked on.

I agree that it's useful to know when a task that should have taken x long takes 5x or 10x; but it may be because the candidate set a higher bar for themselves.

Totally agree. I've almost never completed a take home project (even though I liked the concept) because there were always other companies I liked that didn't demand as much of my time whose interview process moved faster and who gave me offers before I finished the take home interview process.

> I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable

I'm going totally nuts about this particular problem. I have written in as clear terms as possible that I really don't want the candidate to spend more than 2 hours, but I don't feel I am going to convince anybody - especially as this is a (mini)game programming test, and with games you can always add one or ten more things.

We do this at String and Key (and I adopted the process from my prior teams at Digitally Imported. I've posted jobs from both here on HN in the past).

Its a one-hour hands-on programing lab via screenshare - a pretty straightforward task of consuming some data from an API, processing it a little, and displaying it. We use Ruby for our work, and ask the same of our candidates in the lab. No handwritten syntax, exotic/tricky algorithms, or gotchas. Just a task to see how a candidate approaches a problem, deals with requirements, and implements them in our language of choice.

Happy to go into more detail if anyone is trying to implement something similar in their own hiring processes.

I'm interested in hearing more. I'm currently in the process of improving my interview approaches — currently a live coding session with a real-world problem involving an internal dataset.

It sounds like you're on the right track. I think the important parts are:

1) It should be a live exercise (at least mostly live). I'm less interested in the end result than how someone got there, and most people chose to narrate their thought process as they work through the exercise.

2) The candidate should be able to work in their own environment. Online assessment tools can be different enough to introduce friction in a timed interview scenario. I'm interested in seeing how fluid a candidate is with the tools they'll use every day.

3) It should have the same expectations as your day-to-day work. Did the editor have a linter configured, and did they use it? Were tests written for the parts of the lab that introduced nontrivial logic? Is there a README? Is it all one git commit, or did they habitually make commits as they worked?

4) The design of the lab should have a decision point - I'm currently using a lab which could be implemented successfully with either a relational database or some kind of cache store. The strongest candidates recognize there are multiple paths and can justify why they chose the one they did.

This works best for reasonably experienced devs who can actually be expected to create something from scratch from a set of requirements on a short time frame. For more junior people, I'd typically give a version of this lab that I completed myself and introduced a couple of bugs into. The task is to fix the bugs.

Apple did this in one of the sessions last time I interviewed there. They handed me a project in Xcode that had a number of bugs with varying degrees of difficulty — some caused crashes, others were UI glitches, etc. The assignment was to identify the problems and fix them. If you have more than a couple years’ experience the issues were pretty straightforward.

I really enjoyed this session, but I can see how it could be stressful to others.

We used to do this at an old job. We were a consultancy which specialised in a particular e-commerce framework, and were hiring people with experience in it.

So we set up a small project using this framework, and mildly wrecked it. Made some really basic mistakes, made some more subtle mistakes, added a bunch of dead code, wrote good tests for some of the broken stuff, wrote broken tests for some of the good stuff, etc.

Candidates came in, and we asked them to add some simple feature to the app, which we warned them may be broken. We had someone sort-of-pair with them as they work, to watch what they do and keep them on the rails to an extent. We then evaluated whether what the candidate did was sensible: did they run the tests, did they pay attention to the failures, did they find and fix the mistakes, did they get distracted by the dead code, etc. Candidates didn't have to do everything perfectly, but there is plenty of scope to go really wrong. I remember one case where the very first thing the candidate did was delete all the tests. Thanks for coming in, we'll let you know!

The company did quite a lot of rescue missions, where we would pick up a project that another consultancy had started and failed to deliver, and get it live. This interview wasn't just a trick, it was testing everyday skills!

This sounds really cool.

> I think the best approach would be giving the candidate a broken app and have they debug it live

I had a company do this and it went really poorly for me. It was a python app and at the time I wasn't very expert in python. They told me to use my laptop and all I had for python was a vim environment. I knew enough python to do white-boarding problems. But debugging a complex app exercises a whole different set of skills I just hadn't developed in the language. I was also learning vim on the side. I looked like an incompetent amateur.

So there's a language and tooling issue. The bigger problem is that debugging an unfamiliar code base under time pressure is almost as niche as doing algorithm problems under time pressure. I've been on enough company wide on calls that I actually think I'm pretty good at debugging unfamiliar code under time pressure. But if I have someone who understands the code looking over my shoulder, they're usually at least trying to help.

It's still time pressure. It's still a somewhat contrived problem and code base. The interviewer is still comfortably omniscient. And it introduces some more complexity due to unfamiliarity of the language, tooling, code base or problem space.

Ultimately, I'm not sure it's any better than white-boarding.

A lot of these problems are solved with a take home question but then all the problems with take homes rise in their place.

Take-home projects are frustrating when they basically just say "Build an app that does $xyz", and we'll be in touch if we like it.

Obviously no one is going to submit an app that doesn't accomplish the primary task. I feel like you get judged more-so on how you decided to get the initial app up and running, which likely isn't something you'll need to do working there.

> Obviously no one is going to submit an app that doesn't accomplish the primary task.

I would have guessed the same thing before my company started doing homework interviews. But in fact, the ones that don't get to the finish line are sometimes the most interesting ones to review, and they aren't an automatic no-hire. (And we say so explicitly, if a bit more diplomatically, in our instructions.) If someone hands us clean, well-thought-out code and it's clear that they honored our time limit and just didn't quite get there before the clock ran out, we'll still talk to them.

We are a remote company. I give take home projects too. Mostly I give them couple of days but mention they shouldn't work more then 4 hours on it.

Our test is actually quite simple and everyone that did any kind of development should have done these basic things.

But it tells me a log of the current state of the candidate. Most code samples on internet are not production ready code. Security and error handling are mostly not included. So this is where a candidate can shine... but I'm happy if you can string it together working even if it is by googling.

> they shouldn't work more then 4 hours on it

So you're saying spending 3 and a half hours on the assignment is fine. It's not.

Take-home assignments have little to no value for the candidate, they're only feasible if you are jobless or in college. The last thing I want to do in my spare time is to stay focusing on a bogus problem that may end up discarded.

What would you like to do instead?

Talking on-site or on the phone? Is that better spent time

Yes. I worked at Quora and we had a "practical interview" that was exactly this (it wasn't take-home though).


Paid (premium rate) take home tasks is the way to go, after the initial screening.

Take homes are too time consuming.

I would rather spend 3-4 hours working on a take home test rather than writing on whiteboard.

Last year, I did take home test for 5 different companies and all 5 of them invited me to the next round. I yet to have a successful whiteboarding interview.

But did you get any of those jobs? I wouldn't call being invited to the next round successful if you don't get the job. In fact it's a loss, more lost time for the candidate.

Yeah seriously - "rounds" don't mean anything unless they publish numbers, and even then it's sketchy. You need to be able to verify that they're actually there to shrink the pool, not just provide the impression of it. Take-homes are free for the company to issue.

So you're saying you spent 15-20 hours just in order to get interviews. Many, many folks would not have the time for that.

I've had companies straight up pay me for doing a take home project as part of an interview. I think that's the ideal way as it gets rid of the potential for exploitation/excessive burden that can come from take home projects.

This can add a lot of overhead, and some candidates may have jobs that forbid them from accepting outside work.

A good middle ground would be to have the company make a donation to the candidate's choice of nonprofit.

>This can add a lot of overhead, and some candidates may have jobs that forbid them from accepting outside work

How is this enforceable? Even in the case that someone finds out about it, what’s the company going to do? Sue the interviewing company for the rights to a few hours of work?

Well, if the employer finds out you violated your employment contract in this manner they could sue you, or perhaps even the other party that paid you. But IANAL, I'm just saying that contracts forbidding side jobs are not unheard of.

>or perhaps even the other party that paid you

What are the damages?

As for suing you? Again what are the damages?

It’s almost certain there are no damages assuming you are just working on a take home project not something that they are going to actually use to compete with your employer.

Just because something is in the terms of your contract doesn’t actually mean you really need to worry about a lawsuit.

A company might fire you if they find out, but the chance that they fire you is to small to worry about.

And even if you don’t do take home work for pay, you don’t want your company finding out you are interviewing. Taking a token payment doesn’t change anything practically.

IANAL. I am just saying that some employment contracts forbid side work. I imagine many companies would prefer not to get in the middle of that, and so might be reluctant to pay people for work done in interviews.

Some people have contracts that forbid divulging any information or techniques they learned during their employment. This doesn't stop companies from asking questions about previous work projects, or asking them to solve problems on white boards despite that fact that some candidates are certainly violating the letter of their employment contracts to do so.

People have contracts with all kinds of stupid things in them. If they aren't practically enforceable, they can safely be ignored. Any company saying they don't pay for take home projects because of employment contracts is just using that as an excuse not to pay.

There are people out their with contracts that say they can't even interview at rival companies, how many companies out there refuse to conduct interviews because of this?

Or just offer both options. I certainly wouldn't eliminate the opportunity for unemployed people to get a little cash, just because of contract issues that gainfully employed people might have.

There are tax implications, for both employer and employee, to paying the candidate, at least in the UK. It would be quite a lot of headache for not a lot of money.

Donating to charity has tax implications for the company too, but they're much simpler.

I'm not a corporate accountant or anything, but I've been on both sides of this and it's very simple to pay people for outside work in the US.

And for that matter, giving real-but-unpaid work to do as part of an interview may be illegal. So paying for your time may be something companies are doing to protect themselves taxwise, not just for good will.

Yes, but...

Now one worries, is this itself a test? Are they judging my desire to donate.

Well there has to be some benefit to passing on the money.

It's kind of like at salary negotiation. You can impress them with how you really are there for your passion not materialism. Or you can get money.

Well, honestly - if I wanted to donate, I could do so myself after they pay me.

Then I'd be worried... are they watching to see whether I engage in hollow virtue-signalling?

No, they're watching to see if you're worried about whether they're watching to see if you're engaged in hollow virtue-signalling. :P

> A major issue is that they require candidates to have free time outside of work to complete them.

Just like onsite interviews. Any interview process will require candidates to spend time outside of work. The only way to fix that would be a true, universally recognized certification/diploma that would prove that you can write software, but I don't see that happening any time soon.

Just like onsite interviews

And unlike an onsite interview, I don't have to take time off during working hours to do them.

My mental model is that I'm willing to spend N hours in total for the job application process. I'm happy to spend some of those hours working on a take home problem as long as that is instead of and not in addition to spending those hours doing multiple rounds of interviews.

> choose and implement and couple enhancements to a toy app

I think this is a key piece why this kind of project is useful, because that's exactly what they'd do when they start in their coding role: jump into an unfamiliar code base and fix a bug, implement something tiny, make things better along the way.

When hiring for a React role, I went to find an open-source React app and simply asked the candidate to mod it a bit. We then discussed the why and how, but also how they liked the app and why.

We only gave this task to the final candidates, and the ones that had React experience were done in very little time, because typing wasn't the issue here, but reading unfamiliar code and understanding the mechanics of the app.

Writing a hello world app from scratch or coding fizz buzz doesn't tell me enough.

Was this a timed take home project or something with a flexible timeline? That is, was it "You have to respond within 3 hours of receiving this email with your solution" or something more flexible?

I mostly understand why some companies do the x hours routine, and when I've done interviews this way I've historically performed well, but it just felt like it added unnecessary stress. For instance, anyone on the job doesn't have to worry about issues like, "What if I couldn't get the app up and running because I had the wrong version of X installed?" longer than maybe their first day on the job. And if you plan on hiring an engineer for a few years, one day is irrelevant. At best this felt like it favored contractors who were more experienced at jumping into new projects frequently.

In this case it was a flexible timeline. It took a few hours to properly complete, and I was given a number of days to return it.

I'll play devil's advocate and defend whiteboard interviews as compared to take home or online assessments.

Take homes are not very respectful of the candidate's time and, especially if they are not time bound, often leads to a candidate wasting a significant amount of time on these assignments. Plus, there's always the possibility that desperate candidates would cheat and have the assignment done by someone else.

I would only use take homes and HackerRank style quiz for applicants that are from institutions I don't fully trust with quality or applications that are very spammy in nature, i.e from a college/bootcamp that has an abysmal application-to-hire ratio [0].

For candidates from serious institutions I favor a two-step approach: a screening interview with a light coding question and a full whiteboard interview. The screening's purpose is really only to answer questions about the hiring process and check that the candidate can write very simple code by himself when given a trivial problem [1].

Now, a lot of folks do whiteboard interviews wrong. They often expect to get the exact implementation of an algorithm they found in a textbook and for code on the board to compile. This isn't the point of whiteboarding. Doing this only promotes rote memorization. A good whiteboard interview should be a toy problem that can be solved in several different ways by using different strategies or data-structures. The idea is to see how the candidate will break down the problem. Is the candidate able to formulate test cases, write a simple implementation, verify his code and correct the implementation should it fail a test? On the more meta side of things is the candidate able to take feedback and explain why a certain strategy was chosen? Of course it's not representative of real world engineering but it's a good way to peek at someone's ability to debug and reason about programs; these abilities translate well into debugging and design. Especially at the college level, I really can’t make any assumptions on what the candidates know. I’m not judging their knowledge of the standard library of X programming language or the framework-du-jour but their ability to learn it fast.

[0] https://economictimes.indiatimes.com/tech/ites/95-engineers-...

[1] https://blog.codinghorror.com/why-cant-programmers-program/

This is yet another tech interview discussion, so I'd like to highlight a subthread from the one earlier this month:

> I'm actually a little scared to leave my current FAANG gig for that exact reason tbh. I'm fairly certain I wouldn't make it back in the door without more leetcode grinding + repeated loops than I'm willing to do at this point in my career.

>> I'm in a similar position - a tech lead role at a Big N - and have recently been doing some interviews for senior roles at FAANG. The most frustrating thing is knowing I can't really leverage anything I've learned over the past 7 years of my career in the interview, at least at the early stages.

>>> I feel extremely little of what I'm doing in my real, actual, job, helps me in advancing in my career. Unless maybe if I choose to stay at my current company until retirement lol. Otherwise, why bother doing anything more than the bare minimum to get by at work? It would be a far better investment of time and effort to grind leetcode and practice for interviews, instead of going above and beyond to excel at my job. At least until I get into an "endgame company" where I feel it's worth staying long term.


Even as we debate the validity of the metrics measured by technical interviews, it's also worth considering the metrics used to measure engineer worth in the industry at large. This existentialist "fear and loathing in FAANG" discussion was quite illuminating, and a little sad.

There's no rule that says you can't apply for more than one role at a given FAANG. It's not like they blacklist you for not getting hired in one role. They know they have an intentionally high false negative rate, you get to try again.

I think Google blacklists you after 3 failed onsite attempts.

> I'm actually a little scared to leave my current FAANG gig for that exact reason tbh. I'm fairly certain I wouldn't make it back in the door without more leetcode grinding + repeated loops than I'm willing to do at this point in my career.

Makes you question if this is the precise point of it.

Disagree. It makes zero sense for Twitter to make it harder for them to poach Google employees.

The advantage of leetcode is that you study once and then it applies for every job you interview for. Like democracy it's a horrible system except for all the others.

Take home project? There goes 4-12 hours per company that gives you one and sometimes more. Companies have no incentive to cut it down or not give it to even marginal candidates so you'll get a lot more of them than full in-person interviews.

Pair programming? That's basically white boarding with a better white board or, in the remote interview world, just regular white boarding. Sure the problem is more real world, in theory, but then you get issues of knowing the same frameworks, etc. So to do well you need to spend a lot of time beforehand studying their specific frameworks and code bases if they ask you to do an issue on their open source project.

I would argue the problem is it's not a one-time cost. You pay the cost almost every time you want to change jobs, because leetcode/algorithmic interview questions are so fundamentally different to what we actually do day-to-day as developers.

It is very easy to completely forget all about graph algorithms in, say, 3 - 5 years during which time you've been a productive, valuable member of a software development team that...you know...actually builds things customers care about.

Small price to pay to get more and more absurdly high paying jobs every 3-5 years.

Being a productive, valuable member of a software development team is teaching you things that will set you up for success in higher level roles. You won’t be taken for those roles on Leetcode interview performance alone.

>You won’t be taken for those roles on Leetcode interview performance alone.

I think the problem is that for some teams the entire process is some combination of leetcode-style "technical" interviews and then some "culture fit" interviews, without any chance to talk about actual dev work.

Large tech companies will have system design and project oriented behavioral interview which will somewhat cover actual dev work. Other than that it pays to build up your network during your career. You can basically bypass interviews at smaller companies with the right referral.

> You won’t be taken for those roles on Leetcode interview performance alone

You’d be surprised.

>because leetcode/algorithmic interview questions are so fundamentally different to what we actually do day-to-day as developers.

I mean, maybe hard or esoteric easy questions but I don't think that's true for basic things like 'find the first duplicate' or '-traverse- this graph'. When people rail on whiteboard coding, I think they mean obscure algorithmic questions and not fundamental data structure questions.

I don't expect someone to be able to do Dijkstra's algorithm (or even spell Dijkstra, did I do it right?) but everyone should be comfortable enough with a bit of help doing BFS or working with linked lists, hashmaps, no? Am I crazy here?

I don't think you're crazy, but if their work and your work are detached enough from BFS and linked lists, why use them as the assessment criteria?

It's not really that difficult to make quotidian software engineering tasks into interview questions. I've at least seen it done successfully before.

More anecdotes: a nontrivial number of the "algorithmic trick" interviews I've done were questions I knew the answer to beforehand – and often the interviewer didn't care when I told them ("I just want to see how you think" or "Just show me how you'd code it" or sometimes even "OK, just tell me the solution and we'll move on to the next part"). What happens to the people who don't have the same background? Are they really less competent software engineers because their algs class didn't cover making a queue from two stacks?

I've worked both as an electrical engineer, and as a SW engineer/developer.

"Find the first duplicate": In my actual SW roles, this was trivial: Use a hash/map/dictionary/set. However, this isn't a typical Leetcode question - often it will be with additional constraints (cannot use a set, need O(1) space, etc). I don't doubt there are domains where this is important, but I've never had to do this in my professional work.

"Traverse this graph": Probably had to do it a few times. Less than once a year.

BFS: Never for my job.

Linked lists: Never for my job.

Yes, it is good to know what a linked list is and know if your library's data structure is using one, but again: That's not your typical Leetcode question. You'll be expected to write a linked list to solve some problem.

I do get what you're saying: Stuff like linked lists and BFS are fundamental, even if most SW jobs almost never require them. It would be a red flag if a candidate didn't know the basics about them. It's another thing to ask them to code it on the fly.

Going back to my electrical engineering experience: We have to take and use a ton of calculus to get the degree - much more than a CS grad will take algorithms. Yet it's a common trope that most electrical engineers will never use calculus on the job (literally not hard to find one who had a whole career in EE without using calculus). If you're doing research or are at the forefront of a discipline - sure, you'll need calculus. Or if you're doing electromagnetics, etc. The theory is built on calculus, but you don't need to know it to apply the results of the theory.

Would it be a red flag if a candidate clearly didn't understand the basics of differentiation (i.e. rate of change) and integrals (i.e. summing quantities)? Perhaps.

But should I ask someone to do a tricky integral for an EE role that will not require the person to do any calculus? I would be criticized by my peers, and if I kept doing that I would be excluded from interviewing. In fact, even if I ask them to do non-tricky integrals I would be criticized.

Asking people to figure out a trick to solve a linked list problem is equivalent to asking EE candidates to do tricky integrals or differential equations.

thats a great analogy

In the past we had some candidates complain about take-home problems. The primary complaint was that candidates weren't interested in dedicating 4-5 hours of their own time unless they felt that someone from the company was putting in an equal amount of their own time. Some candidates also had suspicions that we were trying to use them for free labor (Not true, we used the same toy problem for every candidate). One candidate even thought they should be able to bill us for their time spent applying (We passed).

Eventually I changed the strategy and offered candidates the choice of the take-home problem or an on-site whiteboard style interview where they solved a subset of the problem in pair-programming fashion with us. Zero people chose the on-site interview. Everyone chose the take-home problem.

No one complained any more after we reframed the take-home problem as the candidate's choice rather than the company's.

> One candidate even thought they should be able to bill us for their time spent applying (We passed)

Ironically, you passed on the smartest applicant of the bunch.

> offered candidates the choice

> reframed

Aren't meaningless shenanigans generally associated with meaningless things?

Do you have any evidence that either strategy really matters? Does it boil down to, "We choose an interviewing strategy that selects for people who look like programmers."

So instead of doing a 4-hour on-site interview, you make a hiring decision based on a 4-hour take-home problem?

Seems you think that 4h test was all, wondering how you got that idea

So candidates actively chose to add a 4h project on top of a 4h in-person interview?

There's probably a one or two hours soft skills interview and more that they didn't mention.

How do you stop people from "getting help" on the take home problems.

Why would you penalize an employee for getting help on a problem? Stack overflow away. Troubleshooting is an essential skill.

I think s/he meant letting a friend do the whole assignment

Precisely and I've heard of people landing gigs at some well known companies who did just that. Without any followup questions.

> One candidate even thought they should be able to bill us for their time spent applying ...

Are you meaning they wanted your company to pay for the time used to solve the take-home problem?

If so, isn't it normally done that way in order to prevent companies potentially exploiting free labour?

> an on-site whiteboard style interview where they solved a subset of the problem in pair-programming fashion with us

Do you offer candidates a remote pair-programming interview?

E.g. screen-sharing session over Skype/Zoom?

I don't mind the idea of take home exercises as long as I have an actual chance of getting in.

If i'm really serious about joining a company and the many months/years to come with them, I can probably spare a day to show it.


It's always surprising to read people complaining about spending even 8 hours on a take-home problem in the comfort of their home, with no one looking over their shoulder, finished at their leisure. Meanwhile, anyone serious about applying to the same company wouldn't hesitate to take PTO and leave work to go on-site for a similar amount of time for an interview.

If you do the take-home problem, do you get to skip the interview? My understanding was that it might, at most, replace a 1-hour whiteboarding interview. The one time I did a take-home problem for an interview, I spent a whole weekend day on it (which felt hard, given that I was making my wife watch our child all day), and then I still had to spend a whole day on in-person interviews.

And then they didn't even ask me any questions about my take-home solution--it was just ignored. It felt like a pure waste. I'd much rather do whiteboarding.

The homework was maybe a capcha to waste your time to show your are acctually applying.

The problem might be that you might need to interview for alot of positions and having a homework for each would add up.

The homework is no cost for the employer and you are not sure how serious about you they are when you do it (if it is in the start of the process) so I wouldn't bother if I had an alternatives to go for.

Except most companies give the take home as essentially a first line screen so the chance of passing is low (as they need to filter out 90+% of applicants with it). So instead of spending an hour per company I now need to spend 8. When you're talking to 10+ companies that adds up quickly. If you also have a family and a a day job it becomes untenable very quickly.

Gotta disagree on pairing, as that gives you an actual computer. If it's your own environment, you're at least comfortable with that. You don't have to worry about penmanship and can get IDE suggestions. It's frustrating not being able to get your ideas down quickly enough like when writing code on a board.

My worst experience so far were pair-programming interviews. This requires so much training on the interviewee side of things that most companies just completely tank this one. They have some fixed expectations in their head and they have not a single clue how to objectively evaluate candidates.

Yup, leetcode at least is fairly easy to make interviewers competent enough at giving especially if you restrict the problems they can give. Although worst one me and my friends have gotten were system design. Friend got interrupted with questions every 30 seconds, told to just answer them directly and then failed for not elaborating deeply enough on things.

Same exact thing happened to me at Microsoft during the system design interview round. Interviewer wouldn't just let me explain and kept on arguing about a relatively minor issue and wouldn't let me proceed and in the end failed me because I didn't come up with a good design. Really the whole 45 minutes were like him repeatedly stopping me and coming back to that one issue.

I agree that take home projects and pair programming are nowhere near ideal, but I think the advantage you mention for the "leetcode" white boarding is one of the main disadvantages. If all you need to do is study these and memorize their solutions then what is it showing us of your actual ability.

I personally when I interview candidates prefer to get to know them and how they think before I even try to understand their ability to problem solve. Knowing the language or algorithms are all easy to learn things. Knowing how to naturally be curious and solve problems is a far more difficult skill to learn and demonstrate, but it is easy to evaluate indirectly.

>Knowing how to naturally be curious and solve problems is a far more difficult skill to learn and demonstrate, but it is easy to evaluate indirectly.

That's what leetcode used to test and then people optimized for it and now it tests memorization. At scale things work differently than in isolated cases. I've also found that most problem solving interviews really test how well I've seen the problem before and how well I can lie about seeing it before. Interview enough candidates and the good problem solvers will look dumb compared to the ones who simply saw it before and can act like they didn't.

I give people the choice of take-home, whiteboard, or code sample. Code sample can be any code they've written for a side-project or even a take-home they've done for another company. Obviously, they can't share actual code they've written for another company, but take-homes are generally fair game... And so that solves the O(N) problem for people who are applying to multiple companies.

Depends how the pair programming is done. Last time I did pairing I was told I could use any language I wanted to solve the problem regardless of whether or not the interviewer knew it. You just need an interviewer who is curious and asks questions about the language you're using to make sure they understand your approach.

No one enjoys whiteboard interviews but it's not clear that these companies have better alternatives. Many of them do take home projects which are time consuming. I'd prefer the employer look at my GitHub profile as there's plenty of code there already. Furthermore, I'm not a fan of "interviews" in general, I find that more information is transmitted by eating lunch with the team and talking like normal people.

Our interview process is far more advanced.

We place the candidate in a room with a large whiteboard for 15 minutes alone with the lights off. I then walk in with a 6 piece dresser [1] from IKEA (obviously keeping the lights out). I explain to the prospect that this particular dresser is missing one piece - BUT we need the dresser assembled then disassembled and placed back into the box [without anyone seeing]. More than HALF the candidates become violent and demand to see my github account - some even suggest I have a mental issue. I am often able to calm the candidate down - this way we can get into the REAL debate: KOPPANG vs MALMs. Any candidate caught assembling the IKEA dresser is quickly dismissed out of spite. This technique is from Boz at facebook - it is how all* senior principle engineers are screened at FB and the reason everyone codes PHP (but swears they do not).

HR has complained that my techinques are "ridiculous" or borderline illegal - but so far my successful candidates perform, tend to have a solid grasp of typescript and quickly move into senior management.


Note that upper management is also interviewed in such a fashion, except with night vision cameras and the entire board crammed into a tiny security cupboard near the elevators. Any candidate caught trying to hide the existence of the IKEA dresser entirely is immediately hired once they have successfully gaslighted the interviewer.

MALM is superior in every aspect.

Here's the thing though. It's a lot more effort to look through projects on GitHub and evaluate them. And even if you can, you then have to figure out how to cross compare that and set a consistent bar vs people who you interview because they don't have a GitHub presence.

People want hiring processes to be somewhat consistent, and in that world it's really kind of a non-starter to replace tech screens with GitHub profile reviews.

yup. and GitHub's useless for value adds like management, marketing, recruiting, or product skills.

for that matter, GitHub is useless if the projects you contribute to are on BitBucket or GitLab. that contributions graph has a lot of assumptions baked in.

It's so backward to see companies shift to these abstracted whiteboard or homework assignments as some sort of standard, considering standardized testing is now falling out of favor in higher education as it is rarely indicative of actual potential success. It strikes me as lazy, just following a trend unrooted in any empirical data.

Maybe the solution is for companies to hire more thorough managers who are capable of vetting a github profile to see if the candidates skills are relevant to the position, not turn the interview process into a performance.

There is not that much code I wrote three, four years ago that I’m still proud of. I figured that would get better with age but it’s only by degrees, and at any rate it may indicate that I’m getting soft rather than better.

So anything on github that I might want you to look at? I’d have to be on a three year replacement cycle, permanently, which is a hell of a lot more work than you might estimate that a reasonable github presence would entail.

I think it is also hard to compare different candidates this way, even assuming they both have a similar level of commitment on GitHub. A take home exercise at least allows them to compare the same thing

> I find that more information is transmitted by eating lunch with the team and talking like normal people.

As an introverted and generally very socially anxious developer, this sounds like a dream compared to a standard interview, which has always felt to me more like an interrogation and generally very aggressive and hostile. I'm confident in my skills and I can communicate well in other situations, but the typical interview environment is cripplingly stressful to me. I would love to just get through a talkative lunch instead.

I like whiteboards, if there is an interesting problem.

Going through my GitHub account however isn't good. It's full of quick one shot things, experiments and so on, very little production ready code and not representative of anything.

I'd much rather do a take-home project at my own pace funded by the company, seeing as how my personal GitHub is a barren wasteland (well...it was before I started working for an OSS company).

In the future, when looking at github becomes a standard hiring practice, companies will start going full closed source so that their talent can’t jump ship.

My former employer, a Fortune 500 company, wouldn't let me volunteer for a globally recognized charity because the boilerplate agreement the charity required volunteers sign, stated that IP created as a volunteer belonged to them, and my employer considered that everything within or outside work hours was theirs. Not that I was volunteering as a programmer anyway.

My current employer doesn't have explicit rules that I'm aware of, but early on, I asked our HR person if I could work on open source outside work and they said something along the lines of "what's open source?" Which, I mean, on the one hand what do you expect from HR, but on the other hand, the entire agency is concerned with operating, maintaining, and developing a software system.

I was under the impression that this kind of interpretation of employment contracts was more or less illegal (or, at the very least, unenforceable) in many states (I'm assuming you're US, programming in Fortune 500, so maybe not applicable?). Sort of fits into the same kind vein as certain non-compete clauses, where companies put them in, but they'd be tossed out pretty quickly in court should things actually reach that point.

If it's not on employer-owned equipment during core working hours, then I believe it's pretty difficult for an employer to enforce that kind of thing. I don't know how important to you working at that charity was or participating in open-source now is to you, but if you're willing to take a harder line of telling rather than asking, you might find that when push comes to shove, there might not be much they can do without drastically escalating the situation (not always an organization's favorite thing to do, even if they can technically win, which they might not be able to).

Obvious disclaimer about being a non-expert on this sort of thing, but really just wanted to point out that the first answer in this situation may not actually be the final one.

My recollection is there was a compulsory agreement saying you would not do anything of a nature related to your work, or if you did, allow the company to decide if they wanted to claim it.

Also, running side businesses was not allowed.

Of course, the normal thing to do was just ignore the rules, because most likely nobody would notice or care. I explicitly asked for permission because I wanted to see what would happen.

Its whether its related work that is the issue, for example if your in a band and write songs they cant claim that.

And its not really the "employment contract" in Anglo Saxon legal systems its pretty much assumed.

This is how I get over that hump. Not only do I not work on anything related to my day job, I mostly build things in my free time that aren't really useful to anyone but myself. Sure my company may be able to claim ownership of my Sean Connery themed programming language but will they? What on earth do they plan to do with it?

They are happy that you can't leave, because you have nothing to legally show to other interviewers and no other prospective project in your drawer. That's not a good position to be in.

I would love to see a push for more intellectual ownership over the code you contribute to your employer. I should be able to take any snippet I wrote and reuse it with impunity.

This might require engineers unionizing, but it would be a boon to the field. Patterns buried in closed source vaults would be disseminated and innovation would surge. It might usher in a golden age of software if everyone were able to freely share their intellectual ideas with their peers and not have to worry about risking their health insurance and home in the process.

Recurse Center has a pretty nice model where they give you a part of a program (small) to bring ahead of time (or you can pick any program you've written) and the interview is going through the code and adding a feature or two.

That kind of interview mitigates the amount of trivia and preparation-gaming while also ensuring that the applicant can solve problems as a team and is a competent programmer. It is cheatable in that one can memorize implementations of some features, though it seems somewhat difficult and not more cheatable than just memorizing a bunch of CS trivia.

I'm not too familiar with them, but why does Recurse Center need to interview people for technical positions?

The Recurse Center is a retreat that you attend with other like-minded programmers. I recommend it if you are interested deepening your experience as a programmer, by working on personal projects as part of a helpful community. There is a selection process as mentioned above, but it isn't onerous.

Well put. The one time i expressed interest but reluctance at leaving my previous job, I was invited on a coffee and chat with a couple lead engineers to help decide. We got along well, talked about technical things, swapped stories about problems solved. Afterwards, i got the impression going through the technical interview would be more of a formality.

I think part of the problem is not everyone is good at giving interviews. But most software engineers can talk about their work once you get them going.

FWIW, I think white boarding interviews are pretty fun.

As an interviewer, how do I know that you wrote that code in your GitHub profile?

Code often doesn't stand alone, there's going to be a commit history, there's often going to be comments and issues on other projects.

Ask questions about the code:

- why did you write this in this way

- explain how this works

- if you needed to add X into this, how would you do it

- are there other ways to achieve what you did here, what are the pros/cons of this?

- what would you do differently if you were to re-write this?

- are there any bugs or common issues, how do you go about diagnosing and debugging them? Could you write the code differently to be more resiliant to those issues?

Anyone who wrote that code (assuming it was relatively recently) is going to be able to talk about it sensibly, and be able to dig into the code and show how certain things are done.

I had a former coworker re-write a library we had open sourced as their own. They had used the original version at work so knew it well, knew the internals by asking lots of questions while working for performance reasons. They renamed things and changed pointers around to their preference. They did this right before they left and had it on github as their work. They could meet all of your questions. They didn't really write the code, but it was MIT, what can you do.

It was very obviously there as bait for people like you and it was removed as soon as they took a new job. It was never used in production in its "released" form.

I had no problem with the person's abilities to code, but for reasons like this I probably wouldn't work with them again, or hire them as a eng lead or higher.

One question is, if someone is doing elaborate deceptive work like this, gets hired and has no problems performing the job, does it actually matter? Some people mostly or entirely code at work, so this looks like a legitimate way to get a "good" project in GitHub that he knew well. It sucks that it's deceptive (which could be a major red flag), but it's also clear enough that he understood a large codebase well enough to explain it and modify and/or refactor it, which is honestly the majority of a SWE job. A lot of people are concerned about the "purity" of how someone solves an interview challenge, and outside of potential ethical red flags, people that "cheat" the system are often putting in a lot of work, and are as likely as anyone else to be capable of doing the work.

No if they had no problems performing on the job then no it's not an issue. Maybe they learned and this provided a good reset for them. Maybe they didn't and similar issues will crop up.

I put the anecdote up as a real-world example of one way the GGP's github reference can be faked.

You would have to dig really hard into their knowledge of the system to catch them, which you wouldn't have time to do.

For instance, I believe the issues would show up in a benchmark.

I don't see a solution for this. What you're describing is someone who's clearly good enough to take a project and rewrite it, and still be able to talk in detail about the code.

I'd bet they could also pass a whiteboard, pair-programming or take-home test too.

That they are willing to lie about it and pass off other's work as their own isn't something that you're going to identify in your typical code interview. Using interrogation tactics to try and weed out the frauds is going to be counter-productive, by driving off all the good candidates.

I don't disagree. You might see some of the issues with this candidate on what they choose to highlight as their career successes, I've definitely caught dishonest people, or people I wouldn't work with via soft questions.

But I'm not sure, I've never interviewed this person.

Have the interviewee submit repos in which they're the only contributor? But, uh, with a take home project, you _really_ don't know that they wrote the code, either...

Do a take-home project but don't make them write from scratch. Give them a mostly working starter application.

Sprinkle in some bugs. Start with easy spelling errors and go all the way to concurrency problems.

Ask them to add a few simple features and fix any bugs they find. Bonus points if they find bugs you didn't add intentionally. Good unit tests are also a bonus.

Give them a couple days or something. Don't count off if they don't get everything done. It's better to do a few things correctly that fix everything with hacks.

Invite them to email questions/suggestions to the hiring team. This person might be on your team so it's nice to find out early if they have good ideas or they hound you with stack overflow questions all day.

I've never had this kind of interview but it sounds nice to me. It's more realistic to real daily work. You can cheat by having someone else do the work but that's always a possibility.

This sounds sadistic to me. I've spent a long time on dumb typos, which become a lot harder to spot when you are stressed and on a deadline. Chances are if you are applying for work, you are probably stressed about not having a job.

You've completely missed my point. It's not about being perfect. I want to work with people who leave code better than they found it but I can't expect perfection. I don't care if they can't find all the typos. I'm sure most software has typos.

Whiteboardimg a solution to n=np or building full applications is crazy and not realistic. Most interviews do that yet most work is fixing bugs and adding mundane features. Why can't interviews focus on the actual skills needed?

Ask them some questions about it over lunch?

What’s your threat model here? People making up elaborate lies about code they’ve written?

> What’s your threat model here? People making up elaborate lies about code they’ve written?

Likely people making up elaborate lies about code they didn't write. It's trivial to push code of which you are not the author.

I can’t imagine a situation where you couldn’t work that out with five minutes of casual conversation about the codebase but maybe I’m naive.

Which is why the interviewer should ask specific and thoughtful questions about the project, so they can gauge the candidate's level of understanding. This concern could also be addressed by introducing a trial period.

>This concern could also be addressed by introducing a trial period.

Why would I leave a stable well paying job for a job that basically says they don't trust me enough to not fire me in two weeks?

> Why would I leave a stable well paying job Well, why are you doing that in the first place? Maybe the new opportunity is worth it or the current one is so bad that it's worth it.

At least they’re honest about it. Unless you’ve got some kind of contract in place (or live somewhere where this is prohibited by law) your current employer could fire you in 2 weeks if they didn’t like your performance, too. They could do it this afternoon even.

I have heard of someone trying and pass off repos forked into their github account as their own work. A quick google or even just looking at the commit history in their fork made it obvious it wasnt true.

Manipulating one or more git repos to claim it as yours then ensuring your lie is not uncovered by googling would be technically challenging enough that those who could do probably don't need to.

"Interesting choice using collections.OrderedDict here, why did you choose that?"

> "Interesting choice using collections.OrderedDict here, why did you choose that?"

Without looking at the commit history, given enough time this is a challenge. I often don't recognize code I wrote myself 6 months ago.

I would assume the best, then in a week when it is clear that they faked their way in you can just lay them off. If they faked their way in but are a great member of the team going forward and contributing to projects, do you really care as a manager?

I think discussing the projects would be a better solution. The interviewer looks over his projects and asks questions about them, the applicant explains his thought process behind the decisions made.

If I'd stolen it, I'd have picked something prettier!

> but it's not clear that these companies have better alternatives.

It obviously depends on your definition of "better". And it's yet to be seen whether whiteboarding is "better" than pulling candidates out of a hat. You'll note that lots of companies that do strict whiteboard-style interviews are also extremely comfortable firing employees. Would you be "better" just hiring anyone whose resume seemed applicable, that could talk like a normal person, and then firing them if it didn't work out?

Different things work for different people, personally white boarding is my favorite. I think the best would be to offer the candidate options. Let them come in for the culture fit portion of the interview, at the end present them a problem and let them choose if they prefer white boarding, pair programming, or take home and submit by the end of the day.

Here we go again perpetuating the stereotype of the whiney privileged engineer put in a position of mild discomfort by the villainous abusive white board interview boogeyman. I don’t mind whiteboard interview exercises. They’re not great, but they often give a very good and reasonable signal that often has a high corollary to work performance. I think there’s bigger fish to fry in refining fair and objective interview processes, I’ll never understand the perpetual moaning around this aspect of technical interviews.

> they often give a very good and reasonable signal that often has a high corollary to work performance

Do you have any statistically significant data to back this claim?

There is statistical data saying the opposite: http://chrisparnin.me/pdf/stress_FSE_20.pdf

That whiteboard interviews are actually testing how you perform under stress rather than how good you are at your job.

Additionally, in my opinion, it measures how good the interviewer is. If you get a bad interviewer or even just one in a bad mood, just pack your bags because you aren't getting it.

Whiteboards are incredibly arbitrary in my opinion, if taken alone. If it is part of a complete assessment, ehhhh.

>There is statistical data saying the opposite

Interesting study and really good discussion on HN at the time, but the opposite of "[Whiteboard interviews] often give a very good and reasonable signal that often has a high corollary to work performance" would be something like "People who do well at whiteboard interviews tend to be worse performers at work." I don't think that's what the study said at all; rather that people do worse at whiteboard problems when they're being watched vs. not being watched.

Also people act as though software is a job without anxiety. There is plenty of anxiety in this job. Sometimes under pressure you really do have to perform or push back and its good to know if people can do it.

> Also people act as though software is a job without anxiety.

Not. All. Anxiety. Is. The. Same.

Ask a firefighter to give a speech in front of 1000 people. This is an entirely different kind of anxiety than running into a burning building. Ironically, it may be the former rather than the latter that makes the firefighter sweat.

Writing code is generally not a performance art. We're not stage performers. Interviews are an unusual situation that aren't anything like our daily jobs. Stress on the job is not the same as interview stress. Work emergencies are not the same as interviews.

We need to take into account human psychology. These factors are all well known, but the tech industry seems to act in deliberate ignorance of humanity.

Also stress on the job isn't the same as stress getting a job.

One you are getting paid no matter what, the other you aren't and are 'performing' in the hopes of getting paid.

It doesn't have to be stressful. Whether or not you have a gun to your head is due to management. Deadlines are an abstract concept created by humans and can always be moved. The best companies keep them very elastic.

The interview anxiety is a very specific type of anxiety - if it occurs during the day job - stay away from such company.

Most noteworthy for me: All of the women failed the whiteboard tests, however they passed tests conducted in a private setting.

Assuming the study’s results are valid, that’s really all the justification we need to get rid of these tests.


if properly trained, interviewers ask questions that build without throwing the kitchen sink at you.

whiteboard interviews (when done holistically) assess for signal, not binary correctness

- do you have a solid framework to build a solution?

- are you considering multiple approaches / data structures / complexities

- referencing similar problems, vocabulary, situations to illustrate breadth of knowledge

sure, the stress component isn't ideal, but there are multiple relevant capabilities being assessed in a very short amount of time...

in other words it's a much better test of tacit knowledge than alternatives

You can confirm the same using a take-home assignment and a code review together with the candidate. Or even with an on-site pair programming. Neither of those require whiteboards.

pair programming, for sure

if done right, whiteboarding should be almost the same as pair programming -- an iterative dialogue

I think its writing exact code down instead of good enough pseudo code - that people hate.

WB interviews should be about problem solving not regurgitation of rote learned algorithms

I strongly believe there is some signal to shoving enough familiarity with at least one language to come up with mostly correct syntax.

I won't ever fault a candidate for failing to close a paren or being unsure of whether the size of a container is .length() or .size(), but if virtually every line a candidate writes is of questionable value, I'd say that means something.


Yeah if a poor nerd makes some decent money, they need to abuse him with stupid whiteboard questions. But no one is questioning the money CEO, executives,managers, lawyers, etc are getting.

What if I like the current interview format?

I am starting to think the interview is a waste of time. Sure, you want to make sure the person is more-or-less who they claim to be on the surface, but you will never really understand their capability for reasoning with your problems until they are working in your process.

One of my earlier jobs dealing with code involved a very brief interview (no whiteboard involved) followed by a 6 month contract offer. The deal was that I basically had 6 months to prove myself, and if everyone felt like it was working out (myself included) this would be bumped into full-time with benefits. The crazy thing is, I was actually the one who decided I did not want to continue after the 6 months was out while they wanted me to stay very badly. I think this can be a really useful checkpoint tool for both sides.

Anyone can misrepresent themselves in an interview and dupe a room full of people for a day or 2. No one can keep up an act for 3-6 months continuously when working with complex code tasks. You will get found out before the term is up, assuming everyone is using this process as an evaluation tool and following up on it regularly.

Arrangements like this are much harder in US because 1) by default there is no health insurance if you’re a contractor, and obtaining your own is incredibly expensive and 2) there are also pretty complex tax implications of being a contractor; there are many other factors as well.

It also puts a candidate at a disadvantage, because looking for a job is heavily one-sided burden.

On a personal level, as both (occasionally) a candidate and a senior hiring manager, I always wonder about white boarding - i believe it works when the interviewer is looking for your thought process, and not code specifically. For example, something as simple as saying “I don’t expect this code to compile” at beginning of interview immediately puts candidate at ease and let’s you understand their logic and not the ability to recall specific function signatures on the fly.

My company does this as policy, except we hire them and put them on full benefits. However there is a 6 month period where you can be fired a lot easier than after that 6 months.

It seems like a reasonable compromise that doesn't leave the job seeker without health insurance etc.

I had a similar arrangement like this in the U.S. but it was for 1 year. When applying for the position it was listed as a term appointment and they were up front about the arrangement (same as OP; after a year, if everyone is happy, the term appointment becomes a FTE). The term position had all the same benefits as a FTE, it just had the time cap on it.

Well, terminating an employement is as well quite easy in US, regardless who initiates it - employer or employee.

It is easy to terminate straight white men under 40. Anyone else is extremely difficult anywhere with an HR department. Protected classes are taken very seriously.

In 2012 I was told Google gets 1 million applications a year. How do you decide which 1000 of those get 6 month contract offer? How many millions of dollars in lost productivity are you willing to lose of the people who don't pass your 6 month test that you had wasting space, taking up training time, having people have to review and reject their code, etc? What will those people do who gave up their lives somewhere else to be part of your company do when you tell them you've decided to not make them employee when their contract is up?

The value is similar to that of internships - the amortized cost of a 6 month 'trial period' should be similar if not cheaper than false positives that are a lot tougher to fire so you avoid dumpster fires like Amazon's mandatory attrition rate. I also wouldn't call it giving up there lives, when they know what they're signing up for and are presumably getting paid a similar amount to the the full time role they're applying for. Now if FAANG companies try to use their reputation to get cheap labor with no intention of hiring I don't have answers to that, but I can't imagine it would be worth the cost.

They still have some selection criteria for internships. It's not like they take the first 100 students at random so you're left with the same issue. You have to narrow the selection. That likely includes interviews.

> The deal was that I basically had 6 months to prove myself, and if everyone felt like it was working out (myself included) this would be bumped into full-time with benefits.

In the UK, we sort of have this in the form of probation - in most permanent jobs, you don't gain full employment rights until you've done a probationary period, which is usually six months. You get full salary and benefits during that period, but you can be fired with no notice at any time (or something like that).

In practice, though, the culture is not to use the probationary period as an extended interview, but more of a backstop against really screwing up hiring. You'd only let someone go after probation if they turned out to be absolutely incompetent or unsuitable, not if they were just disappointing.

It would be interesting to try using probation differently, though. Make the interview more of a screen, employ twice as many people as you need, only retain half of them at the end of probation, and be explicit throughout that this is the plan. I suspect you could implement and communicate this such that it wasn't absolutely horrible for probationers - active and transparent review during probation, one or two months' golden parachute, what else?

In Ontario, the letter of the law says you can be fired during probation for any or no reason (modulo human rights legislation), but in practice an employer can still be sued for wrongful dismissal if the employer terminates an employee without demonstrating cause.

It’s best to have experienced employment law advice when devising any scheme around using probation as a kind of hiring scheme, as opposed to as a last resort if things go very badly.

I will not give legal advice, but for example I would be very careful before implementing a “Survivor: Tech Island” scheme where the company hires more people than it genuinely expects to keep following their probationary period.

I have been told that if I hire someone I think is questionable with the thought that “if it doesn’t work out, we’ll just let them go during probation,” that can backfire as I make a conscious choice to hire them despite my reservations.

Again, INAL, but I would tread carefully around using probation for winnowing candidates out.

IANAL either, but my understanding is that wrongful dismissal is breach of contract, whereas unfair dismissal is illegal [1]. So if you write this procedure into your contracts, and then follow it, it's not wrongful dismissal. To be unfair, for a probation period of less than two years, it would have to be meet one of the criteria for being "automatically unfair", which are basically about the employer doing illegal things, or be discrimination [2].

I would also be very careful before implementing a scheme like this - as you say, you would want to have it thoroughly checked by legal and HR experts.

[1] https://my.ucu.org.uk/app/answers/detail/a_id/280/~/unfair-d...

[2] https://www.citizensadvice.org.uk/work/leaving-a-job/dismiss...

I think you’re right about the distinction between wrongful and unfair dismissal, but if I understand Ontario common law correctly, even in cases where the dismissal during probation is not illegal, and even when you have it in writing that the candidate can be dismissed for any or no reason, you may find yourself on the wrong side of a wrongful dismissal judgment and forced to pony up money.

It’s probably easier to structure such a thing as contract-to-permanent, and I suspect this is why there are a lot of such contracts around. Or at least, I have seen a number of such things in Ontario.

And now the counter-example: My brother is the principal Tuba with the National Arts Centre Orchestra. He had to win multiple auditions, resulting in two finalists being selected. Each of them then went and rehearsed and played a concert with the orchestra, they spent some weeks on contract, expenses and compensation paid. They selected my brother.

Full-time job? Nope! He then went on probation, either for a full year or two, I can’t remember. I’m sure he’d laugh at my worrying about 90 day probationary periods.

If you got 2-3 of those failed probationary periods on your CV though, you'd be stuffed. But that's probably a good thing.

This seems kinda nuts to me. I don't think I would leave a stable position for a probationary position, or accept one if I had more stable offers. "It's ok, we probably won't use it" - uhh, what? No, that's what legal protection is for.

Is it really that common? I'm sure many companies try to hoist this on propsective employees but I'm surprised people accept it.

In the UK, the "probably won't use it" kind of probation is standard. You accept it because you don't get any offers without it.

It really is the case that companies rarely randomly fire people during probation, so employees are fine with it.

Can someone else from the UK verify this? Still seems crazy to me.

Edit: reading around it seems like they're standard, but have no legal weight regarding termination? I.e. you're entitled to 30 days pay regardless. This seems very weird on all sides.

Also from UK and can confirm that 30-day probation is pretty standard.

In 20 years, I think I've only seen employees terminated during their probation 2 times. They were non-dev roles, and both times it had transpired that they were totally incapable of the job (one had also lied on their CV).

Damn. That sucks. At least it isn't common, but I'd like some legal protection against it. Guess it's just a matter of minimising uncertainty for your employer & having the leverage to negotiate your contract.

You'd be setting up a competition between the new hires. That might create something of a toxic work environment.

I've seen this done before. As others pointed out, the lack of health insurance can be a problem in the U.S., although sometimes can be mitigated if the candidate still has access via COBRA.

Even considering the flaws, I think this approach is better than immediately hiring someone full time. I've seen some candidates who seemed perfect on paper and fine in interviews and then turned out to be utterly toxic to actually work with. For the interviewee, the chances of encountering that situation are even greater, since there is one interviewee but typically many future coworkers.

Some personality traits can be very difficult to surface in an interview environment. IMO interviews are worthless for getting that kind of information.

Yeah, there are so many times where I'm asked to solve some sudoku puzzle in an interview. It is really sign of a big interview question.

We already have this extended trial period in tech, it’s called an internship, and responsible companies pay interns. Some internships come through university co-op programs, some through organizations that set this up for people following non-traditional career paths, and some companies “roll their own.”

But these positions are overwhelmingly for people breaking into the field. They have much to gain, and very little to lose by working for a company for three months with an uncertain expectation of a full-time job.

Now consider hiring an experienced worker who is out of a job. If Alice offers them a contract with the company having an option to turn it into full-time, while Bob offers them full-time work with the understanding that a major screw-up would lead to dismissal during a probationary period, Bob’s offer is much lower risk.

Finally, consider an experienced worker with a job. For a non-hypothetical example, they’re working at a famous name, with options, and a YC startup that has grown to 200 employEes recruits them.

It’s going to be very difficult to get them to walk away from guaranteed employment that feeds their children, for a temporary gig with no equity participation And the understanding that it’s an extended job interview.


The above focuses on the candidate’s risk/reward calculation. What about the company?

Well, the least experienced person has the greatest uncertainty. They might be very smart, but unable to work well with people. How would we know without giving them temporary work with people?

The most experienced person has the least overall uncertainty. They “brought the receipts.” Without arguing about how they should be interviewed/tested, if they aren’t a completely toxic person, if they have a non-fatal weakness that pops up, it can probably be managed to be fixed, or accommodated.

The employer’s risk goes down as the candidate’s experience goes up. Sure, it would be a bed of roses if the company could hire senior people on a one-year contract and see how all the intangibles work out, but that usually isn’t necessary, and it’s not going to happen often enough to operationalize that as part of the process.



There is no one-size-fits-all approach that fits all levels of experience, and also scales to large numbers of hires, and also is repeatable.

But the kernel of what you suggest works well for the sweet-spot of high-uncertainty and low risk to the candidate.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact