
Take-home interviews - socmoth
http://blog.triplebyte.com/take-home-interviews
======
brianwawok
Seems good, I like the choice.

In discussions on this topic I see a lot of: "Programming on the spot is hard,
let people program at home"! But then other people say "Why should I program
for free at home, my resume clearly shows I am already a skilled programmer.
All this will do is cater to young people without families, or those fresh out
of school".

I am not looking to hire devs right now, but I am thinking about the same idea
with a tiny tweak when we move to that stage. If the verbal interview goes
well, offer 2 choices. Make it clear neither choice is seen as the better
choice, and both will be judged equally.

* Sit on a computer with me for an hour, and show me how to code a fairly simple program.

or

* Get a small actual work assignment to take home and code. Would expect it to take maybe 5 hours to code. Offer $500 as a 1099 contractor to complete the assignment within the next 2 weeks or so.

This gives both groups a chance. The too busy to do a big programming
assignments can code in front of me for an hour. I should be able to judge
their chops pretty fast. For the people more nervous to code in front of me,
they can get paid a nominal fee to code some small piece of code.

The only group I exclude is the group that doesn't feel the need to show code
to land a job, but not too interested in that group. I have seen too many good
talkers and bad coders to want to risk this group.

~~~
twright0
> Offer $500 as a 1099 contractor to complete the assignment within the next 2
> weeks or so.

Just as an aside, I would actually be more willing to undertake this type of
interview if you weren't paying me. I don't have any ethical qualms about
interviewing while working a full-time engineering job, but I'm not entirely
comfortable taking a contracting job under the table. It may be something of a
symbolic gesture given that it's such a small time investment and small amount
of money, but it feels fundamentally dishonest, especially if you're a current
or plausible future competitor to the company I currently work at.

The official policy where I currently work is that moonlighting is allowed but
must be approved by our legal team, and while I could probably get away with
not asking for permission, I'd be pretty uncomfortable in both cases (asking,
or not asking).

~~~
eru
Just give candidates the option to specify a charity of their choice to
receive the $500 in their stead.

~~~
brianwawok
That is a good option too. Either defer it to a signing bonus or a charity of
their choice.

------
robspychala
From the employees' perspective:

Take home tests are the worst. Company says take home test will take 3 hours
to complete. They never do. Schedule 2x or 3x the estimate. Especially if you
want to impress the reviewer.

You send it over, then the company says no or yes, only to move to new stage.

In the worst case you ruined your weekend and received a no. But the company
just took 10 minutes to arbitrarily reject your application.

From the company's perspective:

I've seen applicants receive friends'/roommates'/spouse's help on take home
tests. Not a good indicator at all even with a glowing submission.

~~~
burger_moon
After doing a handful of these and rejecting several handful of these tests
I'd like to add a little to you comment.

I agree that the time it takes is always muchuch longer than what they state.

Companies that offer these tests before doing an initial phone screen get that
email deleted. Why would I as an applicant who is applying to 10+ jobs spend
time doing this test when I have never even had a chance to interact with a
human.

The tests are sometimes not even close to the actual job. As in the job
description is for a front end developer and JavaScript knowledge needed and
they ask you to write the test using a completely different language. one
company (who adversities jobs on here all the time) asked to do some php
command line scripting for a JavaScript front end position. How is that in
anyway a useful judgement of someone's skills. So wasting people's time is a
big deliminator.

An example of a company I experienced that did the take home 'right' did an
initial phone screen a couple days after applying. Then did another tech
screen which was just basic stuff. After that they asked me to do a take home
exercise and while completing it they continued to move forward with the
application process including setting up travel arrangements. The take home
test was directly related to the job and was given open ended for some
creativity if one chose. The onsite final interview was discussing the code,
so it would do little good to cheat on it because you need to be able to talk
through it.

I didn't even get the job with them but it was actually not a painful
experience for once to do a take home test.

Just my two cents, but I believe there is a good way and a terrible way to do
it.

~~~
pbrb
I agree. I've done really good ones and really bad ones. A good test started
in the office with the hiring manager. Really them just watching me write a
simple CRUD app in asp.net when it was all the rage. This was for an entry
level gig and was actually a really good test. At the end they had an
additional feature for you to add from home. It took a couple hrs and ended up
being a great test to find decent entry level ppl.

The bad one was about 2 yrs ago. I walked into a conference room for the final
round, was sat down at a mac, and asked to write some code in their
proprietary DSL without any documentation or anything. It was maddening. I
almost walked out, but the salary was stupid high. Didn't get the job.
Thinking back on it, maybe it was just a test and they did want me to say it
was ridiculous.

~~~
jaytaylor

        I almost walked out, but the salary was stupid high. Didn't
        get the job. Thinking back on it, maybe it was just a test
        and they did want me to say it was ridiculous.
    

Thinking about it, I see two realistic possibilities:

\- They are being _unintentionally_ stupid by asking you to do this activity.

\- If they _are_ intentionally asking you to do a stupid thing and they expect
you to revolt in the interview and decry the madness. How does that make you
feel about your future day-to-day life at that organization?

In both cases those are not good "A-player" types of people to work for.

No need to torture yourself over more favorable prospects "lost" :)

------
codeshaman
This seems like a wonderful idea.

I hate coding interviews, because I freeze up and I look like a total idiot
and cannot do even the simplest things. Same thing at exams in school or
university - my mind just went blank and into a noisy self-rumination loop. I
guess it's called anxiety.

I interviewed with Google last week and bombed it, even though the problem was
quite simple and I would have solved it wonderfully without all that anxiety.

My mind just can't produce any sensible thoughts when someone's breathing over
my neck and there's a timer. When I'm solving the problem in real time and
talking about it, I have to stick with the split decisions that I verbalize
and build on top of them, even though further down the road I realise they're
not optimal and this contributes to the anxiety even more.

It's hard to 'refactor' your ideas during a 45m coding interview, but this is
exactly the process we go through when we write 'real' code - we try a thing,
then we improve it by refactoring, we optimize it, we find and fix corner
cases, etc.

A take-home problem, on the other hand, would be totally cool. I have time to
think about the problem, come up with solutions, optimize, unit test - do the
real programming thing which I'm being hired for.

I would then gladly discuss and explain the code with the interviewer.

If then I would be given the task to extend the program with a new feature,
then I would be familiar with the data structures and algorithms used and
would probably find it much easier to extend, than starting with a blank file
and figuring it out on the spot.

But I guess not everyone agrees with the home interviews - some think it's a
waste of time and I'd have to agree..

So I guess the optimal solution is to offer the option of on-site coding
interview or a take-home problem.

People like me would take the problem home, build it and shine at it, others,
who's minds are sharpened by the adrenaline would take the on-site 1hr coding
challenge.

~~~
robinson7d
As others have mentioned, the trade-offs can be pretty complicated. Perhaps
the take-home is always better for you, I won't try to argue against your
personal preferences, I'm simply going to list a few reasons why it turns out
worse for a lot of people.

For one thing, as expected, the take-home questions are typically far larger
than in-person interview questions. When you go into the office you're
generally asked, in my experience, much simpler questions; you might get a
simple FizzBuzz-like starter, and a few slightly more difficult ones. All in,
it takes a few hours (Google and the like can be an exception to that, but
I'll get to that after - I'm comparing smaller companies right now).

The take-home work I've seen has generally been suggested by the company to
take 8 hours, but typically been a bit more; they've been along the lines of
build a fairly simple crud (I've had one that was a calendar, another that's a
stream, etc.) with a bootstrap-or-similar UI, back-end validations, and unit
tests for the initial commit, implement these 3-5 features on top of it in
separate commits (tested, also standard bootstrap-or-similar UI, proper
validation, etc.) And then, simply provide access to the repo.

At first, this didn't seem wrong, in fact I figured "well, I guess it's a good
way for them to see that you understand everything". But it came with several
annoyances. For one thing, since each feature is a commit, unless you
specifically try to prevent it they can see things like how long each part
took; even if not consciously, this makes people try to get the work done
quickly, without taking many breaks. It sort of reintroduces the "hard to
'refactor' your ideas during a 45m coding interview" problem. Another
annoyance is that sometimes (and judging by other comments here, even often)
they don't respond at all. The silence is unpleasant, to say the least. In-
person, you can at least try to read tone and facial expressions.

The other problem is one company I applied with threw a 10+ hour take-home
without any warning before the interview. So a bit over an hour in their
office, and another 10+ later. This is difficult if you're working another
job. It's very difficult if you have a couple such take-homes at the same
time, and a full-time, in office job. Scheduling a few interviews around work,
and other commitments, is pretty feasible, but throwing in dozens of hours of
additional coding is more difficult.

With companies like Google, you know ahead of time approximately how long it
will take. You can take time off accordingly. The process is known going in,
and that's great.

Basically, I think that for many people, myself included, white-board
interviews are very stressful and frightening, but not quite as much so as:
scheduling lots of extra time around your life, budgeting X extra hours
because they take longer than suggested, building out the (much larger)
projects, still having the time-constraint/expectancy issues, and after all of
that not always even getting a message/comment afterward.

Both interview processes suck.

~~~
codeshaman
There's a lot of truth in what you write and I definetely see the problem.

One solution is to have the take home interview as the final step in the
process - after having screened the candidate and discussed general
technology/framework/concepts without coding.

When I did interviews, I found that just by discussing things like thread
synchronization and let them explain the projects that they've worked on would
weed out most of the candidates which weren't a fit with the position. Then
the take-home problem would make a lot more sense for both the employer and
employee.

Or another method: You take home a problem, implement it, then present the
solution to _multiple_ potential employers. The problem would have to be
unique (so that there are no ready made solutions online) and trusted by the
participating employers.

And I think this is exactly what OP is doing, and that's why it's great.

~~~
robinson7d
I agree with all of those sentiments.

At the same time, I prefer the idea of presenting previous open source work
instead of having a group of potential employers agree on a project that they
can all judge. One benefit is that the work is less arbitrary, potentially
useful for others. Another is it's more likely to be unique than an assignment
is. Finally, do I want to work for any of those other employers?

Applying to a group that way would seem odd to me; part of me thinks that
choosing companies is important, but another thinks that there's some chance
that that's just because I haven't tried it.

Which, of course, leads me to agree that the OP is great as an experiment - it
might well be a better solution in many cases. The fact that, in the OP, it's
opt-in is a big deal as well. Though I'm still very skeptical of take-home
interviews in general.

------
nsfyn55
Take home interviews are a great indicator of a company's hubris. "Its so
awesome to work here people are going to jump at the chance to do my 3 hour
homework assignment".

The problem with them is fundamental: "You will only get someone desperate
enough to take your exam."

If the person is qualified they will be swimming in opportunities and will
likely throw your exam directly in the trash heap. If they aren't you probably
don't want them working for you. I suppose these might work if the entire
industry colluded on it but then ... prisoner's dilemma.

Maybe a more useful indicator is weeding out the candidates that didn't say
"no thanks." Cause really why are they so desperate for a job and why do they
have all this free time? but then ... ethics.

~~~
Udo
_> The problem with them is fundamental: "You will only get someone desperate
enough to take your exam."_

Lots of people do poorly in whiteboard situations. As an employer, you may
assert that you don't want any of them, fine. But I don't see the problem in
giving people _the option_ of using an alternative testing process.

~~~
ScottWhigham
Wait - where does it say this is an option? It isn't as far as I can tell - it
is how they do it. You either invest 3-9 hours on a "project" before the
interview or you don't get the interview.

This isn't "an alternative testing process" as such.

~~~
Udo
I logged into my TripleByte account and selected it as an option. The UI is
still broken, but yes: it's an option. It's a separate track you can choose to
be on.

~~~
nsfyn55
If its an option and they don't exhibit any bias towards candidates that
prefer a traditional interview then thats fine, but they should make it clear
in the article.

~~~
kwi
Founder here. We definitely don't and the outcome of both tracks will be
treated exactly the same way. As Ammon wrote it: "Anyone who passes our take-
home project assessment will get exactly the same service from us as people
who do the regular interviews. We'll work hard to find several YC startups
they'd be a great fit for, fast track them through the hiring processes, and
handle all logistics of flights/accommodations/scheduling."

~~~
nsfyn55
zing! you got me. I only skimmed the first paragraph. Nah sounds cool. I wish
you the best of luck. Let us know if it works.

------
hanlec
As many other parts of an interview, I've always found the blackboard coding
session extremely strange. When was the last time you coded in TextEdit with
no docs around, no time to think, standing up, and being watched over the
shoulder?

~~~
brianwawok
I have pseudo-coded on blackboards many times, which is really what a
blackboard coding interview should be. If you get a ding for writing .foreach
instead of .forEach, that seems a bit picky ;)

However I realize I may have a bias as I do ok on most blackboard coding
interviews I have done.. maybe been stumped 1 time out of 15 or so in my life?
Some of it is a skill, that the more you do the better you get at it, but
being good at it does not make you good at actual coding.

~~~
vonmoltke
> I have pseudo-coded on blackboards many times, which is really what a
> blackboard coding interview should be. If you get a ding for writing
> .foreach instead of .forEach, that seems a bit picky ;)

In my experience, some interviewers, particularly Amazon interviewers, get
really anal about putting _correct_ code on a whiteboard, sheet of paper, or
bare text document. I personally do not when giving interviews.

My bigger problem, and I seem to depart from the vocal majority on this, is
that my preferred normal workflow does not use whiteboards for anything
besides task lists and diagrams. In 13 years[1] as an engineer I have never
written actual or pseudocode on a whiteboard outside of an interview. I rarely
do it on paper. I have never worked with anyone who did this very much,
either.

[1] Granted, four of those were as an EE, but I still had to deal with code
from time to time

~~~
dpark
> _In my experience, some interviewers, particularly Amazon interviewers, get
> really anal about putting correct code on a whiteboard_

I had that experience with an Amazon interviewer. The guy who told me he
expected perfect code also gave me zero feedback as I worked (despite me
literally asking if I was headed in the direction he expected) and spent the
entire time staring at his laptop. He'd apparently forgotten that he was
scheduled for an interview, and also his manners.

------
nissimk
This is presented as a weeding out technique but it is actually a negotiation
step. This allows the employer to establish precedent that you work off hours
from home. This allows the employer to identify candidates that are willing to
do whatever it takes for the job. These are the same people that won't do
hardball negotiations for salary, employment terms or working conditions.
Rather than weeding out the people who will fail the test, it weeds out the
people who understand that asking for homework is unreasonable.

My last two phone interviews ended with the recruiter / hiring manager
explaining that the next step was a take home assignement or test. I said OK,
but haven't done either one. Next time I will politely tell them that they can
look at code samples in my gthub but I am not spending my time jumping through
their hoops just for the chance to have an in person interview.

It's bad enough that we have to burn an entire day to do an in person
interview...there shouldn't also be homework.

~~~
a3voices
My strategy was to tell them something like "First I'm going to interview at
all the companies that don't require take home assignments, and then I'll get
back to you if I don't have a job by then".

~~~
thirdtruck
This. Happened with my most recent round of interviews. I took an initial stab
at one assignment, moved onto other opportunities, and landed an offer before
getting back to the project work.

At least the take-home work I did perform in my second-to-last job search
provided good, compact example code for my last search.

------
mathattack
I like these great in concept. When I've been between jobs, I've been very
happy to participate in them.

A couple open questions:

1 - Is it reasonable to expect a 10x programmer whom your are trying to poach
to give up so much time? (Or should you give them a $250 Starbucks gift card
or something similar for their time?)

2 - Can you really ferret out cheating? I had a grad school classmate who paid
someone to do his (non-programming) take-home homework for a job interview,
and he got the job. He only lasted 6 or 7 months, but it was enough to be
awful for all parties involved. I don't have a great counter-solution other
than ask for someone to come in to the office to do the work, and even then
you can't tell if they have remote support.

~~~
vonmoltke
> 2 - Can you really ferret out cheating? I had a grad school classmate who
> paid someone to do his (non-programming) take-home homework for a job
> interview, and he got the job. He only lasted 6 or 7 months, but it was
> enough to be awful for all parties involved. I don't have a great counter-
> solution other than ask for someone to come in to the office to do the work,
> and even then you can't tell if they have remote support.

Personally, I think this is where the skill of the interviewer comes in. A
skilled interviewer can bust through bullshit fairly quickly. A big problem in
interviewing and hiring right now is that most of the interviewers are _not_
skilled. Thus, companies try to come up with processes and metrics that
(theoretically) remove the interviewer skill from the equation. Unfortunately,
this seems to be done in a horribly unscientific manner.

~~~
mathattack
I hear you on unskilled interviews.

I guess the challenge in this situation is that the whole reason for take-home
work is that introverted interviewees get flustered in person. Won't this
happen when the review happens?

This still seems like a better idea than "Tell me about yourself" and "How
many golf balls fit in a 747?"

~~~
jimbokun
"Won't this happen when the review happens?"

At that point, the candidate should not be hired.

If the candidate can't communicate while writing code, and can't communicate
about code already written, at some point you have to be afraid the candidate
just can't communicate.

If this is a person who can communicate in any situation except an interview
situation, then it's a Catch 22 where their ability to communicate in a work
environment just can't be verified.

(Unless you secretly record them communicating in some other work context?
Probably not a scalable approach.)

------
thoman23
One thing I always found interesting in these discussions is how the same
people who are quick to cite The Mythical Man-Month when it comes to the
futility of adding new people to a late project will also claim that working
1-3 hours on some trivial problem is "doing free work" for a company. How much
value do you think your 3 hours of work with absolutely no context can
possibly add to the company? A company trying to get meaningful work done by
secretly farming it out to interviewees is beyond ludicrous. Interviewers are
only interested in finding good hires, not in conning you into doing their
homework for them.

~~~
thedufer
I'm not interested in how much value the company gets out of it; it's how much
it costs me that matters.

And anecdotally, 1-3 hours is...optimistic.

------
brobdingnagian
Take home interviews are common in data science, and they are deeply
exploitative. I have been asked to spend anywhere between an hour to a week on
a project, and in most cases they simply decline to hire you without giving
you any feedback. In the worst cases, companies like Knewton and Mattermark
have given take home interviews that not only took a lot of time, but were
related to the companies core business model. In other words, it's free
consulting.

And finally, you aren't fooling anyone when you say that it should take 3
hours, or as long as they want to spend on it, while giving them 3 days. You
are pressuring people to spend as much time on the problem as possible to
"prove they are a good programmer." It's exploitative - you aren't paying us
and the probability that we will get hired is extremely small. Think about how
many companies we are interviewing with.

------
Udo
As someone who is still technically waiting - after a month - for his phone
interview, I like the idea of doing some actual coding. If you have to make
people jump through hoops, at least you might try and make the hoops at least
somewhat meaningful.

Edit: I just pressed the button on my TripleByte dashboard to switch to the
"project-based" track, but then I get redirected straight back to the same
"we're sorry" page I've been seeing all along.

------
hharnisch
Here's the best approach I've seen yet to the take home interview (and
interviewing software engineering candidates in general).

Here's a git repo, a problem statement and a slack channel to ask questions.
You can use any tools you like and spend as much time as you like on the
project for the next week.

\----

Employer's perspective: You get to see some code, see how they approach a
problem, see how they use their tools, and get a sense for what it's like to
work with them.

Candidates's Perspective: You get to use the tools you're already comfortable
with, you can set your own pace, and you get a sense of what it's like to work
with the team.

~~~
mostlystatic
I really like the idea of adding a Slack channel to the problem statement. I
always have questions, and channeling them through a recruiter via email is a
pain.

A lot of take home problems are very unrealistic, so it's difficult to make
the trade-offs that you'd normally make when trying to ship software. Being
able to ask someone on the team what their intentions are would be super
helpful.

My pet peeve is front-end tests that ask you to write maintainable code, but
you're not allowed to use any libraries. So instead you have to create your
own tiny MVC/templating/helper libraries just for the project - but obviously
they won't be as good as something you've spent more than 4 hours on.

~~~
hharnisch
Unless the task was to write an MVC or helper library I'd be concerned the
candidate wasn't using a library of some sort. It's really interesting getting
a sense of where someone spent their time on a project - asking the candidate
that question has been the turning point in the discussion because it a.)
tells you if they actually wrote the code and b.) gives you a closeup view on
how the person thinks

------
throwaway478291
Take-home interviews are a _perfect_ match for startups - they really reflect
actual working conditions. One way or another, you'll be taking home your work
every day! :)

~~~
dntrkv
I see people always saying this online, but the vast majority of engineers I
talk to in-person work very sane hours (9-5, 10-6) and never take their work
home with them.

------
neilk
We (Sauce Labs, Mobile Team) are experimenting with take-home tests as a
first-contact screen. I'm familiar with the take home exam that takes a week
to solve, so we went in a different direction.

We schedule time with candidates, send it to them at the right moment, and
expect them to send in a solution two hours later.

Before we tried it on candidates, we all did our own test, and we confirmed it
could be done in less than two hours. (Granted, we are very familiar with the
problems we care about, but we usually get complete solutions from
candidates.)

So far the results have been fascinating. The test is conceptually super easy,
deliberately so, but with a very wide set of possible implementations. It is
more about tying together moving parts in the right design. The goal was to do
the opposite of the algorithmic brainteasers you normally get in interviews --
it's a miniaturized version of the problems we actually solve all day.
Hopefully, it's a good test of what kind of coder the candidate is.

But we're still working on it and refining it.

~~~
dntrkv
Putting any time constraints on a dev test is a bad idea in my opinion. It's
not that I can't finish the test in the allotted time, but knowing that I am
being timed will make my brain race and do stupid things. Personally, I don't
mind take home tests that take more than 2 hours, as long as they reflect the
actual work I would be doing in that position.

------
VLM
The problem with random or unaligned assignments is you end up with a randomly
skilled programmer. Often not bad, but not on point.

The problem with Rosalind or Project Euler assignments is you end up with an
excellent theoretical math or bioinformatics programmer. Often not bad, but
not on point.

Fundamentally anyone with a degree or experience or a non-trivial github can
write code, but you want to test their judgement, their thought process, their
comprehension, their style, their knowledge about your business domain. Other
than total open field blank slate projects (very little of my time over the
last 35 yrs has been spent in that mode) you usually have existing systems and
code. So give them a "special" sample of your own code. Shouldn't be too hard
to find unless you're literally hiring the first technical employee. Then give
it to them a couple days before an interview and inform then you're gonna
review that code together, they'll present you with a rewritten, redesigned
version, and then review the rewrite together.

If you'll feel bad about making them do "real" work, the best code to send out
is some that has been heavily customized to only work some of the time, not
properly error check, and intentionally somewhat obfuscated, so I sincerely
hope that code thats screwed up to that level has to be intentionally manually
generated for the interview. So strip out most of the failure/error detection
code, screw up some of the code, maybe intentionally cut and paste an almost
identical function in place to see if they clean that up. Some folks like
intentional outright errors, like typos, is this the kind of programmer who
can't write English? Also wipe most of the comments, put some intentional
logic errors in the comments. This can be fun...

If you're looking for non-intro level programmers "everyone knows everyone"
and my latest job we didn't talk programming because I was vouched for as
knowing what I'm doing from years of coworker experience at a past employer.
I'd be moderately offended if someone I worked with for five years asked me to
fizzbuzz, either in person or as a take home test.

------
physcab
I like this idea. I took the Triplebyte quiz mostly out of curiosity and then
decided to stop when I was prompted for a phone interview. I like the idea of
"pre-qualifying" myself for a job even before I'm ready, so that when the time
is right, I can pull the trigger and switch easily.

With a phone interview you have to commit, which is no problem if you're in
the market. But if you're not in the market and you still want your options to
be open, interviewing on your own time makes a lot of sense and reduces the
mental burden and stress of switching.

------
lucasnemeth
I'm not particularly a fan of take-home tests. But comparing that to mumbling
at a white-board, while a software engineer is pretending to be an expert
psychologist at analysing your ~train of thought~, and unfortunately, more
often than not, someone in the board is only paying attention at how much of
an WASP male you are or pass by it, where you need to do a task that you only
repeat at interviews for a job that will not require that...compared to that,
give me take home interviews any day of my life. Yes they are free work, but
at least they are free work related to actual work. Live coding in the
whiteboard performances are not part of the work. And the stress argument
(that programmers must be able to deliver under stress) there are categories
of stress, social anxiety of a live performance is not the stress we deal at
our works. Sometimes I do good whiteboard coding interviews, sometimes I do
bad whiteboard coding interviews, none of the situations I felt the merit was
of the code or my knowledge of the subject, it is not a proof of how well a
programmer does his job, is a somewhat related way of knowing if somebody
knows who to code something and if he does not feel too nervous to code on a
whiteboard instead of a computer to a bunch of strangers testing him. If you
add that to the fact that some interviewers like be to randomly arrogant,
you're missing some really nice but perhaps shy coders.

~~~
francoisblavoet
This is why we use Take-Home tests here at deezer. The part we don't have got
right yet is how to manage the length of such an exercise. Finding a problem
where you have to both demonstrate how you handle several common issues and
that is not too long is pretty hard.

------
beering
One thing that a local company did was to give a simple programming assignment
(given postgres database connection info, create simple REST API using any
language). They would switch up the details of the assignment for each
candidate (e.g. different endpoints for same set of tables, different tables,
etc.) to combat cheating.

The applicant was expected to be able to answer questions about the code
anyways, so hopefully that would've been an effective layer of protection as
well. Don't know if this company is still doing it.

------
neilsharma
Historically, I've tended to do poorly in in-person interviews. I felt my
critical thinking and problem solving skills plummet to a fraction of what I
am capable of. I initially thought that with enough real-world interviewing
experience, I could familiarize myself to the stress, but that never really
happened. Interviews tend to be few and far between, so the familiarity never
really sticks.

I usually perform better on take home interviews, but 90% the time I'm
unwilling to dedicate what are usually days for just a chance to be accepted
at a company I may not even want to join. I think many employers use the take
home interview as a screener without realizing they need to first cultivate in
the candidate the enthusiasm and willingness to complete it.

Some ways companies can encourage me to actually complete the take home
interview:

\- Provide compelling information about the job (estimated pay/equity, meeting
the team, evaluating culture-fit, getting me excited about the problem, etc).
Lots of companies save this step for the courting process that happens after
the technical screen.

\- Pay me a modest amount just to complete it, regardless if I pass or not. I
don't care about the money, but at least it won't _feel_ like wasted time.

\- Make it way shorter, but then it might be useless.

\--

What I'd prefer instead of all this interviewing is still:

1) meet the team, evaluate culture fit, etc.

2) discuss past projects in detail, maybe do a code evaluation if relevant

3) contract for the company for 2-4 weeks at market rate pay. Hell, make a
10-week "internship" out of it, whatever.

4) receive offer (or not)

This is not always practical or scalable, but I've gotten offers 100% of the
time this way, including at YC companies and other startups. And from the
company's perspective, I think it extricates any unreasonable expectations
they have from prospective employees. It's always tempting to look for
unicorns when you have hundreds of resumes to choose from, but when you have a
likable contractor doing good work, there's no reason not to give him/her an
offer. Plus, the employer can evaluate the single most relevant skill in an
employee -- the ability to learn.

------
cblock811
I really hope this works. I never understood whiteboard interviews. They don't
reflect how real development works or how that developer works either. I hope
you post an update when you have enough info!

------
jameshart
Incredible to read so many cases here where people have done a take-home
exercise and heard nothing back. I had assumed that the normal practice would
be to give a take-home exercise as part of prep for an in person interview, so
if you do the work you're at least guaranteed the opportunity to talk about
it.

------
jzila
This is great. I especially love that candidates are given a choice, so if
they prefer the traditional technical interview they can choose that.

Normalizing performance between the two interview types will be challenging,
but I think the benefits to be reaped far outweigh the difficulty of the
challenges.

~~~
eplanit
Agreed. My reaction was "finally!". I really do not approach software
engineering as Performance Art -- and I never write code on white boards when
I work. I've been fortunate to have been asked only once to do such a
performance in an interview, and my reaction was unfortunately (and
unexpectedly) like the candidate they described. In that case I didn't write
anything on the board, but instead explained verbally how I would approach the
problem. They actually gave me a "take home" problem to solve. I was able to
go to my office and produce a solution like I normally do, and then sent them
the sources and test results.

Very glad to hear that folks are breaking free of this long-lived trend. I
also interview candidates on behalf of my clients. Personally, I find the most
effective technique is to choose items from their CV, and have describe in
detail (with a white board) how they did it, what challenges they faced, etc.
Then, pose to them a hypothetical "but what if you were constrained by X or Y,
how would you adapt your solution", etc. It is very easy to spot a charlatan
or liar in these questions. If they really did what they claimed on the CV,
this type of session gives them great latitude to demonstrate and expand on
what their capabilities are.

~~~
cableshaft
You've got to be a little careful picking out items from their CV, though,
especially if you go back far enough. I barely remember anything I did six
months ago, and anything one or two years ago I couldn't explain to you in
anything more than a really broad overview, despite being neck deep in the
code. I've since moved past that, tackled other projects, stuffed my head full
of knowledge of other systems (and hobbies), and all that deep technical
information on those projects is long, long gone.

~~~
qwibbler
If you put it on your resume, its fair game for the interviewer to ask about
it.

~~~
cableshaft
Not disagreeing with you, but just saying there can be legitimate reasons why
they don't have deep knowledge on past projects besides "they're a total
charlatan, a liar!" It's probably cost me jobs before, and I try to reflect on
and refresh my memory on past jobs as best as I can before interviews usually.

I've even gone over notes I took at a couple of my past jobs, and I don't even
remember doing a lot of those things even with it written down in my own
handwriting.

------
sytelus
How does this work if candidate has experience in, say, backend, but haven't
worked much on frontends or mobile apps etc? A specific "take home" task puts
those candidates at advantage who have already worked in related area, have
written code that they can immediately reuse or are experienced in frameworks
that allows them to fast trek. This would be great if the job also required
exact same capabilities for the foreseeable future from the candidate. However
it would overrate or underrate the candidate if the job requires candidates to
work on diverse set of problems in long run (for example, switching from MySQL
to graph problems). It would be interesting if article also gave few examples
of such "take home" tests.

------
erichurkman
That looks great. Are there enough problem seeds to help combat sharing of
answers?

Now we just need you at non-YC startups, too.

~~~
ammon
Triplebyte founder here. I can't be entirely sure until we give this a try,
but the hope is we'll be able to tell if the candidate wrote the code by
talking to them for 45-minutes about what they've done.

------
shockzzz
"Hey! Stop measuring me and give me job! I'm smart you asshole."

~ every software engineer in the world

Anyone else realize we're the only ones who complain about this shit?

~~~
odonnellryan
The interview process if kind-of crazy. I've been looking for a job for the
last few weeks and been on a few dozen interviews. 90% of my experience is
this:

1) They have a very-long interview process. Start to end is months, with weeks
between communications. 2) Each phase, when there are often from 6-10, can
take anytime from 2-8 hours. 3) Projects are always either right off the bat,
and lengthy, or a surprise at the end of the interview.

------
jays
My personal favorite method is similar to this and we use it at my current
company.

The big difference of course is that we pay for the candidates time. Not
everyone has 5-20 hours to burn on a project in terms of free time. It's a big
commitment, especially for the folks with families.

Some key benefits of a project for our interview process:

* We get to see a candidates programming skills with a greenfield project. This gives us a chance to analyze all aspects of development (organizing code, creating tests, using best practices, ect.).

* Candidates get to act as if they are on the team, so they are encouraged to ask questions, get feedback, ect. throughout the project. This gives us a chance to see how well they communicate in addition to their programming skills, which is REALLY important to us, since we're an entirely distributed team.

* The project is not throwaway for the candidate nor us. We usually have an ongoing list of libraries and tools that we need internally for our projects, so the candidate is actually contributing to our mission. __This also justifies paying a candidate for their time __

------
Raphmedia
I would never take this option.

At my current workplace, I have the tools I need to work. Most of my softwares
licences are under my employer's name. I have access to my snippets, my
previous projects and many build processes (minifying my code, preprocessing
my CSS, etc.). My workflow is great.

If you give me a project to do at home, I can't use any of that.

I would have to spent hours working either from my bed, my dining table or in
a coffee, on a laptop. I would also be required to work on this project after
the 40h/week that I already do.

I'll take the regular interview any time.

One of my worst interview process was one time when they required me to do a
project of 20+h in a week. They knew I was employed elsewhere, so they let me
give it Monday instead of Friday. This was ridiculous.

Otherwise, everything was doing very well. However, the experience really
discouraged me from working there.

But I guess it's not all that bad, because I most likely dodged a bullet. If
they can't even give correct deadlines to future employees, I can't imagine
how they threat their own employees.

------
roneesh
I view these as a good litmus test for the company too. If they won't answer a
question about the problem, or think it's very self explanatory, it might mean
they're not the best at communicating.

I recently did a clojure problem where in the gist itself I asked for feedback
and showed my thought process, but crickets until a month later when I
followed up and they said they had moved on. I think it was for the best.

A good preventative measure might be agreeing beforehand on a time to discuss
the solution. I know I certainly will next time.

Also hiring managers need to ACTUALLY read code. There have even been times
I've asked if they've read my code before doing a lengthy drive, they've said
yes, only to realize upon getting there they haven't. If you won't take some
time to assess the code I've written and put my name on, I don't think I
should be expected to drive 3 hours to meet you.

------
efsavage
To me the value in a take-home test is in smoke testing what the applicant
said they are familiar with. We once accidentally hired someone once who
talked the talk but didn't even know SSH.

My test involves a basic task (that is related and already implemented in our
codebase). You have to check out a skeleton project, add the solution, build
it, and deploy it. The test takes me about 30 minutes and most applicants
spend 60-90 on it. Everyone on the team reviews the test and the criteria is
essentially "do you want to work with this person". It's very rare for someone
to pass the test (which occurs between a phone screen and an in-person
interview) and not get an offer. The exceptions have all been around
compensation issues.

------
dandanisaur
The take-home/programming tests in general are just as bad as interview
questions. You're asking senior level engineers to take 3-5 hours (at what?
100-150$ hr bill-rate) to see if they can code?

Do you like them? Check. Do you feel like they are telling the truth? Check.
If you use the 'try before you buy' method: You can easily see if they can
handle the job/handle the pressure.

Also, you probably aren't the only company/avenue they are applying towards.
All of the calls/tests add up.

The question is though, since companies are already using take-home
tests/interview questions... can anyone confirm that these methods are more
effective than a 'try before you by/contract to hire' position?

------
bitshaker
My old employer Thoughtworks does this.

[http://www.thoughtworks.com/careers/application-
process](http://www.thoughtworks.com/careers/application-process)

It's a fantastic way to vet candidates and there are multiple exits to the
process. For obviously bad submissions, it's probably a no-hire, but I've even
seen incomplete assignments get someone to the next level and then a hire.

Some people get sent to ThoughtWorks University which basically trains them to
code, work in an agile environment and how to be a good consultant. Everyone
that goes says it is a great experience.

------
humbertomn
In 2008, I was hired to work on a tech startup from Australia, while I was
living in Brazil. After I went through an initial quiz and a few interviews,
they were willing to give me a shot by giving me (and paying for) a small
project to code. Only after succeeding at it, they hired me, sponsored my visa
and paid for all the relocation costs. In your case, I really think this
approach would be a good bet in cases that the YC company is willing to
sponsor the visa from the potential employee, so giving him a paid project
before even flying him, might be a good filter.

------
tragicthrowaway

        Rather than pick a new project, however, they'll take the same project further, incorporating feedback from the 1st interview.
    

Is there still judgmental coding during the 2 hour interview?

------
zawaideh
We consistently do this in Sandglaz whenever we interview for a technical
position. It most closely aligns with how work is done in real life, and we
give candidates a reasonable amount of time to do the project. We ask them to
document what they would do if they had more time. It is more of a way of
peering into how they organize code, their understanding of technical debt and
prototyping, and their ability to write unit tests.

It's been invaluable in determining candidate fit.

------
mellery451
This sounds like a brilliant idea. I hope that you are also including homework
assignments that cover (1) communication skills (especially WRITTEN
communication) and (2) personality. In my experience, those two areas are far
more critical to success than raw coding skills. Also, everyone should pass
the "no assholes" test, if there is such a thing.

I hope that you expect the same amount of prerequisite work for all of your
candidates (marketing, finance, sales).

------
vinchuco
Wondering if under this model an unethical person could set up a fake
interview process with real assignments to get unpaid work done by others who
want the job.

------
Taylor_OD
Tech recruiter here: I have seen so many companies that lose out on good
developers because they ask them to do a take home test.

Obviously you want to vet the person technically but you should be able to do
that through talking with them. Most developers put off take home tests even
if they are excited about the company and by the time they do it they already
have finals/offers from other companies.

------
bato
I actually did a similar thing as an ops guy (debugging systems as described
in some questions) and really liked the concept. On the other hand I work on
mathematical riddles for fun so maybe I'm the core target for those.

It doesn't replace face to face interviews but I can see it as a good example
of "how would face this real life situation you would encounter if you got the
job?"

------
darrenkopp
This is great, but I feel like take-home interviews are better for a first-
level filter. I went through this when I applied to Thoughtworks, but it
definitely wasn't the only interview. This may have been because I was in a
different state also though, so before they would pay to fly me out, they let
me pick out of a few problems and then email them in within 72-hours.

------
miscfuck
> The project-based track will require a larger time commitment

This is the only part I take issue with. I tried a take home interview once
that took the better part of a weekend and decided I'd never do it again.
There just isn't enough time. Basically, if it takes that long, there needs to
be a very high chance that I'm getting hired at the end, or it's not worth it.

------
deejbee
Why don't interviewers just have a conversation or is that so totally lost
these days? I find it a whole lot easier to just ask the right technical
questions in the first 2 mins. to decide on hiring.

I keep seeing more and more dis-associative non-communication being presented
where the interviewer attempts to do less and less. Its lazy and a crap way to
treat people.

------
alanh
I hate take-home interviews and rarely finish them. Especially when a
candidate has many options or places a high value on actually getting to meet
possible future team members (as I do), a two or three-hour assignment take-
home assignment is a lot to ask. I would rather get to show previous projects
and explain context and what I might do differently now.

------
heyheyhey
> We expect them to spend about 3 hours on the project (or as long as they
> want to spend to show us that they're a good programmer).

3 hours isn't too bad. I had a friend interview with some hedge fund and he
said he probably spent 30+ hours on a take-home project (he didn't get the
job). I thought that was way too excessive for just 1 interview.

------
msluyter
I did one of these when I applied at my most recent position. It took most of
the weekend and actually it was a lot of fun. But I doubt I'd do it again if
it required more than a few hours of my time. Any test that time consuming is
arguably biased against those with families.

Some advice to those doing take home tests: write (good) comments and include
tests.

------
dmitrygr
Gauging ability to operate under pressure is valuable too...ever tried to fix
a bug on a device that was supposed to ship yesterday, under the pressure of
the entirety of the hierarchy above you all the way up to the CxO?

It helps to know if the candidate you hire will handle that, or fall apart
just when you need him/her to do the job he/she is paid for.

~~~
bsder
Um, you really can't interview for that person, thanks. That person is someone
who is already in your company and gets promoted to "trusted lieutenant".
Promotion to "completely trusted lieutenant" is only awarded posthumously. :)

If that kind of situation is happening more than once in a blue moon,
something is very wrong at your company.

------
pinewurst
This sounds like a great alternative and certainly would work well for me.
What I'm afraid of though, is that I'd pass your interview and the
aforementioned YCombinator companies would again want to put me through the
on-site, high-pressure whiteboard technical interview.

~~~
Harj
That's definitely something we've started addressing as we optimize the
process of matching programmers with the right startups. We're starting to
gather data on how the YC companies run their hiring processes (seeing the
degree of variance has been fascinating) and we can use that to avoid exactly
this happening.

------
bluedino
I've done this, and liked the process.

Interview with 2 people for an hour or so. Then you get a Rails project, fix 3
bugs, choose and implement 2 features from a list, then 2 features they
choose. Come back in a few days and discuss why/how you did what you did.

------
Untit1ed
The problem I find with these is that while solving a problem might take 3
hours, you can spend an essentially unlimited amount of time polishing code -
writing comments/javadocs, neatening up the code, writing more unit tests etc.
etc.

------
balls187
Is there data on the efficacy of non traditional coding interviews vs
traditional ones?

I'm all for improving the process, but as someone who at times has struggled
on a coding interview, I believe that the onus to improve should be on me the
candidate.

------
radcam
Why is it ok with everyone that triplebyte is using desperate job seekers as
lab rats?

------
joslin01
I considered a job out in SF working for a decently well-known guy, Andrew
Chen. I had gone through 4 coding interviews progressively working my way up
their ladder to their CTO. Now, each of these I did well on but they didn't
like how fast I went. Despite them saying "Our fastest done was 1hr!", they
didn't seem to appreciate me going fast and came to the conclusion that I
probably didn't know how to write production code -- just a quick hacker.
Actually, it was rather annoying because their "top" iOS dev didn't even
understand basic shit I was doing. He'd have to keep stopping me and being
like wait -- why'd you do that? Yet as I worked my way up, I really enjoyed a
couple of their engineers and CTO. I didn't have to hold THEIR hand during a
code interview.

So I was an iOS developer getting considered for Android position, and they
said "hey it's great you can code fast & all, but can you learn fast?" and
told me to create an Android app that searched images on Google. I sighed
because I had already been through 4 coding interviews, but oh well, this will
be it I thought. Since I wanted to do a good job, I spent two hard days
working on it basically all day every day. Since I was new to Android world, I
had to learn that while being productive. You can see the result of my efforts
here: [https://github.com/joslinm/android-image-search-
example](https://github.com/joslinm/android-image-search-example). I thought I
did a nice job because rather than using a HTTP library, I read the streams
myself to give nice progress indicators for each image (which persisted even
thru phone rotates).

I give it to them and they say, "ok seems to be working." After that, I was
invited to SF to interview with them in person -- actually, interview is the
wrong word, I would be working with them for 2 full days. So we did that. When
I wasn't coding, I was being grilled about my work history (perfectly
acceptable, but why didn't we do this earlier?). Finally, after that, I was
told they had come to their decision. Thinking I was a shoe-in, I went to go
meet with Andrew. Andrew's decision: "You're too entrepreneurial."

Now let's examine this for a quick second. They had come to a pretty fair
conclusion that I was too entrepreneurial after talking to me. They realized,
hey this kid is a go-getter and probably isn't the greatest fit for our #7.
Yet this was AFTER copious amounts of coding and wasting my time doing all
these code interviews and even joining them to code in-house. If they had just
went about a pretty normal interview, they would have discovered this fact way
earlier and not wasted either of our time. So why waste my time forcing me to
code before they do their diligence? Because he's Andrew Chen; you can google
his name and discover who he is. It would be a "honor" to work for such a
prestigious start-up pundit. In other words, he can get away with it.

For those of you considering a start-up and actually have skill.. don't get
taken advantage of by these bogus work projects. The employer wins because
he's not paying you and gets a lot of evaluation for free. And if they do
insist on you working with them for a couple days, you insist on getting paid
for those couple days.

~~~
drdoom
That sucks. There was a time when being entrepreneurial was added to job
descriptions as a nice to have. I went through something similar a long time
ago; the reasoning was just as specious.

Good luck.

~~~
joslin01
Thanks -- specious is a nice word. Never heard that one before. Looks like I
got even more out of that experience ;)

------
geebee
I thought Gayle Laakman McDowell, who wrote "Cracking the Coding Interview",
wrote a good article about the problems of take-home interviews.

[http://www.gayle.com/blog/2013/09/18/companies-who-give-
cand...](http://www.gayle.com/blog/2013/09/18/companies-who-give-candidates-
homework-assignments-knock-it-off)

At the core of the problem is that this approach can be used to burn a lot of
a candidate's time without an equivalent investment from the company.

I've mentioned this before - I applied (maybe 5 years ago) to a company that
first asked me to take a Java test (about 1 hour of work). Then the recruiter
called me and sent me a take home project that should take "5-7 hours". I did
this, crickets chirped for a month, though I did check in with the recruiter
for a while (I gave up eventually). Finally the recruiter called me with a
one-line brush off "we've decided not to move forward at this time…"

Granted, this was a bad experience, and it won't always be like this. But I'm
pretty close to saying "never" to these exams.

I thought Gayle's suggestions were good. She goes so far as to suggest a 90%
passage rate - that you should not be giving these tests to people who are
unlikely to pass then, using them to confirm, not screen.

~~~
ammon
This only applies if the take-home interview is obligatory.

I agree that obligatory take-home interviews are unfair (I feel the same about
trial periods). They work (they're probably more accurate than a standard
interview), but they take too much time from the applicant (and will thus
scare away many of the best people).

As an option, however, I totally disagree. There's a significant percentage of
good programmers who are ill served by standard interviews. Those people want
this.

~~~
geebee
You know, while I agree that these problems are mitigated somewhat by making
the exam optional, I don't think this alone solves all the problems associated
with this approach. I still think that asking candidates to spend, say, 5-7
hours on something is a big deal, even if you give them a choice to take a
different path.

I'm not inherently opposed to technical tests, either in person or take home
exams. I believe that many exam-based institutions adhere to a code (probably
unwritten) that grants certain rights to the examinee. Think about exams in
college, where stakes can actually be quite high.

Think about exams at a university. Typically the subject matter and nature of
exam questions is available in advance. The grader is highly competent in the
field. An associated study path is available for the exam. The exam will be
graded and scored, and those scores will be communicated to the student within
a set time frame. Feedback will be provided to the student. The exam is part
of achieving a lasting credential, such as credit for a course on a
transcript.

None of these things exist in technical exams, and in many ways, this
increases the stress. Merely making an exam "optional" doesn't erase all these
problems. I think this is the core problem with technical exams, they contain
all of the stress for the student, but have none of those rights that I
believe exist in universities and other exam-based institutions for a good
reason.

------
sandworm101
But do you want someone who cannot handle an interview?

Perhaps take-home testing works if you want a "code monkey" who can sit in a
box churning out code according to specifications, but I cannot think of a
company that doesn't want more. The ability to interact with others in
sometimes stressful environments is, imho, an essential skill for all.

There is also the danger that companies will turn these "tests" into work
product. I did an interview last year (legal) that requested "writing samples"
on very particular and timely subjects. Once I realized the game I asked them
to sign a copyright agreement. They got screaming mad and tried to get me to
sign all sorts of NDA junk.

~~~
qwibbler
It seems as part of the take home that you spend 45 minutes discussing your
implementation and defending your choices in person with the interviewer. This
is a much more realistic "on the job" stressor than being able to code out
loud / on a white board.

------
collyw
I usually refuse them unless I have spoken with the technical people first.
You get no idea about the job from HR staff.

------
emmab
> Propranolol is used by musicians, actors, and public speakers for its
> ability to treat anxiety symptoms activated by the sympathetic nervous
> system.

\-
[https://en.wikipedia.org/wiki/Propranolol#Society_and_cultur...](https://en.wikipedia.org/wiki/Propranolol#Society_and_culture)

Can ask your doctor about this

------
serve_yay
Do this instead of white boarding or other coding during the interview.
Please.

------
kartikkumar
I dont really understand the way interviewing is set up in general.
Interviewing in my mind is a two-way street. It’s not just the candidate
that’s being interviewed for the job, it’s also the employer being interviewed
to determine suitability.

In light of this, I think most interview processes don’t make any sense. I
think that a screening phone/Skype call makes sense before either side commits
any time/resources etc. In this process, I think there should always be room
for the candidate to ask questions of the employer. Any situation that is
unbalanced, in which the employer is playing the role of the one with the
final decision is a completely skewed setup.

I interviewed a while back with a YC company and although I didn’t get the
job, I was highly appreciative of the setup. Firstly, there was a Skype/phone
call, which felt really like an open conversation; one in which I was free to
ask questions and understand suitability too.

Then I was invited to an in-house hack. Essentially, I spent the day (a
Saturday) hacking with the founders on a project. The project was a lot of fun
and something that I learned a ton from. We actually ended up working till
around 4am the next day. What I really appreciated is that although it was “an
exam” for me, the investment of time on my part was weighed equally by their
time investment. Also, I was happy to know that they had learned things during
the hack too. I was at ease during the whole day precisely because they
treated me as though I was their colleague already.

Coming out of that experience, I’ve wondered for a while why employers don’t
take the stance that to hire excellence you have to commit time and resources.
Most interview processes that I’ve heard of seem to be a cop out on the part
of the recuiter/interviewer, in the sense that it seems that the priority is
to hire the best people but spend the least time doing it. That to me is the
ingredient for a broken process.

So, in a nutshell, I think talk of take-home interviews misses the point. It
might not be scalable, but my intuition tells me that it would be much more
effective to have the interviewee and company folks hack on something
together. That way, the interviewee gets the opportunity to essentially
interview the company too. I’d definitely preceed this with a few
conversations on the phone/in person, to ensure that the time investment of
hacking together is geared towards the best interviewee/company relationships.

Perhaps this is missing the point too, and perhaps it’s not workable in the
real world. My anecdoctal evidence is simply that have spent an entire
Saturday working on a project as part of an interview process and ultimately
not getting the job, I don’t feel I wasted any time, nor do I feel bitter
about it.

------
joeblau
I've done quite a few "at home" programming challenges [1][2][3] as well as
hacking challenges and most of my experiences have been overwhelmingly
negative. The challenge is that "programming" is about 20% writing code and
80% other stuff (requirements, design, testing, security, documentation,
meetings, beer, etc...). It seems like the proposed solution is not addressing
the problem: Explaining how to solve a programming challenges you may have
never heard of on the spot.

I do agree that interviews add a level of pressure, but what you're trying to
gauge thought process. How does the candidate think about solving the problem;
Can they iterate; Can we lower the space and/or time complexity; Can this be
solved recursively, Are they receptive to input? A take home assignment does't
help you figure any of that out. Usually, the team who is interviewing already
knows the answer to the specific problem and all of the intricacies (based on
having to solve it on an internal project or just because it's a question
they've asked before), but the goal is to see someone thinking in real-time.

In my career, I've never had a time where I go off on my own, work on
something and come back to the team with a solution. Even when I was writing
TI-83 programs in High School to solve all of my science homework and finish
50 minute tests in 10 minutes; I still had classmates asking me about what I
was doing along the way. There is always discussion and I feel like by sending
me off on my own, all of that goes out of the window. You can't verify if I
copied the code from anywhere else, you don't know if I had my buddy who is an
engineer at another company break down the answer for me to spit it back to
you, and you don't know if I paired with someone else to get the assignment
complete. I'm obviously biased based on my experiences doing over 40
interviews in Silicon Valley/SF. My goal for this type of interview question
would be to understand how the candidate thinks; I don't care if you write
your answer in Brainf __k[4].

[1] - [https://github.com/joeblau/sample-elevator-control-
system](https://github.com/joeblau/sample-elevator-control-system)

[2] - [https://github.com/joeblau/sample-top-ten-
tweets](https://github.com/joeblau/sample-top-ten-tweets)

[3] - [https://github.com/joeblau/sample-url-
shortner](https://github.com/joeblau/sample-url-shortner)

[4] -
[https://en.wikipedia.org/wiki/Brainfuck](https://en.wikipedia.org/wiki/Brainfuck)

------
bbcbasic
Software developers huh! What other role requires you to complete an exam to
be considered for a job (or even just an interview). Moreover a 4-8 hour exam
with no syllabus. No 'past papers' etc.

I think this should stop. In it's place, developers can have a pet open-source
project, which they submit with their application. The point is that the same
project can be submitted for a hundred applications if needed, saving the
candidate dozens or hundreds of hours of wasted time.

They could put that time into the project instead and even if they don't get a
single offer, the open source community benefits, and the candidate got to do
something interesting rather than convert roman numerals or aggregate an
array.

I seriously wonder if we should all boycott dev tests. If they give a test,
just give them a github link and say "Please review this as a good example of
my work.". If they can't review code that isn't tied to a silly question maybe
they are not worth working for.

I say this having done some such tests recently myself. (To be fair one of
them said a Github submission is OK, I just didn't feel my Github was good
enough at the time.). I kind of feel bad contributing to the madness!

~~~
Nacraile
> What other role requires you to complete an exam to be considered for a job
> (or even just an interview). Moreover a 4-8 hour exam with no syllabus. No
> 'past papers' etc

As far as I can tell, essentially all of them. Of course, many of them have no
practical way to even attempt to assess competence, so they fall back on
"situational" and "culture fit" bullshit.

> I think this should stop. In it's place, developers can have a pet open-
> source project, which they submit with their application.

Problem the 1st: how do you verify the example project is actually the
candidate's own work?

Problem the 2nd: building and maintaining a non-trivial open source project is
a huge investment of time. There are many competent programmers who have a
long list of things they must / would rather do when they get home from
sitting at a computer programming all day. It's only more efficient if you
send out 100s of applications, but as far as I can tell, competent programmers
rarely spam 100s of applications. If you're just putting your toe in the water
from time to time, taking a day off to do a traditional tech interview grill
fest is a much more effective use of your time.

Problem the 3rd: evaluating a large, unfamiliar codebase is actually really
hard. All unfamiliar code looks shitty, because you don't have the full
context to explain the choices made. _Very few_ people have GitHub pages that
actually look good.

> I seriously wonder if we should all boycott dev tests.

Good luck with that co-ordination problem.

Tech interviews suck, for everyone involved. I still haven't heard a proposal
for an alternative that isn't fatally flawed. (I consider "take home
interviews" to be an incremental improvement on the basic whiteboard problem-
solving test)

~~~
bbcbasic
> Problem the 1st: how do you verify the example project is actually the
> candidate's own work?

Get them to talk about it in the interview

> Problem the 2nd: building and maintaining a non-trivial open source project
> is a huge investment of time...

Not really - in one weekend you probably can create something interesting and
good enough for showing off to an employer. Also it will probably draw you in
and you spend more time (for the fun of it) making it better. But you don't
have to.

For those people who prefer FizzBuzz tests to implementing interesting stuff,
I suggest companies provide the option to do a test for those who don't want
to submit their project code.

> Good luck with that co-ordination problem.

A first stage could be that we say "no tests until at least an interview has
occurred and the candidate has progressed to the next stage". That is in any
candidates own interests regardless of other's actions - so wouldn't require
coordination, just communication of the idea. It is in his own interests
because the time spent on the speculative work could be better yielded doing
something else, such as attending a meetup and generating more opportunities.
Looking at it from a sales perspective of how to spend your time - lead
qualification, lead generation etc. Why spend a lot of effort on an
unqualified lead?

~~~
matwood
Even trivial code is fine on Github. All I look for is that someone can
actually write a bit of code, and a GH repo helps me quickly asses otherwise I
have to do more digging. I've interviewed entirely too many people who are
clueless and leave me wondering how they have ever had a job writing software.

> For those people who prefer FizzBuzz tests to implementing interesting
> stuff, I suggest companies provide the option to do a test for those who
> don't want to submit their project code.

In fact a great personal GH repo could be to implement FizzBuzz in a bunch of
different languages. I think most companies would get an appreciate the joke.

------
joesmo
So you do the test, but you're not guaranteed a position even if your code is
amazing.

Hell, in my experience you're not even guaranteed anyone will read it.

This is just a bad idea in an industry full of irresponsible employers. I'm
not saying that Triplebyte is in any way irresponsible, but the rest of the
industry has already ruined this avenue.

~~~
Harj
Triplebyte co-founder here. We understand how frustrating it'd be to spend
time on a project and then have no one read it, we won't be doing that. In our
process, you schedule a time to talk with us about the code you've written.
We'll go through it together and provide feedback.

We're not replacing talking to another person part of the interview, we're
just trying to remove the awkwardness of having someone watch you code (for
those programmers who want it).

------
brogrammer90
Only desperate people do these tests. I know this first hand because I've been
given a handful of them and can never find the motivation to complete them.

------
starrychloe
I've wasted too much time on take-home interviews. I'll never do one again.
All the contracts I've ever gotten never required one.

