
Show HN: Takehome.io – Time-limited coding challenges for interviews via Git - davidbanham
https://takehome.io/
======
planetjones
I don’t like the timeboxing aspect. It’s putting pressure on the candidate. I
don’t know why the website says there is less pressure when it’s time limited.
Do you have a study or other evidence to support this claim?

For code exercises I set when recruiting I send a pdf out and then expect a
zip file or link to GitHub or bitbucket with the solution. I might give an
advisory along the lines of this shouldn’t take more than half a day. But if
the candidate screws up and starts again that’s ok. Mistakes and wrong turns
are taken all the time in the real world. It works very well. Simple. No
complications. And free.

~~~
davidbanham
Hey, thanks for the feedback.

The idea is that the candidate knows they can do the best they can within the
time limit, and then it's done. No agonizing over whether it's good enough or
they should spend another few hours. No imagining how many hours and days
other candidates for the same role might be putting in, wondering if they're
going to stack up.

I've used this approach with around 100 candidates hiring for various roles in
my own business and for clients. The response I've gotten from candidates has
been overwhelmingly positive.

The problem I have with the traditional "don't spend more than about X time"
approach is that it's not a level playing field. The responses to a challenge
exist in relative terms, not absolute. It doesn't matter whether your response
was great or not. It matters whether your response was in the top 5 of all the
other candidates responding. It follows that you probably need to spend at
least as much time, if not more, than all the other applicants for the role.
How much time the other candidates _might_ be spending is basically unbounded,
leading candidates to spend vastly too much time working on a challenge and
then lying about it.

If the candidate screws up a timed challenge that's also totally okay. It'll
be a great conversation to have in the follow-up interview. "I started down
this road, but if I was doing it again I'd do X, Y and Z because of A, B and
C."

Incidentally, I'd suggest that half a day is way too long. Especially if that
half a day isn't actually enforced. When I'm hiring I usually set challenges
of about 40 minutes, an hour and a half max. I'm not looking for a product
that's ready to ship to production. I'm after a sense of whether the candidate
can actually interpret a problem then string some code together, and something
meaningful we can discuss in the next interview.

~~~
weego
How often in their day job will someone be under artificial time pressure to
solve a task and get it to a point where it's "good enough"?

You're actively removing the possibility of finding out who would think "meh
thats good enough I guess" is the effort needed for production code and people
who would go the extra step of spending time analysing and fully solving the
problem.

2 people with equally average solutions...would one have done more given the
chance? Both? Neither? How can we find out? You've added uncertainty to the
process rather than solved any problem.

~~~
nkristoffersen
Often enough someone will be under real time pressure to solve a task, right?

~~~
janoc
That's more indicative of poor management and planning on the company's side
than reflecting on the performance of the candidate if the time pressure is a
regular occurrence.

Few people perform well under pressure. Do you want to test what is their
mediocre performance under time limits like on a school test or what they are
really capable of?

With this approach you will hire people able to churn out crappy but "good
enough" code in the limited time. Not people who actually can do quality work.

~~~
watwut
No real company has perfect management and a lot of them have poor planning.

> Few people perform well under pressure.

Yes, but majority of jobs have some level of time pressure. That being said,
there is such a thing as reasonable deadline.

> With this approach you will hire people able to churn out crappy but "good
> enough" code in the limited time. Not people who actually can do quality
> work.

Inability to perform under pressure does not imply high skill without
pressure.

~~~
janoc
> Inability to perform under pressure does not imply high skill without
> pressure.

Ability to churn out crap in a time limit doesn't imply competence neither.

~~~
Bjartr
Well, it implies competence at churning out crap within a time limit, which
for a lot of managers out there is precisely what they're looking for.

------
juskrey
My rule of thumb was to decline any interview with "challenges" \- works
perfectly as a filter of perverted corporate climate.

~~~
asadlionpk
I wonder how would you do hiring for your fictional company? (Assuming you get
100s of shitty dev resumes when you post a job opening)

~~~
juskrey
I am actually hiring for my contracting projects - I just have a general talk
and ask for a piece of any recent code. And even this is just a ritual: only
real project in your (not other company) real work environment shows.

~~~
chii
> any recent code

which is unlikely to be available if they aren't the owners of the IP.

So either you have to guess real lucky, or have a probation period of a week
or two with any new hire. Wouldn't you say that's worse than a short take-home
project (for the employee)?

~~~
juskrey
I am not saying I don't have problems with finding people, I am saying here
that companies are creating even worse problems by their naive automated HR
assumptions.

------
profunctor
I thought I would chip in on the pro-time limit side as we seem to be
underrepresented so far. I certainly have been given challenges that
supposedly take 6 hours but clearly take much longer than that. An entire
weekend at least. I don't have that time and I declined but if it really was a
6 hour test they would have gotten an attempt from me. A time limit makes it
fair for those with other commitments.

------
mwcampbell
I like it! We may quibble over what the time limit should be, but I think that
doing coding challenges this way is the most inclusive and fair way to do a
coding interview. All candidates do the work in their own environment, so one
doesn't have to make an exception for, as one example, a blind person who
can't write on a whiteboard. It's also more inclusive for potential
interviewers; consider the challenges involved in a blind or visually impaired
interviewer watching a candidate code in real-time. This way, the interviewer
doesn't have to watch in real time, just as coworkers usually don't watch each
other in real time on the job. Unless the team does pair programming, of
course.

To take it all the way, I think the follow-up discussion should be done
strictly in text, using some combination of IM and a web-based code review
tool. So the interviewer would call out parts of the PR and comment on them
through the code review tool, and the two could have more free-flowing
discussion over IM. Again, this would be just like the way it's actually done
on the job, at least in my dream all-remote environment.

------
dvt
I like the gist of this (I think take-homes are the best way to evaluate a
candidate), but making it time limited seems weird. You're basically taking a
test that you haven't studied for -- that doesn't seem fair (or fun).

~~~
davidbanham
What is an interview if not a test? The code challenge portion of an interview
process is a test that you've been studying for your entire career.

As for fairness, I'd argue that timing it is the only way to ensure it. This
way you know that the playing field is level between you and the other
candidates applying for the role.

To fun, I'd say that the least fun thing I can think of in this context is
spending days wondering whether I had "done enough yet" in order to make my
submission stand out ahead of the other applicants. Much more fun to know that
as soon as that clock ticks over, what's done is done and I can get on with my
life for better or worse.

~~~
xfer
How does your approach differ from the sites being already used by companies
for online coding test? They do the same thing, you login during some fixed
duration(24-48hr) in the weekend, and you do the coding challenges(they have a
web interface to compile your code) within the alloted time.

~~~
davidbanham
I think the main difference is that those sites are geared towards outputting
a ranking or score of some time. Takehome is geared at giving you a way to
qualitatively assess the skills of each candidate and give you something
meaningful to discuss with them in a follow-up interview.

Also, all of the existing sites I've seen have one or more of the following
problems:

* Make you use their editor * Make you install their software on your computer * Make you use some specific language * Make you use their set of challenges

Takehome uses nothing but git. Whatever you put in the repository and how you
get it there is up to you.

That holds true both for the candidate and the challenger. Takehome encourages
challengers to create their own task matching the kind of
work/language/framework/etc that's relevant to their team.

------
zapt02
Don't see the value of the service. It just sounds like a simple CRUD app
where people can create "challenges" and send them out to potential
candidates.

And for so little done by the service, I find the cost to be completely
perverse.

~~~
lozenge
Yep, all it does is wait for the first `git push`, display the challenge text,
then allow further pushes before the time limit and denies them after the time
limit. A per candidate price quickly gets expensive.

~~~
davidbanham
The price is per-candidate to discourage companies from sending out more
challenges than they have time to thoughtfully review. The expected use case
is that you interview the candidate, then if you are both still interested you
issue a takehome challenge, then you have another interview to discuss the
challenge submission.

If you're issuing more than 5 challenges per role you fill something somewhere
has probably gone awry. $100 per role hired is no more than a rounding error.

Also, the timer starts from the first `git clone` rather than the first push.

~~~
Tomino
IMHO trying to educate companies about their process is a sure way to failure.

Also doing interviews before technical challenge is a sure suicide for
companies. In my experience to minimise the waste of time you send out quick
and easy technical challenge (no more than 30 minutes!) and you filter out 90%
of applicants. With remaining 10% you have a phone call and if you have more
than X (magic number of 3 in my experience) of qualified applicants you send
out more thorough tech challenge (no more than 2h long!).

Rinse and repeat until you get to X applicants which you will invite for face
to face interview.

Truth is that almost no-one in tech is qualified to do face-to-face
interviews. Either they are not on the level technically or dont have the
communication and HR skills. Interview in person is mostly to see if you are
not dealing with a weirdo and if that person will be a good fit for the team.

A side note, I tend to issue valuable response to every candidate. If you do
that, they are very likely to re-apply, step up when someone drops out or
recommend to friends. People tend to do/say the same dumm stuff so you can
prepare those replies as well.

------
jakozaur
I use Codility for code screening:
[https://www.codility.com/](https://www.codility.com/)

In most of the Europe take home exams are common during the interview process,
but your online test should be 1h (at most 2h). If you make it longer, very
few ppl will invest that much time in competitive job market such as software
engineering.

Take home tests are way harder in the hottest market such as San Francisco Bay
Area. A lot of candidates will simply drop out of your pipeline if you send
them. IMO there are good for fresh grads, junior candidates or candidates that
would you otherwise reject in resume screening. There are hidden gems, often
in underrepresented minority that simple don't have anything impressive on
their resume yet. Being able to identify them can be your competitive
advantage.

~~~
joezydeco
I've used TestDome in the past,
[http://www.testdome.com](http://www.testdome.com).

The tests are tidy and the in-browser IDE is not bad.

------
zackify
I just completed the task in under 10 minutes for the example socialite one.
It would be neat if the candidate could mark the repo as finished, so that you
don't have to wait the whole time before it is over. Cool idea overall!

~~~
davidbanham
Thanks, that's a great idea. I'll add it to my burndown.

------
ahmedm
We send much smaller coding tests using
[https://remoteinterview.io/](https://remoteinterview.io/). Take-homes require
way too much time commitment from the candidate side. We have tried them but
the candidates would never finish.

With coding tests, we isolate a smaller problem and send that as a fizzbuzz
screener. This scales well as solutions are checked automatically.

------
crsv
I like the trend towards tools like this helping with the software engineering
hiring process. I know that these things can be points of contention in the
community, but in general the problem of reducing the time commitment from the
hiring team while still providing the candidate with a low-friction experience
that produces a meaningful expression of capability isn't going away any time
soon.

------
z3t4
Probably should offer a review service and coding challenges too! A company
that is capable of reviewing and making code challenges surely has the
competence to set up a git server, but there's plenty of companies that can't
or don't have time, so I definitely think there's a market.

------
sciurus
A service where I can `git clone` a repo and _that_ is what starts the timer
for the coding challenge is a great idea.

When I was recently job hunting, I had a bad experience with a company that
wanted me to take a timed coding challenge where I would clone a repo and then
submit a pull request with my solution. What started the timer then was them
opening the repo to me, not me accessing it. This required us to agree on a
time for me to take the challenge. Long story short, the recruiter proved
incapable of opening the repo at the times we agreed on.

Avoiding that need for coordination and chance for the recruiter to mess
things up is a nice benefit of your service.

------
superplussed
This seems useful, and I get the value. I'm wondering how you balance exactly
how much time to allot? Do you try to come up with an average estimate of time
needed and then like 2x that to reduce time anxiety?

~~~
davidbanham
Thankyou! Any person using the site to hire candidates can set whatever time
limit they like.

When I'm choosing a time I'm actually less worried about the complexity of the
exercise than how much of a candidate's time it's reasonable to ask for. If
the challenge is too complex to be done in 40 minutes or so, I make it less
complex.

I also put language in the intro text like 'If you don't end up with a
complete solution in that time, that's fine, just submit it anyway. We don't
expect this to be enough time to write "The Most Perfect Thing".'

If you're curious, sign up and take a look at the "Socialite" sample
challenge.

------
zng00
This is too expensive and it doesn't even let me take the challenges myself
without paying $20AUD, even though is says "Free Trial".

Also, it's basically just 2 forms and some scripts.

~~~
davidbanham
It definitely _should_ allow you to challenge your own email address without
any charge. If you're still having trouble and you'd like to test it out,
please drop me a line on hello@takehome.io and we'll figure it out. The trial
has worked great for everyone else so it sounds like a juicy bug I'd love to
hunt if it's not working for you.

------
caleblloyd
I think this looks really cool. I like the ability to send multiple challenges
to a candidate. I also like that as a side effect, it tests a candidates
proficiency with Git.

I would choose a large time window- 4-8 hours or so. That way it ensures there
is plenty of time to complete the challenge but also that it gets done in one
day, like a normal workday.

~~~
Tomino
By proficiency you mean clone and push?

~~~
caleblloyd
Clone, add, commit messages, push, ignoring dependencies, writing an
instructive README.md, etc. All basic skills you'd expect a developer to have.

~~~
EpicEng
>All basic skills you'd expect a developer to have.

Yeah, if they've used git before. Otherwise they'll need at least a solid 30
minutes to figure out the most basic git usage pattern.

Do you really care about what source control system a person has experience
in? Why? Any competent developer can pick it up quickly enough.

------
sandGorgon
this looks very cool - the single biggest thing that makes or breaks this
space is the strength of the integrated applicant tracking system (ATS).

Do I need to use another ATS ? then my team will just give up on this. If you
have a limited ATS in there (with stuff like emailing from the dashboard,
reading replies from the dashboard), then this is killer

~~~
davidbanham
Currently no ATS at all. I've specifically kept it focused on doing a single
thing well. I appreciate the feedback, though. I'll consider incorporating
something along those lines.

------
abglassman
This is a great idea. I could see becoming a customer of yours if you added:
\- payment in USD \- Ability to provide a post-receive hook to run validation
/ tests / benchmarks, and see results with the notification of submission. I
appreciate that doing this in secure fashion is a nontrivial effort.

~~~
emilecantin
> payment in USD

Honest question: why? Can't your card / bank / PayPal do the conversion for
you? (I say this as a Canadian who often pays USD with my Canadian card / bank
account).

~~~
ascorbic
The bigger issue for me is pricing in USD, and less about the currency in
which payment is collected. I'm in the UK, and while I can generally work out
a rough exchange rate for GBP to USD and EUR in my head, I've no idea what the
current AUD rate is. Making me check is a small but real barrier.

------
north_east_dev
The idea of "take home tests" for an interview is utterly bizarre to me.
Thankfully I've never heard of it my parts, seems to be a mainly US thing?

~~~
Woofles
I think most jobs I applied to out of college had them. It is a good way to
vet if someone can code before bringing them in for an interview and wasting a
bunch of peoples' time. At my company we're currently struggling with not
doing timed tests but not wasting too much of a candidate's time.

We hire a lot straight out of college, which I think is probably where this
type of test is more common.

------
zerr
Integrate the payment mechanism.

------
thrownaway954
i will say that I like your pricing model. Free to try it out yourself and
only $20 (or whatever the currency rate is in AUS) for each time you use it
with no monthly fee.

Personally this is how most online services should be, pay per use.

