
Ask HN: Your thoughts about online developer recruitment tools like HackerRank? - ninadmhatre
I recently appeared for the first time on online challenges as a first round of interview process on HackerRank, well experience was good but as i have participated in only 1 of such process, i don&#x27;t have strong opinion about it. i have been part of live&#x2F;shared coding interview, they seem to have hard limit as you must solve X% of challenges irrespective of how cleverly you solve the them? So not sure if that angle is even considered when judging the answer.<p>Just wanted to thoughts about such tools? do you think they do a good&#x2F;bad job? Or is it proper approach to find the talent?
======
onion2k
In my opinion they test the wrong things for the majority of software
developers. Most people who write code don't need the ability to solve
especially hard algorithmic problems. Ranking solutions to hard problems is
probably useful if you're Google but there aren't many Googles out there. We
like to think that we're solving hard problems (and actually, on HN, we're
more likely to be), but most companies aren't. A large amount of software is
_literally_ just a CRUD form and a database.

For the overwhelming majority of software businesses it's far more useful to
know if someone is disciplined enough to use good variable names than whether
they can write an algorithm to count the number of edges in an acyclic graph.

~~~
zby
You are right that the biggest part of the job is easy - but testing the easy
parts is useless - all programmers would pass it. A test must be hard in some
way if it is to filter people.

Disclaimer: I have a stake in [https://codility.com/](https://codility.com/)

~~~
jzwinck
I assure you this is not always the case. I have worked directly with multiple
people who could contrive the most exquisite solutions to really difficult
problems but whose code was gibberish. Flagging such characteristics during
recruitment would be valuable.

~~~
zby
Interesting - how would you flag it? In what way was their code gibberish?

In my experience people who understand complex concepts can also understand
how to write clear code. Maybe those that you are writing about were not
motivated to do that - or maybe it never occurred to them that this was
needed?

Would you mind also explaining what you mean by this in "this is not always
the case"? Because it does not seem to refer to my comment.

~~~
junto
I guess that concise implementations (e.g. in a one liner) are often a
complete bitch to debug when someone else comes back to it to fix a problem
years later.

It is much more difficult to debug and fix existing code than develop it from
scratch. Hence, if you found the problem hard to code in the first place, then
when you come to debug it again later you've just screwed your future self.

~~~
zby
The problem with people writing too concise code is with their motivation not
with their abilities - and so it is probably impossible to test (at least
directly), because on the test they would have the motivation to do the right
thing.

~~~
st3v3r
No. If someone has to be "motivated" to write clear code, especially given
that these tests are supposed to lead to jobs, then I would have to question
their abilities. Writing clear code is an ability of itself.

~~~
zby
The point was about code 'too concise' \- this is different from 'not clear'.

------
jakozaur
I haven't used HackerRank, but got experience with similar product Codility
([https://codility.com/](https://codility.com/)).

My experience:

1\. Works great if engineers apply to you, much less if you try to poach a
engineer from great company.

2\. Great resumes or credentials sometimes don't correlate with great
performance. Especially there are hidden self-trained engineers, who can build
great products, but never had CS degree or worked in well known company. It
can be competitive advantage if you can give standardized test to everyone who
apply, when most companies wouldn't bother to invite them for an interview.

3\. Still they can save a lot of time for both ends:

a) candidate usually prefer to spend one hour on an online test than to take a
day off for full day of interviews

b) company can offer this test to anyone, time of senior developers is usually
too costly to offer regular interview to anyone

4\. It is not a replacement for regular interview process. You still have to
interview candidate. Most of the time, signal from this tests is clear if they
are used properly (e.g. give mix of tasks with different difficulties).

~~~
arisAlexis
Point no2. The people that built great products and have good cv's don't have
great performance? And why would you want to filter out great product builders
because they don't have a CS degree or worked in a big company before?

------
eisokant
I think HackerRank is a great platform for challenges but I personally believe
it only shows a small part of your skill set. I am a developer myself but also
have hired a lot of engineers in my career. I believe that looking at existing
projects you've done on Github gives a much better overview of your coding
style, knowledge and your ability to learn. It also exposes some of the biases
I know that we have in our own company (for instance, we know that if you have
a functional programming background you'll fit in much better in our code base
and team, even though we don't do functional programming).

All of this said, I think HackerRank is a great place to improve your skills
in solving interview challenges and algorithmic problems.

Disclosure: I am a co-founder at source{d} where we analyse all git based
projects to understand developers through their code.

~~~
planetjones
From experience it's very hard to know how a candidate will perform when
hired, irrespective of the hiring process.

A better recruiting model IMO would be to recruit people on short-term
contracts (e.g. 6 months) with the contractual agreement that the worker will
transition to a permanent job at the end of the 6 months providing their
performance is inline with expectation.

I do use code exercises myself, but it's not some n queens problem or
something - which tests nothing practical (for the positions in my team
anyhow). Rather it's a simple problem, where I look for their ability to test
drive code and clean code. The candidate then doesn't get pressured as they
might in a live situation. If the code is good we walkthrough it in person.

~~~
sumedh
Your solution looks good on paper but if a candidate has two offers one
permanent and the other the 6 month to permanent , I am pretty sure most would
go for the permanent offer.

~~~
dozzie
In my neighbourhood it is normal to hire people for a trial period first, up
to three months (this timespan is regulated by law).

------
escap
"Programming competitions correlate negatively with being good on the job" see
[https://news.ycombinator.com/item?id=9324209](https://news.ycombinator.com/item?id=9324209)
for the more nuanced actual result.

~~~
Waye
(Disclaimer: I have a stake in
[https://www.kattis.com/](https://www.kattis.com/))

Solving a programming challenge is not the same as winning the ICPC. And that
is what a lot of people here are saying it matters how you use these tools. If
you ask developers to solve the very hardest challenges used during the ICPC
you are probably going to waste their time and you might miss out on
candidates who would have done a good job even if they couldn't solve that
challenge. But being asked to have a basic understanding of algorithms, which
is what is needed to solve the average challenges, is something that I think
should be taken for granted whatever position you are applying to.

------
blablablame
I've been asked to do a few of these a while ago while searching for a job.
Don't remember if I did HackerRank though.

I don't really like it. I scored well in most that I did, but as some folks
mentioned, they either test the wrong things, or have horrible UI's.

Also, there are some that end up having multiple choice answers that are
really badly done (bad question syntax, having 2 correct answers, outdated
questions, etc).

I personally refuse to do any of them nowadays, even if I am really interested
in the company. I just can't justify the time and (in case of badly done)
frustration so companies can' save a bit of time.

------
gcb0
their code editor is a joke.

ui doesn't make it clear switching tasks will lose your work, and to make it
worse they silently break the clipboard so you can't "cheat" thus preventing
you from storing your code anywhere. makes experimentation really awful as you
have to keep commenting the whole thing every try

~~~
dozzie
Now you made me want to register just to check if my Vim with ItsAllText
works.

------
madsonlife
I failed an interview a couple days ago, from a hackerrank challenge. The
tools was awesome, but could be better. I got really nervous, and could clear
the challenge 10 minutes after the stipulated time in fact. One of the things
that made me most nervous was that i was not able to debug my code as i can
easily on chrome console. The "click to run your code" gives you a bit of a
harder time. (my opinion)

~~~
jk563
When I completed Hackerrank tests for my last role, I used an IDE and then
copied and pasted the code into the browser.

This gave the benefit of a familiar working environment, and I could quickly
TDD as needed. I highly recommend this approach when faced with these tests.
That and working out in advance an appropriate way to accept input.

------
zippy786
I've been in the industry a while and if a company/recruiter uses tools like
HackerRank for recruitment, I simply ignore them and decide not to move
forward. I think we as a software industry have been following a bad hiring
practice. If you can't tell your fellow colleague is smart enough by having a
tech-talk or giving something that's not a test-format, there is something
wrong with the process or people doing the hiring. Also, it is hilarious how
many times an engineer has to prove himself/herself, and after being hired at
these places I've also seen that the people who do the hiring are not that
good/competent themselves, although this may vary in some other place.

------
asymmetric
I didn't know about HackerRank. The concept seems quite similar to
Stockfighter. Any idea how it compares?

~~~
kbuck
HackerRank's core product is a tool that companies can use to send programming
tests to potential candidates. It's typically used to screen candidates prior
to an interview.

The Stockfighter-like concept is newer for them; it seems like they're also
trying to work the pipeline from the other direction as well (i.e. finding
good candidates for companies, instead of just testing candidates that have
already applied for a position).

------
arisAlexis
I think these tools underestimate the impact stress has on coders. I would
dare to say more than 30% of ability can be impacted. Since coders rarely
write code under extreme stress unless they work for stock market apps I don't
see the point of having such a huge confounder.

~~~
Waye
(Disclaimer: I have a stake in
[https://www.kattis.com/](https://www.kattis.com/))

Not all of "these tools" use stress as part of the evaluation. Kattis is one
example that allow candidates unlimited amount of time. However it should be
noted I think it's possible to increase the time using these other tools up to
some max limit.

------
EvanPlaice
Hacker Rank is to hiring what Tinder would be to dating if profiles were made
up of genital pics and length/depth measurements.

It'll give you a clear measurement of a candidates ability to do CS homework
and not much else.

------
kbuck
I've been on both sides of HackerRank. I helped develop/configure a HackerRank
test for candidates for my team, and recently I've taken a HackerRank test
while applying for a position at another company.

HackerRank is just a tool. Its effectiveness depends on how well the company
interviewing candidates configures it. I think algorithmic questions are the
most popular, but it's completely configurable; you can have it ask whatever
you want. It's also possible to manually review submissions. When we used it
to evaluate candidates, we'd manually review the code for candidates that
scored somewhere in the middle of the range. Depending on what their
submission looked like, we'd decide whether or not to proceed with them.
(HackerRank lets you see each version of the code attempted by the user, in
addition to the final solution submitted.) We actually found it particularly
efficient at finding good candidates; there was a very high correlation
between interview performance and HackerRank score. If properly configured,
HackerRank makes it easier to identify good candidates, which is (IMO) a good
thing for everyone. For companies, it means that they spend less time
interviewing bad candidates, and for candidates themselves, it means that they
might be able to get their foot in the door somewhere where they'd usually get
blocked by the "resume scanner" filter (since the company isn't risking
engineer time/productivity to send out a simple HackerRank evaluation).

That said, HackerRank isn't perfect. My biggest complaint is the lack of
feedback for some failure modes, most notably segfaults and failing non-sample
testcases. For segfaults, it simply returns "segmentation fault" and you're
expected to be able to find the problem (a similar tool I've seen,
coderpad.io, dumps a stack trace). In some algorithmic questions, non-sample
testcases include data that is vastly more voluminous than the sample data
(which is intended to catch non-optimal implementations of the algorithm), but
this isn't obvious at all. It would be nice if the non-sample test cases had
titles (e.g. "extremely large input" or "edge case") so you could theorize
about why yours failed.

People who have experience using HackerRank have a definite edge over
candidates who have never used it before. If you are planning on taking a
HackerRank test for a position, I would recommend trying some open questions
on their site first. I also recommend having a local text editor, compiler,
and debugger ready in case you hit a segfault that isn't immediately obvious.
If your solution fails with "Terminated due to timeout", or your code works on
all the sample cases but fails/crashes on the hidden cases, then your
algorithm is likely not efficient enough (in the timeout case, look for ways
to speed it up; in the 'mystery crash' case, look for ways to reduce memory
usage). Lastly, if you have extra time after completing a HackerRank test, I
recommend making sure your code is as clean as possible and is well-documented
(but not over-documented), in case they decide to manually review it.

~~~
anilgulecha
(I work at Hackerrank) A couple points:

* the stack trace -- We avoid showing stacktraces to candidates as it has the potential of leaking information (a well created error message/stack can leak test case data).

We can and should show the trace (for languages that provide it) to reviewers.
I've added an item to our internal tracker.

* Familiarity of environment: Most (if not all) of the challenges companies created by companies link to a sample test (hackerrank.com/tests/sample). We notice many but not all candidates take this before the actual challenge. Perhaps we should highlight this more.

~~~
ninadmhatre
I could not attempt 1 challenge out of 4 because i didnt get the question, and
needed more information maybe another sample data? I was struggling for 30+
mins to understand the question but couldn't get any clues. I could not
complete last challenge because of not understanding the question/pattern of
question (maybe its my fault but a hint or 2 would have helped) rather than my
coding ability. Maybe you can (as you work there) add 1 or 2 (optional) hints
& make at least 2 samples of test data for tough questions. Anyways, it was a
different and nice experience!

