
Why HackerRank and other coding tests are ageist - forrestbrazeal
http://news.efinancialcareers.com/uk-en/287595/hackerrank-tests-banking
======
carlmr
<<Most damningly, he says the questions posed by HackerRank and similar tests
aren’t necessarily relevant to the work actually done in finance technology
roles: “They’re the sort of problems you’re posed at university. – There’s a
focus on data structures and algorithms, but this is less relevant when your
success as a developer is in building complex enterprise systems.”>>

Exactly on point. Also one gripe I have with hackerrank (I did some of the
challenges to brush up on some language knowledge), is that a lot of the
questions are poorly designed and from obvious amateurs.

I still think that neither whiteboarding nor hackerrank selects for good
developers. In the end you get people that studied hard to solve a certain
class of problem that you'll never need in the real world. It's wasting
candidates time, and it still shows nothing, except that your candidates know
something unrelated to the job.

~~~
jordanpg
Agree, but what is the alternative from the standpoint of the interviewer?
These types of challenges are simple to state, self-contained, easy to
timebox.

For the interviewer for a say intermediate-level enterprise software backend
position who needs to quickly sift through 50 applicants, what is the better
option by way of code challenges?

~~~
pizzaman09
Credentialing. Seriously. Have a Bar Association for developers. When anyone
can do a few weeks of Googling and claim to be a software developer, you're
going to get mostly crap candidates. Offload that process to a formal body,
like lawyers, doctors, and engineers do. You can hire unlicensed developers,
but you do so at your own risk. Then once you're talking to someone licensed
in an interview, you can focus on just getting to know them and talking about
their past work.

Edit: wow, I've clearly touched a nerve. Good thing your doctor is licensed to
prescribe something for it instead of a rockstar named Chad who read about
MongoDB once but can basically build Uber by this point.

~~~
Joboman555
This is maligned and dangerous. It has been shown that credentialing hurts the
marketplace and consumers. Having a credential authority would not cause there
to be a skill bar, it would just require all of us to pay money to a worthless
authority.

~~~
wolfgke
> it would just require all of us to pay money to a worthless authority.

Today these authorities are universities.

~~~
lmm
HackerRank-style test are one of the few ways into a good career for lower-
class people who have the skills but can't afford the credentials.

------
xtracto
We faced exactly this problem at the current company I work for. When the
company was just starting, we had the great idea to use interview questions
"like the ones they make in SV", thus used algorithmics-heavy problems when
interviewing candidates.

Fast forward 4 years, and I realized that the dev team we have is basically a
group of just-graduated Engineers or people with at most 1 year of experience.

My "eye opener" moment was when I realized that by asking for difficult
algorithms (tree and graph traversal, subsequences, etc) the following was
happening: The people that know the most about how to solve these types of
problems are students, or people that had graduated recently. Not only that,
but this groups is a very reduced sub-group from the recent graduates group,
because these people dedicate their Uni free time to solve these problems and
get in ACM, IOI competitions. Thus, they don't play with different
technologies, they oftentimes don't feel confortable with shell, and finally,
the type of systems they develop are optimized to be "used and thrown away".

~~~
jarsin
In my network many people that I would classify as bad to some of the worst
developers I have ever worked with ended up at Google. They are masters of
this type of programming problem solving but could not build a straight
forward, maintainable system to save their lives.

------
bsenftner
I'm in my early 50's. I've been getting paid well writing code since 15, and
have been employed at (arguably) the top of the industry in my fields. Nobody
at my level takes tests; none of us would pass them as they deal with utter
bullshit. Work for a few decades, at the highest levels of the intersection of
business and technology, and you'll find complex and/or sophisticated
algorithms often get removed due to support difficulties. Sophistication
occurs within the core of the product, which needs to embody cutting edge
features and performance, so unless you are the product's core architect,
sophisticated algorithms are taboo. The vast majority of software is user
interface, import/export logic and other aspects that are not the core of a
product. Those non-core features and facilities need to be nimble and simple
because they are transitory, and operated by juniors. Frankly, the world where
HackerStank is valid is a type of software company that is best avoided, as
they treat you 100% like a cog, and not a human.

~~~
expertentipp
If a person in the workplace is asked to implement quick search or some
sophisticated algorithms on linked lists or binary trees there are only two
possibilities: this is a low level and high importance project for
NASA/Intel/Lockheed Martin, or one is being teased.

------
lozzo
I am in my 40s and have a relatively safe job and not looking for a change. A
friend of mine told me to apply for a certain company and I did. They asked me
to free an hour one evening and they put me through hackerrank. And I failed.
I failed big time.

My take away: Surely we all need to revise if we seriously want to apply for a
job yet it did feel like the screening was asking me to see how good I am at
resolving small problems quickly.

Yet, my 25 years of experience told me that we don't need to solve problems
quickly we need to solve them rightly. and that takes "mulling over"

~~~
codyb
I'm only 27 and I suck at these tests. This year alone I've run the gamut
working frontend, backend (Spring), services (clojure), elasticsearch, and a
bit of devops.

I tried one last night after a nine hour work day in JavaScript and it kicked
my butt. First I was thinking solution X but eventually realized Y was
simpler, so then I've wasted a bunch of time because the pressures on coding
quickly instead of "mulling" over as you mentioned.

It was similar with a Ruby one which I prepared for by running through the
Ruby Koans. Lots of time spent looking up syntax, and the vast majority of the
time I'm not starting entire repos from scratch, so a lot of simple but not
oft repeated things can trick you up to.

And, to be quite frank, when I'm at home, and I want to program, I want to
work on my over engineered, terribly underdeveloped personal website where I
get to do things I don't normally do, putz around with erlang and postgres, do
the devops and all that other stuff.

Ahh the afflictions of the affluent!

------
jondubois
HackerRank tests are very effective for companies in that they capture
candidates who:

\- Have great problem solving ability.

\- Low emotional intelligence or self-esteem (don't realize that the kinds of
companies that make you do these tests are exploitative and will treat you
like livestock).

\- Don't have specific ambitions and don't place much value on their own time
(prepared to spend tons of time studying/practicing for the tests).

~~~
jarsin
The question is though how the hell do we find companies that don't do these
test? It seems everyone and their mom are doing it now days, even immigration
officials :)

~~~
charlieegan3
There's this list: [https://github.com/poteto/hiring-without-
whiteboards](https://github.com/poteto/hiring-without-whiteboards)

------
yodsanklai
What I like about these tests, is that everyone can have a shot. You can't
forge a university degree you don't have, or a 5-years work experience, but
you can take a couple of months and work hard to practice your algorithmic
skills. As far as ageism is concerned, I prefer to have the opportunity to
show my skills through technical tests rather than seeing my application
ignored.

~~~
pc86
> _you can take a couple of months and work hard to practice your algorithmic
> skills_

Unless you have kids, or a demanding job, or personal obligations, or you
don't want to make learning sorting algorithms for a job you'll never need
sorting algorithms a full time job on top of your current full time job.

It's just pre-selecting for a group you're in as opposed to a group you're not
(the royal you :)).

------
Majestic121
"For experienced programmers like Adler, the new tests aren’t insurmountable –
all that’s required is an injection of time to learn the new Hackerrank
tricks."

Well, yeah. I do that too and I've been out of university for 5 years.

You have to prepare for interviews, and to resharpen your skills when you're
changing jobs, it's the same whether you're 28 or 48.

I understand finding time is harder when you have a family and all, but it
should not take much more than a few hours to get back to a decent skill
level, and then you get to the interview stage where you can really stand out
with your actual real world experience.

~~~
andrewvc
If you're a successful working programmer you shouldn't need to study to pass
your next interview only to forget it all again after a couple more years on
the job.

That is why these tests are bogus. If they measured useful skills they
wouldn't require any study.

I've been working with a lot of graph algorithms lately. I've already
forgotten several I learned a couple months ago. That is OK.

The more interesting question for interviewees is where they would find the
answer rather than whether they can implement algorithm X from memory.

If a candidate says "I'd use A* and maybe compare it to similar algorithms to
solve the problem" that's enough. There's no reason they need that permanently
committed to memory. Almost no one writes A* twice a year.

~~~
wolfgke
> If they measured useful skills they wouldn't require any study.

I would be careful with this kind of statements. It can also show that most
programming jobs don't actually require a sophisticated level of knowledge.

~~~
joncrocks
I think this is unfair.

As a crude comparison, it would be like a mathematician being asked to recite
his times tables. Something he could probably do, given a bit of time and
practice, but so outside the normal day-to-day stuff they normally do that it
doesn't tell you much other than they spent time preparing. Which is fine if
that's what you're looking to measure.

I think the point that the grandparent is trying to make is that if you take a
group of developers with a good amount of experience and who everyone agrees
are skilled in their field and administer a test with no warning/preparation,
and none of them score very well, are they suddenly not very good developers
or is the test not a very good predictor of whether someone is a skilled
developer?

------
gberger
These are all good points, but how does the author reach the conclusion that
the coding tests are ageist? Ageism was only mentioned in the headline.

~~~
olympus
When he says the tests favor people that just graduated from school. Those
that have been in the workforce for a while (older people) are focused on
"real world" problems and not the academic problems seen on HackerRank. Any
system that favors young people over old people is ageist.

I'm not saying I agree with the argument (old people have been complaining
about young people for a very long time), but the logic is easy to follow.

~~~
Spoom
Bigger software companies also typically require "experienced" candidates
(read: not new grads) to pass a systems design interview, which is very
subjective and typically fairly difficult. This essentially makes the bar for
new grads lower.

The argument is that this is because they expect experienced developers to be
able to enter at a higher level, but the cynical side of me thinks that it's
because they really _prefer_ new grads, and this tips the scales in their
favor.

Remember, Zuckerberg said, "young people are just smarter". I fear that
attitude is pervasive, but hidden under the surface; they're not going to just
come out and say it, as that would be illegal.

~~~
JohnTHaller
You can be a new grad and have experience. I made a point to do work in
college that was appropriate for my work field when I graduated. Appropriate
summer and part time jobs helped quite a bit as well. Having a bunch of
appropriate jobs and projects under your belt (especially a few apps in app
stores and code up on github) goes a very long way.

~~~
Spoom
Sure, I wasn't arguing that they're mutually exclusive. I was referring to the
way that e.g. Google would refer to "experienced" or "industry" candidates.

------
fortythirteen
In my experience, any whiteboard test of generic algorithms is more of a
window into how the lead developer thinks about the people he manages, than an
actual test of programming ability. It almost always denotes a fixed, "I know
best" mindset. It's usually a interview killer for me.

And it's just a poor way to assess skill. Learn about modulus once and you'll
pass every other wizbang somebody else thinks they're the first to give you.

You'll always learn more about a candidate's ability by _just having a
conversation about programming_ with them. Anyone worth their salt will not
only be able to colloquially talk about their skillset, they'll enjoy it.

------
magmasystems
Hello all,

I am a bit late to this discussion, as Sarah Butcher (the author of the
article) just informed me about this thread. This is Marc Adler, the person
that was talked about in this article. It's great to see the discussion around
Hackerank-type coding tests and the hiring process. There are probably some
clarifications that need to accompany the article.

First, when I was interviewed for the article, there was no mention of the
word "ageism". As something in this thread surmised, it was probably inserted
into the title for a bit of "shock value". The article was supposed to be
about the correlation of Hackrank tests and the hiring of experienced people
who have been coding successfully for a number of years on "real-world"
problems. Many of you in this thread fall into that category.

Second, I have about 30 years of experience in the industry. I started out in
the mid-1980s as a Windows developer at Goldman Sachs, writing equities
trading systems, went on to form my own software company (which I had for ten
years and concentrated on programming tools), and then continued as a
developer/architect for more Wall Street companies. Among some of my roles
were Chief Architect of Citigroup's equities division, and Chief Architect of
MetLife (a global insurance company). I am currently CA at another big company
that is not on Wall Street. All of this time, I have been coding. Recently, I
developed an Uber clone in Scala/Akka/Play.

I am not completely against Hackerank-type tests for hiring junior devs or
devs right out of college. But what I am against is the use of these coding
tests to hire people who have 10, 20, or even 30 years of experience. When was
the last time that you used dynamic programming algorithms in your job? When
was the last time that you seriously thought about Big O? In my experience, a
lot of apps use linked lists and hash tables. If you want a nice sorting
algorithm, there is one that comes with one of Microsoft's C# assemblies. What
is more critical is knowing how to find information and how to apply it in
your day to day job. And that's the kind of developer I want to hire ... one
who is resourceful and productive.

-marc

------
lt
Hackerrank is a really good tool to separate the incapable from the capable,
but not a very good one to separate the good from the best (or even the
average from the good).

It's impressive the number of supposedly senior candidates that can't follow
simple instructions from the platform or write a couple of lines of code in
their language of choice to sort the words in a string or something similar.

I fully agree that multiple complex algorithm or puzzle questions are bad -
require a longer time investiment that most candidates should be willing to
devote to such process, are distanced to the reality of most programming tasks
and favor those who enjoy and practice programming puzzles.

------
erikbern
I interview 20-30 people per week and send most of them Hackerrank tests. I'm
a big proponent of vetting that people can write code, but 99% of the
questions on HR are waaaay too "algorithmic". Had to go through almost all of
them to find a few tests that are not about graph algos or dynamic programming
– finally found a question that was based on regular expressions and a few
other ones that I think are a bit more representative of real world
challenges.

I really wish Hackerrank could add more problems or let employers add their
own question. Are there any good alternatives to HR?

~~~
joncrocks
We (the company I work for) use Codility for some of our roles. I think you
can add your own tasks/questions.

~~~
Spoom
Big fan of Codility. They do have a lot of algorithmic questions as well, but
they also have easy ones (which we use), and we tested them on our own staff
prior to deploying (all then-current staff got perfect or nearly perfect). I
would only use medium or above if the question tested something that the job
absolutely needed in a candidate.

They also allow candidates to add their own test cases easily, which, for the
ones that do, tends to be a very strong signal in their favor (they also tend
to do better).

~~~
AstralStorm
Big NOT fan of algorithmic question that can be summed up in one sentence
taking a full page of text description.

In most of these coding test the algorithm is easy, but figuring out the catch
in the description is harder than actual implementation (unless you're
insanely limited and have to reimplement basics from scratch)

------
Xoros
As I understand that having a first selection maybe a good idea to sort out if
a person can program or not, it amazed me that that sort recruiters seems to
not read the applicant resume.

I do relate with the article. For experienced developers, it's probably not
the best way to recruit. Ask something related to the job, not the degree.

------
davesque
This isn't surprising. I remember getting the strong impression when poking
around on Hackerrank (between the UX, question quality, etc.) that it was
designed by people who just graduated from school.

------
yorwba
> When you’ve got a history of having built a high frequency trading system
> for Citadel, Deutsche Bank, Morgan Stanley, you’ll know a lot about latency
> and order flow – but you might not know how to find a maximal duplicated
> substring within a palindrome on a HackerRank-type test.

I have only a vague idea of the requirements in financial technology, but
aren't streams of financial data central to it? And repeating substrings
correspond to pattern repetitions, so they might be important, especially the
longest ones.

I think if you can find the longest repeating pattern in a stream of data, you
should also be able to find maximal duplicated substrings in palindromes.

Maybe it's more about the way the problem is phrased rather than about the
difficulty of coming up with a solution?

------
notyourday
TL; DR summary: programmers... err.. software engineers discover that
fundamentally most of the companies consider them to be be totally
interchangeable, just like janitors are fundamentally interchangeable or
models walking down the catwalk during the NY Fashion Week are
interchangeable.

Since they did not save money for the twenty years that they have been paid
quite well, they dont like the prospect.

~~~
expertentipp
Outside of certain bubbles (i.e. in most of the world) they have never been
paid exceptionally - well, at most.

~~~
notyourday
Outside that bubble they were paid just as well compared to the going salaries
of other people living outside that bubble.

------
jordanpg
Are there any HackerRank-like apps out there that go beyond simple code
challenges?

~~~
Majestic121
You can contribute to open source projects for another kind of challenge

~~~
AstralStorm
Contributing meaningfully to a big OSS project is a way bigger time
investment. Not only you have to understand a lot about the project, you have
to actually work on open issues in it.

And contributions to tiny projects tend to get passed over anyway.

Moreover, certain hiring agreements would consider it breach of contract or
confidence.

Making your own Github or such portfolio faces the same problems.

