Any time a company asks me to do a HackerRank test, I immediately reject them. It's just not possible to have a quality engineering team and to also believe the results of HackerRank tests correspond to on-the-job success.
HackerRank is an attempt to commoditize software labor (literally reducing evaluation of your labor fitness to standardized examples). It communicates immediately that creativity is valued less than standardization, and that your uniformity and compliance are more valuable than your experiences.
It's also a way to position developers as lower-status employees -- you essentially have to capitulate to the judgement of higher-status employees. Even if you ace the code test, it puts you in a defensive position to justify yourself, which inherently reduces your negotiation power. If you submit something that is even slightly unconventional (even if it's provably just as or more accurate than conventional submissions), then your negotiation power is extremely damaged.
For example, think of the difference between actors who must audition and actors who are "offer only" -- they won't respond to your inquiry about hiring them unless you're prepared, based on their previous work, to make an offer already. If you ask Robert De Niro to audition, you'll get laughed out of the room. HackerRank is often like asking Robert De Niro not only to audition, but to do some kind of two-bit community improv class warm up exercise for his audition, then grilling him because he didn't enunciate clearly. Ridiculous.
Employers often say they want to "see how you think" -- when they say this, it's a good idea to run the other way. No one can grok some whiteboard code or some timed code test on standardized examples and draw any meaningful conclusions about "how you think" or how it will relate to job success. Someone who believes they can "see how you think" from narrow, time-constrained examples is going to be a terrible colleague or, worse, a very dysfunctional boss.
I think the trend of cultivating "full-stack" developers (instead of benefiting from specialization and separation of concerns) is the number one problem facing the software industry right now (and folks are largely in denial about it).
This nonsense with commodity interviews via HackerRank is the number two problem, followed closely by the prevalence of open-plan offices and the prevalence of Agile-like workflow management processes.
Using a coding test shouldn't be a red flag. They are an incredibly useful tool for weeding out the applicants who, quite frankly, don't know their ass from their keyboard.
I've done hiring at several companies over the years, and I can honestly say that the signal-to-noise ratio for programmers tends to be very low. Even eliminating the obviously unqualified resumes leaves us with dozens of supposedly 'qualified' developers. Further followup however in most cases (probably 60 to 80% of the time, depending on the seniority of the position) reveals that the applicant is all talk, and can't solve even the simplest of problems.
Eliminating that 60-80% of applicants is an issue. We could have, say, a brief phone interview with each applicant, trying to figure out which ones are garbage and which are good. But that would tie up actual employees for hours and hours, doing something that most of them would much rather not be doing. Instead, using a coding test to weed out the morons can work wonders. We set them all up with a simple problem set, and a day later it becomes very obvious which of the applicants are worth bringing in for a real interview.
Now don't get me wrong - if an employer rejects you because of a spelling mistake or judges you because they say you solved a problem 'wrong', then yeah, that employer likely sucks and you should be happy to have been passed over. But rejecting an employer simply because they asked you to do a test? The only one you're hurting is yourself.
The flip side of this is that most companies are "garbage" as you put it, and a lot of them simply do cargo cult imitations of the more popular companies. If a startup is saying they're going to disrupt the online vegan running shoes market and then tells me they have to weed out the "80% garbage applicants" with some parochial trivia about binary search trees that let's be honest none of us has given a shit about since we passed the exam on it in undergrad, then the code test absolutely is a big red flag to nope on out of there.
Since so many jobs are just shitty talent wasters, and you are just treated horribly, given poor equity terms, not paid what you're worth, etc., it unfortunately means that unless you have special knowledge that some particular job is non-shitty (like a trusted recommendation) then you're just better off erring on the side of nope.
If the candidates look like garbage, why are you screening them?
You can't always tell. You begin part of the process, then the company says how great their Agile teams are, or how "collaborative" their open-plan surveillance workspace is, or they invite you to do a HackerRank test, and only now do you know the company is shit and you pass.
It's the same problem you face with the 60-80% unqualified applicants. I get messages from head hunters, direct recruiter emails on Stack Overflow, traditional recruiting firm phone calls, as well as occasional job listings that I locate through a job search.
80% of these jobs are shitty and need to be weeded out, even when they have plausible-seeming job descriptions and acceptable GlassDoor reviews.
I'm screening them to eliminate the garbage? I'm not sure I understand your point here. We don't choose the applicants we want to apply, the applicants choose which companies to apply for.
The problems with companies that you mention above have nothing to do with the coding test, it's the company itself. Those are perfectly valid reasons not to continue the application process. But eliminating a company because they apply a coding test as a basic level of applicant screening still strikes me as an arbitrary move that does nothing but rule out perfectly valid job opportunities.
Most people don't get to choose who to interview. Your manager or your recruiter does not have enough data to eliminate the 60% unfit. But the first 40% are already eliminated because those candidates did not pass even the first phone interview with a recruiter (think Google hiring process).
My point is that candidates face that exact same problem when trying to weed out bad employers. That's why good developers reject the employers who try to use commodity tests like HackerRank. We do it for the same reason that the employer wants to use the commodity tests to weed out bad candidates. But somehow the employers don't understand they are just signalling how bad they are (apart from a very rare few companies).
HackerRank-like lazy, commodity evaluation is not remotely close to "put[ting] some effort in to find the right job." It's very critical that we don't allow people to make the false comparison between actual interview effort and daft HackerRank time-wasting.
In fact, doing the emotionally difficult task of rejecting an employer who tries the HackerRank commodity nonsense represents putting in more legitimate effort to find the right job and to judiciously choose to complete code evaluations for companies that aren't being lazy and wasting your time.
There's no need to be insincere or intentionally hurtful. Trust me, when you need a job and you've got a lot of life and family pressures building up, but you can clearly see how dysfunctional an employer is and how bad your life will be if you agree to work for them, it can be extremely hard to make the choice to do the right thing and reject them. It's even worse when you get to the stage of an actual job offer and an employer starts revealing how dysfunctional they are when they won't negotiate on basic features that are necessary for minimally acceptable worker health and quality of life.
It seems you really desire to deride me simply because you disagree with me. I don't take it personally, but I will say that the attitude you've displayed throughout these comments defending HackerRank-like evaluations is exactly the kind of attitude that would be indicative of a badly dysfunctional employer, and it's often exactly the kind of dysfunction that HackerRank requests are indicative of. Especially the parts where you try to turn it around and assert that a candidate standing up for minimally acceptable, reasonable treatment is equivalent (for you) to having a "bad attitude." It's quite alarming that you feel entitled to declare candidates as having "bad attitudes" for doing something that simply makes common sense from the point of view of avoiding employers who will waste their time. Contrary to suggesting that the candidate has a bad attitude, it highly suggests that the employer has a bad attitude, bordering on feeling like they are entitled to candidate labor, instead of privileged to have that labor, and somewhat whiny about it too. Definitely a bad signal when coming from someone involved in the hiring process.
Was the sarcasm a bit much? Yeah, probably. But really, everyone makes hard decisions when it comes to jobs. I just find it somewhat amusing that someone would reject a company entirely because they disagree with one single facet of a multi-step interview process.
While we're at it, I don't particularly appreciate many of the insinuations people have made about my team and my employers based simply on the fact that I see a role for automated coding tests in the interview process. But like you said, I don't take it personally either.
Coding tests are just one tool in an interviewing toolbox. Like any tool, they can be - and are - abused. You feel that any use of that tool is indicative of some unredeemable flaw in the company as a whole. I feel that the tool has a genuine role in the initial candidate screening process. We obviously disagree, and are spinning our tires trying to convince each other, so there's really no point in continuing the discussion.
> You feel that any use of that tool is indicative of some unredeemable flaw in the company as a whole.
This is a good summary of my position, except that it's not unredeemable. They could just stop using short, standardized, timed tests as a candidate evaluation, and instead they could acknowledge that there is no effective substitute for actually speaking with candidates, probing them about their experiences, and developing more nuanced understanding. Doing so would be time consuming and expensive ... that's life. Papering over the reality of the situation by pretending like automated, timed, standardized tests can measure the thing you need to measure won't make reality go away.
If a company is verifiably an excellent place to work and has excellent technology culture, and this can be verified ahead of time, then they do have the negotiation power to respectfully require completing code trivia (although most of the firms that actually are excellent don't do it this way even though they could).
Firms that are question marks to a candidate prior to some kind of phone interview to assess the fit, experience, and the nature of the role have no business trying to cheaply avoid the required costs of candidate evaluation. By trying to be cheap about it, they send a bad signal (and also generally don't succeed in getting the candidate pool they want to get).
Short, standardized, timed tests have no place in professional software hiring. Literally none. A company that uses such tests definitely raises red flags. It may be a sign of unredeemable dysfunction in the company, it may be some misguided HR initiative, it may be a totally fine place to work. The candidate can't tell and it's seriously not in their interest to waste effort on whatever the test is going to be.
There are just too many bad jobs ... the better decision rule is to always reject and if you end up rejecting an otherwise good job that somehow ended up using short, timed, standardized testing, oh well. The loss function is not symnmetric. Ending up in a dysfunctional job just because you felt good about acing their code test is a far worse outcome than rejecting an otherwise good job and being overly selective about where to work.
Yup, I still genuinely disagree with most of your points. Just not seeing the connection between using an interviewing tool and being a dysfunctional company. But like I said, spinning tires, etc. All the best to you going forward!
I think the problem is that you continue referring to it as an interview tool just because it is a thing that can be used for interviewing.
Making candidates stand on one leg and recite the alphabet backwards would also be an "interviewing tool" but it's not legitimate, just as short, timed, standardized coding tests are also not legitimate. Playing semantic games about whether it's an "interview tool" is not worthwhile. The question has nothing to do with whether it's logically possible to be used as part of an interview. The point is that it cannot provide the kinds of evidence that people claim it provides, and so continued usage of it for interviews can only be explained by other reasons, which is when it begins to be evidence of dysfunction.
That's not 'the problem'. That's the disagreement. You claim it's not a legitimate tool. I say that it is. We obviously disagree and this is going absolutely nowhere.
> Just not seeing the connection between using an interviewing tool and being a dysfunctional company.
This is problematic because you're not representing my position. It's somewhat of a straw-man. You're saying that I'm saying that the use of some legit interview tool spells dysfunction.
But I'm not saying that. I'm saying that the use of an illegitimate interview tool (timed, standardized tests) is a sign of dysfunction.
Whether or not you see a connection between using an interview tool and dysfunction is irrelevant, because we would both agree that there's a connection between using an illegitimate tool and dysfunction.
So the disagreement happens at least one layer back, at the point where I claim that timed, standardized tests are not actually useful for determining who will be good at a job. The disagreement, and the whole discussion, has nothing whatsoever to do with "just using an interview tool."
I think it's very important to be pedantic about this, because in a lot of cases you keep referring to the timed tests as an interview tool, and speaking about them that way fails to acknowledge that the tests are not capable of producing the kinds of evaluation output necessary for actually determining who would be good at a job.
For example, a great engineer with loads of experience and glowing letters of recommendation about how effective they have been in previous roles might just be a very anxious timed test taker and tend to do badly on timed tests for reasons mostly of anxiety. Since the test is artificial and has no relationship to the actual work they would do if hired, rejecting them based on a failure on the test is a poor business decision.
It doesn't even have to be as extreme as an anxiety issue with test taking. A person could just simply disprefer working in a constrained, timed environment like a browser-based IDE with a literal clock ticking down. They might be a great developer, yet cannot even write FizzBuzz in that situation because it is so utterly and unreasonably alien compared to the manner in which they would do work in a real job.
The tests just simply are not legitimate ways of measuring programming ability. They do not control for all of the bespoke ways a person can appear bad via a timed test, yet still be good, nor do they account for all of the ways a person can appear good on a timed test, yet perform badly on higher-level thinking tasks, open-ended time management, or creative tasks like system design.
Very truly, the tests, and organizations like HackerRank, exist because it's a convenient fiction for HR and management, especially in companies that doesn't have very much of a technology staff to vet candidates. These places are happy to drink the HackerRank Kool-Aid because it gives them all kinds of plausible deniability about hiring the wrong people. It's very similar to the way that management consultants, far from actually functioning in any analytical capacity that benefits their client, really just exist to sort out internal power struggles through means of credential and prestige. It's very much a status-signaling sort of behavior. And it's even worse in the cases when a company adopts something like HackerRank just because some other, supposedly fancy, company adopted it.
Because of all of this and much more, it's just very critical to continue pedantically ensuring that no claim of the tests being valid, legitimate evaluation tools goes unchallenged.
And yet you are misrepresenting what I have claimed about coding tests. I have never once claimed it as useful for determining who would be GOOD at a job. I have claimed they are useful for filtering out applicants who are completely unable to do a job. This is an incredibly useful thing to do when swamped with dozens of similar applications for, say, a junior developer role. But again, this is just a first step. By no means should this test determine who gets hired, it merely helps narrow down the interview pool to something more manageable.
Is there a risk of falsely filtering good applicants? Yes. But that is the exception, not the rule.
Sigh. Why do I keep letting myself get sucked back into this? You see it as an illegitimate tool. I see it as legitimate. We disagree. I'm not going to agree with you, and you're not going to agree with me. So I'm done here. If you'd like to keep arguing by yourself, you go right ahead.
So, if someone fresh out of college who has time to spend lot of time on the sample interview questions he/she is better than someone with experience/expertise in a particular area?
Sure if the experienced person spends enough time practicing, can ace the interview as well. But he/she would rather spend time on interesting stuff than say, how to print a tree spirally and hundreds of other silly questions out there :)
Who said anything about grading results or comparing applicants? This should be a screening test, nothing more. Pass/fail. Any decent developer should be able to knock out decent responses to simple problems in very little time with no prep. The goal is simply to eliminate the ones who can't even do that.
Do you think printing a matrix in spiral order is a simple problem. If an experienced person who spent 10 yrs on file systems didn't get that in a 30mts period, would you eliminate him/her? And oh btw, you are supposed to talk through your thinking process as well in that 30 mts... while youre at it, don't forget that semicolons :)
Using a coding test as the initial barrier to even a phone screen is what I have a severe problem with.
I think coding tests absolutely have their place - after a 30m high-level skillcheck over the phone, let's ask for a brief 30-45m timed code challenge to rule out the applicants who were effective enough bullshitters to clear the phone screen.
But I too have seen this trend that GP calls out re: "auditions" to even make it into the actual audition process, and FWIW it has not hurt me one whit to spend the last few years refusing to complete a code challenge as a prerequisite to anything but an in-person technical interview.
The only way it would hurt you is if the company in question is one you know you want to work for - and I'd consider an up-front code challenge for a few big names out there - but in most cases I simply write back and let the recruiter know my policy.
It's not like they're going to stop sending you leads, if you're a good candidate.
But look at it from the company's point of view - why would they invest time in a 30-minute skill check interview for every applicant if they can just as easily run the code challenge up front and eliminate a large proportion of the unsuitable applicants?
The problem is that you can't "just as easily" run the code test. The code test is unreliable -- bad developers who overfit their knowledge to trivia make it through. The code test also causes the really good developers to simply reject you, so you don't even get a chance to hire them, or even find out who they are.
You're basically saying you want to lop off the bottom 60% of an imagined Normal distribution. Except you're really only lopping off some of the bottom 60%, and some fraction of that bottom 60% gets through, and you're paying the cost of also lopping off the top 5% who think you're a joke of an employer for being way too worried about the costs for you to more substantively evaluate those bottom 60% folks -- especially since you, as an employer, are probably in the bottom 60% of employers anyway, yet are cargo culting to try to act like you only hire the top 0.000001% or something.
You get what you pay for. If you go cheap on candidate evaluation (e.g. lazy commodity HackerRank), you get the McDonald's version of a developer, all while acting like you're being extremely selective.
Gonna have to disagree on that one. If some fraction of that bottom 60% gets through, that's perfectly fine - this is only the first step after all, and there are real interviews still to come.
And if this mystical top 5% can't spare the ten minutes to run through what is essentially an interviewing captcha - for a job they were obviously interested in enough to apply for in the first place - then how do I know they won't consider themselves too good to do their actual work if I do end up hiring then? I'm happy to hire the next 5% down the list of it means they have a good attitude and a willingness to do their job.
And what's the alternative? I waste my team's time setting up screening phone interviews for every halfway decent resume that comes down the pipe? With the number of applications that we get, we'd be losing days of time every time we look to hire, before we even got to the actual interviewing.
If I knew what your test was going in it wouldn't be an issue. Unfortunately, I don't know if I'm getting your 15-minute competency screen or, say, Virtu's 3-hour borderline impossible beating until I actually log in to take it. Since the latter seems to be more common than the former, it pushes me to reject you up-front unless I am desperate.
I understand the reasons why the company would choose to do it. My argument is that they are fixing the wrong problem - and to be honest, could likely be making it worse.
If so many unqualified candidates are making it to the phone screen (which should happen after they've been resume-sorted and google searched), it's an indicator that something at the leading edge of the candidate pipeline is broken.
Asking their potential candidates to make up for their failure is not only asinine but probably locks them into a cycle of mediocrity - highly-skilled engineers are getting more leads than they care to deal with already, they're not likely to take the time to do that upfront work unless there's a very compelling draw. You know who will? Untrained devs looking for a first job.
tl;dr - the candidates that would do unpaid work for the chance of an interview, and the highly-skilled, well-sought-after engineers I'd like to hire for my team, are likely two circles without any overlap unless I have some major draw working in my favor - like signing bonus, or being a Google.
Full-stack developers should be generalist in nature to make sure the issues that cross domains (front/backend, etc) are taken care of. Also, they can plug any gaps in the development that crop up from time to time (eg: someone on vacation, or next feature needs more back-end people). The downside is that they will not be spectacular or as quick at any one domain.
Basically, a full-stack developers specialization is to be flexible and have knowledge and experience in working the entire application stack. Another way of thinking about it is that they are like a general practitioner doctor/dentist, while they generally don't do surgery, they can refer you to the correct specialists who can and take care of any minor/common ailments. This is also why you don't want an entire team of them is that they don't go into depth in any domain.
I don't doubt that the current full-stack education path does a poor job of education on issues that specifically cross domains or when a task is better suited to a specialist.
The desire for this sort of flexibility is a mistake. It sounds good, but it's naive. It's sort of like saying you want a single class in your OO design to be "flexible" so you make the class into an undifferentiated mess. Having a monolith class that "does it all" is a red flag of bad design. It's truly no different when the layer of abstraction is human software developers being organized for complex business problems. Having "flexible" people whose job is a little bit of everything is a major red flag of a bad design (bad management) that took the lazy way out instead of the harder but unequivocally more successful route of actually meaningfully dividing the workload between clean interfaces and having employees mostly specialize. "Sometimes you just need somebody to work across every system" is just flat out wrong. If you find you need that, it means you've been doing things in a very wrong way and patching over it by commanding positions to be full-stack is only going to perpetuate the problem.
This analogy is correct, except ... you're not Robert De Niro, and neither is almost any programmer in the valley. Just like almost no actor is Robert De Niro, and most actors are lucky to even be able to audition for commercials.
Your company is not Robert De Niro either! And, to boot, you don't have to be way up at the level of a Robert De Niro to be an offer-only actor. Basically, anyone with a decent and verifiable acting record is offer only. It's actually pretty common. And some actors are offer-only for some kinds of work, such as voice-over work if that's what they're known for, but not for feature films.
Anyway, I think your reply totally misses the point. The companies who are so obsessed with needing to weed out the bottom 60% are themselves well in the bottom 60% of employers. What gives them the audacity to claim that certain developers are too low skilled, if the company itself is not any good?
A good developer is going to see that and immediately walk away. Unless the company is already verifiably a great place to work with great technical prowess, then their obsession with getting rid of "garbage" programmers makes them seem extremely dysfunctional.
(Just look at how people here on HN are speaking about other developers -- so now you're "garbage" if you're in the bottom 60%, even though that still places you well into the top 5% of humanity in terms of technological skills and quantitative thinking. It's absurd!)
HackerRank is an attempt to commoditize software labor (literally reducing evaluation of your labor fitness to standardized examples). It communicates immediately that creativity is valued less than standardization, and that your uniformity and compliance are more valuable than your experiences.
It's also a way to position developers as lower-status employees -- you essentially have to capitulate to the judgement of higher-status employees. Even if you ace the code test, it puts you in a defensive position to justify yourself, which inherently reduces your negotiation power. If you submit something that is even slightly unconventional (even if it's provably just as or more accurate than conventional submissions), then your negotiation power is extremely damaged.
For example, think of the difference between actors who must audition and actors who are "offer only" -- they won't respond to your inquiry about hiring them unless you're prepared, based on their previous work, to make an offer already. If you ask Robert De Niro to audition, you'll get laughed out of the room. HackerRank is often like asking Robert De Niro not only to audition, but to do some kind of two-bit community improv class warm up exercise for his audition, then grilling him because he didn't enunciate clearly. Ridiculous.
Employers often say they want to "see how you think" -- when they say this, it's a good idea to run the other way. No one can grok some whiteboard code or some timed code test on standardized examples and draw any meaningful conclusions about "how you think" or how it will relate to job success. Someone who believes they can "see how you think" from narrow, time-constrained examples is going to be a terrible colleague or, worse, a very dysfunctional boss.
I think the trend of cultivating "full-stack" developers (instead of benefiting from specialization and separation of concerns) is the number one problem facing the software industry right now (and folks are largely in denial about it).
This nonsense with commodity interviews via HackerRank is the number two problem, followed closely by the prevalence of open-plan offices and the prevalence of Agile-like workflow management processes.