For a long time I worked for a company that used a coding test as the first stage of the selection process. We advised candidates not to spend more than about 90 minutes on it, although I'm sure spent more. There was also no time pressure: we had some people take weeks to return the completed test due to exams, holidays, family commitments, or whatever, and it was no big deal.
Anyway, this all seemed perfectly reasonable until I went for a job I'd been approached about by an agency with a similar test at the beginning of the application process. They'd asked me to build a simple calculator and said it should take no more than 2 hours.
No big deal, but it was the weekend before I had 2 hours free where I could do it due to other commitments. Pushy recruiter didn't understand this but, whatever[1]. Should have been a red flag straight away.
Anyway, I put together a working solution with comprehensive tests, send it off to the recruiter and on Monday hear... nothing. That's right: nothing. Gave it a few days, got in touch with the recruiter and still nothing. No, "sorry, it wasn't good enough." No feedback on what might have been wrong with it. Nothing.
I was absolutely incensed. The whole incident gave me a different perspective on pre-interview tests, to the point where I'm no longer really interested in doing them for "unknown" companies. I made an exception to work with an organisation where I had several friends, so was confident that even if I didn't pass I'd get some decent feedback on why.
The most significant effect, however, has been on my own approach to recruitment. At the moment we don't, and I have no plans to introduce, programming homework as a pre-screener. Rather we take a much more personalised approach. We're also in the situation where we get few enough applicants that we can afford to talk to all of them, although this won't scale and will have to change.
Still, it's hopefully uncontroversial that before you invest the 5-6 hours (across 2 people) required for a 2 hour technical interview (in our case an exercise where we work together on a problem), you do need to some sort of pre-screener to weed out the people who really can't code: https://blog.codinghorror.com/why-cant-programmers-program/. I wouldn't say this is 99 out of 100 programmers, nor anything like, but it's certainly the majority. We therefore do a half hour telescreen, which is a fairly informal chat about what the candidate has been working on, what we do, and then - of course - there's a very simple programming task at the end. This takes an investment of roughly 90 minutes on our side (again, 2 people). We're explain the format to candidates in the invitation, which gives them an opportunity to decide whether they want to go through with it or not.
Even if candidates don't pass, and it's usually on the technical task that they fall down, we give them feedback, including pointers to how they can improve their skill level.
(Apologies, this came out much longer than I intended.)
[1] There's some merit to the "well, do you want a new job or not?" mode of thought but not much. E.g., in my current role I advise candidates not to do our technical interview after a full day of work because they're not likely to perform at their best. Obviously not always possible, but it's about ensuring they set themselves up for success: if you dislike your current job enough to be looking for another one don't you want to make sure that when you get an interview you do everything you can to do well in that interview? This is obviously very different to expecting candidates to jump through hoops under time pressure just because "commitment".
Anyway, this all seemed perfectly reasonable until I went for a job I'd been approached about by an agency with a similar test at the beginning of the application process. They'd asked me to build a simple calculator and said it should take no more than 2 hours.
No big deal, but it was the weekend before I had 2 hours free where I could do it due to other commitments. Pushy recruiter didn't understand this but, whatever[1]. Should have been a red flag straight away.
Anyway, I put together a working solution with comprehensive tests, send it off to the recruiter and on Monday hear... nothing. That's right: nothing. Gave it a few days, got in touch with the recruiter and still nothing. No, "sorry, it wasn't good enough." No feedback on what might have been wrong with it. Nothing.
I was absolutely incensed. The whole incident gave me a different perspective on pre-interview tests, to the point where I'm no longer really interested in doing them for "unknown" companies. I made an exception to work with an organisation where I had several friends, so was confident that even if I didn't pass I'd get some decent feedback on why.
The most significant effect, however, has been on my own approach to recruitment. At the moment we don't, and I have no plans to introduce, programming homework as a pre-screener. Rather we take a much more personalised approach. We're also in the situation where we get few enough applicants that we can afford to talk to all of them, although this won't scale and will have to change.
Still, it's hopefully uncontroversial that before you invest the 5-6 hours (across 2 people) required for a 2 hour technical interview (in our case an exercise where we work together on a problem), you do need to some sort of pre-screener to weed out the people who really can't code: https://blog.codinghorror.com/why-cant-programmers-program/. I wouldn't say this is 99 out of 100 programmers, nor anything like, but it's certainly the majority. We therefore do a half hour telescreen, which is a fairly informal chat about what the candidate has been working on, what we do, and then - of course - there's a very simple programming task at the end. This takes an investment of roughly 90 minutes on our side (again, 2 people). We're explain the format to candidates in the invitation, which gives them an opportunity to decide whether they want to go through with it or not.
Even if candidates don't pass, and it's usually on the technical task that they fall down, we give them feedback, including pointers to how they can improve their skill level.
(Apologies, this came out much longer than I intended.)
[1] There's some merit to the "well, do you want a new job or not?" mode of thought but not much. E.g., in my current role I advise candidates not to do our technical interview after a full day of work because they're not likely to perform at their best. Obviously not always possible, but it's about ensuring they set themselves up for success: if you dislike your current job enough to be looking for another one don't you want to make sure that when you get an interview you do everything you can to do well in that interview? This is obviously very different to expecting candidates to jump through hoops under time pressure just because "commitment".