A 1000 employee company I know almost never makes hiring mistakes and has almost no turnover. One of the first things they do in the process is on-line assessment testing through a 3rd party. It could be for a specific skill or general aptitude. Score less than the 95th percentile and they disqualify you. They pay up front to eliminate the posers so that they can focus on legitimate candidates.
I've considered doing that, but I always worry that if I put too much of a barrier up front, I'll lose the superstars who know they can get a job somewhere else with less effort.
Is this company a very famous or desirable place to work? Up-front screening could work very well in that situation. But for less well-known companies like mine, candidates are less willing to go through a lengthy process to interview with us (except the desperate ones, and we don't want those).
I don't know if I'm a 'superstar', though I don't particularly favor the term, but I'll offer my own view on the onerous hiring process.
I won't do it. That is to say, I won't do it for just any company. I try to determine if a company has their act together by interviewing them back. I don't mind (and expect) some degree of white board coding or something, but there's a level of nuttiness that isn't worth my time for a company that maybe would be good, maybe not. If it's 'just a job' I have other options.
You can get away with more onerous hiring barriers if you pay a better salary or are otherwise a more attractive candidate (top reputation, etc) for employment. If you really are getting much better hires, you should be able to pay better I suppose. I think paying for code samples, or contracting for a few hours of work, etc, are also more respectful ways to get work from a non-employee and signals to the candidate that you're serious and they're at the final step.
Losing the superstars usually isn't that much of a problem. There are other superstars out there, and most of the top programmers I know care very deeply about being able to work with other smart people and not having to put up with incompetence. A rigorous hiring process is a plus in their book.
Besides, given a population of superstars that want the easiest interview process possible, vs. a population of superstars that want to work with other superstars, which one would you rather work with?
I'd be more concerned with the test not measuring what you want it to measure. It's possible to test well and still not be able to code yourself out of a paper bag, and it's possible (though somewhat less likely) to code well and not test well.
