"Previously, he was a failed investment banker." AFTER EDIT: and remember, he introduced the word "loser" into the thread by how he chose the title for his blog post.) Maybe he'll be a winner in the future, and I wish him well, but I wouldn't advise following his hiring procedures. He has hiring authority at a business corporation, but he hasn't done even elementary research on company hiring procedures. He writes, "But then it was time for him to interview with me. I didn’t ask him very many questions about sales, advertising operations, invoicing, collections, or any of the handful of other tactical skills we wanted. I just grilled him on the bottom fourth of his resume — you know, the one about hobbies and college."
And he thinks that by doing that he is identifying winners, even as he is proud of his ignorance of about a century of research on company hiring procedures. There are many discussions here on HN about company hiring procedures. From participants in earlier discussions I have learned about many useful references on the subject, which I have gathered here in a FAQ file. The review article by Frank L. Schmidt and John E. Hunter, "The Validity and Utility of Selection Models in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings," Psychological Bulletin, Vol. 124, No. 2, 262-274
sums up, current to 1998, a meta-analysis of much of the HUGE peer-reviewed professional literature on the industrial and organizational psychology devoted to business hiring procedures. There are many kinds of hiring criteria, such as in-person interviews, telephone interviews, resume reviews for job experience, checks for academic credentials, personality tests, and so on. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.
EXECUTIVE SUMMARY: If you are hiring for any kind of job in the United States, prefer a work-sample test as your hiring procedure. If you are hiring in most other parts of the world, use a work-sample test in combination with a general mental ability test.
The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well. One is a general mental ability (GMA) test (an IQ-like test, such as the Wonderlic personnel screening test). Another is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. (But the calculated validity of each of the two best kinds of procedures, standing alone, is only 0.54 for work sample tests and 0.51 for general mental ability tests.) Each of these kinds of tests has about the same validity in screening applicants for jobs, with the general mental ability test better predicting success for applicants who will be trained into a new job. Neither is perfect (both miss some good performers on the job, and select some bad performers on the job), but both are better than any other single-factor hiring procedure that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.
Because of a Supreme Court decision in the United States (the decision does not apply in other countries, which have different statutes about employment), it is legally risky to give job applicants general mental ability tests such as a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of hiring procedures. The Griggs v. Duke Power, 401 U.S. 424 (1971) case
interpreted a federal statute about employment discrimination and held that a general intelligence test used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring procedure had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Note that a brain-teaser-type test used in a hiring procedure could be challenged as illegal if it can be shown to have disparate impact on some job applicants. A company defending a brain-teaser test for hiring would have to defend it by showing it is supported by a validation study demonstrating that the test is related to successful performance on the job. Such validation studies can be quite expensive. (Companies outside the United States are regulated by different laws. One other big difference between the United States and other countries is the relative ease with which workers may be fired in the United States, allowing companies to correct hiring mistakes by terminating the employment of the workers they hired mistakenly. The more legal protections a worker has from being fired, the more reluctant companies will be about hiring in the first place.)
The social background to the legal environment in the United States is explained in many books about hiring procedures
Previous discussion on HN pointed out that the Schmidt & Hunter (1998) article showed that multi-factor procedures work better than single-factor procedures, a summary of that article we can find in the current professional literature, for example "Reasons for being selective when choosing personnel selection procedures" (2010) by Cornelius J. König, Ute-Christine Klehe, Matthias Berchtold, and Martin Kleinmann:
"Choosing personnel selection procedures could be so simple: Grab your copy of Schmidt and Hunter (1998) and read their Table 1 (again). This should remind you to use a general mental ability (GMA) test in combination with an integrity test, a structured interview, a work sample test, and/or a conscientiousness measure."
But the 2010 article notes, looking at actual practice of companies around the world, "However, this idea does not seem to capture what is actually happening in organizations, as practitioners worldwide often use procedures with low predictive validity and regularly ignore procedures that are more valid (e.g., Di Milia, 2004; Lievens & De Paepe, 2004; Ryan, McFarland, Baron, & Page, 1999; Scholarios & Lockyer, 1999; Schuler, Hell, Trapmann, Schaar, & Boramir, 2007; Taylor, Keelty, & McDonnell, 2002). For example, the highly valid work sample tests are hardly used in the US, and the potentially rather useless procedure of graphology (Dean, 1992; Neter & Ben-Shakhar, 1989) is applied somewhere between occasionally and often in France (Ryan et al., 1999). In Germany, the use of GMA tests is reported to be low and to be decreasing (i.e., only 30% of the companies surveyed by Schuler et al., 2007, now use them)."
Integrity tests have limited validity standing alone, but appear to have significant incremental validity when added to a general mental ability test or work-sample test.
I've watched your hiring FAQ evolve through a number of threads here on HN, and I'd love to see it wikified. Even if you don't open it up for general editing, it'd be nice to be able to bookmark it, check it out from time to time, view the revision history, and so on. (Of course, that doesn't necessarily mean an actual wiki. Could be Google Docs, a GitHub gist, etc.)
So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.
Sure, but I doubt that advice would make a difference in the author's tale. In the opening anecdote, it seems that the candidate would easily pass such a test if one were present but that he was rejected because he wasn't good at conversation or showing any passion for his hobbies.
Regardless of how effective (or not) it is as a recruiting tactic, I can muster a little sympathy for the author's position as I would find it quite a dilemma whether to give the thumbs up to someone who couldn't engage in an interesting conversation during interview.
> "It's fine to disagree, but it's a pretty lame attack him based on a selected portion of his bio."
As opposed to attacking an entire blanket group of people based on their ability to answer personal questions in an interview?
tokenadult didn't start by calling people "losers".
It's lame when know-it-alls write blog posts about how their unscientific, unverified approach to some task ought to be adopted by everyone. It's even more lame when successful people ascribe their successes to such unscientific hoo-hah (even though we all have them), proceed to spoon feed them to everyone within eye-shot, and point at people and call them losers on arbitrary criteria.
The world is full of successful jerks - the jury is still out on whether your boss ever really had an effective hiring system.
That is what the author is espousing - judging people on a selected part of their bio, while disregarding any success they've had in the actual work they do.
Sure, we could replace winners and losers with some term like "A players" and "B players." But even judging past success has its flaws; I feel that society as a whole tends to put success, or "winning", on a pedestal. How many great minds have we benefited from who had significant flaws in their time, or went unrecognized?
People are probably a lot more complicated than we think. We try to sort them out into categories as best we can, but we should remember that our understanding of another person's value is limited at best. Calling out people as "losers" because they don't fit your purpose is immature.
Winner and Loser have personal traits attached to them. Loser almost implies that the individual had some problems personally and not in the way he executed the task.
Success and Failure imply that the individual did task X, Y , Z to achieve a result S or not achieve result S. It has more to do with an individuals execution at the instance.
Loser almost implies that the person is doomed and some how naturally incapable.
Do you happen to know if the author went to college?
He seems to have this cute idea that you pay $120k to a institution of higher learning, then you dedicate all of your time to academics, then you emerge with a well-refined body of knowledge in your particular field. I thought this is how it worked too, before I attended a college.
Maybe his point is that you shouldn't be taken seriously if you pay $120k and spend 4 years doing something where you're not deeply intellectually invested, but it's hard to have the confidence at 17 to be the contrarian that doesn't attend college.
Yes. But then, we use a gamified version of a work-sample test, so I would know that despite the fact that the candidate doesn't really care about the difference between Python and Ruby, he can exploit blind SQL injection and reverse engineer a network protocol. Which is good, because the latter ability is predictive of success, and the former is predictive of lots of stupid unproductive arguments.
This would be a nitpicking debate except that it gets to the heart of what makes the interview strategy in this article so dumb. There are lots of great programmers --- maybe some of the best --- who don't care all that much about language design or what language they'll be working in. There are, more importantly, a WHOLE MESS of programmers who can DEFINITELY talk your ears off about what the best language is... and then fail fizbuzz.
It depends if I'm hiring him to program or repair cars. If someone left college with a CS degree and a realization that he never wanted to write a line code ever again and got an apprenticeship as a welder instead, I wouldn't hold it against him if 6 years later he'd forgotten most of what he knew about programming.
So this candidate partied his way through college and bluffed his way to a degree, what difference does that make if you're not hiring him to do anything related to his degree? I thought the prevailing notion here was that most of the stuff taught at college was largely pointless when compared to the hard-won knowledge gained in the trenches of the professional world.
I would rather hire a CS grad who couldn't answer the question, "What's your favorite programming language?" than someone who could. A decent CS grad wouldn't have a preference, there are different languages for different purposes depending on what you are trying to accomplish. There is no one language fits all language. I'd prefer a CS grad who has a vested interest and appreciation of multiple languages, wouldn't you? Why hire someone with a raging hard on for Java when you can hire someone with an appreciation for Java, Python and C++?
I don't really know what my favorite programming language is, i'm currently most efficient in perl and java, but that doesn't mean that they're my favorites. I've not found "the language" yet.
Ignoring that he has built a successful company (while needing to hire a ton of people to accomplish it) seems very selective to me.
Also saying that "Maybe he'll be a winner in the future, and I wish him well" makes me think he completely missed that, or perhaps has never heard of Bleacher Report. The intersection of tech people and sports people does not seem very large.
