Hacker News new | comments | show | ask | jobs | submit login

>If you only get canned questions

>If they give you an IQ test or similar, run.

I ran across this study a few months ago that says IQ-like tests, structured interviews (canned questions), and work-sample evaluations were some of the few useful interview tools for selecting good employees [1].

[1]. http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%...




>I ran across this study a few months ago that says IQ-like tests, structured interviews (canned questions), and work-sample evaluations were some of the few useful interview tools for selecting good employees [1].

They're great for determining which employees will be productive

They're entirely terrible for figuring out which employees will be decent people to spend 9-5, 5 days a week with.

Realistically you need a bit of both


I've found that well-designed (i.e. open ended) standardised questions help normalise comparison between candidates. Especially helpful if you have many candidates in a pipeline and/or a pool of interviewers that compare notes.

Work samples are my #1 competence filter. They've also formed the basis of great interviews.

As for your second point, sometimes you want to hire a change agent. It takes skill and practice to distinguish toxic personalities from the merely disruptive. Again, standard questions designed from a perspective of empathy and experience have helped me.


Curious about OP's assertion on this as well. Only one I ever got was from a company widely regarded as one of the best to work for in Atlanta (I ended up not taking their offer so I can't say).

Unrelated to that -- one of my personal red flags is poor handling of the post interview follow up. If a company drags their feet, gives conflicting signals about next steps, etc., they probably are a mess.


Speaking as someone with a cognitive science background (and yes, probably too lazy to dig up specific references), I'll say that there exist a lot of poorly designed or poorly implemented tests of "intelligence". Even the concept of intelligence is poorly defined, and lots of different measures are just proxies. Even if the test is good and has some psychometric utility, often giving the test is a giant pain to do well.

Even if the test is good, it's probably calibrated on WEIRD [0] subjects, so is really only good as another kind of gatekeeper - you can probably use old-school IQ tests as an excuse to reduce diversity. People who aren't WEIRD may not do as well on these tests (even though they're just as "intelligent" (parenthetically bracketing the ill-defined term for a moment)), so you have your reason that you only hire white dudes. But that's a cynical extension of logic there.

Most people, when approaching a psychometric test like one of intelligence or personality make the first incorrect assumption that modern cognitive science can actually form a construct like "personality type" or "intelligence" that's stable across all cultures and norms, testable, and repeatable. That's just not really true, and by and large what makes it to public consumption is pseudo-science.

Diverting just a little bit here, but MBTI [1] is another good example of bad use of psychometrics in business. There's about as much science supporting Myers-Briggs tests as there is astrology, yet people still use MBTI for actual decision making. Types aren't shown to be stable and they're not well-clustered (meaning you can be a mix of introvert and extrovert, or show signs of both depending on the day, context, mood). Zodiac Sign and MBTI Type should only be used as pick-up lines in a bar. "Hey, babe, I'm an INTJ, so I'm not going to say anything else."

There are good uses of psychometrics, but they're rare enough that I take any reliance on any kind of psychometric during hiring as kind of a bad sign.

0: http://blogs.scientificamerican.com/primate-diaries/the-weir... 1: https://www.psychologytoday.com/blog/give-and-take/201309/go...


To be fair, I think that Raven's matrices probably get around this. If someone explains the process to you, you don't actually need to be able to read to do well on the test. https://en.wikipedia.org/wiki/Raven%27s_Progressive_Matrices

I do find it surprising that Raven's appears to be more susceptable to the Flynn effect (people keep getting better at IQ tests, for an unknown reason).

Completely agreed on the MBTI, but its surprisingly difficult to convince people that's its useful.

FWIW, I agree with most of what you said above, I was just pointing out a counter-example. (Psychologist/psychometrician here).


Yes, I agree that Raven's matrices are pretty good at measuring something in a way that's not super-reliant on language. I'm not inherently opposed to the notion of generalized intelligence as a construct either, I just think measuring it is pretty tricky. The Flynn effect is another great example - these instruments are nuanced, and it's important to really understand them in order to interpret their results.


Totally agree, I'm somewhat in sympathy with Shalizi's argument that g is just the result of iterated factor analysis. That being said, done well, these kinds of tests can provide useful data, especially if you're willing to randomly hire to calibrate performance (which no one ever is, sadly).


I find the "WEIRD" thing a bit tough to support, at least with regard to IQ. East Asian populations (Hong Kong, South Korea, and Japan right at the top) seem to consistently score higher on IQ tests than "WEIRD"os. Maybe the less pathetic and self-hating way to look at it would be "EIR" populations. Even then, it's hard to argue that spatial and linguistic reasoning are not universal values; even if you're uncomfortable that some populations have lower averages than others. I feel as though this must come from a lack of respect for the obvious personal value of Education, Industrialization, and Riches.


You're making two different arguments here and kind of tying them together. First: you're claiming that there actually is some kind of universal "spatial reasoning" skill. Usually people who claim that also believe that this skill is somewhat generic - it applies to lots of different contexts and transfers easily from (say) visualizing rotations in space to finding your way around a 3d environment and so on. There's actually a fair bit of research that questions this supposition (take a look at embodied cognition to challenge the theoretical model, or stereotype threat effects to challenge the assessment model).

Secondly, in this context, the assumption is that not only does this generalized attribute exist in humanity and we can measure it, but this attribute causes success in $job. That's a further assumption that I'd call into question. Sure, plenty of people who do well on $test do well on $job, but those could easily be covariate or epiphenomenological relationships rather than causal ones.

I don't doubt that for some values of $test, there's some discriminatory power predicting the presence of success, especially in more monocultural environments (affluent, often-white, often-male, often-young startups, to caricature). But I do believe that this kind of test would suffer from strong false negatives as many kinds of people who would otherwise be excellent programmers (or whatever other $job) are rejected for not doing well on $test.

That's why I worry about anyone who puts real stock in these kinds of tests - it is (generally) indicative of a kind of science blind spot in the user. People are critical and skeptical of all kinds of other scientific claims, but psychological measures tend to get a pass.

I imagine the fact that a lot of us tend to score well on IQ tests has a lot to contribute here. When I found out how many standard deviations above the mean my IQ score represented, I was pretty excited about how awesome I was. When I realized that the test was probably baloney, I had to figure out more interesting ways to shore up my self-esteem. Personally: I bake bread and fish for compliments on the quality of my sourdough.


I prefer stronger evidence than "I don't doubt" when positing an international raciosexual conspiracy. Especially if that conspiracy would presumably have heinous and far-reaching consequences.

I learned, as an adult, that I had taken an IQ test while young and had scored in the 99.97th percentile. I think that I got that not everything benefits from what IQ tests measure. That said, I had already started seeing great success in a knowledge industry. It is clear that you also have great linguistic intelligence. I don't think you could completely dissociate your IQ test results from that intelligence.


I don't think you could call the general lack of diversity in tech a "raciosexual conspiracy". Rather, it seems a lot more like the result of classic pressures - institutionalized racism and sexism, combined with the general tendency of people to prefer other people who look like them.

The fact that certain groups of people tend to test well (or, taken conversely, that there are so many reasons why otherwise "intelligent" people don't test well) just exacerbates this problem and I think the central claim is still reasonable: things like IQ tests aren't necessarily strong predictors of tech skills, are dangerously close to pseudoscientific when misapplied, and have enough other theoretical problems that they should probably not be utilized during a hiring process.

As to my own linguistic skills, I contend that a lifetime of reading as well as a MS (in computer science), a Ph.D (in cognitive science and education), and ten years afterwards in academics including a professorship has prepared me as a writer. Did I have some initial Potential that gave me a head start? Maybe. If nothing else, I had the head start of my general introvertedness and a love of both reading and geekiness. A short test that purports to measure a fixed potential somehow inherent in someone is going to be pretty flawed.

But my "I don't doubt" phrase was more along the lines of ceding a central point: there probably do exist certain inherent characteristics that vary between people and provide some kind of predilection or head start. I'm just not yet convinced that the scientific community has really identified them yet, or that they're really able to effectively measure them yet. Instead, we get proxies that have a very high false negative rate, especially among otherwise-marginalized groups.

Anyway, four paragraphs is probably enough here.


Could there possibly be a chance that with your majestic IQ might make you a bit biased on the subject? Mind you, I have a uselessly high IQ, too, and I still think they're are an insult to hard working and bleedingly knowledgeable people who could outperform you and I, easily. I'm just not sure what the value of the tests are, considering the discriminatory ease.


Thank you! It's refreshing to hear that from anyone with a cognitive science background, as you put it.

I had an MBTI test pretty much forced on me a while ago (senior engineer would be unhappy if I didn't take it). I argued very forcefully against it, saying it was like astrology, that it made me feel like the company was asking me to participate in some religious ritual when I was an atheist and so on. I'm not in that company anymore and one major reason I ran was that they allowed pop psychology like that to dictate decisions.


It's akin to needlessly using technology to solving a problem or requiring someone to write a hugely time consuming code sample. Sure, theoretically it's a great idea, but since they're never done right and they never address the issues that affect the fringe.

The worst I've seen was where I had to do 100 questions and half were spatial intelligence (gameable and wtf). It looked like an incredibly expensive piece of software. Didn't get the job, thankfully, but the company's now known for having a policy of not being able to talk about work out loud if you're in IT. The other places that gave me tests had dissimilar, but equally insane workplace policies.

Rambling list: -At some level, the person interviewing you (or management) doesn't trust their intuition to find the right candidate.

-It's an easy way to throw someone out of a pool despite all else. There's a reason it's illegal to do IQ tests.

-Mental health issues such as dyslexia or ADD.

-I've taken maybe five intelligence tests for work and none of them have been the same. Where's the standard?

-Companies have a weird tendency to use puzzles such as the "three light bulbs" one. All that does is test if you've heard it before. "I want to see how you think" makes no sense when the question's on paper and there's 100 questions just like it.

-A lot of these tests are gameable since you can just practice beforehand.

-Test anxiety sucks.


Which company?


Coca Cola: That recipe must be protected by state of the art custom techonology


HN user tokenadult has a canned comment about this that they reposted for YEARS on this site.


Anecdotal: My first job out of college had an IQ test (along with other interviews, tests, and a presentation), and it was a great job.


IQ tests are useless. I got a printout from one that says I have an IQ of 130, after I was sent a link to do the IQ test at home. Obviously, I called my flatmate and we did it together. We spent most of the time of the test taking the piss, else we'd have hit 200 or so.

The company who gave me the test offered me the job after a very brief interview, in which they said the test was not important. It was very clear from their behaviour that they thought it really, really was (important) and they were about to hire some sort of genius. I ran. I'm still running.

In my current position, at Big Financial Corp, I had to take an "aptitude test". There were three parts of it: literacy, numeracy, logic in that order. I suck at arithmetic so after the literacy bit I wanted to get straight to the cool and fun logic puzzles at the end, but the time we were given was like a third of the time the test was meant to take (allegedly, they wanted to see how well we perform under pressure) and I wouldn't have the time to do all three parts equally well. So I thought, screw it, I'll approximate. I clicked through the numeracy test mostly at random (I concentrated clicks on the first three options), hoping to get a 50/50 score and dove right into the logic test at the end. I had great fun and I got hired after all so it can't have been that bad. They haven't told me how well I did because they don't like to um, foster antagonism. Or something.

Point being: IQ tests are stupid. You don't need to have a high enough IQ to pass them. You just need to be smart enough to beat the test. So, useless.


An above average IQ is usually good, but not too high. 115-120 seems pretty good. Above that, I've found a strong correlation with either a) being a dick, b) having obnoxious social quirks, or c) being dangerously over-confident. YMMV


"In their communication with me all these people have been civilized and polite, without weirdness or negativity. All in all the top scorers seem to confirm what has slowly been dawning to me over the past years: that, within the high range of intelligence, higher I.Q.s tend to come with a greater likelihood of being psychiatrically and socially normal."

http://iq-tests-for-the-high-range.com/statistics/iq170.html

Also, please read about IQ and rational thinking: http://www.scientificamerican.com/article/rational-and-irrat...




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: