
Lessons from a year's worth of hiring data - slyall
http://blog.alinelerner.com/lessons-from-a-years-worth-of-hiring-data/
======
sneak
> The most significant feature by far was the presence of typos, grammatical
> errors, or syntactic inconsistencies.

I've always used attention to detail when writing (although I give a pass on
grammar/spelling to some extent to non-native speakers) as a proxy for general
professionalism/intelligence with relatively high accuracy.

If you can't be assed to write accurately and consistently in your native
language (and/or don't avail yourself of the myriad automated tools to achieve
this end), how can anyone expect you to be accurate and consistent in any
other endeavor? It's not like language and written communication is
unimportant; indeed it may be the most important skill any information worker
can have.

If I were considering hiring a native speaker of English, a single homonym
misuse (its/it's, effect/affect, et c.) or lack of subject-verb agreement in
any pre-hire communication (emails, CV, their public-facing website, READMEs
on Github) would be sufficient for a NO HIRE in my book.

~~~
lusr
While I agree that immaculate attention to detail is important in documents
you're creating for formal consumption (e.g. CVs/resumes), I don't see how the
same extends to emails, websites and READMEs.

I'm a native English speaker and despite being well aware of homonym misuse,
my brain doesn't always play the game and occasionally I find myself using the
wrong word in my writing with no rational explanation.

Similarly, I often find myself leaving out words in my writing that I don't
notice are missing even after re-reading a paragraph many times. (I fall for
"PARIS IN THE THE SPRING" almost every time it comes up.)

Alternatively I'll change the wording in a sentence and not notice stray words
are left over in the wrong order, again even after re-reading. Usually I have
to do something else and look back at what I've written to read it with "fresh
eyes". I imagine dyslexic people have similar problems.

Despite these issues I've been complimented many times on the quality of my
technical writing and my code, so I don't consider my natural language issues
as much of a disability when it comes to programming or technical work. That
is, until I encounter somebody who holds your beliefs.

It seems obvious to me that there's a vast difference between somebody writing
a CV using obviously incorrect spelling or random formatting, or inconsistent
capitalisation, punctuation, tenses, etc., and somebody who makes one or two
typos in a blog post on their personal home page.

~~~
tehwalrus
It is almost impossible to proof read a document you wrote yourself for typos,
specifically repeated/replaced words, immediately (while you can still
remember writing it.) I don't know if this is true in all languages, but it's
definitely true in English.

This is because your brain subvocalises (reads back) what you _meant_ to say,
not what you actually wrote, when you re-read and can still remember the
sentence in your head.

I know of at least one example of a Deputy Headmaster, who was also my maths
teacher, accidentally typing in a swearword to a school report. My Dad (the IT
manager, whose job it was to check all the reports before they were posted)
got him to read it back aloud (a day or so after he wrote it) and he read out
the sanitised/corrected version, aloud, from a piece of paper with the
swearword staring him in the face. He was _very_ embarrassed when my dad
pointed it out.

In cases like that, you wish you could define a specialised dictionary without
the swearwords, so that dangerous typos jump out at you!

to GP: tooling can only get you so far, and you're battling against your own
brain the rest of the way. A 2AM README typo is forgivable; a CV/cover letter
issue (where you were supposed to be concentrating, and/or getting someone to
proof read for you) less so.

~~~
sneak
Typos are one thing, misuse of homonyms or lack of subject/verb agreement is
another entirely. I wouldn't begrudge someone a "teh" in a readme THAT much
(though, again: where the fuck is your spellcheck?), but a misuse of "your"
for "you're" immediately makes me start to question if they have a clue what
they're doing.

------
jonnathanson
_" As soon as you get someone who’s never been an engineer making hiring
decisions, you need to set up proxies for aptitude. Because these proxies need
to be easily detectable, things like a CS degree from a top school become
paramount."_

No doubt this is true to a some extent, and perhaps even to a large one. But
let's not give the HR drones too much credit here.

How many of _us_ have ever had to sift through a stack of 100 resumes or more
in, say, a week or less? It's not an easy task. It's especially difficult for
hiring managers, because they have day jobs to perform. They may not _mean_
for things like "Google," or "Harvard," or "L33t CS Degree" to become proxies
for our honest, thorough, intellectually rigorous analysis of every resume in
the pile. But these things become a sort of shorthand.

HR types seem more prone to overemphasizing the letter of the law, to putting
pedigree on a pedestal, and to thinking as un-differently as possible. But
given a thick stack and a few measly hours, I doubt most of us fare
_significantly_ better.

If we're serious about moving toward a better hiring process, we need to start
by recognizing the limitations of the resume itself as a normative tool.

~~~
leeny
Author of the original post here. I agree completely and recently wrote about
this problem: [http://blog.alinelerner.com/silicon-valley-hiring-is-not-
a-m...](http://blog.alinelerner.com/silicon-valley-hiring-is-not-a-
meritocracy/)

TL;DR It would be awesome if candidates had the option to do a coding
assignment and submit a writing sample in lieu of a traditional resume/cover
letter.

------
Peroni
_I interviewed roughly 300 people for our back-end /full-stack engineer
position_

Q1. - Given that most companies interview less than 10% of the total number of
applicants, I would love to know how one job received 3,000 applicants.

Q2. - Why did you have to interview 300 people to fill one job? There is a
serious flaw in that strategy. If you spent one hour with every candidate you
interviewed, that works out at just under 6 weeks of back to back interviews.
That's not taking into consideration the logistics involved in filtering and
processing these 300. All that aside, do you not consider it to be
exceptionally flawed that you had to interview 300 different people in order
to find one suitable person for the job?

~~~
leeny
0\. Peroni, thank you for reading this. I've been a fan of your writing for
years, and your take on hiring was one of the things that ultimately helped me
make the decision to switch from coding to recruiting.

1\. Yeah, over the course of a year, we got several thousand
applications/resumes for this position. The interview rate was roughly 1 in
10.

2\. We interviewed 300 people and made offers to 6, i.e. the hit rate was 1 in
50, or 2%. Moreover (and fortunately!), each individual filtering round and
subsequent interview took much less than an hour.

~~~
Peroni
Thanks for the kind words. I feel exceptionally guilty for my harsh feedback
now.

I'm still bothered by the 1 in 50 ratio. Maybe I'm misinterpreting it. Does
that 50 include people who get phone screened or is that 50 exclusively people
you have met face to face?

If it's the former rather than the latter, then consider my incredulity null
and void. If it's the latter then you should drop me an email as I would jump
at the chance to help you to halve that number.

~~~
leeny
Thank you :)

The 50 includes people who got phone screened. The onsite to offer ratio was
something like 3 to 1, if memory serves. I will update the original post to
make that clearer.

------
mswen
Interesting results that seem to challenge some received wisdom. However, N=1!

What this study tells you is the systematic relationship between resume
detectable attributes and getting hired at this particular company. If this
same analysis was carried out at 29 more companies we would have N=30 which is
considered a small sample size. To effectively do that the author would need
to put out a detailed description of methods for coding resumes and ideally a
means of determining inter-rater reliability and so on.

Interesting conversation starter - but please no one make any important
decisions based on this.

~~~
mcherm
Depends. From the point of view of the applicant perhaps N=1 as only one
company's hiring was investigated. But from the point of view of someone
scanning resumes, N=300 (there were 300 applicants considered).

~~~
mswen
From the perspective of lessons learned, that is generalizable truths, one can
only generalize to that company. So the reader of this article can only really
say, this is highly relevant and accurate if I intend to apply for an
engineering position at this particular company. Beyond that it falls into the
same anecdotal category as any other advice dispensed by recruiters or HR
personnel who have responsibility to screen applicants entering into the
interview process.

The big difference is this recruiter has gone to the trouble of actually
measuring a number of factors and assumptions and can say with great
confidence, this is how it works in a particular company. In that regard it is
certainly more weighty than the type of blog post that says "I have been doing
screening for 5 years and this is what matters to me. And, by the way it seems
to correlate to hiring decisions at the back end of the process."

------
tnuc
Excellent piece.

I was getting sick of reading about "cultural hiring" on HN. Finally someone
has some numbers instead of trying to justify why they should only hire people
who look and sound like themselves.

~~~
myth_drannon
actually hiring based grammar mistakes is " cultural hiring". I assume they
were not hiring immigrants based on the stats presented.

~~~
mcherm
I know lots of immigrants with excellent grammar and spelling. A few who even
bother to get someone else to check their resume for that stuff.

~~~
myth_drannon
Professional proof reading costs money. If you are modifying your resume for
each job you are applying, it will cost you a fortune. And asking your friends
who speak English natively(very few immigrants have those) is not so good
since most native speakers have grammar problems( unless you have a friend
with a Phd in English lit).

------
ronaldx
Amongst applicants mentioning GPA: 30% have GPA 3.8 or higher

Amongst job offers mentioning GPA: 67% have GPA 3.8 or higher

Author's conclusion: GPA doesn't seem to matter

Alternative conclusion: If you mention your GPA, 3.8 or higher gives you 5
times the chance of being hired.

*Edited for my bad.

~~~
jasonlotito
Be careful with those numbers. He mentions that half the applicants didn't
list a GPA. He also mentions that because of this, their are likely biases in
those numbers. He makes mention of this in the comments.

Basically, the data isn't clear.

~~~
ronaldx
My point is that the author's conclusion is way more than a stretch.

------
ndonnellan
Nice data! However, I'd love to understand the relationship between those
attributes and quality of employee. Perhaps a government funded study where
you select 100 resumes at random and hire all of them for 6 months? Piece of
cake.

~~~
leeny
Yeah, tracking on-the-job performance would be much more interesting than just
looking at whether or not someone got an offer. I hope to be able to do this
kind of study in the future with the advent of more data.

------
cmarschner
Nice article! Two thoughts on the issue of grammar/spelling mistakes:

1\. The anecdotal argument: i know a great developer, very detail-oriented,
math/stats genius, great team player, elite company background. But he's
dyslexic. Let's not hope you would apply your filter there.

2\. Great teams need a balance between detail-oriented and not so detail-
oriented folks. I know quite a few excellent programmers who can get lost in
the details. People with less focus on the details are often the more
visionary ones, the ones with the big picture. Truly great people can switch
between the two, but they are rare.

------
7Figures2Commas
> Of all the companies that our applicants had on their resumes, I classified
> the following as elite: Amazon, Apple, Evernote, Facebook, Google, LinkedIn,
> Microsoft, Oracle, _any Y Combinator startup_ , Yelp, and Zynga.

I'm not sure an "elite" filter is a great filter, but it's not surprising that
it is used. What is surprising is that the OP considers "any Y Combinator
startup" to be an "elite" company.

More than 500 companies have passed through Y Combinator and I doubt that most
of them would be instantly recognizable as Y Combinator companies on a resume
unless Y Combinator was mentioned. Grouping _all_ former employees of Y
Combinator startups into an "elite" category seems like it requires an
unnecessarily big leap of faith on the part of the prospective employer.

------
leeny
Hey guys, I'm the author of the original post. Happy to answer any questions.

Also, for some more discussion, see
[https://news.ycombinator.com/item?id=5919819](https://news.ycombinator.com/item?id=5919819)
(last time this piece was on HN).

------
victoriap
>>having worked at a top company matters

does that mean that it makes more sense to know how top companies hire, rather
than small ones, since small companies just shadow decision made by bigger
ones and check for typos and grammatical mistakes?

~~~
LaurensBER
To be honest, most of the top programmers or engineers that I know have
terrible, terrible grammar and spelling. A number of them do not speak English
as a first language but others just don't seem to care that much.

Would I base my hiring decision for a new programmer or engineer on the amount
of spelling mistakes in a resume? I would give it thought but I would be more
interested in actual real life working experience.

~~~
jasonlotito
> To be honest, most of the top programmers or engineers that I know have
> terrible, terrible grammar and spelling

On their resume? Or just in emails they are sending out or in chat?

------
DocSavage
> I ran this analysis on people whom we decided to interview rather than on
> every applicant; roughly out 9 out of 10 applicants were screened out before
> the first round.

Although the author acknowledges this will taint the results, it seems like a
major filter especially when you are crafting rules for screening by non-
engineers. For example, the results for "Offer Likelihood as a Function of
Highest Degree Earned" could shift quite a bit if those with higher degrees
were basically passed through the first cut.

Why not run a subset of the analysis using the full set of applicants?

------
kriro
It's somewhat interesting that grammar/typos etc. matter this much. It's a
little hard for me to believe because I'd think "projects, projects, projects,
get stuff done" trumps all.

Did you adjust for native speakers vs. non-native speakers in any way?

Also maybe I'm naive but why can't engineers do the HR even at big companies?
Imo for knowledge workers you really can't afford to outsource HR to people
that have no domain expertise.

~~~
RougeFemme
At the big companies, HR doesn't handle the complete hiring process, but a
good HR department can do effective pre-screening for the hiring manager. I
like the fact that the OP points out that if you provide sufficient criteria
to HR, they can effectively screen resumes. (The problem that I've seen -
which can be avoided - is that either no criteria are given to HR or the
criteria are too tight.)

Why should an engineer spend time pre-screening for typos? And, if the hiring
manager is adamant that the candidates must be proficient in
language/methodology X, why should an engineer waste time looking at resumes
that don't even mention language/methodology X? (Maybe the hiring manager
shouldn't be adamant about lanauge/methodology. Maybe judgement should be
used. So that's a criterion that should not be passed on HR, in that case. If
it is, that's the fault of the hiring manager, not HR.)

Also, a good HR department can keep you from getting into legal trouble with
illegal questions/behavior.

------
riggins
this is an extraordinary claim.

stop and think about it for a second.

if it's true, then whatever trait leads one to being punctilious about
spelling also makes one a great engineer.

forget the guy who's really good at linear algebra or can write his own
compiler ... the guy who is perfect with his spelling will be the better
engineer.

I'd love to see more research on this because it would be really fascinating
result if true.

~~~
gaius
In an age where word processors highlight mistakes and offer suggestions,
errors of the kind that would be caught automagically are akin to checking in
code that doesn't compile.

I don't really care about email (esp. if written on a phone) but a CV is, as
others have said, a formal document.

------
xdd
Ultimate Troll: Wrote plenty good code, frequent speaker, founder of several
companies, hacker.

Send resume that has grammatical errors, or syntactic inconsistencies. Turn
down by technical recruiting, then post with details.

