
Lessons from a year’s worth of hiring data (2013) - xasos
http://blog.alinelerner.com/lessons-from-a-years-worth-of-hiring-data/
======
brokencup
Previous HN discussion on this post:

[https://news.ycombinator.com/item?id=6326477](https://news.ycombinator.com/item?id=6326477)

~~~
soham
In fact, this is the 4th time (See here:
[https://hn.algolia.com/?query=http:%2F%2Fblog.alinelerner.co...](https://hn.algolia.com/?query=http:%2F%2Fblog.alinelerner.com%2Flessons-
from-a-years-worth-of-hiring-
data%2F&sort=byPopularity&prefix&page=0&dateRange=all&type=story))

It's a well researched article, so I don't mind per se.

[me: [http://InterviewKickstart.com](http://InterviewKickstart.com)]

------
wdewind
> Bemoaning that non-technical people are the first to filter resumes is silly
> because it’s not going to change. What can change, however, is how they do
> the filtering. We need to start thinking analytically about these things,
> and I hope that publishing this data is a step in the right direction.

This is only true for very few companies (massive ones like Google). If you
are running and engineering team of < 50 I think it makes a hell of a lot of
sense for resume review to be on the engineers. 10 minutes a day of resume
reviewing gets you through a ton of resumes (it takes me less than 60 seconds
to determine if I want to continue talking to someone from a resume) at very
little cost. You don't want your engineers handling scheduling etc., but
resume review isn't really much of a time sink until you are getting tons of
inbound all the time, which many companies wont ever get to.

And this:

> As you can see, “good” resumes focused much more on action words/doing stuff
> (“manage”, “ship”, “team”, “create”, and so on) versus “bad” resumes which,
> in turn, focused much more on details/technologies used/techniques.

Is highly biased by the fact that she was hiring for a web dev company.
Resumes including words like "systems", "C++" and "algorithm" were considered
_bad_ because they received no offer. You don't really need the distributed
systems guy who can write highly performant C++ and actually understands how
to apply algorithms at a standard web dev job.

~~~
ryandrake
I've always thought the best resume format is "Action -> Result" rather than
simply listing your responsibilities and/or technical knowledge. People get
hired based not on what they did or know, but based on how what they did
helped add value to their company.

Example:

    
    
        WEAK
        ----
        - Responsible for development and maintenance of the FooBar software suite
        - Skills applied: Java, HTML, CSS, REST APIs
    
        STRONG
        ----
        - Added BarBuzz feature to FooBar software suite, which contributed to 25% increase in product sales and winning 2 industry awards.

~~~
Swizec
I agree, action -> result always sounds more impressive.

But to a point it's always felt disingenuous to me as far as engineering goes.
Sure, I implemented the BarBuzz feature. But a marketing person gave the idea,
a product person specced it out, a UX person designed it, a UI person made it
pretty, and all I did was the final implementation.

Did I really _add_ the feature, or did I just implement it?

~~~
ryandrake
Everyone else who touched the feature is going to also say they added it, on
their resume. You have as much of a claim to it as they do. Your job title and
the size of the company is clear, so the reader can infer the size of your
contribution from that.

------
T2_t2
The grammar one worries me. I wonder if that was corrected for country of
birth, e.g. if this is an apples for apples comparison of native English
speakers (USA, Canada, UK, Australia etc) or if it includes non-native
speakers.

If there are people of non-English decent, I wonder if they would be more
likely to have errors, especially grammatical, that are understandable. If
that is the case, this may simply be a correlation to poor interview
performance, and a bias towards the singular group of native English speakers,
and that they may have let great programmers go.

Now, I'm not saying it is true, I am just saying that is a possibility that
would alarm me when errors on a resume is such a huge indicator, and wonder
whether that is not something they should be correcting for.

~~~
leeny
Author here. I get this counterpoint a lot and for good reason. My best
response is that a resume isn't something you come up with on the fly. It's a
document you have every opportunity to show to people, get feedback, and
improve upon. Therefore, if you're not a native speaker, you should presumably
have the self-awareness to mitigate that disadvantage by enlisting outside
help.

~~~
T2_t2
I understand those arguments, but it doesn't addresses the issue for a very
specific reason. What if the problem is the way internally the team works?

When measuring end-to-end, any part of that may influence the final outcome.
If, and I am not saying this is true, if there is a tendency to have a few
more errors (especially grammatical with it's its) in a non-native tongue,
success outcomes MAY point to a bias in interviewing, or in how people
interact internally.

Or not. It is just something I wondered, and something it doesn't seem like
you've looked at specifically, and understandably as well, because who wants
to open themselves to accusations of bias with their own data?

------
rch
If _having worked at a top company_ is more correlated with success than any
combination of one's own metrics, then I'd look for ways to improve the
recruiting process.

~~~
AnimalMuppet
Well, it's maybe an admission that the top company's hiring process is better
than your own. Which, if you think about it, might be reasonable.

Still, yes, it's a sign that your own isn't top-company-worthy yet...

~~~
trustfundbaby
> Well, it's maybe an admission that the top company's hiring process is
> better than your own

Or maybe the most naturally talented candidates self select into the bigger
best known firms from the onset?

~~~
PopeOfNope
Or they're suffering from brand recognition. Google and Disney have a
reputation for excellence, so just having it on your resume biases the
interviewer into projecting those attributes onto you.

~~~
collyw
Yeah, having worked for a couple of Fortune 500 companies, 1 was fine to work
for, the other was absolute hell.

------
MCRed
Some comments: \-- If you're running a good company, then the resumes will be
filtered by technical people, not HR. There really aren't too many of them (if
you're big enough to be getting thousands of resumes, then you have enough
engineers to do a first pass.) And don't just give the resumes to these
engineers, you need to train them on what to look for.

It is already changing that Engineers filter the resumes and not HR. Good
startups do this.

Hiring -- especially for a startup-- is absolutely the most valuable thing,
and it is a valuable use of engineers time (but only have engineers who want
to do it and care about the hiring process do it.)

"Before I share the actual results, a quick word about context is in order.
TrialPay’s hiring standards are quite high. We ended up interviewing roughly 1
in 10 people that applied. "

A non-technical person (yeah yeah, I don't believe for a second she was ever
an engineer, that's vanity talking) is eliminating %90 of the applicants and
they think that's a "high bar"? No. That's randomness.

\-- Top Company.

I love that she thinks having worked at a "Top company" like Amazon is an
indicator of success. I pick amazon because I've worked there and its in the
news lately for being totally poorly managed. That poor management means the
engineering side of the house is a total and absolute mess. Bugs I fixed in
2006 are STILL BROKEN. Because they were regressed due to mismanagement. The
QA team that was focusing on that area is totally disbanded. This areas of the
site has not improved at all, and has in fact gotten worse over the past 10
years-- and it's critical- it's product search!

So, they will hire people from Top Companies (and put them thru the
incompetent HR filter) over better engineers with good side projects.

Great.

~~~
deciplex
> _A non-technical person (yeah yeah, I don 't believe for a second she was
> ever an engineer, that's vanity talking) is eliminating %90 of the
> applicants and they think that's a "high bar"? No. That's randomness._

I wish more people could wrap their heads around this. Her process could be
rolling a D10 for every CV she receives, then lighting the CV on fire unless
it passes the saving throw, and she'd still be able to say "we ended up
interviewing roughly 1 in 10 people that applied". It says absolutely nothing
about the effectiveness of her methods.

I've been involved in hiring in the past, and discussed this issue with people
having this mindset, even using this exact example with the D10, but I don't
think I've ever gotten through. Some people will insist on believing that
_any_ highly selective screening process is automatically good, without any
further introspection.

~~~
WildUtah
I just divide the pile in two and throw half the resumes I get in the trash.
Why would I want to hire unlucky people? They could make the whole company
unlucky. Plus, it makes our hiring process more selective.

~~~
deciplex
In seriousness though, my hypothesis is that they are at least certain that
they are not making the candidate pool any _worse_ by their screening, i.e.
they are not selecting for unqualified candidates. Even if their process is
barely better than 50%+1, they are at least certain it is not worse than 50%,
and consider that good enough. If they took 1000 candidates, 100 of which were
qualified for the position, and whittled that down to 100 candidates, 11 of
which were qualified for the position, they'd honestly consider that a win.

------
matheweis
This seems pretty reasonable; "having worked at a top company matters" could
speak more to the hiring practices of "top company" than to anything else.

A candidate who worked at a "top company" was (probably) already prefiltered
by "top company"'s hiring practices.

~~~
mathattack
The same is true from name brand schools. They filter for good test scores and
other things.

------
aidenn0
Holy selection bias Batman!

So the people with resumes so awesome that they could land an interview
despite having no CS degree did well in interviews? Color me unimpressed.

------
wuyanbo
I can't comment on the typo thing as it never seems to be a problem in my
team. But here are some thoughts for the hiring culture in China. I am a
technical manager and currently taking care of a team of engineers in a large
online education company. I do recruiting myself.

> having attended a top computer science school doesn’t matter.

Matters to me. Unlike in U.S., the selection process for college/university
entrance is more effective in China, especially for top schools. I am not
saying those from less top schools are not as good. It is just more efficient
for the resume screening. The education system has done this much better than
a short interview or resume.

> listing side projects on your resume isn’t as advantageous as expected.

Somewhat agree. It does have some impact, but I will read the code. It is
advantageous for me as I can see his previous work in deep details.

> GPA doesn’t seem to matter.

Hm... here nobody lists GPA unless it's very high, and high GPA means nothing
to me.

> having worked at a top company matters.

It does. Former employees from Baidu, Alibaba and Tencent (BAT) are more
welcome, for the same reason as the top school thing. The selection there is
much more strict and cruel. But I often ask the question 'why you leave B/A/T
while they pay better and are generous in stock options?'. The question
implies my concern that the candidate leaves B/A/T for some weakness that may
also have bad influence in my team.

The branding of top company is a double-edged sword. While you are enjoying
the branding, you must be prepared to explain why giving it up.

~~~
happywolf
I was an engineering manager in one of the big Internet companies in Shanghai
for 2 years.

As a foreigner (Singaporean), my own experience was those graduating from top
Chinese schools actually is a dis-service to the whole team since those
graduates tend to think too high of themselves, and thus are arrogant,
slacking, demanding high benefits (salary, holidays, etc.), but their outputs
don't commensurate with their attitude. Definitely there are good graduates
from those schools and your experience may differ, but so far this is my
observation.

As such I now no longer hire graduates from top Chinese colleges. Instead,
those from second or third tier schools are more humble and hardworking.

No, I am not suppressing any salary. My staff (who came from a pretty not-
well-known school) got a monthly basic salary of RMB25K, which I think isn't
low in any standard, even in Shanghai. But I think he deserved every cent of
it and I am happy to pay.

------
mathattack
The more I read this, the more skeptical I become. All this shows is "What
characteristics are most likely to get you an offer?" This is a little
important from a narrow process metric, but isn't valuable to the company's
bottom line.

2 more important questions: 1 - "Which characteristics are more likely to
appear in high performers than low performers and non-hires?"

2 - "Are we over-weighting or under-weighting certain characteristics in our
recruiting process based on our knowledge of question 1?"

Question 2 is actually much harder to answer. For example, if you find no
correlation between GPA and performance, it isn't that GPA doesn't matter,
it's that you're already weighing it properly in the performance decision. (It
could be that you ignore it, or it could be that you give it tons of weight,
but either way, the lack of correlation to performance post-hire means you're
doing the right thing)

------
bootload
I've read and re-read this post a number of times. It resonates because hiring
is broken and companies are looking for ways to triage the process. Yet when I
look at the data points being captured, I ask _' what is really being
measured?'_ Ability to work and ship product or Hoop jumpers?
([https://news.ycombinator.com/item?id=10073663](https://news.ycombinator.com/item?id=10073663))

A new hire at a Startup can and should look more like what I found at
lighttable, cf _' Meet the new guy'_: [http://lighttable.com/2015/01/13/light-
table-hack-night/](http://lighttable.com/2015/01/13/light-table-hack-night/)

The progression from plugin contributor to new hire is measured in actual
results and peer interaction, not through second order measurements. Adapt or
die.

------
justboxing
>> listing side projects on your resume isn’t as advantageous as expected

I am surprised by this "finding". Why would side projects not be advantageous?
Don't side projects establish that the candidate is actively keeping up to
date on technologies and is so passionate about it that he/she works on it
even outside office hours?

~~~
tclark225
Side projects are probably much more advantageous if you get to talk about
them in an actual interview. I suspect they don't matter to much on resumes
because of the issues others have brought up here

~~~
xasos
I think that's the main reason why you'd have them. To incite discussion in
your interview, and discuss what and why you like to build stuff.

