
Job hunting is a matter of Big Data, not how you perform at an interview - Roonerelli
http://www.theguardian.com/technology/2014/may/10/job-hunting-big-data-interview-algorithms-employees
======
ddebernardy
Best quote from the entire article imho:

> The data suggested that the success of teams had much less to do with
> experience, education, gender balance, or even personality types; it was
> closely correlated with a single factor: "Does everybody talk to each
> other?"

> Ideally this talk was in animated short bursts indicating listening,
> involvement and trust – long speeches generally correlated with unsuccessful
> outcomes. For creative groups such as drug discovery teams or for traders at
> financial institutions, say, the other overwhelming factor determining
> success was: do they also talk to a lot of people outside their group? "What
> we call 'engagement' and 'exploration' appeared to be about 40% of the
> explanation of the difference between a low-performing group and a high-
> performing group across all the studies," Pentland says.

> It was important that a good deal of engagement happened outside formal
> meetings. From this data, Pentland extrapolates a series of observations on
> everything from patterns of home-working (not generally a good idea) to
> office design (open and collegiate) to leadership. "If you create a highly
> energetic environment where people want to talk to each other right across
> the organisation then you have pretty much done your job right there."

So true.

~~~
asdfologist
The article didn't make clear though, whether having more engaging
conversations actually _causes_ team success. If a team is successful, that
may boosts morale and lead to increased levels of conversation.

------
anjc
It's been known for a long time that standard interviews are a poor indicator
of future job performance, so probably any analyses of data regarding
behavioural modes, a person's intrinsic motivators, cultural fit etc, will do
a slightly better job. Presumably all of this will still be subordinate to an
on-the-job trial anyway, though.

It'd worry me that, although this guy says you need to increase behavioural
diversity but minimise value diversity, that you're effectively just
minimising the pool of potential employees, rather than figuring out ways that
a larger pool of people could fit. Or in other words, whether it's Myers-
Briggs, Belbin's roles, IQ tests etc, it seems that evaluation tools are
trying to quantify the diversity of people, and pick off, with increasing
accuracy, the exact archetype that aligns with organisational goals. But if
you assume that people are diverse, and that employment roles are diverse, and
that there's a large pool of both, would it not be a better idea to focus on
quantifying the differences in attributes required for your employment roles,
so as to maximise your pool of potential applicants?

I'm willing to bet that someone who's slobbish and lazy and unpleasant could
play a valuable role if you could quantify the requirements and goals of
positions in your company, for example. And does value alignment matter for
all roles in all companies, or is it just a phenomenon arising in the last
decade, being an intuitive way to maximise employee investment and increase
profit? Why would financial staff need to want to 'change the world' or some
bullshit to work at crappy Startup X?

~~~
Malarkey73
I'm not a regular interviewer but I've been on a panel for a few recently. For
one job in particular the standout candidate on paper gave the worst interview
I have ever seen.

He was arrogant, confrontational, couldn't answer simple questions and was
half an hour late - and not apologetic at all. He was just weirdly and
inexplicably unpleasant to everyone.

Perhaps he would have been great in some role or would have aced a
psychological test .. but I'm glad we did an interview so we could reject him
out of hand.

I'm sure there is a place for widespread empirical evaluations .. but
ultimately if you are employing someone to work with you there has to be space
to say: "Err, No! - I don't care what the computer says".

~~~
silverbax88
Or he took one look at the place, decided he (or she) did not want the job and
tanked the interview.

~~~
Malarkey73
We're all lovely people and we served nice coffee and delicious biscuits.

~~~
bryan_rasmussen
eww, biscuits!

------
munrocape
Now any analysis applied to a data set is now "Big Data." While easy to just
resolve it to being how the media treats most science stories, I had hard time
understanding the demarcation. I now go by DevOps Borat's definition - "any
thing which crash Excel" [1]

[1]
[https://twitter.com/DEVOPS_BORAT/status/288698056470315008](https://twitter.com/DEVOPS_BORAT/status/288698056470315008)

~~~
gaius
It's worse than that. People will run Hadoop on 20 nodes to do stuff Excel
could handle easily, so they can join the "big data" fun. It's just CV-padding
really, ironic given the discussion.

~~~
001sky
[https://twitter.com/Lumonade/status/289171814444331008](https://twitter.com/Lumonade/status/289171814444331008)

"@DEVOPS_BORAT got heavy duty @dev ops gig available in San Mateo,ca and
Seattle area! $130-$140+k doe email me Ryan.Lum@greythorn.com"

does this mean it works?

------
tom_jones
I'm suspicious of this technology. While I concede that it probably has some
statistical and even empirical merit, those in HR will always do everything
they can to make sure that "troublemakers and journeymen" get kept out. But
unfortunately, most people who have had the same job for a long time are
either just trying to pay their mortgage and smiling through gritted teeth
long enough to see their kids alright or have no imagination/ambition and
those who switch jobs regularly are often simply tired of taking shit from
people. I think you'll find that "troublemakers" usually have tried to alert
their line managers to the real flaws in the company's systems and been told
to shut up. Look at the businesses this data is being taken from, Law
Companies, Corporate Banks etc. It is obviously yet another toy to allow rich
kids to recognise their own. What has this got to do with those who do the
vast majority of the actual work?

------
chroem
This is exactly the sort of thing that I have been absolutely terrified about
for a while now. When you're being hired, companies often tell you that
they're going to run a professional background check, and I am concerned this
may mean they buy data from Google, Facebook, cell carriers, etc.

Unsavory political views? Get blacklisted.

Don't get along with a relative? Get blacklisted.

Indulge a porn habit more than the HR manager likes? Get blacklisted.

I cannot stress enough how dangerous this is.

~~~
codystebbins
I think this is cynical and forgets the reality of competition. In this 'big
data hiring' future companies still need to compete for talent. Companies
which hire on the basis of arbitrary factors like political views, porn
habits, family infighting risk losing talent to competitors. There are
incentives to find the right factors of a good employee, just like there are
today. In fact a lot of these concerns can happen with the human interview
process today.

The algorithm to hire people in this future is a competitive asset of a
company and they would spend lots of time & money to narrow down the actual
important factors. I highly doubt companies would just all settle on the same
algorithm given the incentives to compete.

Also if you are concerned about this info impacting employment, just don't
post it online. Not much different than today.

~~~
quanticle
> In this 'big data hiring' future companies still need to compete for talent.
> Companies which hire on the basis of arbitrary factors like political views,
> porn habits, family infighting risk losing talent to competitors.

Yes, but you forget the old banking maxim: it is better to fail conventionally
than it is to succeed unconventionally. I think it's very likely that hiring
managers would pass up the potential increased talent of an employee if that
talent came coupled with increased risk that the employee would do something
to embarrass the company or do something to make the company look bad.

>Also if you are concerned about this info impacting employment, just don't
post it online.

So what do I do when the lack of online information about me in itself becomes
a drawback?

------
lifeofanalysis
There is obvious up trending of terms like 'big data', 'predictive analytics'
and 'data mining'. I have worked in this area since 1998. So here are a few
thoughts:

\- Good analytics (I'll combine the three terms into 'analytics' for the sake
of simplicity) requires an understanding of the tools, as well as a
significant understanding of statistics so that you know which analysis to
pick. But in addition, it requires a lot of creativity (see my examples below)
and a significant amount of time to analyze/slice/dice data in a zillion
different ways.

\- This is a huge opportunity. Much, much bigger than people realize and much
bigger than past trends of new technologies like client-server in early
nineties or web apps of 4-5 years back. Why? Because it has the power to
affect business processes very powerfully.

\- Example 1: I spent 10 months working for a $5B shipping company analyzing
data from their Marketing department. I combined it with several hundred
global data sources. I worked on over 100 hypotheses. At the end of it, I came
up three specific actions that their existing customers take about 6 months
before going to a competitor. The Marketing department was thrilled. They
spent $17 Million coming with a plan to tackle this. It has been a few months
since then; and they have not lost a single customer. This is a powerful
proprietary competitive weapon for them now.

\- Example 2: I analyzed 10 years of power meter reading data for a large
utility company. I combined it publicly available data sources of power
consumption of major appliances and census data on family composition/wealth
for various neighborhoods. I was able to reliably predict the lifestyle of
every family, down to whether the person living in the house streamed a movie
on Friday evenings and a whole lot more. So the company decided to use this
analysis to change their Direct Mailers with very specific, personalized
offerings. Their response to the first test mailer sent to 10,000 people?
Twenty seven percent!!! They predict that a significant portion of their
profits would come from DM's.

~~~
lifeisstillgood
I am ... stunned.

What level of sophistication did you have to reach in the shipping company
case. Was it, hypothetically speaking, the client starts using more than one
other alternative shipper (fairly low sophistication) or was it multi factor
stuff at levels of statistical skill that you need a Phd to grasp?

And thank you - these one off comments are why HN is such a valuable place to
contribute to.

~~~
lifeofanalysis
It was very sophisticated analysis. But let me emphasize two things:

\- When you start, you are taking a leap into total emptiness. You explore a
thousand different avenues, most of them are dead ends. Your day consists of
massive amount of mental effort to stay focused, to stay sane, and to make
good assumptions. Then you change tack/analyses continuously. You keep
adding/deleting datasets. This is not theoretical statistics but sometimes you
use arcane things; so you definitely need to be very strong at Stats. My
personal dream is that one day, when I have time and money, I will use these
methods to come up with a Meta- Statistics approach to empirical analysis --
i.e., to use Analytics to predict, based on the problem definition and
available data sets, which methods to use.

\- You have to have patient clients. Jumping 10 months into a project with
absolutely NO guarantee of success is a huge leap of faith, financially
speaking; but the rewards can be huge. One day you are still nowhere, and
literally the next day, everything clicks, you check your conclusions once-
twice-thrice, make a presentation to the client, and, boom!, your project is
over.

~~~
lifeisstillgood
Thank you. This does sound like an entire industry waiting to exist rather
than a clever one off.

So how are you planning to go from tenuous projects to repeatable revenue
(sorry someone asked me some hard questions today - paying it forward!). I am
guessing that a 5bn dollar shipping firm that just stopped multi-million
clients walking out the door is getting some serious payback - even if your
daily rate was enormous.

So what about things like multiple clients simultaneously (hiring interns),
building a capability maturity matrix to help clients increase their data
sophistication (and naturally you charge a monthly retainer)

Just interested in knowing where you are taking this.

~~~
lifeofanalysis
No, no such thing for me. I like to be the independent, incorruptible voice.
Besides, I would hate dealing with employees. Man's gotta know his
limitations. :)

~~~
lifeisstillgood
Enjoy your freedom :-)

------
ThePhysicist
Honestly, I think a good tech journalist should be able to critically review
what his interview partner claims and not take it for face value, which the
author of this article seems to do most of the time. Today, all start-ups that
do any kind of data processing advertise themselves as "big data" companies
that use "advanced machine learning", but from my own experience most of them
rely on pretty trivial algorithms behind the scenes.

Also, some of the numbers in the article really make you scratch your head:
Achieving more than 95 % accuracy when ranking a large number of student teams
in an eight-months long business plan competition, based solely on the results
of a simple online questionnaire taken at the beginning of the competition?
This just seems too good to be true considering the data sources they have at
hand, even assuming that they use the most advanced machine learning in the
world.

Of course, if you test your algorithm many times at different competitions you
will achieve a perfect or near-perfect prediction accuracy for some of them
(by pure chance), which however doesn't mean that you can achieve this kind of
accuracy consistently (which is where the business value lies).

~~~
jqm
"Achieving more than 95 % accuracy when ranking a large number of student
teams in an eight-months long business plan competition, based solely on the
results of a simple online questionnaire taken at the beginning of the
competition? This just seems too good to be true."

My first thought exactly.

------
klunger
A few months ago, I met the founder of a company in the recruiting business.
They aggregate online profiles of people, both applicants and other people
happily employed at their current jobs. Then, based on the combination of
their LinkedIn, StackOverflow, Facebook, FourSquare etc, their algorithm ranks
folks according to the desired characteristics for a given position.

It sounds interesting in theory. That is, until I got to asking about how they
quantified softer qualities that employers look for, like an applicant's
social skills or potential for a client facing role. Apparently, to determine
this, they look at the number of "check ins" people do at locations that are
not their home city while employed. Their algorithm assumes that the person is
traveling for business and is therefore trusted to meet customers.

There are so many assumptions in this one example that it makes me question
the integrity of the whole system. An algorithm is only as good as the person
designing it. Maybe Evolv really is better than these guys at finding
quantitative markers for softer skills, but I remain skeptical.

~~~
Balgair
And what if you have 2/4 of those? Or have their privacy to max status? How do
they account for those? Like you said, the algorithm is only as good as the
writer.

~~~
klunger
Actually, I asked about that, since my privacy settings is at the max on
Facebook and maybe I used FourSquare once? I also never anchor Facebook posts
to a location.

His response was basically that they are "trusted partner" (quotes because I
can't remember the exact term, but that sounds right) of Facebook and so they
get all the data somehow. Maybe they pay for it? I am not sure how it would
work because it couldn't be anonymized for their service.

Anyway, he basically said that, when you combine the Facebook location based
posts and FourSquare checkins (and probably location anchored Tweets too as
well as others I am missing), there are so many millions of these happening
everyday that some of those people are bound to be qualified for the position
a company is hiring for.

...Which brings us back to problematic assumptions. Garbage in, garbage out.

~~~
Balgair
Well now, I was unaware FB was selling my data in this way. Looking back, it
makes sense, but still, that's a little chilly to me.

But yes, you end up selecting people that:

1) Have smart-phones

2) Use them, and not have their kids mess with them

3)Use many of these services

4)Are not caught up in the SV non-compete bubble

5)Are good enough nonetheless

6)Are looking

This limits the use of this idea incredibly. Not to mention all the regular
Federal issues of race, sex, creed, and orientation. If I were hiring, unless
this were incredibly cheap, I'd stay far away from it. It's just plain creepy.

------
Fr0styMatt
As someone in the job market right now, I'm refreshed by the article's
admission of how much chance plays a role in whether you land a job.

I'm currently working on my CV and there's about as much conflicting advice as
to what the 'ideal CV' is as there is about what the ideal diet is. It's
somewhat frustrating but at the same time it just shows the amount of chance
and variation involved in the whole process.

At least in the industry I'm going for (games / Unity3D for what it's worth),
actions & side-projects seem to speak louder than words.

------
ytNumbers
Today's Dilbert is forecasting where this new approach to staff efficiency is
heading.

[http://www.dilbert.com/dyn/str_strip/000000000/00000000/0000...](http://www.dilbert.com/dyn/str_strip/000000000/00000000/0000000/200000/10000/6000/900/216903/216903.strip.sunday.gif)

------
coldcode
While I like the idea, I don't care for the side effects. If everyone uses a
big data based algorithm, and you are an outlier, you will never get a job.

~~~
cruise02
That would only be the case if everyone used the same algorithm with the same
inputs.

------
jqm
Probably this science is in it's infancy, and as data grows and algorithms are
refined, it will go places we can scarcely imagine at the moment.

I can hardly wait for the day when a baby is born and "garbageman" or
"engineer" is stamped on it's head and it becomes futile to argue with
empirical truth about "best fit".

Actually, kidding aside I think companies and employees may both benefit from
this research if it is applied properly. But if a better form of these tools
were available to our current system it would likely produce some very bad
effects. I think maybe we have some important decisions about humans and their
role in society coming up soon. Because technology never seems to go back in
the bottle.

------
lifeisstillgood
I love the reverse idea here - take an assessment test and then release that
data out to the public - presumably as anonymised as possible.

Then folks can take the same test and see if they should run screaming from
the interview

------
morgante
I stopped at "collecting the data of all our private moves on the internet and
applying their logarithms accordingly." Apparently programming is all about
writing logarithms.

~~~
walshemj
And your not worried about collecting private data about job applicants?

~~~
seabee
Seeing shady behaviour isn't a reason to stop reading. Seeing bullshit is.

------
ewinters123
"Past performance is not a reliable indicator of future results" this
expression should be used more often when data has a big human element
attached to it.

------
waps
Another way for big companies to more accurately exploit their workforce under
market-based conditions. If this software goes through, the only possible way
to get a raise is if the program believes not getting a raise would actually
cause you to quit. There's only one reliable way to make the program think
that : make sure you actually would quit if you don't get it.

In other words: expect a massive increase in job hopping as people find the
only possible way to negotiate with this program : quit.

> By morning, he says: "If a customer has thousands of people in similar job
> types, our system can predict accurately on a given day which individuals
> are most likely to quit." In response, Evolv then offers employers "what-if
> types of analysis" by which if they change certain incentives – a bonus,
> training scheme, change in environment – they can see exactly what effect it
> is likely to have on a particular person's behaviour. In this way Evolv
> advertises average reduced employee attrition rates among its clients, who
> include one fifth of Fortune 100 companies, of up to 15%.

This sounds horrible. It would force employees into quit-to-improve-working
conditions dynamics. Constantly interview, at a non-ridiculous rate. If you
get offered better conditions, either Evolv will offer you the same at your
current position, or you should quit.

Of course that's already mostly true : my advice working for a fortune 100
company that isn't Google or Facebook : prepare to quit after 1 year or less.
Regardless of whether you want to stay or not, have a serious discussion with
your boss about quitting after 6 months at most.

I wonder if it would defeat the negotiation tactic used by "Evolv" here. If
you can call it a tactic, that is.

~~~
icegreentea
Once you have a sufficient percentage of a job market gated by these types of
algorithms, the individual worker literally cannot win. Adversarial approaches
to 'beating' the algorithm can be detected and compensated for within the
algorithm. Ironically, it'll just be right back to forming unions, except
you'll be negotiating against algorithms.

Seriously, almost any type of 'optimization' on the half of
companies/employers right now is bad news for employees and job seekers. We
call it optimization because its nice and clean. But the effect of every
optimization is to squeeze out every last bit of productivity out of a given
work force as possible.

Honestly, I hope this type of thing is never developed and deployed to the
degree to which some of the proponents in the article wish it to. They can
paint starry eyed pictures of a future where everyone gets to work the job
that fits them the best, but all I can see is a future where everyone who has
a job is scared shitless of losing it, since it'll blackmark them forever.

~~~
jib
Thats a highly cynical view. There's plenty of optimization problems that
aren't about squeezing out productivity at the cost of anyone, but rather
about improving productivity and employee satisfaction at the same time.

For instance - lets say that I realise I have a problem with turnover that I
want to fix, because my costs of replacing staff are too high. To address that
I want to spend 1M on retention activities in a year. Should I spend that on
additional vacation days, or should I spend that on extra events for the
staff, or on extra training opportunities? If I spend that money, will the
impact on my turnover costs be positive enough to warrant the spend?

That's the kind of discussions that always pop up. Being able to quantify the
impact would make it easier to do the right thing that both benefits employees
and the employer.

~~~
icegreentea
Oh, I agree that there are ways to improve productivity and employee
satisfaction at the same time, and some companies might choose to do that.

But let's be real - there are likely more people that jobs that they are
willing to do in the US for the short to medium term. As the 'jobless'
recovery showed, companies were perfectly able to squeeze out additional
'productivity' (ok, I'll agree that the way we measure market performance
isn't really that great, but it's the metrics that we largely agree to play
the game by) while cutting labour force, and maybe even per-worker pay, and
likely driving down employee satisfaction, other than how glad they were to
still have a job.

As long as 'hey, look be glad that you have a job' is a legitimate threat, for
the majority of workers, they're really out of luck, cause again, let's be
real. The right thing that benefits both employees and the employer is not the
same thing as the right thing that benefits both the employees plus the guy
you just fired and the employer.

------
michaelochurch
_Evolv data undermines certain truisms, among them the idea of the serial job-
hopper. "The number of jobs you have previously had," Simkoff says, "and even
whether you're employed or not at the time of application, has zero
statistical correlation with how successful you will be or how long you will
stay."_

The job-hopper stigma isn't about imputed low skill or merit. It's about
social status. The person who is presently unemployed has (temporarily) low
social status. The person with 5 jobs in 6 years, it is perceived, failed to
achieve high social status at any of them.

The problem with humans is that most don't make decisions based on value-add
potential, but on social status. They see Harvard on a resume and want to hire
that person, to be socially "closer" to Harvard. It's not about whether
Harvard graduates are better hires or not; that question is irrelevant.

Job-hopping might seem like it could be a high-status behavior, in that the
best people get bored quickly and always have other opportunities, so they
don't put up with abuse. After all, the serially fired job hoppers are maybe
1/10 of that set. It's not so, because the people who make hiring and
promotion decisions are in corporate in-crowds, and part of being an in-crowd
is the necessary assumption that _everyone_ wants to be in an in-crowd. The
job hopper may be individually excellent, and it may be that he'd be a 5+ year
employee if given high-quality work and colleagues, but all his paper says is
that he never stayed long enough to join a corporate in-crowd, and that even
if he was invited into one, he made the "wrong" decision to leave it.

~~~
dkarapetyan
I have heard people say that serial job hoppers have commitment issues which
is a little weird because why would anyone commit to a shitty job. I tackle
the issue directly by addressing it honestly when people ask why I'm looking
for a new job only after 6 months. I guess so far I've given good enough
answers.

