
Is there a STEM worker shortage? - gandalfgeek
http://blog.vivekhaldar.com/post/61027056759/is-there-a-stem-worker-shortage
======
ekidd
I've sat in front of enormous piles of resumes, desperately hoping that one
candidate in 10 might be able to code fizzbuzz or reverse a linked list. Yes,
the candidate pool is awful on average, but this is a survivor effect—the good
candidates tend to get hired, and a handful of unqualified candidates keep
spamming their resumes.

But as an economist once told me, the market clears. If you're realistic about
the market price, you can absolutely hire qualified people. That's practically
the definition of "market price." Price signals work; when programmer salaries
get ridiculous, plenty of competent people take an interest in programming.

You can save money by offering other benefits, hiring outside Silicon Valley,
being better at evaluating candidates than your competitors, not
discriminating based on age/gender, or by figuring out how to work with
talented remote experts. But you have to make an interesting offer if you want
talented people.

------
anovikov
Easy explanation is that there are a ton of people holding a STEM-related
diploma but absolutely inept in this profession not even trying to compete. I
know that from every class of software engineers in Russia, couple guys make a
successful career, couple more become mediocre coders, couple more try for a
while give up, rest not even bother...

~~~
SomeCallMeTim
Exactly. I've interviewed recent UC Berkeley EECS [1] graduates and found they
couldn't write a line of code, and Berkeley is a top 10 engineering and top 10
computer science school (or at least was a the time; haven't checked rankings
recently).

[1] Electrical Engineering and Computer Science; their B.S. degree in computer
science. I think they have a B.A. as well.

~~~
angdis
Come on... do you _REALLY_ mean they can't write a line of code? I find that
hard to believe.

~~~
SomeCallMeTim
Probably given very controlled circumstances of a language they were really
familiar with, they could have written something.

But they took my programming test and got a big fat ZERO on it. Some of the
questions were very open-ended ("What do you NOT like about your favorite
programming language?"), and I didn't even know they HAD a wrong answer until
these two recent grads took the test, but they were very creative. ("I can't
think of anything.")

The only code I actually asked them to write was in a very simplified language
that was completely defined ON the test. They only needed to know IN GENERAL
how programming languages worked. And I'm not talking about picky syntax
problems, but a lack of even an attempt to write the code.

FWIW, the people I hired tended to completely ace the test. The one time that
I ignored the results of the test because the guy talked well and seemed
otherwise like a good fit, it turned out he couldn't do anything we'd asked
him. There didn't seem to be many in the middle: Either they really got it and
could ace the test, or they really didn't and they'd bomb badly.

This isn't news; haven't you heard of the Fizz/Bang problem? [1]

[1] [http://www.codinghorror.com/blog/2007/02/why-cant-
programmer...](http://www.codinghorror.com/blog/2007/02/why-cant-programmers-
program.html)

~~~
anovikov
Really curious, may i have a look at the test? I was hiring people for almost
all of my career (on my first job, my first boos quickly realized i was too
good to just let me type code) and i know things are really bad, but not as
bad as you put it... Maybe your test is too hard?

~~~
SomeCallMeTim
My test is obsolete, AND is likely designed to get a different kind (class?)
of programmer than you need (statistically speaking; I have no idea what you
do).

At the time I was doing video game development. I still am, but WHAT you need
to know as a game developer is different now than it was then. For instance, I
still wanted people to know SOME assembly language concepts (and Berkeley EECS
grads were required to take an assembly language course, so that wasn't
unreasonable).

Game development is still harder than most kinds of software development
("most" defined in terms of number of people who practice each kind), but the
specific knowledge required has changed a lot. There are probably awesome
developers today who have never tried to code a line of assembly language.

This fact makes me sad, actually, because I feel that my grounding in assembly
language gives me a perspective on WHAT the computer is actually doing that
developers who have never seen a language lower-level than C can't really
have. And I feel it's a healthy perspective to have. But I'm not filtering out
applicants based on whether they know assembly language. At least not any
more.

Seriously, though, one of the questions was "What do you not like about your
favorite programming language?" At the time the question may have been "What
do you not like about C/C++?"; I can't remember when I changed it. This is a
question that is designed to gauge what programming concepts the developer in
question is challenged by, or what corners of the language they have trouble
with.

If they answer "Pointers give me trouble!", then they're too green, but if
they answer "overloaded template and normal function selection priorities in
C++ are too hard to remember," then I know they've been doing relatively
complex things with the language. So it's a question designed to give me a
glimpse of their current skill and expertise; none of the answers I imagined
people would say are "wrong," even though some might make me not want to hire
them.

But to answer "I can't think of anything wrong" showed such an amazing lack of
imagination (or fear of revealing the above) that it was the only time that I
felt they'd managed to give a "wrong" answer to the question. If you can't
come up with anything wrong with a programming language that you are saying
you're skilled in, then you're not sufficiently analytically minded to work in
games. Or at least not in a small indie studio where everyone needs to be
awesome at something.

------
robbles
One issue I keep seeing in these kind of studies is the use of the "STEM"
acronym itself.

I'm sorry, but with the current job market, this kind of grouping makes NO
sense. The jobs and opportunities are mostly in the middle two letters -
technology and engineering.

Science and mathematics are still tied to academia enough that only the real
go-getters with the ability to self-educate and work out how to transfer their
skills to other fields will easily find careers. Despite the flaws in most of
the institutions offering engineering degrees, an education in
engineering/technology is still the most straightforward path to a career,
with the most applicable job skills being taught.

~~~
drewying
I have a good friend who is a mathematician, masters degree. She works for an
insurance company doing statistics as an actuary.

This is just based her anecdotes, but from her stories it sounds like the
world of professional mathematicians/staticians is suffering from a talent
shortage that completely dwarfs what we experience in the technology sector. A
good mathematician is rare gold in the finance/insurance worlds right now.

~~~
robbles
That's a great example - Your friend sounds exactly like the people I
mentioned who "work out how to transfer their skills to other fields". I think
that these kind of people are outliers who underestimate the difficulty
involved in doing this for others, probably because they're very talented
people. What most people want to get out of their degree is a straightforward
path to a job, not a stepping stone that they can use creatively to build
their own career.

I suspect that the issue with that particular talent shortage might be similar
to the software engineering one - it hinges on the definition of "good
mathematician". If I graduated with good grades from a top university with a
degree in mathematics, will I automatically have a good chance of being hired
by a financial institution? Or do I need to somehow acquire domain-specific
knowledge first and have the right personal network?

------
Balgair
A couple of friends of mine decided to 'walk a mile in their shoes' and became
hiring managers. Craigslist is a wonderful thing, especially since it is very
cheap to post a job listing. My buddies put out a very generic job posting for
the types of jobs they were looking for. It was a way to gauge the heat of the
job market, to see the competition, and mostly to see what it was like. Their
hypothesis was at least 100 resumes. And being good scientists, they posted in
a lot of big metro areas.

14 total resumes came in, 7 from India. In 10 major US metro areas, and this
posting lasting a cumulative 3 months among the areas they got 14 resumes in.
5 Cover letters. 3 of the guys were ok, and hirable.

It was a shock. They thought they were in a really hot market and the
competition was fierce. That must be the reason they get no calls back. Nope,
not at all. At least in the Craigslist sphere, there were no job hunters to
speak of. Personally, I don't know what to make of the data either. Lots of
jobs were posted, but no-one was there and answering. The only thing that came
to mind was that MOST of the craigslist postings were fradulent, just like my
friend's.

The job market is stranger every-day.

~~~
eli_gottlieb
_The only thing that came to mind was that MOST of the craigslist postings
were fradulent, just like my friend 's._

This is the basic assumption, yes. It's why most of us don't job-hunt on
Craigslist.

~~~
Balgair
Ok, so where do we all job hunt, and how can we do this again? The info there
is of great value for applicants.

~~~
hackula1
I typically find a company I like, then send them an email expressing
interest.

~~~
Balgair
Myself as well. Also, just driving over and handing them a resume has worked
really well for me. I did this to 12 companies and got 3 interviews. Best ROI
yet.

------
gopher1
This guy linked to the IEEE Spectrum article, but I don't think he read it,
because they address the issue about defining STEM workers, and despite his
anecdotal evidence, they came to the conclusion that, yes, the STEM shortage
is a myth.

~~~
bowlofpetunias
No, they don't address the issue in that article, they carefully circumvent
it. Like all shortage-denial articles, they focus on education rather than
actual practical competence.

There is no shortage of STEM graduates. There is a shortage of people actually
capable of doing the jobs which, _amongst other things_ , require a STEM
background. And for most software development jobs, the latter isn't even the
most important qualification.

The STEM shortage is a myth created by people who want to deny that there is a
shortage of qualified knowledge workers by equating "qualified" to having a
STEM degree.

Nobody actually in the business ever complained about a STEM shortage. We
complain about a shortage of competent people.

~~~
hackula1
> We complain about a shortage of competent people.

As an aside, software development is one of the most internally brutal and
self-critical industries out there. We tend to consider anyone outside the top
20% to be entirely, unredeemably incompetent. It creates a huge drive in our
industry to keep getting better, but I do wonder if the d* measuring contest
might be holding us back at the same time. The ridiculously competitive nature
of our field has already been attributed to unintended consequences like
blocking out women and older people. I personally was drawn to the field due
to its competitiveness, but I do wonder if sometimes we let "perfect" be the
enemy of "good".

~~~
noonespecial
>We tend to consider anyone outside the top 20% to be entirely, unredeemably
incompetent.

The trouble is, in software, single individuals in that top 20% are capable of
doing things in a short time that individuals from the bottom 80% couldn't do
in 1000 man years. Software is funny like that, some people can think
different _kinds_ of thoughts.

So in a way, for certain values of incompetent, this is actually true.

~~~
eli_gottlieb
In which case, merely adding more software developers won't help the problem.
You need to add more people of the quality of the current top 20%, thus
expanding that to "top 30%", "top 40%", and eventually the actual majority,
hopefully.

~~~
noonespecial
I'm not quite sure why I collected the downvotes, but you're spot on. For
example, I'm short and relatively uncoordinated. I have a few chronic physical
problems in addition to this that make me "unredeemably incompetent" at the
sport of basketball. I could work my ass off for the rest of my life and still
never come within shouting distance of Michael Jordan. I simply cannot perform
at an NBA level no matter what I do.

If there were a shortage of talented basketball players, it wouldn't matter
how many copies of myself you were able to get into basketball training
programs, you'd never get an MJ. We need a better system to find MJ's of STEM
and allow their talent to work for us.

------
daurnimator
This sounds to me like the issue is more university programs: people
completing a STEM degree SHOULD come out of it able to take on a STEM job.

From the hiring side, I agree with the dearth of quality candidates, and on
the job training is often the current solution.

From my own university experience: the wrong things were being taught, and
depth was pretty much out of the question. I can only imagine its worse
elsewhere.

Only due to my personal side projects did I end up getting the knowledge
needed for a job.

~~~
epoxyhockey
_This sounds to me like the issue is more university programs: people
completing a STEM degree SHOULD come out of it able to take on a STEM job._

To be fair, college is not trade school, nor is it supposed to be trade
school.

 _on the job training is often the current solution_

10-15 years ago, this was standard operating procedure for college grads, as
were decent training budgets to keep all employees up to date on skills. Only
more recently have employers demanded a turn-key candidate while training
budgets have been cut back or zero'd out.

It's pretty worrisome for the future. Many individuals in this thread
complaining about lack of qualified candidates may find themselves unqualified
to a future employer if they don't value training.

------
trekky1700
It's a concerning trend. Lot's of people go into the fields looking for high
salaries and job opportunities, and then have no interest and thus talent in
the field. There's so many totally inept people with even graduate degrees in
computer science.

At the same time, look into some of the top University's CS exams and they're
writing code on paper. They're not training effective and talented
programmers, they're training effective exam writers.

~~~
dnautics
This problem exists in the "hard sciences" too. There happens to be an intern
in our lab, who is very motivated to do science, and actually fairly
competent, but there are just some really critical things that are missing she
just can't do basic mental arithmetic - it takes her minutes to figure out
what volume to add to do a 1:20 dilution. Then she writes it down so that she
doesn't have to do that calculation again in the future. She's always asking
to make sure her procedure is correct, and has never creatively come up with
an experiment on her own. This is compounded by self-doubt, there are
procedures that need to be done _extremely quickly_ , because there's an air-
sensitive compound, and I've told her she needs to be snappy, and if she needs
to, practice it 20 or 50 or 100 times to get the whole thing done under a
minute or two, still no, and I think it's affecting the results, which of
course, increases her self-doubt, so she tries to be more careful, which makes
things slower, etc etc (she's not my intern so I can't be too aggressive in
giving her criticism). This student has a master's degree and was applying to
PhD programs at Harvard. I think Harvard rejected her, and the best grad
school she got into had "people worse than her" (which I don't doubt, like I
said, she's competent overall), so she's applying again this fall.

I can't take her aside and tell her, "you are not PhD material". I leave it to
the reader to figure out why. She would be fine in industry as a lab tech. I
presume something similar happens at a lower level where people who are techs,
shouldn't even be at that level.

Contrast this with a high school intern in the other lab, who successfully
built a giant 6 meter bioreactor that is actually being used to process waste
at an industrial facility, and totally took down a professor that asked a dumb
question during her "presentation on what she did".... I am terrified that she
will leave STEM - she didn't get into as good a college as she should have.

~~~
trekky1700
Wow, I didn't realize it was quite so bad everywhere. This really shows how
desperately reform in the education sector is needed. Especially the last
part, a student brimming with talent turned away and potentially changing
fields because of it. That's scary.

~~~
dnautics
maybe I'm being pessimistic, she got into an good school that I respect, she
just - deserves more.

EDIT: I guess I could be more specific.

In the state of california, there is the Cal State system and the UC system,
the UC system is considered to be the "top tier" system, and the CS system is
widely considered to be "slightly less than". She got into a CS engineering
program, that I happen to think creates a substantial number of engineers that
are superior to some of the UCs, but that may or may not be the appraisal of
people who hire her down the line, especially as we move towards a more
credentialed system, or if the system gets flooded with less competent
engineers and candidate selection trends toward the superficial.

~~~
dschuler
I went to a Cal State for undergrad, and had a great experience. None of my
engineering or math classes were 'seminar-style', I always had access to my
professors, who generally knew me by name. Most of the students come out being
able to put _something_ together on their own. I ended up going to to a
private school for my master's, and was surprised to see so many people who
wouldn't know which end to hold on a soldering iron. BUT - the student body
was generally much more engaged and interesting, and I ended up having a much
better time there. I just wasn't blown away by their academic superiority :)

~~~
dnautics
without naming names, there are DEFINITELY programs/whole schools in the UCs
that are worse than many Cal States, in my opinion. Nonetheless, my opinion in
this world doesn't count for very much.

------
dnautics
>The flaw is in the assumption that every STEM degree holder is a qualified
candidate for a STEM job.

There is also a flaw in the assumption that every STEM jobholder is qualified
for a STEM job.

~~~
hackula1
IDK, I could see this the other way around too. If you are currently doing the
job, then you must be qualified for it, or are soon to be fired. There are
probably some small percentage of people who just slide under the radar, but I
would think if you were entirely incompetent and unable to do your job you
would not last long.

~~~
dnautics
sometimes it's not trivial to fire people, because lawyers could get involved.

~~~
walshemj
For gross incompetence it's not really that hard in either the USA or the UK
of course this just means your HR dept has some performance issues it needs to
address :-)

~~~
dnautics
depends on the state in the US. And let's say the HR dept has performance
issues. How do you fire someone in HR? They're the ones that know all the
rules and all the laws that could get you in trouble. Where I work slid by,
managed to can the HR head, but they were only able to do it because 1) she
was super-overpaid and there was a budget crunch - which can be a legal
problem for nonprofits (overhead) 2) there was a pretty serious SNAFU
involving federal funds that happened under her watch, so she could have been
in legal trouble herself.

Also, gross incompetence is not necessarily the problem... Death by a thousand
cuts.

~~~
walshemj
Then the board is incompetent.

------
seiji
Local anecdote: my last N interviews have gone from HR contact to phone
interview to phone interview to take home project to (being flown out for) in
person interview to ... rejection.

I'm getting worried I'm actually incompetent now.

------
MrBuddyCasino
tl;dr: there is no shortage, its just that most candidates are crap

~~~
angdis
And the real reason that the "candidates are crap" is that too many employers
no longer spend the time and resources to train their workers properly.
Instead employees are expected to get hired with insanely specific skills and
"hit the ground running" working for some clueless line and project managers
who don't give a damn about what happens outside of their small-minded
metrics.

~~~
bowlofpetunias
As a hiring manager, it happens very, very rarely that I encounter someone
worth training.

When we're talking "crap" candidates, we're talking a complete lack of both
talent and passion. It would be a waste of time and effort to try to train
these people.

Education is part of the problem. I don't expect higher education to result in
immediately applicable job skills, but I do expect it to weed out the
fundamentally incompetent.

------
tekalon
The shortage is in jobs that are willing to take graduates/applicants that
know that basics, or even has experience, and then TRAIN them on what they
want. Those that can pass STEM are usually smart enough to be trainable.
Companies are expecting schools to teach current skills (not just theory) or
students to be 100% self motivated in learning those skills on their own.

~~~
bcheung
Yes, they are expecting that. Imagine that.

------
bcheung
I'm the lead developer at a tech startup in Silicon Valley and I interview
about 2-3x a week. There is definitely a shortage of competent workers here.
We will typically extend an offer to about 1 out of every 5 candidates because
they just can't code or don't have the experience in the technologies we use.
I've learned that resumes are very puffed up and that having a college degree
really doesn't make much difference in terms of a candidate's competency. I
disagree with the article about salaries being stagnate. In startups,
especially for web developers, competition amongst companies is fierce and has
driven up salaries.

Many studies use college degrees vs open positions as their data but I think
this is a flawed model because what I have found is that college degrees and
education / competency is only a loose correlation at best.

~~~
hackula1
1 in 5 sounds pretty good to me. You could hire someone new every ~1.5 weeks.

------
ifben
"... how many grads do not have a decent grasp of elementary algorithms and
data structures, and are not comfortable with code."

This is more a fault of computer science education than the student. I was
taking 16 credits of upper division CS courses at [not renowned but not bad
university]. Half-way through the term I had yet to do any programming. I
realize that CS is about more than programming (my four classes broadly were
proofs, computer hardware, methods of software engineering, and stats for
scientists/engineers). But the only way to become good at programming is to do
it consistently. One or two big projects towards the end of the term leaves
you brushing up on old skills instead of building new ones.

I'll be attending Hack Reactor this fall because I want to become a good
programmer, not vaguely familiar with everything CS has to offer.

~~~
rizzom5000
CS isn't about programming at all. While CS can help make a programmer a
better engineer, it's not intended to teach programming skills; which is much
more a vocational skill than a science.

I think the primary reason so many development shops ask for CS grads, is not
because they should have learned programming at university - it's because
they've already been selected for intelligence by getting into a more
selective hard science university program. If programming skills were more
important than intelligence, these jobs could be filled with folks who have a
6-month certificate from ITT -- but hiring shops have evidently found out that
they get a better product from CS departments.

I kind of feel that CS departments should really emphasize these points to
prospective applicants so they can go to HackReactor instead of wasting their
time in CS programs - but unfortunately they have little incentive to do so.

------
kbenson
Is it possible that industry automation reduces the workers needed for some
positions, freeing them to look for other positions, thus increasing the pool
of candidates? That would go some way towards explaining why projected STEM
job candidate numbers are always below reality.

Automation affects _many_ jobs. Just because a job isn't replaced entirely by
a robot or program, doesn't mean it hasn't been rendered significantly easier
or less time consuming, reducing the personnel required.

CAD is probably one of the oldest examples of this. What's to say that similar
changes aren't taking place in many industries now?

------
xfour
We discuss this every other day on here, do other professions, I'm just
curious.

------
eli_gottlieb
There are several different _localized_ shortages, but also a _general_ glut.

If you're trying to hire well-trained software engineers with high-level CS
knowledge in the Bay Area, _good fucking luck_ , and make sure to pay _plenty_
because you're getting _nothing_ otherwise.

If you're trying to hire medical device engineers in North Carolina (pulling
this out of my ass, but you get the point), there is a glut of unemployed
medicine and biology grads to choose from, many with top-flight skills and
work experience.

Why do we keep employing these overly large categories that conceal more than
they reveal?

------
robotcookies
This is a view I've held for a while... that the shortage isn't with people
holding square pieces of paper with fancy latin on it proclaiming competence.
Rather the shortage is a talent shortage and degrees are a very poor indicator
of talent.

There was another article on HN a while ago and it stated that only about half
of STEM workers have a STEM degree. Meanwhile, 75% of STEM degree holders
didn't have a STEM job. Maybe the problem lies with the process of getting a
degree?

~~~
vonmoltke
I agree the university process is part of the problem, but you need to be
conscious of where the numbers come from. As the full Spectrum article
mentions, different groups have different definitions of "STEM worker" and
"STEM degree" that can vary the numbers significantly, so you may be comparing
apples to oranges and not realize it.

------
moron4hire
So what you're saying is that it's all the university's fault setting
standards too low and graduating too many students.

------
rearry
techinsurgent.com

------
rearry
vivek haldar....

------
stuaxo
No.

------
rearry
robertlindsay.wordpress.com/2013/06/04/america-rip/

~~~
eli_gottlieb
Get out.

~~~
dctoedt
> _Get out._

Meaning? (I can think of several possibilities.)

~~~
eli_gottlieb
Meaning, I clicked into that blog post of his, and he's a racist, fascist nut.
He should leave, right now.

