
The Myth of the Science and Engineering Shortage - vitaminj
http://www.theatlantic.com/education/archive/2014/03/the-myth-of-the-science-and-engineering-shortage/284359/
======
mehrdada
I think it is important to avoid using the term STEM and specifically dissect
it in the discussions about shortage. Computer Science is a very peculiar part
of STEM and not all of the STEM fields face the same challenges. As Hadi
Partovi of Code.org put out eloquently in his testimony before Congress[1],
there is a difference between enrollments in Computer Science and STEM in
general. For instance, in US high schools, it seems like there is no shortage
of students in biology or math, but CS is underenrolled.

[1]: [http://www.c-span.org/video/?317093-1/house-subcmte-
hearing-...](http://www.c-span.org/video/?317093-1/house-subcmte-hearing-
private-sector-stem-education)

~~~
mjn
_I think it is important to avoid using the term STEM and specifically dissect
it in the discussions about shortage. Computer Science is a very peculiar part
of STEM and not all of the STEM fields face the same challenges._

I think there's a bit of a rhetorical issue with this, though. The dominant
narrative is that we have too many people doing _soft, fuzzy, weird, useless_
liberal-arts degrees. Studying things like literature, history, philosophy,
political science. What we need is more people doing _hard, rigorous,
mathematical, technical_ STEM degrees. Studying things like physics, biology,
mathematics, computer science, chemistry.

If you admit that there is no shortage of mathematicians, though, you
undermine the whole strategy of "we need more STEM, less liberal arts",
because mathematicians are the rhetorical core of STEM: rigorous, technical,
mathematical, non-fuzzy. If it turns out mathematicians are about as useful as
historians (useful in principle, not directly in a major applied shortage),
the whole narrative fails.

To a certain extent I think the whole STEM construction is based fundamentally
on trying to hand-wave across this gap: math and physics are prestigious and
perceived as hard/rigorous, while computer programming is in demand. The union
(not intersection) of these two fields is STEM, which perceives itself as
rigorous + hard + in-demand.

~~~
Malarkey73
To add to this I was very surprised recently trying to hire
Statisticians/Mathematicians for a computing data-sciency bioinofrmatics role
to find that a couple of PhD students had no computing skills at all. No DB,
no linux, no scripting.. just stat applications on Windows. Curiously a few of
the best candidates I saw were from "soft" subjects like ecology, geography,
social science - where they have real cutting edge statistical chops but were
also used to dealing with databases, mapping tools, and modern computing
infrastructure.

------
ChristianMarks
The Silicon Valley cartel conspirators helped to depress wages and increase
unemployment. Universities have relied on cheap student and postdoc labor.
Their game-theoretic strategy is cheap talking their serfs into the belief
that they too may become masters.

ADDENDUM: downvoters can take a warm piss on a power line.

~~~
mmorett
I normally just lurk, but I signed in _just_ to upvote him. Still laughing...

------
jedanbik
I must study politics and war that my sons may have liberty to study
mathematics and philosophy. My sons ought to study mathematics and philosophy,
geography, natural history, naval architecture, navigation, commerce, and
agriculture, in order to give their children a right to study painting,
poetry, music, architecture, statuary, tapestry, and porcelain.

John Adams US diplomat & politician (1735 - 1826)

~~~
lifeisstillgood
brilliant - except the implication that liberal arts should be all we need by
now

~~~
jedanbik
Rather than an all or nothing proposition, I saw this quote as an elegant
framing of what the liberal arts are: nice-to-haves in the wake of ought-to-
haves in the wake of die-or-haves. In the popular discourse, we aren't always
afforded that sense of perspective.

~~~
lifeisstillgood
Agreed

------
stale2001
What world is this article living in? I got a job making $100k as a software
engineer, right out of college (top-tier I guess). And then I realized that I
was worth more than that, and 8 months later I'm now switching over to a job
making $150k in total compensation. And TBH, I was middle of the road compared
to my classmates.

~~~
jonrimmer
Why do you think all these silicon valley CEOs are throwing their weight
behind schemes to get kids and adults learning how to code? It isn't because
they really believe there's some social or personal value in being able to
program, it's because they want to flood the job market with developers and so
crash wages. What's sad is that many programmers are actually buying it, and
volunteering their time to help support and evangelize these schemes.

~~~
jkrems
CEOs who think just throwing more developers at a problem will solve it are
what natural selection is for.

~~~
aviraldg
More developers might not solve a problem faster, but they will drive down
wages (even for skilled developers)

~~~
ranci
There is a lot of crying and worrying about wages on here right now. If you
are from CS or SE, you don't have the right to cry about a saturated job
market driving down your wages. They are plenty high. If you want people with
real problems, look at the graphic design industry. $10/hr can get you your
pick of the litter among several applicants who have over 20 years experience
in one of the biggest cities in the United States. Then imagine what it must
be like for recent college graduates in the field.

~~~
collyw
Wages are plenty high in the States, or places like London. The rest of us are
not on anything special.

------
_Robbie
I like this article, but it does not delve very deep into why the current
perceived shortage exists. On the public policy side, there is probably a
belief that more STEM professionals will increase the economic competitiveness
of the nation, even if there is a surplus. From technology businesses'
standpoint, it is in their interests to have a surplus of STEM graduates in
order to maintain or reduce wages.

~~~
tsotha
>On the public policy side, there is probably a belief that more STEM
professionals will increase the economic competitiveness of the nation, even
if there is a surplus.

Except that it doesn't work, and it should be obvious why. There's no reason
to bust your ass in college to enter a field the government floods with
foreigners. The more people the government imports the less attractive the
field looks to domestic students choosing a major.

I can't complain about my treatment as an EE and later a software developer,
but knowing what I know now I doubt I would go into anything technical if I
were graduating from high school today.

~~~
matteotom
As somebody graduating from high school in a bit, I have to ask: what would
you do differently today? It seems to me that, as somebody who can do comp sci
reasonably well, that's my best option right now.

~~~
tsotha
If I were going to do it today the first choice would be business degree and
MBA from Harvard or Yale. I'd be willing to borrow what it took, because the
payoff is there. In the US today this is the ticket to the C-Suite.

If I couldn't get in I'd try to get an ROTC scholarship for as prestigious a
place as I could manage and graduate without borrowing money. Either business
or liberal arts, though the latter case is a minefield of worthless programs,
so it would require some research.

------
DonGateley
Time to demonstrate once and for all the fraud that the H1B visa has always
been. All it does is push older engineers out of the field so they can be
replaced very cheaply.

That needs to stop but having the best congress money can buy pretty much
assures that it won't.

------
eranation
There is no shortage of bad programmers, there are plenty of medium
programmers, it's also not that hard to find decent programmers. It's harder
to find good programmers, but much harder to find great ones. The thing is
that a great programmer can do 5X more than a good programmer, that can do 2X
more than a decent one, that can do 2X more than a medium one. It's not just
the number of features they can write in a given time or the fact they have
less bugs or test their code better, or simply can do things no one else can.
It's not even their ability to design better. It's their ability to do all
that and influence the others to follow. I think there is a shortage in that
kind of developers, and there is no school that teaches these soft skills.

~~~
x0x0
Sorry, but that's nonsensical.

1 - virtually no supposed 5x or 10x programmer makes even 2x more. If people
are really 5x, why on earth don't I see $300-$500k salaries? Particularly
given the fall in communication costs ala Brooks, it would be an enormous win
for employers

2 - at least in the bay area, there would be tons and tons of highly qualified
candidates if moving to the bay area wasn't a financial nightmare (cost of
living is horrific and the pay doesn't come anywhere close to making up for
it); and if buying a decent 3 bedroom home and having kids didn't nearly
require winning a startup lottery -- or at least enough for a good downpayment
to get to a conforming mortgage.

3 - I'm in my 30s, and in my cohort of developers I know a number (all of them
very skilled and in high demand in the bay area) who have moved to the midwest
or austin because financially they're so much better off

4 - the majority of complaints about unavailability of developers,
particularly in the bay area, are after the fact justifications to (1) cover
employers not paying salaries commensurate with the cost of living, and (2)
the ability to import (cheaper, more easily controlled) h1b labor

4a - with a side of companies ducking their responsibilities to america, the
state they are in, and their communities to help create the employees they
need. Now obviously I don't think 1-4 person startups have any such
responsibilities, but somewhere between that and google/fb/hp companies have
responsibilities to their communities and countries that bay area companies in
particular almost completely duck. For example, why isn't facebook or google,
in lieu of whinging about difficulty hiring (while illegally restraining
wages, ain't that beautiful) running hacker schools themselves? It couldn't be
because they'd rather let someone else pay for it and cherry pick the winners
(saving money two ways)?

~~~
edanm
"1 - virtually no supposed 5x or 10x programmer makes even 2x more. If people
are really 5x, why on earth don't I see $300-$500k salaries? [...], it would
be an enormous win for employers"

Some reasons you don't see higher salaries:

1\. It's very hard to measure the difference in programming ability directly.
Good programmers also tend not to realise just how much better they are, and
are generally unwilling/unable to demand higher salaries (not because they're
programmers, but because most people _by default_ aren't good at these
things.)

2\. Good programmers tend to cluster around good companies, one of their
advantages being that they're surrounded by people at their level. Think
Google, Facebook. Over there, if everyone is as good as you but making the
same as you, then you don't feel there is a disparity.

3\. Some programmers DO make 300-500k. You just don't hear about it.

4\. More importantly, some people make 300-500k in roundabout ways. E.g. some
great programmers work as freelancers and make that amount, because that's one
way to solve the problem of companies being unwilling to pay so much more.

5\. Some programmers with more business-fu start consultancies and startups,
making lots of money _that_ way in a non-obvious way.

~~~
fredophile
I don't buy the hard to measure argument. CEOs at large companies make a tonne
of money. This is usually justified by saying that they are more skilled than
the average CEO but this kind of skill is also hard to quantify. Why does this
argument work for CEO pay but not programmer pay?

If there was really a shortage you'd see salaries for programmers going up.
Instead there are documented cases of companies colluding to keep pay down.
Based on the high salaries you could argue that America has a CEO shortage and
needs to increase visas for foreign CEOs.

~~~
edanm
"CEOs at large companies make a tonne of money. This is usually justified by
saying that they are more skilled than the average CEO but this kind of skill
is also hard to quantify. Why does this argument work for CEO pay but not
programmer pay?"

Because it's much easier to measure performance of CEO's, at least
artificially. If Google does well, then you (supposedly) know that Larry Page
is doing well. That's how the market treats it, at least. But can you really
tell me that you're able to tell me which of the 10's of thousands of
engineers that Google has is responsible for that success?

Also, there's the issue of leverage. Just like a programmer can build software
that is used by millions, and therefore has a lot more leverage and creates
more wealth, so a CEO usually influences even more customers than the
programmer. Paul Graham has an article about exactly that - the idea of a
startup is to give e.g. programmers a lot more leverage than they usually
have.

And arguably, the fact that a few engineers can build WhatsApp and earn
billions is an example of that in action - getting more leverage by building a
company, proving success, then gaining the money it gives. But the average
exceptional programmer, while being 10X better at programming, won't
necessarily also be better at business or money generation, etc.

Lastly, the fact that there are cases of companies colluding to keep pay down
proves the opposite of your point - that salaries _should have_ been even
higher!

~~~
fredophile
|Because it's much easier to measure performance of CEO's, at least
artificially. If Google does well, then you (supposedly) know that Larry Page
is doing well."

Originally I wrote a much longer reply but after re-reading it it came across
as much more anti CEO compensation than I intended. I don't view short term
market movements as an accurate reflection of company or CEO performance. If
you willing to use easy to calculate but broken metrics I can supply a few for
programmers. How about lines of code? This also ignores under performing
executives getting big salaries and severance packages. No one is going to pay
me millions if I get fired.

"And arguably, the fact that a few engineers can build WhatsApp and earn
billions is an example of that in action"

I view employment at a startup like buying a lottery ticket. Someone is going
to get a large payday but that doesn't mean they're better than someone that
doesn't get one. Even with the possibility of a big payout your expected
return is pretty low.

I think I stated the collusion argument badly so let me try again. If you're
in charge of a major corporation and experiencing a talent shortage would you
tell your HR department to exclude a large portion of the local candidate
pool? That's what those companies did.

~~~
edanm
You raise interesting points. I'm honestly not knowledgeable enough to tell
you if it makes sense to judge a CEO by the short-term, or even long-term,
performance of their company. I don't think anyone really knows just how much
is the correlation there.

What is clear is that, assuming CEO's _do_ affect the value of a company
significantly, they'll tend to affect it much more than even the best
programmers, especially at a company the size of Google. That's just an issue
of leverage. So a (possibly semi-broken) method of judging whether you made
billions for the company will still spit out compensations of millions.

As for startups, I was talking specifically about the founders, not employees.
And yes, I agree that startup's succeeding, at least on the Whatsapp scale, is
very similar to a lottery. But smaller startups/consultancies can make much
more money for the programmers who start them, who have skills that the market
cares about more than just programming well.

------
PhantomGremlin
Fundamentally, I don't accept that there is, in general, a shortage of "good
people" in the USA. I responded earlier [1] to someone who thinks there is.

Instead of continuing to follow up with just that one person, I'd like to
propose an interesting "experiment" for hiring managers:

    
    
       Do an Ask HN, linking to a recent representative
       job posting you haven't been able to fill
    

I'd like to see all the relevant details, such as:

1) the job posting, of course

2) company name, location, salary range

3) are the usual requested perks applicable, e.g. flex time, work from home,
etc?

4) what kind of resumes are you seeing? How many? What, in general, has been
wrong with those people?

Then I'd like to see HN posters dissect the information. Let's find out if the
requirements are realistic. Let's find out if the company's reputation is
toxic.

[1]
[https://news.ycombinator.com/item?id=7452376](https://news.ycombinator.com/item?id=7452376)

------
Malarkey73
Are wages purely determined by supply and demand though? There is a tale I
remember about an economics professor who dumbfounds his class by pointing out
that there is a huge surplus of economics students across the country who all
want to work for Goldman Sachs. But the wages/bonuses don't fall to reflect
the oversupply.

I think the same is true for doctors and dentists. Do very few people want to
be company executives or politicians? Is there a scarcity? I don't think so.

Sometimes people are well paid because they are setting their own salaries
(many believe bankers have hijacked banks from the shareholders), others are
setting the rules within society (politicians), or interpreting those rules
(lawyers), or just getting close to those who do (lobbyists, bureaucrats,
special interests).

In short the wages of workers are dependent partly on their position of power
within society .. of which supply and demand is just one element.

~~~
jacalata
the set of econ students who want to work at Goldman is not the supply: the
set of econ students that Goldman thinks will make them money is the supply,
and I bet very few of them are unemployed.

similar: set of waiters who want to be actors is huge, set of known actors
that a studio can say 'his face on the poster will bring in an extra 50
million cinema tickets' is miniscule. That's why all the waiters are acting
for free in film student projects and Tom Cruise is a millionaire.

~~~
Malarkey73
No. I don't believe a word of that. And your analogy undermines your own
argument.

If you had said Michael Jordan or Lionel Messi ... but Tom Cruise? Really? Tom
Cruise is a perfectly good Tom Cruise (a good actor) but a million other
waiters could have been him.

There maybe Messi like geniuses in politics, banking, business but the idea
that bankers are all paid Millions because "they are worth it" is to me ...
laughable.

They are paid millions because they can get away with it.

------
dleskov
Out of all those graduates, the number of which we are told is 2-3x the number
of open positions, are really qualified to fill in those positions? Passionate
about their field of study? Intended to get a job in that field in the first
place, as opposed to getting a degree in something they can (afford)?

Our own experience is that even internships do not always yield good results.
And the level of most applicants, many of whom are recent graduates, makes me
sad. They manage to find work at some other company in the end, but only
because we are in IT and IT is big at our place...

------
keypusher
All I know is from my own experience, and that experience is that I have
recruiters calling up at least once a week and a few more via email looking
for software engineers. I never got that kind of attention when I was working
a cash register in retail, and my friends with liberal arts degrees certainly
aren't getting it either.

------
baldfat
I do not work in IT. Family and friends think I should be a millionair because
I am "so good with computers."

Worked in IT in the 90s for 18 months as IT Manager and well I went bald at
22. Did programing when I was 17 and well I woke up with ideas on how to do to
fix my issue. To much stress and to little pay for the jobs.

Don't see the change for a "good" IT job for most people. When students start
asking about the industry from people that work it they usually get a negative
response and told to do something else.

------
dblacc
I thought the whole shortage for demand argument was to do with the lack of
__talent __that was being produced, rather than simply the number of people
who had a bachelors degree.

~~~
firstOrder
I don't know how many jobs I've seen out there where they have an incredibly
high bar to hire someone, and then it is 99% CRUD work.

Some of the AI jobs at Google, yes, they need good people. Amazon needs some
good engineers at the top guiding the creation of EC2 architecture. But most
jobs ask for a much higher caliber of people than they actually need.

~~~
collyw
I don't get why CRUD work is so under appreciated here on HN.

I do CRUD work (at least I think I do - I create and maintain the database and
front end for a DNA sequencing centre). I have in in depth understanding of
SQL - my database returns some moderately complex results. I have solid
database design and optimization skills. I know Django in depth. I know Python
well. I have a solid understanding of Linux, and know enough to get Django
running on the server. I have read up a fair bit on NoSQL, but have yet to
find a suitable use case for it. I can do some Javascript / HTML / CSS.

Bioinformatics on the other hand seems to be considered a lot more brain
power, when most of it is just scripts to count stuff an plot it on a graph.
Its' (often) an order of magnitude simpler than what I do. The majority of
bioinformaticains I know have never used a debugger - so the code can't be too
complex - or they are WAY smarter than me.

Or when people talk of CRUD, do they mean the sort of thing you should use MS
Access for?

------
frozenport
How does it look for other fields?

There is a misunderstanding on how jobs are made. Consider what these look
like for history of English majors? STEM is in demand if you look at the areas
with predicted job growth.

------
michaelochurch
Well, there _is_ a shortage of competent software engineers willing to
sacrifice as much as corporate executives think they should, for what
corporate executives think such people should be paid.

In the same vein, there's a shortage of supermodels who want to fuck obese,
unemployed men with halitosis and general poor hygiene.

~~~
jshen
I think you're wrong. I have the hardest time finding good engineers, we pay
very well, we don't expect anyone to work crazy overtime, etc.

Can we make this more concrete? What sacrifices do you think is being asked of
competent developers? What pay do you think they are offered?

~~~
mehrdada
> we pay very well

How do you know, objectively? Have you tried paying twice (or more) the
current market rate, for instance, and binary search for a point where you
have enough happy hires and failed? Perhaps the market rate is artificially
lowered.

~~~
jshen
The problem isn't that good engineers turn done our offers, it's that very few
good engineers come in the door. The ones that have were very happy with their
offers.

Edit: Didn't realize you weren't the person I originally replied to. Let's
reverse your question. What is the objective basis for the claim that
engineers are asked to sacrifice to much for too little pay?

~~~
firstOrder
The objective basis that you are offering engineers too little pay is that
they are not taking your offers. Unless you don't believe in one of the first
things they teach in modern economics - that at a certain value (price),
supply will always meet demand.

~~~
FLUX-YOU
Could just be a run of bad luck or not enough advertising on their part.
Supply can't meet demand if supply doesn't know demand exists.

~~~
firstOrder
I agree about making an effort looking for people. I know nothing about the
company being discussed here. But for other companies, I know some that
contact a (crappy) headhunter or two, or who post an ad on Craigslist, and
then expect top tier candidates to roll in. For some CRUD job working in class
C office space for relatively not much money etc. They're shocked when they
don't have their pick from dozens of top tier candidates.

If your methods don't work - try going to a local programmer meetup. Chat up
some of the programmers. Encourage them to apply. I go to programmer meetups
as a programmer, and all the time I meet other programmers who are out of
work, or who are unhappy at their job and looking to work elsewhere. Most are
employed, but there's always a few who are looking.

What is happening is this - in 2008/2009 the economy was tighter, and
employers could get by with a CL posting, or talking to a headhunter or two.
Now that unemployment rate is less than 10%, you don't have dozens of good
programmers checking CL for jobs every day. You have to do a little more work
to find people. Just like THEY had to do a little more work when unemployment
was double digit.

~~~
jshen
Not only do we go to meetups and conferences, we host them. At some point you
may want to consider the possibility that I know what I'm talking about.

