I am confused as to the methodology by which IEEE extracted that conclusion from BLS data. On noticing my confusion, I attempted to verify every number in the article. I have just spent an hour with OES data from 2012 and 2011, since I believe the IEEE was looking at the 2012 data released in 2013 rather than the 2013 data which be released later this year.
I remain confused as to how to generate any number that the IEEE has quoted in this article.
I have considered the possibility that the IEEE has accessed to unpublished data from the BLS for 2013. However, if I interpret the numbers they are quoting as from 2013 with the delta from the year previous, I am unable to reproduce the BLS' numbers from 2012.
I am confused by the magnitude of the discrepancy. For example, the IEEE claims that "The number of employed software developers, the largest IT occupation segment, increased by only 1.75%, to 1.1 million, a gain of 19,000." I have difficulty reconciling this with the BLS numbers from 2012, which report that employment in that segment is 1,397,780. The one-year gain in that segment is, according to the BLS, over 100,000 jobs, again assuming that we're looking at 2011 to 2012.
I have taken the liberty of repeating the analysis while intentionally making several easy processing mistakes in interpreting the 2012 data, for example looking at the numbers for only the private sector. I have not yet found any combination of data sources or mistakes which produces the numbers the IEEE is claiming.
The general form of Patrick's point is that the BLS employment statistics don't always map to our common-sense understanding of terms like "engineering" or "software" jobs. So if you want to really understand what's going on, you have to dig into the definitions and how the data is gathered.
An example that received some discussion on HN last year was the employment numbers for the movie industry. Since many (most?) people who work on movies are technically independent contractors working under contract, they are not "employees" and therefore not included in some measures of film industry "employment".
Actually, the article says the rate for software developers is 2.7%, and is referred to as "elevated."
Would it be fair to say that some percentage of that percentage probably aren't qualified to find work as developers, and that contributes to the elevated number?
Based on my experience interviewing at 3 separate software companies, I think that's fair to say.
Looking at any shortage by the number of applicants or job-seekers assumes that the bulk of the applicants are qualified to do the work.
This is not even remotely true in practice. A large proportion (over 50%, honestly) of the developers I've interviewed in my life can't tell the difference between a tree and an array, much less apply them intelligently. The majority cannot articulate the performance and storage differences between arrays and hash tables. These are the most basic qualifications for writing code.
Without excluding this massive, massive pool of yahoos, I don't think the statistics here are ultimately that meaningful.
> These are the most basic qualifications for writing code.
That seems like a very strong assertion. I don't have formal CS/data structures training; I've read about all of these structures in the past but probably wouldn't be able to answer your questions about them to the level you seem to expect. (I could tell the difference between a tree and an array, because they look different, but frankly I have no idea where it would make sense to use a tree; they aren't offered as a native type in any of the languages I use day-to-day and I've never missed them. I know a bit about the difference between arrays and hash tables because it's one of the most common criticisms of PHP, but again; to the extent that I use PHP--and I avoid it like the plague--their implementation of "arrays" using hash tables has never once affected me. It is the least shitty part of that language.)
So anyway, I confess my ignorance. And I'm very open to learning more about the structures you're talking about (seriously). But I (and many of the programmers I know) have been successfully writing programs that work for people who want it (and gladly pay for it) for a relatively long time without meeting your most basic qualifications.
When I say that someone can't tell the difference between an array and a tree, I really mean someone does not know what a tree is. The number of interviews I've conducted where the candidate has heard of trees before, but cannot articulate, even in a basic way, what they are, is depressing.
To be more clear, the basic level I'm expecting is "it's a way to store data, where one object has children objects, which in turn have children objects", or something along that theme. Really, if you articulate in some way that it's stuff pointing to other stuff, you're already further along than half the people I've interviewed.
A lot of people can't even get that far.
These questions aren't designed to defeat you, they're questions that have depth so that you can get some idea about the candidate's level of understanding of data structures. Once you get past "it's an object with children" we can go deeper - what is the complexity of fetching an object out of a tree? What of sorting? Disadvantages and advantages in application? Cycle detection? All of these are nice to haves - but the basic requirement is that you have to know what a tree is.
Cool, okay that sounds pretty reasonable. Although I've successfully interviewed candidates without even asking them about trees, and they turned out just fine. I think there is a general tendency (and I don't intend to make assumptions about your specific case) towards overestimating the level of computer science knowledge required to crank out operational code for 99% of programming jobs. I think it's important for teams to have that knowledge within them, but not necessarily replicated through every individual member. (I know, I know, "A players" and all that... but the best teams I've known or been a part of, in programming or otherwise, had a couple of A players and a lot of B's who had the potential to be A's.)
EDIT: I'm more distressed by the number of applicants I've come across who've never written an SQL query, or have no idea what a database schema should look like (never mind noSQL stuff). I generally prefer working with programmers who understand and appreciate big picture design principles over the ones who sweat the details (because in my experience the former are usually able to pick up details pretty quickly, while the latter are not always able to appreciate the big picture design decisions being made) but in bigger teams there is presumably room/requirement for both types.
How do you know SQL and not know what a tree is ? How about we ask a simple SQL design question that you can't answer without knowing about the datastructures : what operations does an index accelerate, and which are unaffected and why ... How do you answer that without using the word hashmap/hashtable ?
It is entirely possible to use a database without understanding its internals. Look, I'm not encouraging incompetence--not by a long shot--I'm just challenging the assertion that everyone who "writes code" (the words 'potatolicious used, which started this sub-thread) needs to have the same knowledge base.
I've worked with a lot of academics who write code for math and stats without any real computer science knowledge (including no more than a rudimentary understanding of the most basic data structures). I have a business mentor who makes millions of dollars with a piece of legacy web software he wrote himself--I'm certain he could tell you that certain fields are indexed in the DB because the SQL performs better that way, but I doubt he would use the word hashmap/hashtable at all and that doesn't undermine his credibility in my eyes one bit. I have friends who write embedded code all day but couldn't write an SQL query to save their life; that is obviously fine for what they do, and it doesn't make them any worse of a programmer.
My example in the parent comment was obviously relating to web programmers, where the necessity of being able to write a usable SQL query is of far greater importance than the ability to explain how the database is actually executing that query. All I'm saying is that if we are able to deflate our own self-importance just a little bit (and accept the idea that the role of an employer might also be to act as a mentor), suddenly the pool of qualified applicants becomes bigger.
