That's what's so mind-blowing to me about our terrible process today. It aggressively tries to find and dispatch unqualified candidates, so much so that it forcefully repels talented people. And it fails miserably at screening out people who can't code! It is the worst case scenario of processes.
It's also really important to remember that the problem is People Who Can't Code, not "People Not Smart Enough To Code". Time and again I've "worked" with people who could very effectively critique my own code, who could participate in design meetings down to the bits-and-bytes layer, and, having taken responsibility for actually implementing some portion of those designs, could not commit usable code to save their life. I don't know what the deal with these people is, but they are out there: plenty smart, totally fluent, and completely ineffective.
I was talking to Erin this weekend about the phenomenon and realized that these people not only exist, but are actually favored by the way the market works. A smart person can last many months in a dev role before anyone realizes they're ineffective. Then, it takes many, many more months to manage them out of the team. By the time the story plays itself out, the bad team member has more than a year on the job, and when they parlay that resume line item to another prominent job elsewhere, their previous teammates aren't outraged but instead relieved. They end up with gold-plated resumes and tons of credibility.
1) Young, fresh graduates who still remember well the CS material. Or those who have a lot of free time and can study for these interviews (for example look at this Quora link http://www.quora.com/How-much-time-did-you-spend-preparing-f... the first reply mentions spending 28.5 hours per week for 5 weeks, which is 142.5 hours total) From a senior software engineers perspective that's a ridiculous commitment for something where an arrogant asshole interviewer can shutdown a whole onsite interview process because they're having a bad day or have some favorite question they think is the One True Question for determining if someone is a software engineer.
2) The software industry tends to overvalue youth. It has a very high salary at start, but also a very quick salary cap. And most companies have no technical ladder, and even those that do it's a joke compared to the manager ladder. All of this is not surprising. Since the industry is so heavily biased towards hiring and rewarding youth, older experienced engineers are not valued.
3) Senior software engineers have heavy pressure to leave the software field. I'm feeling this now. I'm a senior at my current place and my salary is basically capped. There would be little to no benefit going to another company. And even if I wanted to, I have a family now and don't have
the free time to study for software interviews. My options are basically go into management, consulting, or start a business. I have no desire to start a business or go into consulting because I simply don't like doing those business related things. I want to be an engineer not a business person. But if I just stay an engineer, I will probably get laid off someday due to ageism. And if I'm an old engineer who is laid off, job mobility is basically gone due to ageism and the CS heavy interviews that I don't have the time or motivation to study for. So the only safe path is management. A lot of senior engineers face this decision. Which again reduces the number of mentors and experienced engineers to teach the new younger engineers.
4) And doesn't anyone notice just how wrong whole concept of studying for an interview is? There's a whole book publishing industry dedicated to technical interviews such as Cracking the Coding Interview. The technical interview has become like the SAT. Many could just study to pass it, but be horrible engineers.
(I'm having a pretty bad day, which I hope doesn't show through too much in that summary).
In that context, "give take-home tests" and "ask better questions" sound like the exact opposite of good answers.
I think this is a problem that's unique to very large corporations. Not only do thousands of people interview there, meaning questions and answers are likely to be traded verbatim online, but there's also actually an incentive to cheat your way in: you can hide out for 1-2 years and acquire a horde of cash and resume fodder.
Google have actually made this problem worse for themselves, too. By overvaluing their interview process, they've made cheating even more lucrative. Cheat your way into even an offer there and you have something valuable to put on your resume!
I suspect you haven't seen a lot of cheaters on take-home tests before because at a company like Matasano, there's little incentive to cheat your way in. You certainly can't hide out in a corner doing no work there.
Google is no more a proof of the value of algorithm tests than Github proves the value of manager-free organizations.
I was a bit horrified at the acquisition coaching comment. I did not realize that happened but of course it makes sense. Can't manage an acquihire if your devs can't pass the algorithm tests at the big acquirers. Shame.
It is actually more profitable for me if I say that these algorithm tests are BS and completely study-able, and even the worst engineer can pass them. That easily translates into "here! buy my book and even YOU can get a job at Google!"
It's actually a much harder sell when, really, it's more like studying helps someone perform better, but only to an extent.
Oh, and the acquirers encourage studying and prep, just as most of the top tech companies do for their normal dev (and non-dev) candidates.
To be clear: my experience is that the algorithm interviews, when done properly, work reasonably well to identify intelligence/problem solving and coding skills. Those are more important at some places than others. They can work well for certain places -- but not at all. I do not think that all companies should use this process. It's not right for all places, but it's reasonably good for some.
As for Google being proof of the value of algorithm test: I think you're pulling that from a twitter conversation very much out of context.
Context: Some people were arguing that the algorithm interviews offer zero value - that you'd be at least as well off, if not better off, hiring at random.
If they truly believe that (and they appeared to be sticking to this argument), then it's absolutely relevant that not just Google, but basically all the top tech companies, use this hiring process. Could a randomly selected group of engineers build the technology at companies like Google, Facebook, Amazon, etc? I suppose that's what they must believe, and I find that pretty absurd.
If they had argued that these interviews sort of work, but aren't nearly as good as some other interview processes, then you're right that the company about Google and other companies wouldn't be relevant. And I also wouldn't have brought it up.
Given that there is no correlation between job performance and interview score, are you saying this based on your gut instinct, or do you have objective data somewhere?
My best estimation is that the Google interview process is just a glorified fizzbuzz presentation, and that the hiring decisions are more or less made by arbitrary factors, including the nebulous "culture fit".
While it may be amusing to work through the details of doing a merge sort on a linked list in 30 minutes, it's certainly not germaine to the quotidian experience of an SWE at Google.
That's not a correct assumption. I suspect that you are referring to the (infamous) Google studies on this that were alleged to show this. The articles about this got many details wrong, and the study was fundamentally flawed. Remember: they only studied the people who did very well. That's like finding no link between exercise and health when you only study people who run regular marathons.
Merge sort on a linked list is a bad interview question, so I agree with you there. Good questions are problem that actually make you design a new algorithm.
I disagree with your assertion that lambasting Google's hiring practices is an easy sell in the market. The last time I tried I was dismissed out of hand by a talent consultant.
That... wasn't my assertion at all.
This has been going on for ages. A lot of coding bootcamps and similar things are just interview prep programs. A lot, I mean a lot, of people I have worked with can nail interviews but can't do real work for shit.
Unfortunately to land a job, you have to go through this process. Developing interview skills is parallel to being able to do actual work.
How? If the person is not delivering stories it's pretty obvious. I've seen a few people get the boot after three weeks.
Team accomplishments can be embellished, stronger team members can cover for the weaknesses of weaker team members, but admitting that a decision you made was incorrect gets punished. It's obviously better to hire someone competent in the first place, but the next best alternative, for the manager, is to live with their bad hiring decision.
As engineering managers, we were expected to "manage out" about 10% of the lowest performing staff on a yearly basis. The way that worked was basically that the 10% under-performers were to be put on strict personal improvement plans, and were under no circumstances given raises at all, and it should be made clear that no raises would ever come their way unless they improved.
That was it. Sounded harsh if you were used to 10% a year raises, as many of the good performers got. Not so harsh for those who were used to live in fear of getting "exposed" and fired.
And as a manager, if I were to designate a lot of my team as under-performing come review time, it'd reflect badly on me, so managers also had an added reason to push for good reviews for their reports: You'd end up getting paid more by getting good reviews yourself if your reports were helping create a good impression through rankings that you as a manager could trivially fudge (we did "360 degree reviews" were the manager and report could both designate some co-workers to review the person, which was of course trivial to game for those who wanted).
There was also an implicit expectation of a bell curve. If you had an under-performing team, that'd be awesome news for at least some of the weaker members on your team (and conversely, I several times had reports who got short-changed because I was told too many of them were designated "above expectations" even though my manager and other people who had worked with them agreed, and so several of them were ranked lower to get the "correct" spread - it was absolute lunacy).
Unsurprisingly, come hiring time, I felt like we were under siege by an army of ineffective developers hoping to find refuge somewhere safe and hard-to-get-fired.
There are scarily many places like this for ineffective people to hide. Sometimes for many years.
I ran into a C programmer that did not get the relation between pointers and parameter passing. It was some scary awful code. Damn prep for code review almost got me in a car wreck.