Hacker News new | comments | ask | show | jobs | submit login
One hiring filter that works (chrisstucchio.com)
208 points by yummyfajitas on Apr 6, 2012 | hide | past | web | favorite | 189 comments

I call BS on this one. I'd even say that this will filter out the top level of folks (†).

If you give me a synthetic puzzle to solve I wouldn't bother applying to your organization. It's much better to craft a problem to the specific domain that your company is dealing with. Then you can test for both domain expertise and problem solving ability in one fell swoop with the extra bonus of it not seeming pointless.

† The important addendum on this is that you're not going to have the top level of folks sending out blind applications unless that person is in the process of making a lateral jump into another sub-field. And even then probably not. But if you give a top applicant a synthetic puzzle after your recruiting contacts them, expect a large portion of them to thumb their nose at you. Similar note on top folks, though this is mostly pedantry -- the top OSS projects aren't on GitHub and similar. The largest and most prestigious of the OSS projects host their repos themselves.

To clarify, I wasn't discussing recruiting Linus or Guido.

I'm actually more likely to apply if there is a puzzle. I've sent a few resumes out blindly, it usually goes into the black hole. If there is a puzzle, I figure my odds of getting a response (assuming I get it right) go way up.

Of course, if a correct solution results in a recruiter replying and asking me to submit to resumator, I'm probably ignoring your company in the future. (Yes, this happened to me once.)

"Not trying to hire Linus" is a bit of a straw-man. There are a lot of folks that fall between "Linus" and "have to actively look for a job". Probably tens of thousands of people in the programming field.

The important point that I'm getting at, however, is that the dynamics of hiring shift dramatically as you move up the skill / connectedness scale. As you move towards the higher percentages potential employees are not competing for employers, but employers are competing for hires. And the dynamics of hiring are dramatically different at that point. If you're designing heuristics for hiring, you don't want to create a low-pass filter, and I believe that synthetic puzzles are exactly that.

The point of raganwald's post was on the surprisingly high cost of false-negatives. What I'm postulating is that your suggestion is just yet another way to reduce the false-positives rather than a way to reduce the false-negatives.

If it increased the number of overall respondents as well as the number of respondents who were above a certain bar (passed a phone screen), then even if it created other false negatives, it still seems to have helped.

Good developers don't just go around submitting their resumes to every job in existence, and not filtering out false negatives was insufficient in and of itself to produce a good applicant pool. If doing something that produces false negatives (which, yes, are undesirable) nevertheless convinces enough developers to consider your company who never would have considered it otherwise, then it seems like the advantages have outweighed the disadvantages.

Obviously, an even better scenario would be to produce the same level of developer interest without also producing any false negatives. Perhaps by investing sufficient resources in networking or in making his company famous to other developers, he could accomplish this. This would still be a possibility to explore in the future.

But just because his tactic has not produced the best possible outcome does not mean it has not produced a positive one.

I don't understand. He tried one approach that didn't get many good candidates as a percentage of total applicants. He tried another approach and got more candidates with a higher percentage of good ones. One sample is far from conclusive but I don't see any BS here. Maybe its not an optimal solution for all top-level candidates but that doesn't mean its not a sufficiently good one for many.

Exactly. There's nothing stopping OP from using this method to get a bunch of applicants while also using other networking methods to reach out to the current top talent.

Most top level folks rarely respond to job ads, as they tend to get hired away before they'd realized they were ready to move, so I wouldn't worry too much about filtering them out in the first place.

"the top OSS projects aren't on GitHub and similar"

True, however if you're a contributor to a top OSS project, you'd tell me that when you apply, and send me an example of one of your contributions that you're particularly proud of (at least I hope you do, otherwise I won't know!)

Puzzles and challenges are useful when the candidate doesn't have any good examples of their work that they're at liberty to share with you. I'm very wary of hiring someone without seeing any of their work. We've been taking this approach for awhile now, and it's worked out well for us.

Why do these "top applicants" not want to do coding pre-screens? If I do a coding pre-screen, it means I will get to work with only people who are capable of doing it. That bar may be high or low, but it is often better than a bar that does not involve any actual programming. It also means that I won't spend a huge proportion of my time with the company talking on the phone to people who can't fizzbuzz, which is a pretty good perk if you ask me.

So far, 2 of 4 companies I have applied to have had a coding pre-screen and 1 of 2 companies that I have worked for have had a coding pre-screen.

I can't tell if you're being intentionally dense or just naive.

Linus copied the kernel tree to GitHub when the kernel git repo was compromised. It is not the main repo. Of the others, only Rails and JQuery could even be reasonably be called "top" projects (Bootstrap? Seriously, you list a 7 month old project with 2000 commits?) and even those are a stretch. Major projects are well into the hundreds of thousands of commits. It's ludicrous to even muse of Bootstrap being in the same league as e.g. Mozilla, WebKit, et al.

"Top" projects are better defined by the number of users than the number of commits. By this standard, Bootstrap is a very successful project.

If you define success by the number of people who have actually checked out and built the code, Bootstrap is vastly more successful than Mozilla & WebKit.

Why am I either naive or dense? I was just pointing out some obviously large and important OSS projects on github.

How to hire C players: Do what everyone else, including recruiters and HR departments, are doing.

How to hire B players: Implement serious recruiting processes and plan to spend a lot of time on it. Get creative with hacks like OP's.

How to hire A players: Identify them through never ending networking, research, and keeping your finger on the pulse of the industry. You'll never find them the same way you find B and C players.

I think hiring A players is actually more complex than that even. To get A players, or even the best B players, the most important thing you can do is to make your company somewhere those people will want to work in terms of the work, the culture and the facilities.

If you're doing that then it's possible they'll find you, if you're not then nothing you can do is going to help.

It greatly depends on what you call an A player. Linus is extremely talented and well known, but in some tiny organisations his actual programming talent may not be above average. Most programmers working for the NSA are probably about what you would expect at a large organization, but I suspect they have a few people that would rock your world.

And the thing is Google, Microsoft, and GM also have some amazing teams simply though blind luck. The kind of thing where a few friends all end up working for a manager they like on some interesting little problem.

Some people have only worked on such teams and don't understand why so many projects fail, and others have never even seen them and wonder how complex software ever get's written. But, so few people have a truly understand the range from inspiring though dysfunctional people and teams out there that talk of A, B, and C coders / rock stars becomes almost meaningless.

PS: One of my favorite Real Programmers was a systems programmer for Texas Instruments. One day, he got a long distance call from a user whose system had crashed in the middle of saving some important work. Jim was able to repair the damage over the phone, getting the user to toggle in disk I/O instructions at the front panel, repairing system tables in hex, reading register contents back over the phone. http://www-personal.edfac.usyd.edu.au/staff/souters/Humour/R... or the modern equivalent http://bellard.org/jslinux/tech.html

One thing to note is that we are talking about humans[1]. So there are no absolute A's, B's or C's. These are just values for the given _time_.

If you're a decent company, you shouldn't hesitate to invest in B,C players so that they become A players.

[1] - Of course I'm talking about humans who are mentally healthy.

I am surprised how many people completely ignore this. I see tons of people blog on and on about the bright eyed, bushy tailed genius programmer out there who needs to be acquired. I see very few about companies actually trying to make an effort to look into their organizations and seeing how they can, if not try to bring their "C" players to par, at least bring their "B" players to "A" level[1]. Although I don't have any experience training other people (apart from improving myself), I am not especially convinced that software engineering and programming skills are hard enough or impossible that relatively smart people can't be with proper coaching be helped to excel at it [2].


[1] Implicit assumption that the person is actually interested in becoming better.

[2] I understand that this takes a "toll" in productivity because you could be out there building stuff instead of helping these people improve. However, once a company reaches a certain size, I should think there is enough "boring" work for the company to staff "B" players to handle those, while simultaneously making them spend another chunk of their time working on becoming better.

> How to hire A players:

One thing missing in your list: Have 'A' leaders.

This is very important. A player under B or C leader will soon be looking at the exit sign

Yes, exactly. My advisor in grad school also managed 100+ PhDs in one of the national labs, and that was one lesson he imparted in almost the same words.

He went one step farther: If you have B and C leaders in your org, it's basically all over because few organizations are capable of taking the measures to correct that problem.

Does this conflict with the mantra, 'Hire people that are smarter than you.'?

Smarter does not mean better leader. Often great leaders are indeed intelligent, but the skills that make someone a better leader do not necessarily correlate with being "smarter". Instead, great leaders possess the ability to motivate and read people in an effective way.

Conversely, you've probably encountered highly intelligent people that have the people skills of a bug.

Management is orthogonal to code design/dev/maint.

No one (ok... almost no one) has no management nor coding skill, rather each of us has a mix of the two. The great organizations I've worked in had extremely smart people in management and in development, but the type of brilliance was different.

So yes, hire someone smarter than yourself.

In my experience it does. "Hire people smart than you" is a management ethos, which translates to delegate the hell out of a task.

But software development specially in small teams doesn't work that way. Many a times, junior programmers require direction and even help in completing their tasks. So it comes down to two problems (three, third one later)

1. Either you give someone sub-standard work to maintain that you be able to help them. This will lead to demotivation eventually

2. You accept you are not in position to help them. This generally leads to friction and a bad boss will try to hide incompetence by architecture discussions, meetings etc

Smartness is subjective. Which leads to the third problem. How can you assess skills of someone better than you? You can only go as far as grilling a candidate to your own competence. So a B, C player will only be hiring other B, C players and that will soon degenerate the organization.

What is 'smart' though? I think the mantra is basically saying "hire people who fill in the gaps". A good leader is someone who facilitates identifying and filling in the gaps, which has little to do with "smarter".

Not really. Leadership is about giving clear direction and encouraging others to rise to the occasion. That can happen if the "leader" is not as good technically as his subordinates.

More than it can happen, if your programmers are good it should happen.

A leader is spending a fraction of the time hands on that a programmer is. Given the developments in technology and the additional time the developers are getting with it, if they're not better than their leader in a few years, you've got the wrong programmers.

You bet. I never see the answer to: Why would people smarter than you want to work for someone dumber than them?

there are two classes of boss, those you look to for technical leadership and some degree of mentoring, and those you expect to do the actual "management" stuff so that you don't need to. for the latter category, i'm not looking for sheer brains so much as i am looking for good leadership qualities.

> and those you expect to do the actual "management" stuff so that you don't need to.

Sounds like your answer is: When there are items that smart people want to outsource, they often go work for people stupider than them rather than hire people to take care of it.

Do I have that right?

no, i think you're falling into the common technical-person trap of assuming that raw intelligence is the only measure of competence. i don't want to work for myself, and i don't want to hire people - i want to be competent at my job, and work for someone who is competent at theirs. that someone may or may not be more intelligent than me; it's largely irrelevant as long as they're smart enough to do their job well. (and i may or may not be more intelligent than they are; it should likewise be irrelevant to them as long as i do my job well)

> no, i think you're falling into the common technical-person trap of assuming that raw intelligence is the only measure of competence.

No, I am looking at the logical consequence of "hire people smarter than yourself". If A is smarter than B, then B is less smart than A.

Therefore: "hire people smarter than yourself" implies hiring people who are willing to work for someone dumber.

Meh -- "smart" doesn't work on a single scale.

Your boss can be smarter than you at his job, and successfully hire you, smarter than he is at your job. That's an odd way to word it, but the fact remains -- there are generally different skillsets and competencies involved, and you are looking to build or be a part of a team with complementary talents.

certainly. it's just that the word "dumber" has connotations of "stupid and therefore not worth working for".

How to hire A players: Offer way above current market value.

How to hire B players: Offer market value.

How to hire C players: Offer below market value.

You do two things when you offer more money.

1) You are more careful about who you hire. 2) You actually attract good people who would make a move for that kind of money.

I would love to hear an inverse argument against this.

Silly money isn't a good recruiting tactic in my experience (from the original dot com boom days - which was the only time I ever had silly money available :-):

* It motivates B and C players more. So you get a lot more chaff to filter out in your application process. You also get many more D players (the ones who are knowingly incompetent and are actively trying to deceive you into recruiting them).

* The major motivation of A players usually isn't money (once you reach the "fair wage" stage). So you don't attract that many extra A players.

* Some of the A players who do get tempted by the extra cash discover that it's not quite as wonderful as they expect and leave for more interesting work. Wasting large chunks of your time and money.

* It may turn some A players off. There's a unwritten rule that you get more cash to make up for dull work. So offering silly money implies to some that the work is going to be exceptionally uninteresting.

The absolute best way I know to recruit A players is:

0) Understand the role that needs filling and the kind of person that needs to fill it

1) Have a really interesting job

2) Know how to access the A players

(I'm amazed at the number of organisations I come across that skip step 0...)

Silly money may not be a good recruiting tactic, but the importance of understanding what the market actually is for the A players cannot be understated. So to add to your excellent list

3) Be prepared to be truly market competitive in salary - your target A player may have a 20 year consistent track record of getting sh*t done, respect their sense of self worth.

Being immodest I consider myself an A player, when a startup offering "competitive salary and benefits" low-balls my current income by 50% they've wasted my time and theirs

> the importance of understanding what the market actually is for the A players cannot be understated.

This can't be emphasized enough. What you consider "silly money" probably isn't silly money to an A player. In fact they may even find it low.

Nope. It was definitely silly money..... ahhh... the 1990's :-)


Indeed I'd argue that it's an implicit part of performing step zero - understanding the role and the person. If you don't understand what your potential employee is looking for in a role - be it money, responsibilities, working environment, or whatever - then you're never going to find 'em.

For example I'm continually amazed at the number of companies who decide that they need an uber-geek with a ten year track record of delivering e-commerce platforms (or whatever) - and then go about recruiting them as if they were unemployed. Sigh.

The finance industry tends to offer way more for people with a strong technical background to do quantitative analysis work.

To use real world examples, no amount of money you offer Salman Khan now would pull him away from his non-profit Khan academy to go back to the hedge fund industry.

If you're going to point to your step 2, then that sounds like a tautology. "Attract good people who would make a move for more money. Offer them more money"

That's actually an excellent point. Uber personal success would completely remove the desire to make a move for money.

Isn't that what acquihiring is about though? Purchase the talent that wont make the move for money.

I suspect you can get by with only the A players who are not yet loaded.

I don't know what GCHQ pays for programmers or mathematicians. I imagine that Cliff Cocks (invented public key crypto before RSA) didn't get the many millions of dollars that RSA did.

So; offer an environment where people want to work. Give them interesting work. Add a dollop of patriotism.

Work environment definitely matters. If the environment isn't any good, then the pay had better be great.

Ultimately, if you had an interesting product, a great environment and offered extremely competitive salaries then you would most likely end up with interest from more people in the "A" range.

The HUGE peer-reviewed professional literature on industrial and organizational psychology has many articles devoted to business hiring practices. There are many kinds of hiring screens, such as resume reviews for job experience, telephone interviews, in-person interviews, checks for academic credentials, and so on. There is much published study research on how job applicants perform after they are hired in a wide variety of occupations.

The overall summary of the industrial psychology research in reliable secondary sources is that two kinds of job screening procedures work reasonably well (but still below the .50 level, standing alone). One is a work-sample test, where the applicant does an actual task or group of tasks like what the applicant will do on the job if hired. Another is a general cognitive ability test (an IQ-like test, such as the Wonderlic personnel screening test). Each of these kinds of tests has about the same validity in screening applicants for jobs. Neither is perfect (both operate at about .4x level in validation studies), but both are better than anything else that has been tested in rigorous research, across a wide variety of occupations. So if you are hiring for your company, it's a good idea to think about how to build a work-sample test into all of your hiring processes.

For legal reasons in the United States (the same consideration does not apply in other countries), it is difficult to give job applicants a straight-up IQ test (as was commonplace in my parents' generation) as a routine part of a hiring process. The Griggs v. Duke Power, 401 U.S. 424 (1971) case in the United States Supreme Court


held that cognitive ability tests used in hiring that could have a "disparate impact" on applicants of some protected classes must "bear a demonstrable relationship to successful performance of the jobs for which it was used." In other words, a company that wants to use a test like the Wonderlic, or like the SAT, or like the current WAIS or Stanford-Binet IQ tests, in a hiring process had best conduct a specific validation study of the test related to performance on the job in question. Some companies do the validation study, and use IQ-like tests in hiring. Other companies use IQ-like tests in hiring and hope that no one sues (which is not what I would advise any company). Companies outside the United States are regulated by different laws.

The author's point, however, is wider than what you discussed. Filtering is one thing, which it sounds like the literature you mentioned covers. But filtering makes the assumption that the candidates have already decided to apply. The author of this piece found that he got significantly more applications when he added a test. So it's not just filtering, but also advertising.

How specific to the job does the validation of IQ test to job have to be to pass the legal test?

Could someone do an (awesome) academic study which shows that all programmers benefit from g, or at least that all developers in x language do? Or add some specific problem where g is hugely useful (algorithm design?) and then make that problem a requirement for many programming positions?

There would be a huge benefit to technology hiring if we could start using a cheap, standard test result vs. relying on proxies like school admittance (where they can screen for IQ), or expensive work-sample tests (which burn more of the applicant's time, selecting against busy/good candidates).

I'd rather advantage poor, possibly non-US, high-IQ, busy applicants over either graduates of elite schools (correlated with rich) or people willing to spend the time to take a 4-40 hour unpaid assignment.

I applied at Company A that had an test similar to an IQ test, and they returned back that I would be better suited to an entirely different job than what I had applied for. It also took them 3 full weeks to get this information back to me, then another 3 months to determine that they wanted to interview me for the original position I had applied for. At that point, I was already working at Company B making 10k more than Company A said I would be eligible to earn after working at their company for 3 years.

Yeah, I'd pretty much say that the IQ test was pointless in this case and luckily that was the only company I had run into using it.

EDIT: Forgot to mention that the test also took 3 hours to go through and had no limit on the questions that you would get. There was no possible way you could finish every question on the test.

This is interesting - do you have a good academic reference for that summarizes this literature? Maybe like a handbook chapter?

I think the the study he is referencing is the 1998 paper by Schmidt & Hunter. We studied it in business school. I couldn't find an actual copy of the paper but these two articles have abstracts of the findings: http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&id=1... http://www.onetest.com.au/awms/Upload/documents/whitepapers/...

Schmidt, F.L. & Hunter, J.E. (1998) The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings,” Psychological Bulletin,

Thanks. I had read the conclusion I mentioned in monographs about psychology I was reading for other research interests, and I'm glad you cited a key review article. I found the article's full text


via Google Scholar.

I've never been involved in hiring at a startup, but for 8 years I was involved in hiring as a prof in academia.

In that context, there is a peculiar and consistent gap between the best hiring strategy and the strategies institutions actually use.

For hiring faculty whose main focus is research, the best strategy is targeted attempts to hire superstars -- the research equivalent of a Guido van Rossum, say, or a Linus Torvalds, or someone near that level -- or early-career people where you have very good reason to be confident they will be future superstars.

It sounds obvious to say "go after superstars". But what's strange is that in any given field only a very short list of institutions actually do it. At most academic institutions it's surprisingly difficult to convince the institution to go after superstars.

I think the reason for this aversion is instinctive. If you're hiring it's much more comfortable to go the conventional route, advertising a position, interviewing a short-list and so on. This produces a situation where most of the interviewees want the job pretty badly. This gives the hiring committee a lot of power, which is a comfortable position to be in.

Unfortunately, it also produces lower-quality candidates than identifying the best people and going after them directly. A superstar will be interviewing you more than you are interviewing them. You'll need to get creative in convincing them that you have something to offer, something they can't find anywhere else. This is much less comfortable.

A useful heuristic: if most of your job offers are being accepted, your standards are way too low.

So with all the how to hire threads lately, may I make a suggestion:

Focus on how good your company is on running an engineering department. Especially if you're a startup.

Do you have a VP of engineering who knows what the hell they're doing? Do you insulate the devs from sales people making ridiculous promises? Is your code base in good shape so someone can come in and make a difference without having to learn the complete spaghetti stack? Do you have a well designed deployment procedure that has dev, test, and production environments?

How do you approach scheduling? I'd say this is the number one issue that leads to bad experiences for developers, so tell us up front that you have this under control. Do I ever have to hear a business guy say "I don't see why that would take more than a few minutes..."

In other words, prove that every problem isn't solved with "developers work more hours."

Say something that at least implies you know that these issues exist. They are much more important than ping pong tables and calling each other ninjas. I bet you'll find plenty of pro developers who are sick of their current employer if you do.

Imagine if developers actually learned something about running a software company just from your job ad itself. That would be amazing. I would point to JoelOnSoftware as proof it would work.

Providing you're testing the actual skills that you want, this works well.

But if you're throwing out maths puzzles when you actually want people to write Java business logic then I can see you ending up with the wrong people.

The math puzzle we used required only prime numbers and modular arithmetic. Any competent programmer could wikipedia it and solve in 20-30 minutes (and many did).

So it was like FizzBuzz, then?

May I please see the puzzle?

It seems like this criticism should apply to FizzBuzz too, and that's why I'm inclined to dismiss it. (Specifically: how often does a Java business logic programmer invoke the modulus operator?) But I think I would agree with a more general point of avoiding difficult Project Euler style problems in favor of simpler ones that nevertheless need a written program to solve (or hardcore paper and pencil math skills).

FizzBuzz isn't about creating a correct compilable piece of software in a particular language.

FizzBuzz is useful for finding the people who cannot even begin the problem; the people who couldn't craft even a vague approach in pseudocode. Some people can't sketch out a few loops and modulus tests.

You know that the person who isn't sure of the precise syntax of a modulus operator can pick it up. You also know that the person who is baffled by FizzBuzz probably will need too much hand-holding.

This reminds me of a stupid NH job posting where they said they want, and then there were two hashes: Googling for those hashes revealed in the first snippet first standing for "good", second for "developers".

The message was that anybody who has the common sense to google an unknown string is qualified duh?

The first puzzle job ad is nice, but it quickly wears off, and would start to bring bored "oh, not again" feelings. And many people just aren't good at making puzzles.

It's the same as with paint-drawn NFS ad.

I used to think I knew something about hiring. I worked in a fairly large engineering organization and was involved in all our hiring. I did college recruiting, intern programs, phone screens and interviews. I mention that for background when I say: you can't reliably make good hiring decisions on the basis of a couple hours of your/the interviewees time. It just doesn't work. I'm personally a fan of hiring within your network (for engineering positions). You can't beat a genuine recommendation from someone who's spent months working shoulder-to-shoulder with the candidate.

So please, when you hire people, have some humility. Know that your decisions are affecting someone's life in a major way, and you're probably making the wrong choice anyways, just do the best you can.

I wonder how much of this is due to novelty factor. I doubt that one could reach such good numbers, if asking for solving puzzles becomes a common practice.

There is something to be said for the novelty probably getting more interest, but I think that giving puzzles around the real work the hire will perform tells them as much about your company and the problems they can help you solve, as it tells you about the candidates ability to do so. A win/win as they say.

I already ask about 10 technical questions in each of the interviews I conduct, and having a problem online for each candidate to hack on would be really useful.

Now that I think about it, I may even consider using something like this to drive internal improvement projects. Some people just love the challenge, and the 'a' people always step up, you just have to ask.

And perhaps tedious.

I think this post misses one of the key takeaways from "unlucky". That post isn't only finding good candidates. Its about casting a wide net and doing the laborious part of filtering out good candidates solely on the basis of their qualifications.

This seems like a good way to only get good candidates but you may also have some very qualified people who don't want to play this game and find it unnecessary. In some ways its the equivalent of throwing out resumes with misspellings.

Yup, this worked really well for us too - we asked candidates to submit a 20-line "binary chop" search over a given data file (well, that was the solution, we described the problem and restrictions such that that was the only right answer). All the interviewees were good when we did that, and it also seems like a good test for recruitment agencies (i.e. which ones understand what we're trying to do).

We did get one answer that was both incorrect, too slow, and massively over-engineered. I got emailed a multi-megabyte Rails project, boilerplate, tests for correctness and all. Bit It Didn't Solve The 20-line Problem. He was the guy we wrote the test to avoid!

What if companies just open-sourced their code? Then a programmer could look at the very system they would be paid to work on and a company could look at examples of real work. No more puzzles and arbitrary tasks. And if the system is of any interest and a community develops, that's a source of new programmers. The company gets access to a talent pool and hackers get to know if it's work they want to do.

The obvious downside is that competitors get your code too. But is there any credible analysis or evidence of how bad a risk this is? Because the risk of not finding good programmers is pretty bad.

I suspect that people are still way overestimating the value of source code in and of itself, relative to the value of the team that produces it. If so, that's a market inefficiency and open-source might provide an edge. It would also be more rational and satisfying from the point of view of the actual work. Imagine where mathematics would be if mathematicians still kept their proofs secret.

Can anyone point to examples that shed light on the effect of open-source on hiring?

I worked for a startup that used to advertise its open positions as "Silver Java Guru" or "Java Geek".

Needless to say, we got one awful applicant after the other. Most could Fizzbuzz. Some could not. And others must have cheated their way through the phone interview - when you got them in person, it all fell apart.

One thing I noticed from the process is that little things like spelling mistakes, incorrect formatting, etc., in a resume was very telling. Without fail, the kind of person who had typos in his resume was the kind that failed our programming tests, miserably.

If you are looking for a good candidate, and ready to spend the time doing it, why not look around the developers who have done something in the field you are active on.

For example, if you are looking for a Front-End guy, why not look around web magazines, Web 2.0 products and services, Github, Dribbble and pick the guys you like and contact them.

Part of the trouble is that the best people are well employed and busy working. They are in high demand. They aren't going to do puzzles or beg you for a job. You have to find some way to meet them and convince them to work with you.

This reminds of the Google billboard test from years back:


No discussion of rockstar ninjas, just saying exactly what we are looking for.

Like I posted in the "who's hiring", that doesn't work for me because it doesn't tell me whether you are what I'm looking for. What do you do? Why should I care about you?

Like it or not, this particular hiring filter seems to have caught on bigtime. The only downside to it is that there are people like myself who love to take a stab at solving puzzles regardless of whether they actually want the job or not. In the past 2 years, I solved online puzzles & scored interviews with the founders of 3 yc companies ( greplin, poundpay & freshplum) & 4 giant financial shops ( BlackRock & 2 other hedge funds I can't name/nda). In some cases like BlackRock, the puzzles are quite involved & can take a week to complete. Puzzles are a lot of fun....I just wish the associated job was equally interesting.

Do you have a source for where you find these interview puzzles? I would like to take a few and see how they are set up.


Before I look at a resume I send every applicant a coding problem with simple instructions. It looks very simple at first, but actually requires some thought. It is fairly real-world, but unusual. I even provide a sample test case, and the instruction that the solution must pass all test cases. It would take the ideal candidate less than half an hour to code and test. Acceptable candidates less than an hour. Despite having all the time and resources they need to solve it, 85% to 93% of the candidates do not get it. I don't even glance at these resumes.

Have you considered that 85-93% of the candidates don't want to do an hour's worth of unpaid work for you before you've even given them an interview?

It's not 85-93% not responding, but 85-93% responding with crap that took some effort to produce. Also if you're actually interested in the position, you're going to be doing more than an hours work (if that's what you want to call it) in the interview process. Lastly, it would be work if I were tricking people into doing something I used as R&D or actual production end-product. This exercise simply establishes qualification.

What is the response rate? How much contact do you have with the applicants before sending them the problem?

Lately these are contract positions, and being in a corporate environment here I'm dealing with an agency that deals with other agencies. I still do some direct (usually by Craigslist ad) recruiting. I only submit the screening task to people who have already responded to the position posting. Very few people, maybe 5-7% do not complete the task, and I assume they found another position in the interim.

It is an interesting idea, but this may be limited in its use for small startups that hire only 1 or 2 programmers or for someone willing to wait. My reasons being: Only a small sliver of developers visit sites like HN. They are mostly busy working, going home to kids and such. Getting the news out that you are hiring, to this larger set of candidates is a real problem. Which is where the traditional methods (including job web sites) become crucial. If you use any job borads (like Monster), or a headhunter, it is hard to pipe a problem through them. They are not setup for this. If you had asked me to solve a problem just to find the email to apply to a job, I would have not done so, just because I do not have time to do it. If I were a kid out of college, or did not have time constraints, I would have loved to. But where I am today, I would rather send the resume in and talk to the hiring manager.

Good puzzle, as opposed to a bad one, is an excellent attention getter for good programmers. They can judge you and your intellect by your puzzle and be attracted to work with you. Ie. your input makes the recruitment a two-way street.

"we hadn’t really planned for so many good applicants so we were very slow to respond to everyone"

So they weeded out the really bad applicants and as a result were left with so many good applicants that they're now in the same situation as the original article.

The key takeaway for me is not how people filter their applicants on real criteria, it's what they do when they have a large amount of seemingly equally skilled applicants. The original article demonstrated the 'unlucky' solution, I'm not sure what this article demonstrates since they still got too many applicants and they're spending time responding to all of them.

So they weeded out the really bad applicants and as a result were left with so many good applicants that they're now in the same situation as the original article.

The original article talks about finding 4-5 people worth talking to out of 100 people, 50 of whom can't fizzbuz.

We were in the situation of 20 interesting people out of 40 candidates, all of whom could fizzbuz.

If it were truly unmanageable, we'd probably have used weak Bayesian filters. E.g., degree, rockstar ninja, etc, all the things Raganwald originally advocated against using. I gather this is what Google and Facebook do.

Having a lot of highly skilled applicants is a much better than having a lot of unskilled applicants. In the former situation, you pretty much can't go wrong...even if you use completely arbitrary filters, it doesn't really matter.

This article demonstrates a way to get a lot of highly skilled applicants-that's extremely useful.

I fully support this mostly because I absolutely love doing the puzzles :). (So make more of them!)[1]

One of my favorites from a few years ago was when reddit was hiring. One of the more fun aspects of it was:

git clone our public repo and tell us the most commonly used word

Another fun one was (I think? This was a few years ago):

find me a right triangle, the length of which's sides all add up to: $some_number.

[1]: In fact, this would be an awesome website if somebody hasn't made it yet: be a "recruiter", sortof, except you just post programming/math puzzles, we get to solve them, and then you get to match us up with employers.

see interviewstreet.com

The only jobs that caught my eye were the ones that offered some type of challenge to apply (Apply Via API, Etc.) If I'm going to end up talking to someone from Human Resources that's going to ask me the "Man Hole Cover" or "Lightbulb Question" then forget about it, I'm not interested in working in more stuffy corporate environments.

-(void)fizzbuzz { int i = 1;int t = 3; int f = 5;int h = 101; int z = 0;

    // here we do a for loop using i and incrementing it until it's equal to z
    for (i = i; i < h; i++) 
        /* we check if both i/t is equal to z and if i/f is equal to z
         if both are true we log fizzbuzz in the console */
        if (i % t == z && i % f == z) 
        //check if i/t = z and log
        else if (i % t == z)
        //check if i/f = z and log
        else if (i % f == z)
        // if none of them are true we just act like we're learning to count
            NSLog(@"%d", i);


Pertinent passage: FizzBuzz was presented as the lowest level of comprehension required to illustrate adequacy. There's no glory to be had in writing code that establishes a minimum level of competency. Even if you can write it in five different languages or in under 50 bytes of code.

Every time fizzbuzz comes up, someone has to post a solution it seems, but why does everyone write such huge fizzbuzz programs?

  for(int i=1; i <= 100; i++)
    printf((char *[]){"%d\n","Buzz\n","Fizz\n","FizzBuzz\n"}[((i%3==0)<<1)|((i%5)==0)], i);

Some people value readability, some compactness / efficiency.

Why they do it one way or another is an occasionally interesting follow up question.

I'm in the readability camp, I'm one to comment my code even if it's something simple and I also like to use verbose variable names. I always keep in the back of my mind that if someone had to pick up my code without knowing how anything was implemented they might appreciate my efforts (even if it's just a personal project that I don't plan on anyone seeing). You can't please everyone I'm sure for as many people that like it compact there are people that like the whitespace.

I posted it as a joke: http://www.codeslate.com/2007/01/you-dont-bury-survivors.htm...

However, yes it is huge it's Objective C with whitespace what do you expect? ;)

A bit long, isn't it?

(Thats https://github.com/kevinlawler/kona)

[Edit: Found a way to shave off three chars]

Do you want a critique of that? I'm mainly interested because I'd be interested in a critique of my critique... We use Fizzbuzz as our lowest bar question and I'm curious to know what people consider significant points and what are not.

Significant point- Does it work? Non-significant points- everything else.

Depends on what you're looking for.

Does it work is the most important thing by a mile but if you've asked them to wrie production ready code then you've got to ask "how would I feel if one of my developers checked this into version control and how would it fair in a code review?"

Maintainability, readability, efficiency and most significantly the candidates ability to explain why they took the approach they did.

I wouldn't hold any approach against someone so long as they could talk intelligently about the trade offs and why they'd done what they'd done.

Even really simple problems are a chance to see whether someone thinks about things.

Your first comment should say "until its equal to h" not z

Shouldn't it say until it ceases to be less than h?

Your comment supposes that the loop will increment without interference. While you'd hope that was the case, it's not necessarily true.

Obviously with the loop as coded your comment is correct (that'll be the first condition that terminates the loop) but it could raise a question about whether comments should reflect the routine as written or just the section they're immediately referring to.

Or indeed whether comments for something so self evident are even useful or necessary... Something I'd definitely ask in an interview if I was presented with this code, not looking for a specific answer, just looking for some indication that the candidate had (or could come up with) some vaguely informed thoughts on the subject.

It took me more than four months after college to find my first job as an intern. Six months later, I got offered a better job so I quit. However during the transition I was offered some well paid freelance gigs so I didn't end up taking the second job. After 4 years of freelancing and moving out of a big city, I got tired of working alone so I joined a small team in a non-tech company. Six month later, I was getting bored so I looked for work in a major city. I found one startup that did something that sounded interesting, so I sent them an email even though they weren't advertising opened positions. An hour later I got a response, and two weeks later I was working for them. The past 3 months, I got offered 2 jobs and Facebook contacted me for the second time.

This isn't even the Silicon Valley -- I live in Europe. Sometimes I wonder if I'm smart or not. I know I probably wouldn't be able to answer all of these trick questions I hear about. I think I'm much better at abstracting concepts and seeing the whole picture. I love programming, but so far I haven't found a place where I can have as much fun as when I am creating software on my own, free of constraints and in control of everything. When I work on my own, every now and then I stop coding, sometimes for days, thinking, sketching, until I know exactly what I'm going to do and can explain why in plain English.

Anyways, I think younger programmers might enjoy taking time answering these kinds of question, but I'm the kind of person that wishes he didn't have to sleep so he could spend more time working on his own projects, so I don't bother. If you want me to be interested, you'd have to find a way to convince me that I would be working in an environment that feels like I'm working on my own project. I want to work with motivated people that care more about their product than getting funded. I don't want to work in an environment where the number of closed tickets is a metric for productivity.

My field (classical music) has a similar application screening problem (lots of applicants, very few good ones). Our chosen screening process, the audition, is imperfect for many of the same reasons given for why puzzle solving sometimes falls short when it comes to screening for developers:

- The ability to perform on audition day doesn't necessarily correlate with the abilty to perform consistently in concert settings. - Great solo players aren't always great ensemble players. - It is very difficult to predict how an auditionee will mature over time.

That said, our field has yet to come up with anything better, and I think the organization I work for has developed a particularly good screening process:

On audition day, applicants are asked to perform some excerpts from literature our organization plays regularly, a prepared solo selection, and to do some sight-reading. This is pretty standard stuff for the clasical music industry and this preliminary round is done from behind a screen to help combat biases on the part of the panel. The excerpts are important because that's closest to the work applicants will be doing on a day to day basis if selected. The prepared solo helps show the candidates potential to excel in solo situations. The sight reading is crucial because it is a basic and necessary skill and correlates very highly with the applicant's quality of musical training.

The second round (generally 5-15 applicants from the initial pool of about 100) is where our organization does things a little differently. It consists of more excerpts and sight-reading, but this time applicants are asked to make adjustments to how they performed the selections (a little faster or slower, a little louder or softer, etc.). Observing how the candidates respond to these requests shows how nimble the musicians will be in a rehearsal setting, and also shows how well-prepared they are and how well they will take direction.

In the final round (generally 2-5 candidates), applicants are asked to perform alongside members of our organization. This demonstrates more responsiveness, how well the applicant listens, and how well the candidate performs in a group setting.

With that in mind, I often wonder why those looking to hire developers don't use a multi-stage "audition" process.

Step one could consist of some thing along the lines of "Here's a piece of code like what you might see in our organization. There's a bug in it. Find and fix the bug and send the revised code back to us." Step one could also consist of a short puzzle-type problem. Step one could be 100% automatic and would require the same amount of time for 100 applicants as it would for 1000 applicants.

Step two could be where applicants are asked to show samples of something they've coded in the past. This could also be where they are asked to perform a more difficult bug fix and a more difficult puzzle. In this round, applicants are told that their code will be read for documentation quality and use of good style.

Finally a final round could be done in-person where an applicant is asked to solve a problem with existing members of the team. This could be done in conjunction with one on one interview. This would help tell if the candidate is someone the team actually would enjoy working with, and how the candidate would be able to contribute.

I know I wouldn't mind submitting to such an application process because I could see how each step is relevant to what the company is looking for. It also shows that the company takes hiring seriously. Hiring is supposed to be hard. Knowing that every one else on the team has been able to excel in those circumstances gives me greater confidence that I'd be joining a great team.

Do you really see many talented programmers submitting to the process you're suggesting?

Interviewing is a two-way street. While you're trying to figure out my coding skills, I'm trying to figure out if I'm interested in what your company does and if I'd like to work with the people there. Personally if I knew I'll have to spend an hour of my time on interview challenges just for the chance to get some face-time with an engineer, you'd never even see my resume.

Based on what I have seen, the turnover rate of musicians in a symphony orchestra is much slower than the turnover rate within software development organizations.

That is a very good point. Most top-tier orchestras have tenure for their permanent positions and it's not uncommon for musicians to stay 20-40 years.

As a result, the consequences for a bad hire are extremely high. Many orchestras have the audition winner perform with the group for a probationary period of a season to help guard against this. That said, I read a lot on HN about the consequences of a bad hire for a an early team member, so although the timeframes are different, the fear is similar.

The long tenure (and prestige of the organisation) probably also contribute to the willingness of candidates to subject themselves to such a process. Getting someone who is currently employed to sacrifice multiple days of (paid or unpaid) holiday for your hiring process is presumptuous, particularly if they're actively looking for a new position and thus probably going through this with multiple companies simultaneously.

As you say, it's all about making the relevance of the process obvious to the candidate. Also, companies should be as accomodating as possible in terms of timing of appointments, etc.

Totally off-topic, but I'm really curious about what brings you to Hacker News? I wouldn't have expected a professional musician to spend time on a software-tech/startup site.

I'm a recording engineer, which increasingly means "data wrangler". A couple years ago I taught myself bash and then python in order to write a script to convert our archives from wav to mp3 (and inject metadata from a crystal reports file). That project has evolved into a web interface for searching and browsing our audio, video and photo archives. I probably spend about 3/4 of my time making recordings and 1/4 writing and maintaining code.

I don't remember what first brought me to HN, but I stay to learn about open-source projects I wouldn't otherwise find out about. The next version of my archives site is built with Flask and Twitter Bootstrap, both of which I learned about here. Also, my transcoding scripts use Kenneth Reitz's Envoy for the system calls.

Also because I realize that as much as I love music, my decision to become a recording engineer was probably ill-advised. There are days I strongly consider giving it up to move to the Bay Area and try developing full-time, hopefully at a startup.

Was a recording engineer too and stopped because it broke my ability to sit down and appreciate listening to music: I had to analyse it "professionally". Mick Jagger said once he would only listen to classic on the radio. I can believe that. In some areas of human creation being an amateur is necessary. A simple example: China has an enormous corpus of classical painting and poetry, no part of it has been created by professionals.

Add: to me writing code is an activity that can be enjoyed professionally, unlike music.

Do you have a link to your encrypted application?

Yes, this is a bit annoying. First he relates a not entirely interesting story about his HR problem, then says he solved it by including an interesting puzzle. OK, cool story bro, NOW WHERE'S MY PUZZLE?!?!? I'm not looking for a job, or looking to hire, but like most Hacker News readers I like puzzles. Now I'm angry and have to find a puzzle somewhere else.

My previous blog post (submitted here yesterday) has two.


But I'd suggest that except in special cases, posts with titles like mine are probably not worth reading if you are uninterested in hiring or finding a job.

Well, you never can tell. When someone says they've found a novel test for weeding out bad applicants, that sounds interesting. When the test is described with a handwave as an interesting puzzle that the best applicants leap at the chance of solving before reading the rest of the interview.... and then you don't even give us the puzzle, but rather launch back into the HR aspect... well, that just seems cruel. It's like reading a murder mystery where the detective details how he figured out who the murderer is, and then never tells anyone. From my authoritative position as a random anonymous internet jerk, I assert that if you allude to a puzzle, you ought to at least include it.

I am apparently puzzle obsessed today. Slow day at work.

I just bought that C++ book. Thanks!

The puzzle can be found at http://meshcapital.com/application.tar.gz

Note that they left the puzzle on the server, but that is no longer the puzzle used for recruiting. Before I left I wrote a more difficult one.

I have no idea if they are currently hiring, so please don't bother them with job applications unless you see a job ad.

Here's a math puzzle.


((First Row and Fifth Row are not well defined, but that's not particularly relevant to the problem.))

I don't know if it's still up, and I don't work there anymore.

I wonder how many high quality applicants he would get if he just described what the fuck the job is about.

I often look through job ads out of curiosity, and it's remarkable how generic most of them are. Maybe they promise "high profile clients" or "working with exciting technologies". Maybe they even promise "a friendly atmosphere". I don't understand why anybody would answer such a job ad.

I'd be interested how many well qualified applicants they would get for a job ad that actually described what they are looking for: "We look for someone to work on the back end of our fizz buzz cloud service. The task will involve factoring numbers and counting digits."

Have you ever stopped to realize that there is a class of people who like solving problems like these that have a definite, deterministic solution but who are utterly incapable of shipping code or a working product?

I wonder when people stop searching for holy grail of simple, quick, universal and always effective hiring method. Even probation doesn't provide you with 100% accuracy of performance prediction. And everything else you'll use will be less effective. The real challenge is to make a proper, conscious choice of the tool taking into account the job itself (and its context) as well as balance between accuracy and resources spent. The discussions like this remind me discussions about which programming language is the best. Is there answer for such question? :)

Hmm. I remember telephone interviewing for this position. As one of the applicants in the first group(i.e No puzzle just a simple job description posted :-)). I don't remember it verbatim, but the couple of questions i remember was (1. Why did you decide to apply?(i.e: from the post) I mentioned that the post referred to a entropy reduction algo and implementation. 2. Do you like HPMOR? :-P Just a reference. I don't have a strong opinion about the whole experience.

This is fairly true in my experience.

I find solving puzzles at codeeval much more interesting than crafting custom cover letters, and the ease of applying means I'll more likely apply to a company there than elsewhere.

OTOH I don't think I'd classify myself as an "A" player, at least not yet. The more I learn the more I find I need to learn &tc.

(haven't heard back for anyone yet though. Probably a bunch of good applicants inundating their hiring team as well.)

I suspect this works even better in areas outside of traditional development, like forensics, crypto, or data science. Hackers with specific skill sets, especially where those skill sets usually get applied to private data sets, often love the opportunity to mess around with alternate approaches on public data sets. And if it means an opportunity to show off, even better.

If you're simultaneously someone who regularly reads Hacker News AND cannot FizzBuzz, shouldn't you pretty much kill yourself?

No, you should send yourself to the museum as an example of extremely rare specimen ;) Or learn yourself on codeacademy.com

I like this because it isn't a "gotcha" trick. That's the way this method backfires (in my opinion).

I found this very intriguing and decided to try. Here is my version of the filter: http://www.careesma.in/employer/joboffer/view/id/124996

Any thoughts/feedback welcome.

If I have to spend a lot of effort just figuring out if I even want to apply to the job/company, I'll probably just look somewhere else. On the other hand, if I'm not looking for a job but like puzzles, then maybe I might do it.

As someone who occasionally solves hiring problems with little intention of applying, it was nice to hear that my hobby is less annoying than I had imagined.

i remember doing this.

i mostly did the puzzle because it was a fun distraction.

another thing is that it self selects for people who consider problem solving fun. its not necessarily the most predictive of job performance, but it is better than nothing and you'll also be filtering for personality (no opinion on whether that's a good or bad thing).

sure it might weed out some good people, but it probably improves signal/noise ratio.

How about putting a link to the real job post instead of "instructions on where to send your resume" hidden in the application?

What is this "Fizzbuzz" which only talented star programmers seem "able to do"?

Fizzbuzz is a very simple problem described here: http://www.codinghorror.com/blog/2007/02/why-cant-programmer...

The point is not that everyone who can program FizzBuzz is talented. The point is that programmers who can't solve FizzBuzz are probably extremely untalented.

Great way to keep recruiters out of the process too.


What is fizzbuz?

Why are people so unhelpful? Is fizzbuz cryptography?

It's an old blog post meme from jeff atwood http://www.codinghorror.com/blog/2007/02/why-cant-programmer.... Simple googling for fizzbuzz programming finds it. EDIT: fixed wrong author name . thanks lazugod

Although that's Jeff Atwood, not Spolsky.

Oh, you're right. Thanks.

I think companies need to do a better job of listing what technologies they use without making it sound like it's required to have mastered all of them. People who've taken Haskell or even Scala into prime-time production are extremely rare. People who would be excited to try, and who could be trained up into that role are a lot more common.

I'm getting to the point where I'm realizing that I'm only going to grow as a software engineer if I'm extremely selective about the work I take. I'd go into management before I'd work in Java or C++ full-time. (Of course, I'll use these languages if small projects require it, but if you're a "Java shop" I'm not interested.)

I wrote about this effect here: http://michaelochurch.wordpress.com/2012/01/24/how-any-softw... . It's long, but the TL;DR of it is that it's very easy for a company to become good at hiring if it gets its core technologies right.

I'd go into management before I'd work in Java

Tried that. Now I'm back coding Java, Groovy, JS and looking into Scala.

The perfect management role for me is one where I can shape product direction, mentor some younger guys, and still code. The problem is that in all the companies I've seen, management ends up being I sit in meetings all day and get completely cut off from coding or even having deeper tech discussions. I just don't feel like I'm still creating at that point.

What you describe is what has kept me in my current position. It takes a small company to make it happen, though.

I'd go into management before I'd work in Java or C++ full-time.

I suspect that statement is more absolute than you intend it to be. Would you not work on a new language if the compiler for that language generated C++, which meant that the language's runtime was implemented in C++? Which, of course, implies that you will have to write a decent amount of C++ code.

If you would work on such a project, then I suspect you mean "If the problem and solution itself are not novel, then I have no interest in using Java and C++."

Fair point. Actually, you've nailed it. C++, for example, isn't some horrible language without purpose. It has a niche that it fills well (although I'd prefer C) but most C++ jobs aren't in that niche.

My experience is that language adjustment is up to a 0.5-point drop (the scale is here: http://michaelochurch.wordpress.com/2012/01/26/the-trajector... ) in an engineer's ability. This is on a 3.0 scale where most engineers are between 1.0 and 2.0, so that's a huge change. Compensation tends to be linear in proficiency but exponential in value-add. 0.5 points is about $50,000 in market pay but a 2-4x (depending on the type of company) difference in productivity.

Sometimes it is worth the drop, such as when one's learning a dramatically new and more effective approach, or solving a fresh class of problem (e.g. real-time software, which is done in C). That isn't the case with enterprise, "pattern"-afflicted Java. Instead of solving the problem appropriately and well because I know how to do it, I have solve problems inappropriately and badly because I don't know the crappy idioms that people invented because they don't have appropriate tools.

On that scale above, I put myself around a 1.7. If I set foot in a typical C++ or Java shop, I'd drop to 1.2 or 1.3. Which means I'd risk falling to the level of a junior programmer and being typecast that way. Which means I wouldn't get the benefit of the doubt if I say, "This project wouldn't be complete shit if we'd done it in an FP language" (in some nicer words, of course). Your first 6 months determine whether you have technical credibility or not. If you don't, you can't blame the tools, even if bad tools are (as they often are) the cause of the problem. The unenlightened tend to think of all tools as equally good (all languages are Turing complete). No, not so. A 3x difference in CPU performance may not matter for most problems, but the 3x difference in human performance that hinges on whether one uses the wrong tools certainly does.

The drop to 1.2 is something I'd tolerate if moving to something genuinely interesting, knowing it'd only take a year to pop back up to the same or even higher level. An example might be high-performance C++ in weather forecasting, bioinformatics, or artificial intelligence. I'd gladly become a junior again for a year for that kind of stuff. On the other hand, most enterprise Java/C++ bullshit isn't really worth learning, not for a creature that has 60 years left to live instead of 600,000 (#ShitYouCantSayAtWork).

So yes, my language bigotry is more directed at the typical C++ or Java job than at the language itself.

I'd suggest that you're being a bit ideologically biased against Java. Ain't no easier language to do concurrent programming, aside from Clojure (jvm), Scala (jvm) and I hear Erlang (don't know much about it). Every serious project in Clojure or Scala that I've seen ignores their concurrency stuff and goes straight to Java.

It turns out that if you implement a bunch of Callables and ExecutorServices with LinkedBlockingQueues or SynchronousQueues.. it's basically functional programming. Don't get too hung up on the syntax.

That bias is very much justified, I'd say.

      scala> (1 to 10).par.foreach(x=>print(x*x+","))
Let me know how you'd parallelize the above in Java, how many lines of code it'll take, what syntactic constructs you'd end up using, what libraries you'd import, and finally your my mental model would be - in terms of locks and semaphores and so forth. The thing with scala is that concurrency - both data parallelism ( above example ) and task parallelism ( using actors in scala, a slightly longer example) are baked right into the language. A newbie in concurrency whose mental model doesn't incorporate thread pools or executor services or synchronous queues can progress leaps & bounds in scala vs java when it comes to concurrency.

Ideological bias is never justified. On to the justified bias:

For that problem, I'd just do it serially, because it's only 10 numbers.

Oh, it was an example for really big sets of numbers? Like really really big? Then I'd skip the .par.foreach and deal with java.util.concurrent directly, because it's worth my while if I have a serious problem worth optimizing.

"A newbie in concurrency whose mental model doesn't incorporate thread pools or executor services or synchronous queues can progress leaps & bounds in scala vs java when it comes to concurrency."

A newbie in concurrency needs to understand how many threads are in their threadpool, or else they should just do it serially. I'm not talking rocket science here, it's just a single integer argument.

concurrency stuff

You're talking about at least 3 distinct models: direct thread management, with improved locking routines (java.util.concurrent, see recent Prismatic thread on HN), actors/green threads (Akka and erlang BEAM), and STM/guarded read/writes (clojure, and you should read about Microsoft's attempt to graft STM onto C#, Joe Duffy's blog)

java.util.concurrent's entire point is to make direct thread management unnecessary. Note the list of classes I mentioned didn't include Thread.

Actors and STM are great thought experiments but real-world production stuff tends to always circumvent them for the java.util.concurrent stuff.

>Actors and STM are great thought experiments but real-world production stuff tends to always circumvent them for the java.util.concurrent stuff.

Its a mixed picture, if you want to be honest. There are places like bofa where I've pushed into production a cvar processing module on 500,000 loans which uses only actors. No low level tinkering with util.concurrent, no Threadpool setting, no Executors, no synchronized anywhere in my code. In the past at GS, I wrote concurrent code where I explicitly specified the number of threads & we had a ThreadPool manager & internally it used the util.concurrent classes. At LinkedIn, I believe they started with actors & switched parts of it to Doug Lea's framework ( which was the sort of the foundation of util.concurrent). I have seen financial shops where they do half & half - the quants seem to like actors but the swing programmers like to have control on threadcounts, & they tend to do things like naming threads & creating parent-child relationships so swing can paint reliably & they can debug threads by name when it doesn't. I did the same in the past when most of my work was on the swing side, creating custom blotters to show trade tickets & the like. But now I primarily do quant model building work with actors & no more util.concurrent stuff. If you want to construct a lattice model to price structured options on a 100 different strikes per month for 5 years in advance, you can find no better friend than an actor. Actors are certainly not a thought experiment, very much a reality.

Great post, yeah I overstated a little bit and actors are certainly valid for certain cases. I was just reacting to what I saw as an overstatement on the other side.

You've made this point twice now ("Every serious project in Clojure or Scala that I've seen ignores their concurrency stuff and goes straight to Java") and I'm curious to hear some examples as well as your understanding of why this would be.

For examples, the most recent ones I've seen are linkedin's kafka (scala), and bradford cross's recent post regarding clojure at prismatic (and other posts I've seen regarding both).

For why, I'd assume that they found the abstractions were getting in their way. java.util.concurrent is sort of the minimum abstraction (IMO) between nicer constructs and OS-level threads.

My problem with Java is the language. Scala and Clojure are great.

The platform is excellent technology. The language is mind-numbingly boring.

When Scala and Clojure projects go "straight to Java", it's generally for specialized purposes (where those languages don't have native maturity) or in already-discovered bottlenecks. I think some degree of that's inevitable. Like database tuning, it's just something you have to do when you get to a certain point where (a) your problem is well-defined, and (b) shaving 20% off latency is worth doing. I consider these good problems to have. If I have to rewrite a class in Java for performance reasons, fine. I'd never start a multi-thousand-line project in Java, though.

Also, I really don't think anyone mature minds writing code in underpowered or difficult languages. C++, I find, is actually a lot of fun to write. It's reading other peoples' code, in large amounts, that becomes intolerable in these languages. If I have to spend a day to answer a simple question like "how do I initialize this object?", that's bad.

Interesting fact: Closures were cut from the 1.0 release for time. Gosling wanted them in, but Java's original purpose was to run in machines like automatic coffee pots and TV "set top boxes", so they were far from essential for that domain. The folk history (which I once believed) of Java being deliberately underpowered so that teams of hundreds of mediocre programmers could be stapled together in it is false. It was, however, co-opted by the "let's commoditize developers" crowd in the late '90s, and we've had to deal with FactoryVisitorDelegates ever since.

If reading code is your problem rather than writing code, I find that Java excels. Most of the complaints are about writing code in Java.

You know how awesome that type inference is in Scala, and how stupid and annoying it is in Java to type stuff like Map<K,V> myMap = new HashMap<K,V>? I totally agree with how annoying that is. Here's a related example in scala.

def myThing = something.getSomeThing()

What is the type of myThing?

Java has weaknesses for sure but reading code is actually one of its strengths, precisely because of the boilerplate. Note that I'm not defending EJB or any other atrocities here, just the core language.

Lastly, I agree that syntactically-nicer closures would be better for Java. But it already has closures,

final Something variableImClosingAround;

Runnable myClosure = new Runnable() { public void run() { variableImClosingAround.doSomething(); }};


Now assign to the closed-over value :)

This comment with all its hand-waving reads like it comes straight from 1990.

Hand-waving? There's code in my comment. No code in yours.

Yes, the sad all stuff which has been used to argue over a decade now. Do you think people are stupid? I don't think there is anyone left who didn't hear these straw men. People moved on, maybe you should too?

What the hell are you talking about?

> Java's original purpose was to run in machines like automatic coffee pots and TV "set top boxes

from my time at sun, i remember speeches by Gosling that included microwaves and refrigerators ( i vaguely recall an example of the fridge telling you via java applet that you had run out of milk ). the hotjava thing ( web browser written entirely in java http://en.wikipedia.org/wiki/HotJava) created some anticipation around Word & Photoshop & Powerpoint written in java, which were brought up when Sun hosted a gigantic industry conference in Vegas. The Java Chip ( picojava http://www.jopdesign.com/perf.jsp ) was supposed to revolutionize the industry by running java bytecode natively. Honestly I think the dumbing down began once EJB's crept in. There was some nasty visual shit called the Beantool which automatically inserted getters & setters by mandating every EJB to be a Bean, and from then on things got downhill really fast.

A lot of people learned to trust their BS detector.

Mine had already been primed with XML "XSLT can map anything to anything!", and Enterprise Java was just more of the same.

It's funny that Javascript and WebGL are cleaning up in the browser, and Java applets are almost nowhere to be seen.

i actually find c++ more pleasant to read than to write. when writing c++ i am constantly annoyed at the tedium involved in things that would be a few lines of ruby/clojure/ocaml (my top 3 hobbyist languages), but once the tedium activation barrier is crossed and the code is written, it usually ends up pretty clear and straightforward to read and maintain. this is especially true in google, where coding standards are taken seriously and refactoring is a culturally valued operation (that was actually one of the main draws the job had for me).

I don't think I'd take an enterprise job. However, When someone mentions C++ my assumption is we're talking about fairly low level programming. Java though, I tend to think of as being strongly enterprise-y.

I assume my reaction to C++ is that way because my C++ experience is entirely in embedded software / consumer electronics (Strongly C-driven but way more C++ than you might imagine).

>People who've taken Haskell or even Scala into prime-time production are extremely rare.

Not that rare, actually. I have a hard time finding companies that want to use Scala.

I currently write Scala at work. As far as I can tell, it's a lot like C or C++, except that you have to use while loops instead of for loops and you have to perform unholy incantations to fake arrays of structs. If you don't do this, you can either try to allocate a few hundred million tuples and OutOfMemoryError, or you can allocate one array of a primitive for each field of the struct and give up on the idea of having decent cache locality. It's less capable of TCO than gcc, which is pretty hilarious for a functional programming language. Generic programming in Scala is lots of fun until you try to use an Ordering[Double] or an ArrayBuffer[Double] in a critical path.

Manageable parallelism is nice, though.

Almost every problem you mention is a problem of the underlying runtime. Are you sure you bash the right thing?

This is true. The only things that are trivial to fix are the absence of for loops and the use of ArrayBuffer[AnyRef] to implement ArrayBuffer[Double]. It might also be possible to make TCO of mutually recursive final methods work, but it's probably more trouble than it's worth.

Aren't for loops already fixed since months? (see https://github.com/scala/scala/commit/4cfc633fc6cb2ab0f473c2...)

Regarding ArrayBuffer: It is not as easy as it looks like, because of the conflicting nature of Arrays vs. Generics (yet another JVM problem). Amusingly I tried using Manifests to do exactly that last week. I hit the wall because the Manifests seemed to spread arbitrarily. Will try with TypeTags/macros when they become available.

I'm glad to learn that they're fixed for Range. Thanks.

Matching problem. Hard to find jobs in good languages, because they're rare. Hard to hire in good languages, because the people are rare. Fuck Java/C++.

"I think companies need to do a better job of listing what technologies they use without making it sound like it's required to have mastered all of them."

A good way to do this is a humans.txt, whose purpose is not just to credit the developers but also the technologies in use. It's funny that a lot of companies are crazy about hiring right now, but haven't taken 30 minutes to add this simple file to their web server.

Interesting read: skimmed/bookmarked.

I think it's the 2 surfers outpaddling the shark scenario: you don't have to be great at hiring, just slightly better than other companies in your demographic(geographic and "itch yoru scratching". You do have to be great at dev retention.

Note: the below may apply more to contractors, but if you read through it I think you might find it useful...

in order of confidence, here is how you hire someone for the job. (perceived)

100.000% - The job is done already, just like something you already had the same guy do and it was perfect, now you pay him.

95% - The job is done to your satisfaction, you know the guy and he's solvent and vouches for what he's done. He hasn't done just this, but maybe something similar for you. You're not sure if this works as advertised yet...

90% - The job is done to your satisfaction, you know the guy and he vouches for what he's done. He hasn't done this for you but did it for someone else you know, who vouches for the guy.

85% - The job is done to your satisfaction, the guy comes recommended only by people you don't know, but warrantees what he's offering and seems like he cares about his reputation and about you, seems solvent and in a situation to fix it if it's not what you thought you were getting.

80% - The job is NOT done (none of it is), but you know the guy (rest per 100.000%)

75% - The job is NOT done (none of it is), but you know the guy (rest per 95%)

70% - the job is NOT done (none of it is), but you ... (rest per 90%)

65% - the job is NOT done (none of it is), but... (rest per 85%)

60% --> the job is not done and you have a guy offering to do it who comes recommended by people you don't know, you haven't worked with him, and he doesn't seem like he's in a situation to warrantee what he's going to do or care utmost about his reputation and about you. However, he brings proof he's done it before (portfolio).

55% - the guy comes with recommendations from people you don't know, but brings proof only of things OTHER than what you want him to do for you. He is not in a position to guarantee what he's going to do for you.

50% - the guy comes with recommendations from people you don't know, and doesn't bring proof of anything he's done.

BINGO. This is where you are if you get a stellar resume and cover letter with references from people you don't know.

Now of these resumes, let's go through the hiring filters that work... 40% of the time... 30% of the time... 20% of the time... 10% of the time...

Ready? Okay, you know what, let's not go through all that. If you are trying to filter the hires so that you're getting the best of it like this, well, what can I say. Good luck pal.

A, B and C are never really defined in these discussions, so I'm going to use my perspective:

C programmers can't do FizzBuzz[0]. B programmers can, and LOVE PUZZLES. A developers are the ones whose idea of a "puzzle" is solving the fundamental business problem via software. B programmers think that super elegant engineering is great, and don't care about business so much.

These B players are also the ones who often will torpedo a candidate by asking them trick questions (trick programming questions, irrelevant or obscure interview questions, etc.)[1] These questions are great for making the B player feel superior. Puzzles are a good way of doing this. This way if the candidate is hired, the B player's ego is mollified by knowing he was able to trick the candidate during the interview, and if the B player decides the candidate isn't deferential, he can claim he doesn't have the engineering chops for the job.

I sometimes wonder how many of these complaints about people "not being able to do FizzBuzz" are from these B players. (I've never used FizzBuzz as an interview question, I don't ask programming questions, but I've never managed to hire someone who couldn't do FizzBuzz for a programming role.)

At any rate, one of the things that makes interviewing at a given place a total roll of the dice is when you see evidence that they don't know what they're doing. An HR person trying to screen you who doesn't know the difference between java and javascript, for instance.

But another signal to me, is these puzzles. These puzzles scream to me "This company is run by B players!"

For instance, this is how I knew Google was not a company I wanted to interview with (even though they pursued me relentlessly for about 6 months, and still pester me periodically)... before I had the inside info that confirmed what the puzzles indicate.

I also don't believe in the "A people hire A people and B people hire C people" mantra, because I worked at Amazon (A C organization if there ever was one) and Bezos liked to say this (a D manager, in my book... but obviously he thought he was an A.)

I'm sure to some people, I'm a B player, or a C player (certainly the puzzle fanatics might think that) but I have seen a lot of companies get lost spending all their time working on clever engineering and not solving business problems. I like engineering problems and solving them, but the business problems are the really interesting ones-- its amazing what software can do to solve these problems, and often, non-engineering management has no way to know, which is why specs often don't work because thy ask for only what they think is possible.

So, I'm not being a snob here. I'm just saying- I use these puzzles as a filter. I wouldn't apply to such a company (if I was still working for others.)

[0] Theoretically, anyway. I'm a bit dubious about this claim-- all the programmers that I've seen who were a bad fit and "couldn't' do the work" were actually good programmers, just dealing with bad management. I have met a lot of non-programmers who think they are programmers and who only can do things that fit their pre-determined ways, e.g.: They're trained in a set of foundation classes and can't solve problems without using those classes (which have already solved things like sort, etc.) I assume these are the people we hear about when someone says "can't program fizz buzz".

[1] For example, once I was interviewing for an internet infrastructure job and got quizzed on Win32 API details (like "what are the parameters for this obscure graphics call?" "I don't know, haven't looked at it in years." "It says here on your resume that you know Win32!". Similar example- interviewing for a Mac development position in the PowerPC era, mentioned I'd done assembly in Z80 back in the day and had a little familiarity with 8086 assembly-- getting questions about the Pentium instruction set. I could answer broad questions, because I had a high level exposure, but I only claimed to study 8086 back in the day, not be writing assembly currently, and it was for a Mac development job, and he wasn't even talking about the PowerPC.

>A developers are the ones whose idea of a "puzzle" is solving the fundamental business problem via software. B programmers think that super elegant engineering is great, and don't care about business so much.

I couldn't agree with this more, although I would say "B programmers love to solve every problem. A programmers love to solve the problems that matter."

I have worked with many people, some quite brilliant, that simply can't prioritize issues. If they perceive something as broken or sub-optimal, they absolutely must fix it, no matter how minor the problem. If you can reign them in and keep them focused, this type of programmer can produce great things. But it takes a lot of management time and energy. I'd rather work with someone who can see the larger picture and focus on solving the actual problem, even if the implementation isn't 100% perfect.

I'm curious why you think Amazon is a C organization. They make fantastic products. Their main store, AWS, and the kindle are all pretty great and well ahead of their competition.

I've heard bad things about working for them, but as a consumer, I love Amazon.

I'm talking about the quality of the products and the management. The Store does great because they have good customer service-- I don't fault them there-- but it is still chicken wire and duck tape from an engineering viewpoint.

AWS has gotten a lot of hype, as has the kindle, but I don't think either of them are ahead of their competition. My point though, was internally.

Amazon is full of B and C players who think they are A players, and that goes all the way to the top. (There are some A players there too, of course, but they often leave within a year or two.)

i think you're mixing several different things together, based on your personal prejudices. by your metric, i'm definitely a B programmer - i am far more passionate about making sure a product is elegantly engineered than about the specific business requirements of that product. all that means, though, is that i need to recognise where my strengths and my limitations lie, and am thereafter a pretty valuable person to have on your team. i'd never try to be a product manager, and i have no ego problems when it comes to working under a lead programmer, or deferring to someone who understands the customer better than i do when arguing over features. i will also make sure that the code that gets checked in not only solves the problem at hand, but is easy to maintain, extend, reuse and integrate into future code (which, at the end of the day, is what elegance is all about).

i also enjoy puzzles, and would probably be one of the people who solved the puzzle in the interview screening test just because it was a fun thing to do. i would never, however, use tricks or puzzles in an actual interview, and my focus is strongly on encouraging the candidate to put his or her best foot forward, and supporting them while they do so. an interview that lets me get a feel for the best the candidate is capable of is far more valuable than seeing who does and doesn't get tripped up by some random trick or either-you-get-it-or-you-don't puzzle.

and finally, i now work at google, and my interview was pretty straightforward algorithm problems - no "how would you move mount fuji" conundrums or random rec.puzzles-style brainteasers. i'm not sure if google actually did use that sort of question at one point, but they certainly didn't last year. (anecdotal evidence, yes, but i had six different interviewers, and surely at least one of them would have trotted out a puzzle if it was still a prevalent practice).

Honestly, the egotism displayed in this post is rather laughable. You claim companies like Google or Facebook don't hire A players, not realizing you're doing the same thing you claimed B players do (write others off who don't fit their arbitrary criteria as a way of feeling smug about themselves).

Any criteria that says Google doesn't hire A players is a joke, or completely meaningless.

> Bezos [...] a D manager, in my book... but obviously he thought he was an A.

Can you expand on this? Bezos seems to be doing a spectacular job at Amazon.

Yegge's now-famous rant paints a really different picture. https://plus.google.com/112678702228711889851/posts/eVeouesv...

The lesson to be learned here is if you can do something different and get people to look at and talk about your posting, more people will apply because more people will see it. If the place all of those 'more' people see it is a place frequented by A players (such as HN), more A players will apply.

In other words, this was a marketing/PR win - not a win in terms of how you are asking people to apply.

Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact