
Programming Puzzles Are Not the Answer – How Spreedly Does Work Samples - ryandaigle
http://engineering.spreedly.com/blog/programming-puzzles-are-not-the-answer-how-spreedly-does-work-samples.html
======
matt_s
I applied a few months ago to a position at Spreedly and welcomed the code
sample process. It is only one part of their hiring process, like the article
mentioned. I loved getting feedback on the results too, it turned out someone
had a better score and that's ok, I knew exactly why and actually learned
something about that problem. Most jobs you apply for and don't get you just
hear deafening silence back.

It also gives you an idea of what the company focuses on - delivering working
software. I'm not saying companies that don't do work samples don't focus on
that but as a candidate it makes you wonder if playing games is part of the
job. You gotta talk about real code, look at real code and discuss it.

I have also been a manager and sat on the other side of the hiring table and
it is hard to judge the technical chops of people. We used quizzes of deep
technical items that should have been known by someone if they had done the
work before (e.g. MTU settings for a db cluster) and that helped weed out
people but it really wouldn't give a sense of if they were a 100% good fit,
you need other ways to check that too.

I like the analogy of developers to chefs and cooking. Programming puzzles is
like judging a chef on cute tricks to slice an onion. Would you hire a chef
based on their onion slicing tricks or their ability to cook an amazing meal?

------
peter_gibbons
I recently applied for a software engineer position at spreedly. I looked into
what spreedly did and thought I was a perfect fit, and I still think that. I
thought I did good on my work sample all things considered (see below).
Spreedly’s metrics did not agree. One thing that I really liked about spreedly
was the fact they told me why I failed and what I failed on. Other companies
are treating their hiring process like a trade a secret; if you’re trying that
hard, it’s probably not working IMHO. Most of the spreedly work sample was
great. You had to dig into source code, reason about the code base, etc. What
I did not like was the gotchas. There were not a lot, just a few, but it’s
part of the reason I wasn’t considered. I don’t have time for games, pure and
simple. I’m never going to put the time or effort into a code sample that I
would as if I was getting paid. Not when I have real work to do, not when I’m
playing the job application lottery and I might go bust vesting too much time
with one company’s code sample. It seems almost narcissistic of a company to
ask for that much. It’s kind of analogous to the fruitless question: “Tell us
why you want to work for Hooli?” The other feeling I consistently walk away
with is that I’m a guiena pig; I’m a beta tester for your hiring process and
that does not feel good one bit.

All of that said, I have to say spreedly’s work sample was the most realistic.
During the process I did feel like there was a purpose to what I was doing,
which is great for someone like me who has a hard time doing “puzzles” and the
like because I see no utility in it.

Do I think blind hiring works? It could, but not the way it's being utilized.
When I first heard of blind hiring, I thought this is great. No bias, no bs,
just code. I've been on the hiring side of the table before and 90% of the
applicants were full of it in regards to what they put on their resume vs what
they actually knew. The worst part is after you spend the whole day
interviewing worthless candidates you have all your regular work that has
piled up during the process that must be done. A lot of time and energy gets
expended in that process and wasted. I truly understand both sides. As a tool
to filter out "those people", it's great. However, blind hiring seems like it
being used to try and find the perfect candidates. Perfect candidates do not
exist; just as perfect humans do not exist. Great test takers do exist though.
Personally, I have never correlated good test takers with good employees or
good programmers. In fact, it’s always seemed to be the opposite from my
experience.

    
    
      So, what’s a better way to skin this cat? I honestly don’t know, but if I think of it I’ll be sure to make a proprietary app and sell it to all the companies I’ve done blind samples for : )

~~~
ryandaigle
OP here.

This is great, Peter, thank you for this perspective!

If you walked away with the impression that there were gotchas, then we have
failed. While we do have specific criteria we're looking for (as we're trying
to make the grading of each work sample as objective and consistent as
possible), I wouldn't call those items "gotchas". It's also not the type of
thing where if you miss one small item, you're disqualified. The grading
allows for several missed items and is only looking for some total of all
allowed points (across 33 different criteria). So while some of the items you
didn't get credit for might have appeared trivial, or like a gotcha, there
should be enough allowance in the process to not let one or two misses scuttle
the whole effort.

> Perfect candidates do not exist; just as perfect humans do not exist. Great
> test takers do exist though. Personally, I have never correlated good test
> takers with good employees or good programmers

We are totally with you re: perfect candidates. Work samples are only one part
of the hiring process after all, and are meant to ensure some base level of
technical ability. Others parts of the process, such as our structured
interviews, are meant to flesh out some of the nuances I believe you're
referring to here. We're not looking for a perfect candidate, but we are
looking to understand what makes up each candidate to know if their strengths
are a fit for Spreedly and if we can live with the weaknesses (and vice versa
for the candidate evaluating Spreedly!).

We really do appreciate your effort, both in writing up your thoughts here,
and in applying in the first place. This is how we get better at this kinda of
thing!

~~~
peter_gibbons
Just wanted to add. Some of my comments are not directed at spreedly but about
the whole shimmy shaw in general.

Oh, and remember... next Friday... is Hawaiian shirt day. So, you know, if you
want to, go ahead and wear a Hawaiian shirt and jeans.

