Intriguingly, that's the opposite of what's popular on HN. HN and /r/programming posters prefer the "Look at what I've done. Here are my projects" approach which is the y-intercept because it describes where they are. Asking them a question they've never heard before and observing their approach is a way (albeit imperfect) of seeing how fast one can deal with a novel problem.
Not necessarily, it can also describe the slope, based on the timestamp of the projects.
4 years ago I did this, I did this last year, last month I tried my hand at this. However, it requires the interviewer to have the skill to extract this information during the interview and recognise the slope.
"Look at what I've done. Here are my projects" is definitely not the Y-intercept "Look I went to <fancy school>" is the Y-intercept. With projects and experiences you can start to plot someone's work according to it's own Y-intercept and guess the slope. The hard part is assessing someone's slope when all you have is it's Y-intercept ( freshly out of school ).
Concerning asking tricky question, you can do it the Google way ( How many frogs does it take to build a space elevator ) but even they decided to stop these kind of questions or take a lot of time and find something your candidate can work on, even if the technology is known to him, seeing how fast he can appropriate himself the codebase is really a good way to judge your future new recruit potential
"Look at what I've done" is neither the y-intercept nor the slope in this increasingly stretched metaphor. It's a definite integral (area under the curve).
You could try to "guess the slope" if the slope were static, but it's not. Aptitude and track record are, in my experience, not correlated. I can get specific about this if you want, but I don't want to be tiresome about this subject (you can find similar comments in my history).
A short preemptive summary is: the last 10 amazing hires we made would not have been distinguishable from their online portfolios.
They appear to be the only software company on Earth to do work sample tests. It's working out really well for them. The process is extremely well laid out here.
matasano.com/careers
We'll get on the phone and talk to you about the company and what our work looks like. At the end of this call you should have a good idea of what we do, how our hiring process works, and answers to questions about Matasano. Most importantly, you'll have a contact at Matasano to talk with and bounce questions off of through the duration of our process.
We do 1-3 technical phone screens. You'll talk to a senior Matasano team member who will ask you about your technical background and talk you through scenarios and concepts from our day-to-day work. If you've been doing app security for 5 years, you'll be talking about your past projects; if you're a developer, you'll be talking about code.
We do a web app challenge. Most software written within the last several years is web code. Everyone on our team needs to be able to deliver a solid web pen test. When you're ready, you'll be given an instance of a vulnerable web application and an hour or so to break it. We timebox challenges to avoid taking too much of your time. You're doing this on your own schedule, in your own comfortable setting.
We do a custom protocol challenge. Every Matasano team member routinely runs into exotic network protocols. We'll throw something at you that you're unlikely to have worked with before and watch you reason your way through breaking it. This challenge seems to be everyone's favorite; candidates routinely tell us how they particularly enjoyed it. That's great! It's part of our day-to-day work here. Like the web challenge, it's timeboxed and you're doing it remote.
We'll have you write a fuzzer. Everyone here writes fuzzers. We'll give you a file format. In the language of your choosing, you'll write a fuzzer for it. This gives us a chance to see how you code and to see what types of things you automate testing for. Like the other challenges, this one is time limited and you can do it remote.
We've talked. We've done phone screens. We've answered questions. You've done challenges for us. At this point we both have a pretty good idea whether you'll be happy working with us. If that's the case, we'll bring you onsite for an in-person interview, which concludes our hiring process.
