One of Ousterhout's examples of this principle is something that comes up frequently on HN:
"Before I came back to academia a couple of years ago I was out doing startups. What I noticed is that when people hire they are almost always hire based on experience. They're looking for somebody's resume trying to find the person who has already done the job they want them to do three times over. That's basically hiring based on Y-intercept.
Personally I don't think that's a very good way to hire. The people who are doing the same thing over and over again often get burnt out and typically the reason they're doing the same thing over and over again is they've maxed out. They can't do anything more than that. And, in fact, typically what happens when you level off is you level off slightly above your level of competence. So in fact you're not actually doing the current job all that well.
So what I would always hire on is based on aptitude, not on experience. You know, is this person ready to do the job? They may never have done it before and have no experience in this area, but are they a smart person who can figure things out? Are they a quick learner? And I've found that's a much better way to get really effective people."
Intriguingly, that's the opposite of what's popular on HN. HN and /r/programming posters prefer the "Look at what I've done. Here are my projects" approach which is the y-intercept because it describes where they are. Asking them a question they've never heard before and observing their approach is a way (albeit imperfect) of seeing how fast one can deal with a novel problem.
Not necessarily, it can also describe the slope, based on the timestamp of the projects.
4 years ago I did this, I did this last year, last month I tried my hand at this. However, it requires the interviewer to have the skill to extract this information during the interview and recognise the slope.
"Look at what I've done. Here are my projects" is definitely not the Y-intercept "Look I went to <fancy school>" is the Y-intercept. With projects and experiences you can start to plot someone's work according to it's own Y-intercept and guess the slope. The hard part is assessing someone's slope when all you have is it's Y-intercept ( freshly out of school ).
Concerning asking tricky question, you can do it the Google way ( How many frogs does it take to build a space elevator ) but even they decided to stop these kind of questions or take a lot of time and find something your candidate can work on, even if the technology is known to him, seeing how fast he can appropriate himself the codebase is really a good way to judge your future new recruit potential
"Look at what I've done" is neither the y-intercept nor the slope in this increasingly stretched metaphor. It's a definite integral (area under the curve).
You could try to "guess the slope" if the slope were static, but it's not. Aptitude and track record are, in my experience, not correlated. I can get specific about this if you want, but I don't want to be tiresome about this subject (you can find similar comments in my history).
A short preemptive summary is: the last 10 amazing hires we made would not have been distinguishable from their online portfolios.
They appear to be the only software company on Earth to do work sample tests. It's working out really well for them. The process is extremely well laid out here.
matasano.com/careers
We'll get on the phone and talk to you about the company and what our work looks like. At the end of this call you should have a good idea of what we do, how our hiring process works, and answers to questions about Matasano. Most importantly, you'll have a contact at Matasano to talk with and bounce questions off of through the duration of our process.
We do 1-3 technical phone screens. You'll talk to a senior Matasano team member who will ask you about your technical background and talk you through scenarios and concepts from our day-to-day work. If you've been doing app security for 5 years, you'll be talking about your past projects; if you're a developer, you'll be talking about code.
We do a web app challenge. Most software written within the last several years is web code. Everyone on our team needs to be able to deliver a solid web pen test. When you're ready, you'll be given an instance of a vulnerable web application and an hour or so to break it. We timebox challenges to avoid taking too much of your time. You're doing this on your own schedule, in your own comfortable setting.
We do a custom protocol challenge. Every Matasano team member routinely runs into exotic network protocols. We'll throw something at you that you're unlikely to have worked with before and watch you reason your way through breaking it. This challenge seems to be everyone's favorite; candidates routinely tell us how they particularly enjoyed it. That's great! It's part of our day-to-day work here. Like the web challenge, it's timeboxed and you're doing it remote.
We'll have you write a fuzzer. Everyone here writes fuzzers. We'll give you a file format. In the language of your choosing, you'll write a fuzzer for it. This gives us a chance to see how you code and to see what types of things you automate testing for. Like the other challenges, this one is time limited and you can do it remote.
We've talked. We've done phone screens. We've answered questions. You've done challenges for us. At this point we both have a pretty good idea whether you'll be happy working with us. If that's the case, we'll bring you onsite for an in-person interview, which concludes our hiring process.
I used to think this was right (trained in the school of Joel), but now I'm not so sure. I look at it this way: the best predictor of future success is past success. School is pretty non-predictive of your success in industry. General "intelligence" is as well. The best thing I can look at is: do you have a track record of tackling problems and delivering solutions? If you do, you'll probably keep doing that. If not, you're at best a gamble.
But what if you just need the job done? Sometimes you're hiring for a position you expect to make a deep impact on the success of your startup, sometimes you just need the project written. In the latter case Ms. Intercept might get the job done in a week while Mr. Slope will take six months in some cases. All the aptitude in the world won't compensate for experience when time is a factor. Experience knows how to do the job on day zero, even if they aren't any better at it after they're done. The time you save from hiring Ms. Intercept as your first programmer might even buy you some extra time to find exactly the right person to be your superstar hire.
> And if the Y-axis is something good, depending on your definition of something good, then I think most people would pick the red trajectory over the blue trajectory (..unless you think you're going to die before you get to the crossing point).
When hiring, I agree that aptitude is probably a better signal than experience. However, I also think aptitude is much harder to assess during an interview. It's pretty easy to look at someone's work history, github, and ask them a few questions about what they currently know. I'm not sure if I know a truly effective way to figure out their aptitude in a short amount of time.
There was an article about it making the rounds a few days/weeks ago. A guy basically takes on a class of 30+ people for two days when hiring, teaches them something, then hires the one who did best. The rest are still happy to have learned something.
Another good approach might be flat out asking "Tell me about the last time you learned something new"
Good points. I would add to his last point though that hiring at least one person with a high y-intercept can increase the slope of your other hires significantly. (It's a lot easier to learn something quickly if you have someone around who already knows a lot about it.) And people who both know a lot and learn quickly are expensive and rare, so I expect the optimal hiring process would look for a large percentage of relatively inexperienced fast learners, along with a small (but non-zero) percentage of more experienced people, even if they aren't likely to develop as quickly. (Perhaps consultants would fit well in that role too.)
Ah, Ousterhout. Guess he's at Stanford now; he was a prof at Berkeley when I was there. He had an amazing knack for presenting CS in an entertaining (and, at times, hilarious) manner.
It's a bit of a false dilemma to choose between experience or aptitude.
In the start-up period, though, aptitude is a better strategic choice in that you need people who can grow as quickly as the venture.
Later, you need the people with experience, even if they're a bit slow. They're the ones that have been bitten by the edge cases, the deadlocks, the XSS openings - and know to avoid them.
Really? I'd rather have the experienced but dull people to put together the minimal product first. Having them onboard gives me more time to find and hire the right aptitude people, and having them there _first_ keeps the inexperienced people from sticking in all the deadlocks and security holes in the first place.
Do people really want their minimally-viable product to be full of deadlocks? I personally do not. And I don't want to hire promising but new-to-the-industry people who have to check stackoverflow to remember the difference between a pointer and a reference, or who spend hours trying to debug why something in their python program is iterating over a string.
Bottom line, I guess I don't see how anyone can want to bring in aptitude in the absence of experience. People with both would be fine, or a combination of both types of people would be fine. Aptitude without experience is not going to work.
I happen to disagree regarding the hiring. I prefer to put more emphasis on the system itself, and not the developers, who should be replaceable. (I say this as a developer myself, who prefers to be replaceable.)
When a developer forgets to test their code before pushing it to production, we often blame the developer. But the real problem is lack of automated testing, lack of processes, and too much responsibility for the developer.
With a good system in place, you hire people who have all the prerequisite knowledge (the languages, patterns, experience with similar solutions to the ones they'll do, and preferably good team spirit that matches your culture). The rest can be learned on the job. But once again, focus on your onboarding materials!
In short -- you should always look to be optimizing the system. THAT is your "slope" if you will. Except it's not a slope, it's an exponent! Because it builds on itself week after week. And you don't risk that one developer somewhere messing up your code.
We say: people live lives, companies create products.
Probably everyone knows the math; I just think they're exaggerating the relevance of this model to real life.
I'm with the commenter (here) who said that real life is more likely to be an initial-value-dependent or path-dependent PDE. Someone else said having a high-IV person on your team will raise the slopes of everyone else which seems also right: the issues are multidimensional, not affine 1-D.
Ousterhout is taking sides (smarter > more experienced) which is fine; he can make that argument. But using y=mx+b as x→∞ doesn't count as an argument; it's rhetorical flair, not rhetorical substance. The substance of his reasoning seems to be "That's my opinion based on my experience in my past jobs".
Analogies are inexact by definition, which doesn't mean that the speaker is being intentionally deceptive by using one.
I don't think anyone took this beyond the anecdotal evidence provided. We can argue motive and hypotheticals all day but the fact remains that this is just a blurb thrown at fresh undergraduates.
It's a motivational speech (delivered via heresay on Quora) not a thesis.
If you read the full answer, you'll realize that he points out you'll pick the better slope, unless you think you'll be dead before the two lines intersect; so he clearly understands it.
If you're going to pick apart the math, we might as well pick apart that not everything can be modeled well as a linear function.
Why do Quora articles continue to make the front page when we can't read them? I'm greeted with http://i.imgur.com/WTeSt8s.png No, I won't sign in with Google, nor Facebook.
How do the people upvoting this read the article? Do they sign in? What possible benefit is there to signing in? They're basically holding content hostage.
When I defended Scribd, people came out in droves to point out how wrong it was to hold unique content hostage. I'll admit, it made me rethink my position. But it's strange to see that Quora doesn't get the same stick.
EDIT: If the HN homepage had a popup saying "Login with Google or Facebook to read all of HN," would you tolerate it?
To be fair, when the Scribd links were first implemented, 1) in-browser PDF was much worse than it is now; and 2) Scribd's interface was better than it is now. Over time, browsers got better at showing PDFs, and Scribd got more cluttered and weird.
I can't speak for others, but in my case, "difficulty of use" also refers to "you have to go through all these additional steps just to get to the document". I can't remember if that was only for grey-area, restricted-access postings, or if it applied to all Scribd documents, but I was conditioned to assume the latter by the time I saw them on HN.
Opening a PDF in browser is more convenient. That way the PDF behaves like another browser tab. Opening a PDF in an external viewer could be only one click and one alt+tab more, but it's less convenient and reduces usage. And it would leave one extra file in your downloads folder.
Speaking only for myself, I visit Quora so frequently that I pretty much always have a login cookie stored, so I never see the login page.
That said, given my very public and vocal arguments against "walled gardens" in the past, why do I tolerate Quora? For the same reasons I tolerate G+, Facebook, etc. I annoys me and tickles a sore spot regarding the way I want things to be, but the value of the content is sufficient for me to "grin and bear it". But if there were a really good Quora alternative that was more open, I'd work to promote and support it. I don't know of one offhand though, especially considering network effects (that is, the value of Quora isn't the Quora software, it's the people posting and answering).
I have a similar position. If the content is valuable enough then it's tolerable to have a gate. Quibb is another similar platform, but I don't know of anything with the quality of Quora answers - Stack Overflow is the closest, but it's very specific and Quora covers a lot of content the various Stack sites don't.
Exactly. I really don't understand what's wrong with signing in to access content, it shows the site owner I'm actually interested in engaging. Facebook isn't going steal my soul when I send them my credentials, you know.
There's a button right there that says 'close'. I assume you just didn't see it. Granted, I'm not a fan of this either - but it's hardly holding the content hostage.
I never noticed the Close button before, so I tried it w/o the ?share=1 in another browser. I get "Close & Read First Answer". All the answers after the first were blurred. I think this is the first time I've seen a site A/B test how best to annoy people.
Worse than being simply annoying and deceptive (since there are plenty of ways to get around it if you care), it's also literally useless.
For example, I just went into Incognito on another machine out of curiosity. I unclicked 'I am 13 or older' and then clicked on random grayed-out whitespace, and it showed the full article. Unless they kept my home IP from previous requests, but at that point, why bother with the age gate? Lawyers?
Not sure how anyone can defend this deceptive UX bullshit.
That's probably your problem. I signed up for Quora and can read the posts fine. Also, there's a 'close' link under that dialog, which you could have clicked without logging in.
> If the HN homepage had a popup saying "Login with Google or Facebook to read all of HN," would you tolerate it?
Yes. Why not? It seems childish to rail against this. If a site is useful, I create an account. I did that for HN, and it doesn't matter if the sign-in mechanism is proprietary, Google based, Facebook or OAuth. In fact, I trust sites that use external, federalised SSO more because I know they didn't roll their own authentication with associated failures.
"Before I came back to academia a couple of years ago I was out doing startups. What I noticed is that when people hire they are almost always hire based on experience. They're looking for somebody's resume trying to find the person who has already done the job they want them to do three times over. That's basically hiring based on Y-intercept.
Personally I don't think that's a very good way to hire. The people who are doing the same thing over and over again often get burnt out and typically the reason they're doing the same thing over and over again is they've maxed out. They can't do anything more than that. And, in fact, typically what happens when you level off is you level off slightly above your level of competence. So in fact you're not actually doing the current job all that well.
So what I would always hire on is based on aptitude, not on experience. You know, is this person ready to do the job? They may never have done it before and have no experience in this area, but are they a smart person who can figure things out? Are they a quick learner? And I've found that's a much better way to get really effective people."