Consensus seems to lean toward this being satire. However, that’s actually part of the problem with online discussions of interviewing right now: Everyone loves to read and write about how developer interviewing is flawed, but no one wants to go out on a limb and make suggestions about how to improve it. It’s easy to be a critic and it’s fun to be smugly dismissive of those companies who rejected you after long interviews, but it’s hard to actually step into the hiring manager position and find the best candidate without disappointing some people along the way. Writing satire or criticism is an easy way to win clicks and views because you’re not going to alienate people with the tried and true “all interviewing is bad” claim.
Sadly, many of the implied alternatives to coding interviews are becoming a regression toward credentialism: People who have already secured good jobs, impressive internships, or prestigious university degrees would much rather coast on those credentials than be challenged for a few hours during an interview. This feels good if you’ve already made it or you grew up with the right connections, but it’s far from ideal you’re a promising developer who doesn’t yet have the right credentials on your resume to unlock the job. To many people, the coding interviews is a chance to break the traditional mold and land a job that would have been out of reach otherwise.
> Everyone loves to read and write about how developer interviewing is flawed, but no one wants to go out on a limb and make suggestions about how to improve it.
I've spend this year writing about hiring[0], both to talk about the problems and provide suggestions on how to improve the process. My suggestions are sometimes high-level, and sometimes very detailed. Some examples:
- Be specific about the requirements [1]
- Tailor the requirements to the role you're hiring for [2]
- The types of non-technical issues that can cause a candidate to perform poorly, and how interviewers should correct for them [3]
- Let people work independently [4]
But above all, my primary thesis is that it comes down to looking for strengths, not trying to discover weaknesses [5]. Having been on the hiring side, this is the specific suggestion that I believe will have the most impact, especially to those who want to land a job that would have been out of reach otherwise.
> But above all, my primary thesis is that it comes down to looking for strengths, not trying to discover weaknesses [5].
I strongly agree. I think a generation of software engineers has been trained to calibrate their interviews by looking for reasons not to hire someone, instead of looking for reasons they should.
This perpetuates all sorts of biases in the hiring process, and it insidiously serves as a form of self validation as well.
The problem is that you can find a reason to hire almost anyone. Now most companies just need someone to write boring line-of-business applications so maybe that's okay, but if you're working on anything novel hiring the wrong person can be very expensive. I don't think there's any intrinsic negative in keeping a high bar for hiring.
Looking for strengths isn't the same as lowering the bar. If you have specific requirements, you will maintain those requirements. But, the goal is to push aside weaknesses that don't matter (such as nerves during an interview, or lack of truly unrelated knowledge).
Instead, the goal should be to ensure _qualified_ candidates can demonstrate the strengths they have in meeting your requirements.
So basically, there are two pieces to looking for strengths:
1. Determine which requirements truly matter. Even for highly technical jobs, interview skills are not what makes a successful employee.
2. Find ways for those who meet the bar to demonstrate that fact.
I think most people vastly overestimate the novelty of what they're working on. Most people writing software at Google actually aren't doing more than gluing APIs together on a day to day basis. In that sense it's much more important for them to have a solid understanding of reliable infrastructure and system design than algorithms and data structures. Indeed, this shows in the way they interview for people who aren't new grads.
The author has stated that this article was not meant as satire, even though some parts are exaggerated[0].
I do not like much of this advice, even if some of it may be true. For example, I believe that the sum of a team can be greater than its parts, and hiring for talent alone can result in a poorly functioning team.
I think it is and it isn't. The author definitely believes in testing for people's IQs, so I think the whole section on "identifying talent" is meant to serve as a reductio ad absurdum.
Some of the author's tweets about identifying talent, IQ, and temperament:
“ People who have already secured good jobs, impressive internships, or prestigious university degrees would much rather coast on those credentials than be challenged for a few hours during an interview. This feels good if you’ve already made it or you grew up with the right connections, but it’s far from ideal you’re a promising developer who doesn’t yet have the right credentials on your resume to unlock the job.”
Perhaps we could interview different people differently?
I don’t know why we have this obsession with subjecting experienced candidates to the same whiteboard gauntlet as a new grad. In fact, after a candidate has a bunch of experience you’d better be screening for other factors, since the role of an experienced developer can/should be dramatically different than someone with a
few years of experience.
But no, still with the “challenges”. If you can’t do obscure puzzles with the same fidelity as you could six months out of college, it’s a no-hire!
I can't tell you how many people I've interviewed that were "experienced" developers that were just terrible terrible engineers.
My favorite interviews were when we were trying to hire a senior level engineer, and we were only interviewing people with 10+ years of development experience. We'd ask people to build a priority queue. Items with a higher priority need to be handled before those with a lower priority, but otherwise items should be handled in order.
Can you guess what the most common methodology people chose was?
Did you guess an XML document?
These are people with degrees in comp sci, some claiming 15 years of experience as a developer, trying to get a role at a top 10 bank in New York City.
Of course technical interviews shouldn't be the only test used for hiring, but they absolutely should be included. Maybe it's demeaning that you have to jump through these hoops. Or maybe you're bad at them. Or they're "beneath you" in some sort of way. But don't blame the interviewer, blame the hundreds of other people trying to get a job that they aren't qualified for.
Most arguments I've seen claiming the incompetence of senior engineers are circular.
A: We need programming quizzes even for senior engineers, because many turn out of be incompetent.
B: How do you know they're incompetent?
A: They failed our programming quizzes!
The reality is that you're testing for stage fright, not programming skills. An audition is not a good criterion for choosing people who aren't stage performers.
Stage fright does not lead a senior level engineer to build a priority queue using XML.
Assume good faith: let’s presume he knows some candidates get stage fright and has ways to help mitigate it. Or that despite stage fright, such tests are still a good signal (e.g. the signal is so strong the false negatives are a worthwhile cost, or stage fright might be highly correlated with poor performance, or stage performance could be highly correlated with talent!)
I think that the idea is to optimise for preventing false positives (bad hires), at the cost of a higher rate of false negatives (good hires incorrectly weeded out). Interviews are imperfect: candidates can fail for a vast range of reasons that are completely unfair.
Also an applicant could train to avoid stage fright, just as they might train to learn algorithms if applying to Google. AFAIK many real actors have all the symptoms of stage fright, but they have learned to deal with it to be successful.
It's not nitpicking. I want to draw attention to the fact that the so-called "false negatives" that everyone claims are acceptable casualties of the system are not just random false negatives. Audition-style coding interviews systematically discriminate against people who tend to experience extreme nervousness during auditions but otherwise perform normally on the job.
Note that this thread started with "Everyone loves to read and write about how developer interviewing is flawed, but no one wants to go out on a limb and make suggestions about how to improve it." Now you're saying applicants can train for this, applicants can train for that, but what happened to the original point about how to improve the process instead of making candidates take difficult steps to adapt to the flawed process?
I know I am being oppositional, but your style of argument comes across to me as deep diving on points where you can be correct, or changing the subject. I am trying to argue against your original thesis which I will paraphrase as “a quiz detecting incompetence is actually detecting stage fright” https://news.ycombinator.com/item?id=24758092
Anyone who thinks XML is a valid solution to a priority queue is incompetent (even if they were prompted by the recruiter as said in sibling comment). https://news.ycombinator.com/item?id=24758234
Stage fright is real, and it is unavoidable in an interview i.e. even if coding quizzes were removed then the problem still exists, so arguing just against quizzes is silly IMHO.
> everyone claims are acceptable casualties of the system
I would say instead that casualties are unavoidable - interviews must be imperfect because time is limited (regardless of how much we may try to improve interviews.)
> systematically discriminate against people who tend to experience extreme nervousness during auditions but otherwise perform normally on the job.
Interviews are by definition systematically discriminatory: one is trying to select one person. The data is dirty, and if you make interviews more inclusive on one dimension you are discriminating against others on that dimension. They must be systematic as per anti-discrimination regulations, or else you will get sued to your cost.
How to improve interviews to reduce false negatives is hard, but I think you are oversimplifying it down to “quizzes are bad”, which is at odds with many smart professionals who say they are good for detecting incompetent developers. You have not suggested any improvement that detects incompetence.
> Note that this thread started with "Everyone loves to read and write about how developer interviewing is flawed, but no one wants to go out on a limb and make suggestions about how to improve it."
I can also cherry pick a quote further up-thread: “To many people, the coding interviews is a chance to break the traditional mold and land a job that would have been out of reach otherwise.”
> your style of argument comes across to me as deep diving on points where you can be correct, or changing the subject.
There's only so much that can be said in a HN comment. I'm trying to avoid writing a whole treatise.
> Stage fright is real, and it is unavoidable in an interview i.e. even if coding quizzes were removed then the problem still exists, so arguing just against quizzes is silly IMHO.
The conclusion doesn't follow. The problem still exists, yes, but I think everyone would acknowledge that casual chatting is easier than coding quizzes? Thus, if you tend to experience performance anxiety, then tasks that are more difficult to perform are more strongly negatively affected by performance anxiety. Compare with other issues that affect performance, e.g., drunkenness. Being drunk will affect your speech, but it severely affects your ability to more complex tasks such as driving a car.
> Interviews are by definition systematically discriminatory: one is trying to select one person.
That's not what systematic discrimination means. It means discriminating against a certain class of people who all share the same characteristic. Some examples are gender, race, and age, but that's not an exhaustive list.
> You have not suggested any improvement that detects incompetence.
I fundamentally disagree that this should be the main purpose and focus of hiring and interviews. I think rampant, exaggerated paranoia is one of the reasons that tech hiring is so messed up. The irrational fear of the "bad hire". Few other industries are so afraid of this. The reality is that it's impossible to avoid bad hires, and trying to be perfect in this respect will distort the whole process. Professional sports spend vastly more time and money on talent evaluation and acquisition than tech, but still they make "bad hires" all the time. Wasted #1 draft picks. Multi-million-dollar contracts for busts. It's just expected, a cost of doing business. They shrug it off and move on, and keep making mistakes too, because perfection is unattainable.
"How can we drop coding tests... while still being ultra-paranoid?" is not a question I can answer. My answer is to not be so paranoid, and just accept the inevitable mistakes, a more relaxed process which also allows you to find the "diamonds in the rough".
> I can also cherry pick a quote further up-thread: “To many people, the coding interviews is a chance to break the traditional mold and land a job that would have been out of reach otherwise.”
I don't know why it's cherry picking to restate the original context into which I first replied in this thread. In any case, I already replied to that quote too.
I am really struggling to follow your threads of thought.
I am going to throw in a patio11 spanner instead: ‘In a way, every scaling startup is an experiment in empirical microeconomics research on “What parts of the typical corporate form are necessary and which are pageantry which we only keep around due to anchoring, the sunk cost fallacy, and tradition?” Every time a startup bites the bullet and hires a VP of Sales, a lifecycle email copywriter, a retirements benefits administrator, or a cook, count that as a published result saying “Yep, we found this to be necessary.”’. https://www.kalzumeus.com/2020/10/09/four-years-at-stripe/
Failing an interview is embarrassing and leads to stage fright. I've seen no evidence that stage fright is a significant factor ruling out good candidates. If you are fluent at coding then a coding interview is just like any other interview, if that scares you then a normal interview would hurt you just as much.
If you aren't fluent at coding then why are you applying to a coding job? Would you apply to an English writing job without being fluent in English? If you just have a few years of English from school doing an interview in English would be extremely stressful and likely lead to stage fright. That doesn't mean that making interviews in English is a problem, the problem is that the person isn't fluent in English.
I remember that English fluency was a big barrier when I originally applied for jobs. Just speaking took a lot of mental power and the interviews were very stressful, but now that I'm fluent it no longer takes any effort. I went through the same stages in programming as well. I did a technical interview with a few months of coding experience and it was very stressful, now I don't even have to think.
Being asked to perform a complex task while someone else is both watching and judging you is way different from just talking through a non-technical interview. It’s an entirely different experience. Speaking a foreign language is hard, especially under pressure, but it’s absolutely nothing like trying to solve a brain teaser while the clock is ticking.
When you talk to someone you waste maybe 50% of your time/energy talking about what you do, meaning you effectively have 20 minutes instead of 40. Doubling the time you have available will make more people pass, yes, that isn't evidence of anxiety being a major factor.
Edit: Also looking at the data people spent more time on the problem in the private setting than the public. Their stats could therefore be as simple as the students not caring about the result and therefore they quit earlier in the more uncomfortable environment instead of doing it properly and checking their solution. This wouldn't be a thing in real interviews since there a lot is at stake so people wouldn't quit early, at least in my experience not a single candidate I've interviewed quit early.
Edit Edit: For example, their very impressive "All women in private completed it, all women in public failed" stat can also be described as "Women in private spent on average 16 minutes on the problem, women in public spent on average 7 minutes on the problem". Maybe if they had more at stake and therefore more stress they would have spent the entire 30 minutes and therefore done just as well?
Anyway, my point is that just because the abstract says they have clear evidence that stress is hurting performance these studies are still mostly garbage. Don't take it as proof just because there is a study.
Lots and lots of people have severe, physical, visceral responses to anxiety that can often be not only distracting but downright awful, feel-like-you’re-dying kind of feelings. This kind of anxiety response is entirely independent from day-to-day ability.
Common sense is often wrong. I have very bad stage fright, I have went mute in front of class several times. I still don't believe it is a significant factor filtering out good candidates. It might unfairly filter out some mediocre ones though.
> Most arguments I've seen claiming the incompetence of senior engineers are circular.
Exactly!
If someone has been implementing and delivering products for years/decades and they "fail" a whiteboard puzzle, which is more likely?
A: The whiteboard puzzle doesn't actually measure anything correlated to delivering products.
B: They can't actually develop anything, despite having repeatedly done so.
I want to hire the people who can develop and deliver products, not the whiteboard puzzle specialists because I've never been at a company where solving whiteboard puzzles is something we do as part of the business.
As someone who has worked at a few of the "top 10 banks in NYC" for most of my career (and still am...albeit trying to desperately get out), I have to say that the quality of much of the engineering talent at these firms, and the typical candidates that would apply there, are pretty low. Yes, I realize that the above statement can be construed as spitting on my own face in a way. So I am not surprised.
There may be a few small niches that attract good quality talent, but otherwise I don't know why anyone would opt to work as a technologist at a bank as opposed to say, a bonafide real tech company (all the banks are eager to say they are "tech companies" nowadays), or even a hedge fund.
> Can you guess what the most common methodology people chose was?
> Did you guess an XML document?
If that is your most common experience with asking that question, maybe it is time you consider whether you are communicating what you want properly to your candidates.
The real WTF was probably the hiring agency that the hiring manager insisted we go through. I'm pretty sure they were coaching their candidates, but we couldn't break away from that agency for some reason (cough kickbacks cough). The answers were suspiciously similar.
But I don't know how much better I could have described what I wanted:
"I want a class for holding a queue of items in memory. The queue will only need to hold a few thousand items, and you don't need to worry about persisting it in case of a power failure or anything. The only wrinkle is that when an item is added to a queue it has a priority. Higher priority items need to come out of the queue before lower priority items, but items with the same priority should come out in the order they were inserted."
I don't get it either. How did they suggest using an XML document to act as a priority queue? I actually would have found that highly amusing!
My naive contribution: A linked list with pointers to the tails of various priorities. The grabber just points to the head and pops things off the top as it goes.
> How did they suggest using an XML document to act as a priority queue? I actually would have found that highly amusing!
I tried. It got tiring. Sometimes people would loop through all possible priorities plugging them into an XPath query and pop the first element of the highest priority that matched. Some would do two XPath queries, one across all nodes to get the max priority and a second to query the first element (ignoring the lack of a need to run the query a second time at all). Some would insist that XPath has some function that would let you select a node with an attribute that has the maximum numeric value.
Did you ever actual hire any of those people? Did you ever verify that they actually had the experience they claimed? Did you follow up with their references? Were they able to walk you through projects they worked, or discuss solutions implemented?
Every job gets applicants that aren't qualified. Only software engineering out of every single non-performance based (musicians, actors etc...) industry out there has white board interviews.
Actually, yeah, we did. The hiring manager got tired of waiting for us to find a good candidate, so he just picked one that he had screened and seemed best on paper.
It was an unmitigated disaster. Constantly had to double check his work and redirect him from doing things bad ways (like using mutable objects with changing hash codes as dictionary keys kind of bad).
Eventually I quit the job (for other reasons). On the last day he cornered me in the elevator and offered me $100/hr to do his work for him under the table so that he could claim it was his.
> Did you ever verify that they actually had the experience they claimed?
Employer would only verify employment dates and eligibility for re-hire.
> Did you follow up with their references?
Yeah, but when you ask someone for references they're not likely to give you people who wouldn't speak well of them.
> Were they able to walk you through projects they worked, or discuss solutions implemented?
Yeah. I can walk you through projects that I was involved in as a junior as if I had been the lead because I paid attention during meetings.
> So you had one bad experience with one person who didn't do a whiteboard?
It was a bad experience with someone who utterly failed a whiteboard.
> How many bad experiences have you had with someone who did? I've had enough that I've lost count.
> I don't think a leet code question would necessarily have screened stuff like that out at all.
I'm sure I have too, but I never advocated for only whiteboard interviewing. There should be a spread covering all sorts of aspects of their abilities.
> Did you actually talk with these references about what it was like to work with him?
Of course. I have a wide variety of coworkers with a wide variety of opinions of my skill-set. I'd select the ones that I think would resonate best with you. Barring that I have half a dozen friends that would tell you whatever I want them to. I fail to see how references are any sort of magic bullet.
> That's $200k a year. I find it pretty hard to believe that he was serious.
I was another senior engineer on the project, so I definitely could have covered the work. I had deeper experience, and given his quality I could have churned things out more quickly. I suspect he wasn't offering me $100 / hour at 40 hours per week. The job had a lot of paperwork and bureaucracy involved. I suspect if I was just doing the coding parts I could have covered it at about 10 hours per week, which is a lot more feasible.
>Of course. I have a wide variety of coworkers with a wide variety of opinions of my skill-set. I'd select the ones that I think would resonate best with you.
Could it be that you have people who will give you good references because you're a good developer?
>Barring that I have half a dozen friends that would tell you whatever I want them to.
So if I pull those friends up on linked in, they're developers too, and you'll have some plausible overlapping work history?
> I fail to see how references are any sort of magic bullet.
It's not a magic bullet. But faking 10 years of development work history and multiple decent quality references is a fairly high bar. It's not going to catch everyone but neither is whiteboarding. The goal shouldn't be to catch everyone.
Sometimes you make a bad hire. Despite every test you put someone through, you'll get false positives. The problem is that if you ratchet your test up to far to avoid false positives, you end up turning off good engineers, or failing good engineers.
I think the problem is that the company wasn't willing to just admit they made a mistake and fire the guy, not that he was hired in the first place. America (assuming you're in the US) has nearly unrestricted at will employment. Why are you so afraid of hiring one bad employee? If you can't identify bad employees after a few weeks of working with, you have zero chance identifying them before you hire them.
Maybe instead of spending months trying to find the perfect candidate, just do what literally every other industry does and hire experienced candidates based on reputation, resume, talking to them about the job and past projects, and fire them if it turns out they were completely lying to you.
Conceptually I know what a priority queue is and how it works but I haven't built one, though I've used many. How satisfied would you be if an interviewee implementated it in pseudo code?
Thrilled. Probably a strong hire. I don't expect anyone to know what one is off the top of their head, but if I explain the goal to you I expect you to figure out roughly how to accomplish it without using XPath.
Now someone is going to use “Implement a priority queue using XSLT/XPATH given this XML data” as an interview question for a new hire... filtering for people willing to take the job even after being asked to do that!
Literally came out of our work queue. Couldn't find a satisfactory off-the-shelf component for it, needed one built. Afterwards we liked it as a technical interview question.
If I was doing it today I'd use some sort of balanced tree using the priority to navigate the nodes. Each node would have pointers to the head and tail of a singly linked list. Insert when the priority isn't in the tree is a simple tree insert with head and tail pointing to a new single-element linked list, insert when the priority is in there is adding it to the tail of the right node and moving the tail pointer, removing is finding the right-most node, taking the head and advancing the head pointer by one element. If the list is now empty then delete the node.
But none of it matters. I don't care if someone would build it the way I would. I would never expect that. I don't even want an optimal or semi-optimal answer. I'd have accepted a singly linked list with O(n) insertion time and O(1) pop time. Or a plain list where new items are added to the end and a pop is an O(n) seek through the list.
What I care about is whether the developer come up with any solution, describe the tradeoffs that they are making, why they are making them, and what research they would do if any to find a better solution. If i could have this conversation that we're having right here, right now with them then they'd probably pass my hiring bar.
A binary heap has some advantages over using a balanced tree that could have significant performance impacts (and vice versa). Particularly in a situation where you need to roll your own priority queue.
That's why I said it's likely the person who built it would done a bit of research. When hiring someone I care that they remember enough to realize that and are capable of doing the research, more than that they remember exactly how to implement some data structure.
>If I could have this conversation that we're having right here, right now with them then they'd probably pass my hiring bar.
I think that's a totally reasonable. My problem is that this isn't what people are complaining about when complaining about white board interviews.
99% of the time the interviewer is asking for the answer to a very specific problem that is highly biased towards new grads who would have just seen it on a test.
And even if they say you don't care about the optimal solution they will always hire the person who gets the optimal solution in 15 minutes b/c they memorized it versus the person takes 30 min to bang out something that kind of works.
Here's my approach:
I ask someone else on my team to come up with an interview problem, so I haven't seen it before hand. Then I work with the candidate on hashing out a solution.
Also lots of questions emerge when one is trying to be constructive.
Hiring is meant to have fidelity. Fidelity to what? To the extent there are company values, nobody explicitly “tests” for them in hiring. Certainly when I worked at US Engineering, which had an explicit values statement, I don’t recall being interviewed for how I “exhibit integrity in everything we do” or “ensure a safe working environment.” I doubt Amazon is like “so let’s talk customer obsession. when is the last time that you stalked a customer and got obsessed with making their life easy?” (I mean you can question the value as well, but to the extent that Amazon tells you its core values, are they a part of hiring?)
How much fidelity are we talking? Use the “nines” metaphor. Do you want five nines, one bad hire in a hundred thousand? Probably you need a month-long “interview” process then, you’re basically at the question of “let us contract some work out to you for a month and if we like you we will hire you.” But, consider: are more nines even better? If we have BS metrics that do not properly reflect our interests then we only find out about them by hiring people we would not have rationally hired. Maybe you only want one nine, or more radically even half a nine (that is 68%, if you have never heard of it), of fidelity. What if you baked that fidelity in? What if a unilateral consensus to not hire still generated a 10% chance of hiring and a unilateral consensus to hire still generated only a 90% chance of hiring, so that interviewers had to know getting into this that absolute sureness was not a goal?
That brings up an even better question, which is: can you have a good hiring process without a good firing process? How do you give yourself the one-nine process and then make sure that people who were retroactively not a good fit get a decent severance but are done away with?
I have mentioned elsewhere that Bayes’s theorem suggests that evidence should be measured more or less in decibels; this is covered explicitly in Jaynes’ Probability Theory ch. 4. But consider [1] where researchers found that they did not see substantially more correlation between different interviewers evaluating the same candidate for 20 minutes, and other interviewers watching videos of those interviews over only the first 20 seconds (the initial greetings of the video). So can we actually trust interviewers to give us, say, +10dB of evidence that this person is a good fit? Or if we get 5 interviewers do we get a range from “meh, pass” to “hire immediately for twice what they’re asking” as much as if we had done job interview “speed dating”?
Constructive advice is hard. One wants to instead force the hiring committee to watch talk after talk by Alan Kay until the roughly-every-other-talk discourse Alan gives about how “we live in a pink world and then we have a deviant thought but then we kersplat it until one day in the shower we have a kerpow that launches us into a new blue plane of existence—but that idea like most is more likely to be mediocre or terrible, so we have to then come back to it with something like science so that we can discard terrible ideas” becomes somehow ingrained in the hiring committee and they start trading their “kerpows” and evaluating them and coming to a better process than this “pink world” we see with our rose-colored glasses.
Because without that sort of systematic re-invention, aren’t we just guessing?
[1] https://www.researchgate.net/publication/313878823_The_impor... . Side note: in tracking this study back down I got to see a lot of interesting takes on it but the most common consensus is that “we make our first impressions in that first 20s and then do not budge from them” which I do not think is warranted given the data. That is, while a significant correlation coefficient is observed and it is comparable to the correlation between different long-form interviewers, at no point is that correlation coefficient, like, 0.9 or 0.99 or anything like that, suppressing interpretations that these first 20 seconds determine the interview. For that matter if that was your goal you would want as a control any other 20 seconds in the interview—you would want to prove that it was the first 20 seconds that determined the overall interview impression, not that people were judging based on some je-ne-sais-quoi demeanor that is visible throughout the interview.
> but no one wants to go out on a limb and make suggestions about how to improve it.
That's not really true, plenty of suggestions on how to improve it. Many of those based on experience doing it for the long haul.
Personally I always advocate for a technical discussion of their experience. Read the resume in detail. Talk about it. Pick some past project to go deep. Also let them pick a favorite past project to go deep in detail. It means the interviewer needs enough technical depth to follow along. This works. I've never said "yes" to a hire and later thought that was a mistake, in ~25 years of doing interviews.
Just the other day here someone was saying they received over 50 resumes for a role so they had to interview over 50 people. No. Don't do that. You can't go deep, so it just leads to meaningless puzzles. Read the resumes, pick the top 5 or so, interview them personally. Hire the best of those. Done, move on with getting work done.
> This feels good if you’ve already made it or you grew up with the right connections, but it’s far from ideal you’re a promising developer who doesn’t yet have the right credentials on your resume to unlock the job.
I've never understood why these two different groups, long experienced developers and unproven newcomers, are competing for the same jobs. That's the insanity of the system.
I suspect the answer is that many people believe (mistakenly IMO) that experience doesn't really matter, and "raw talent" overrides everything else.
The problem is that "long experienced developers" are often worse than "unproven newcomers". They aren't any better and instead of being a "lob of clay", they are "molded steel" and all the bad habits will never go away.
True senior engineers from reputable companies are of course different than newcomers and they are interviewed quite differently at big companies. But the problem is that what candidates understand as "senior" and what FANNG companies understand as "senior", often have little to do with each other. And the reason is simply that most people who apply for a senior role, wouldn't even pass the entry level interview at a FANNG company. It's just the sad reality. And this does not mean the entry level interview is wrong. It just means that these "senior" candidates were instead just junior developers who never developed beyond that from a skill perspective (except in their own heads).
And at FANNG companies, to even be considered for a senior position interview, you MUST have the credentials on your CV to prove it. However, a lot of candidates will never be interviewed for a senior position, even if their CV is "ten pages" long, because its apparent that this experience will be unlikely to have prepared you for a senior position at a FANNG company.
Being a senior engineer at a FANNG company is demanding and difficult on many more levels than just raw coding. You could be the best coder in the world, but that won't make you a senior engineer. It will make you a top mid-level engineer and you will never get beyond that without acquiring non-technical skills. Most people simply won't be able to do this job, no matter how long they stay in the industry. And I can assure you that there is little similarity in how interviews are conducted for these positions vs. college grads.
A senior engineer is essentially a developer who wants to still code, while also being a manager and a leader.
I would add that many of the "senior" engineers in SV received that title because they joined a startup 6-months before the company secured major funding and massively scaled. Seniority/familiarity with the startup's stack != senior.
Part of it is the conveyor belt of new frameworks and technologies, and binary litmus tests from first-line recruiters.
Someone who could very well have invented javascript yet doesnt know ExpressReactVue[*]JS or framework du jour cannot get past first-line recruiters who are searching on Linked In keywords. Thus, both groups are in the same bucket.
I wouldnt be surprised if some recruiters turn down python 3.6 candidates because they dont have python 3.7.
My prediction: Somebody with a realistic chance of inventing whatever the next javascript will be will do fine. Recruiters who screen them out are hurting their company not that candidate.
I agree, i was exaggerating a bit but there is a spectrum between, say, Brendan Eich (on one extreme) and an average code camp graduate (on the other extreme). Think about those in the 75 to 50% region.
- "but it’s hard to actually step into the hiring manager position and find the best candidate without disappointing some people along the way"
Hear me out: the quality of (hiring) managers in tech is ridiculously bad for the money they make. Little quality, poor ideas, and little personality. Mostly, they are of the nerdy type, like the vast majority of people working in tech, so by inclination and culture are not leaders, but followers or the aloof type, and they much prefer to work on their own than in teams. It pays off to go with the flow in corporate.
Source: I had managers (including Directors and VPs) at some of the tech companies everybody talks about that were they to take any decision in which life and livelihood of people are at stake, it would be one of those dreams after which you wake up drenched in sweat and you make the sign of the cross you are just working in a tech company.
Let me clarify. Most managers I have seen in tech are sub-par. So, it might be that it is not that the hiring problem is a process problem, or a market problem, but a manager-quality problem.
Of course, I can present only anecdotes from my own experience; but I can confidently say that, before getting into tech, I was expecting the quality of managers and decision-makers to be much higher, especially considering the money they make. Instead, I have noticed the usual skewed distribution: a bunch of brilliant decision-makers that move the company in the hopefully right direction, and a ton of minor managerial figures who are surprisingly terrible at being managers, when not total frauds.
> Consensus seems to lean toward this being satire.
I think "How to interview engineers" is a set of actual Slava Akhmechet's recommendations about hiring, sprincled with occasional satire about hiring games.
there's plenty of suggestions and there are companies / teams that adapt those suggestions. Just big companies and startups, and clueless management refuse to. I can't control how my company hires people but I can control how I interview internal candidates.
I usually really like what Slava writes, but I didn't like this one at all.
Just a few things I disagree with:
- 'Talent', as it is defined in the article, is not the most important factor in most jobs (though granted he writes that this article doesn't apply to most jobs)
- IQ is not a good measure for intelligence. Intelligence is way too diverse to be broken down into a linear scale, even when it comes to just engineering or 'talent', as he calls it here.
- Personality is way more important than he makes it out to be. It can't just be broken down into 3-5 factors. This isn't a scientific process. In my experience, this is where most of the value of personal interviews is.
- I found the last section disgusting to read. He's advocating for manipulating the candidate into accepting the job by asking successively harder questions, even though you've already made up your mind. What a great start to your business relationship. And then justifies this by saying that 'people want to suffer'. This is almost like negging in romantic relationships. Just awful.
Slava, if you're reading this, I'm disappointed. I respected you for your experience and your writing but this is making me quite sad.
EDIT: As lots of people have written here in the comments: On a second read, I agree this could indeed be satire. If so, well played Slava, well played!
I don't think so though, unfortunately. This is what the 'everything is measurable and science has all the answers' mentality brings, if taken to an extreme. These days, science, data-driven approaches and 'measure everything' are more religion than anything else. Even if this article is satire, I think there's enough people out there that think this way.
> He's advocating for manipulating the candidate into accepting the job by asking successively harder questions, even though you've already made up your mind. What a great start to your business relationship. And then justifies this by saying that 'people want to suffer'. This is almost like negging in romantic relationships.
And hazing.
Some form of the hazing seems very common for dotcom hiring right now, whether or not (as the article alludes at the start) the people who are mimicking it from elsewhere known the reasons or effects.
>- Personality is way more important than he makes it out to be. It can't just be broken down into 3-5 factors. This isn't a scientific process.
Eh, some would argue the opposite... (Probably hundreds or even thousands of psychologists & academic professors of psychology)
"In psychological trait theory, the Big Five personality traits, also known as the five-factor model (FFM) and the OCEAN model, is a suggested taxonomy, or grouping, for personality traits,[1] developed from the 1980s onwards."
"The five-dimensional structure was robust across major regions of the world. Trait levels were related in predictable ways to self-esteem, sociosexuality, and national personality profiles."
"Personality was related to career success controlling for general mental ability and, though adulthood measures of the Big Five traits were more strongly related to career success than were childhood measures, both contributed unique variance in explaining career success."
There's intense critique of the Big Five model within psychology, and with what we've learned about the reproducibility of psychology studies , I'd be shocked if those studies you linked were reproducible.
I will ask successively harder and harder questions in an interview, time permitting.
However I do it only so that I can potentially give a hire recommendation for a higher level with tens+ thousands/yr more compensation.
Also because I'm always just one of 6+ interviewers, and a committee of 6+ completely separate people makes the actual decision, I can't even know if the person is being hired anyways.
The part about a candidate with multiple offers rejecting ones that felt “too easy” is not entirely unfounded. One of the things I’m personally interested in is professional growth so similar thoughts have definitely crossed my mind as a candidate
> Intelligence is way too diverse to be broken down into a linear scale
The article is written with this in mind, that's why there's multiple tests (coding and written!) for both "talent" and "judgement."
Intelligence isn't really very diverse, when you need people who are s-m-a-r-t smart. That means you need smart people, not half-smart differently-abled people.
As someone who has worked with engineers all day every day for the past 15 years, one thing that I don't see "measured" is how well that engineer can work cross functionality. I consistently see great results when cross functional collaboration is high and poor to catastrophic results when engineers are empowered to do everything themselves or where the org structure maintains engineering as an island unto itself.
> one thing that I don't see "measured" is how well that engineer can work cross functionality.
Large tech companies will measure for this. From personal experience I know that Google, Facebook and Netflix all test for cross functional leadership at the senior+ level. By the time you're interviewing for senior or higher roles, only two of the six interviews are actually about coding at these companies. The rest are behavioral and system design, with specific questions used to calibrate how hands on you are and how much you're used to working with many teams effectively in a large org.
>IQ is not a good measure for intelligence. Intelligence is way too diverse to be broken down into a linear scale, even when it comes to just engineering or 'talent', as he calls it here.
I see everyone saying this is satire, but I don't think so. Satire takes a commonly held belief and exaggerates it to the point where problems are easily visible. How is that happening here?
I can almost see it in the "six hours" part, which I think is the weakest part of the essay, but it doesn't fit because the usual motivation for long hours and hard questions is not the negging/hazing explanation here, but because you want multiple perspectives on the candidate and because you want to establish the candidate's limits. It's useful to know the difference between "Candidate is good enough to work here" and "Candidate is insanely talented". The only way to get that is by asking questions that differentiate between good enough and insanely talented.
Other than that aspect, the essay seemed straightforward. It seems true that it would be hard to make the questions the author asks and easy to fake "Tell me about a time you took initiative" style questions.
I think the interview described here seems challenging and designed to hire quick thinking hackers, but it doesn't seem absurd or obviously wrong or terribly flawed as I would expect from a satirical piece.
The author believes that we should screen for IQ directly and that long technical interviews are second-rate proxy for that. The piece is meant to get the reader to buy into a set of "natural" or "obvious" priorities and then show how absurd most interviews are in light of those priorities.
I suspect the author is looking for a reaction like: "Wait, if I agree this is absurd, then why _don't_ we just administer IQ tests and be done with it?"
Some of the author's tweets about identifying talent, IQ, and temperament:
It's not obvious to me that testing IQ is a bad thing. Maybe it's not a good indicator for job performance, but that seems like something you'd need to discover empirically.
I think the fact that it does not seem "absurd" or "obviously wrong" is the revelation. It should, but it doesn't. It's clever writing. Imagine a world where engineering interviews were sane and then writing this article in that world, it would be "absurd" and "obviously wrong" and clearly satire. In our world, the real world.. it's not so clear.
Why should it seem absurd or obviously wrong? It seems like the author wants a certain kind of programmer and wants to minimize false positives. If you want the top 1% (according to your own measurements about what "top" means) then it only makes sense that you'd have an interview process that more than 99% of applicants couldn't pass. "More than" because you're also making cuts to avoid false positives which will reject some good candidates.
In my view this strikes people as unfair because it excludes most people. It probably excludes them. I think I'd struggle with the tic tac toe problem and would probably need 30-60 minutes on it (hope to try this afternoon to see). It feels bad to see people say the bar is something you can't reach, and that makes people upset at the author. The author is an elitist and people don't like that.
Furthermore, the bar is arbitrary. No good programmer needs to implement a tictactoe counting program rapidly so it feels dumb to judge on that. This is like saying though that IQ tests are meaningless because I seldom need to guess which shape comes next in a sequence. However, just like predicting a sequence tells us something about the quality of your cognition (probably) quickly writing an arbitrary program tells us that you have some level of coding, thinking, intelligence, etc.
You may have that level of talent and fail the test, but you can't lack that level of talent and pass the test. At least, that's the theory in this essay. It seems mostly correct to me, barring an otherwise incompetent person who just coincidentally practices lots of tic tac toe problems.
Given that hiring a bad person is a lot costlier than failing to hire a good one, it seems logical that some people would need to structure hiring like this.
I don't think this is a good bar for FAANG companies. They need too many people. I don't think this is a good bar for companies where software is mundane not core competency stuff, you don't need high speed hackers. I'd doubt this is a good strategy for underfunded startups, you probably can't afford extensive interview processes with high bars or to hire the folks who would pass them. This seems like it might be a good pattern for a well funded software startup that needs to move fast.
Because you could replace the entire thing with a 2 hour IQ test.
> but you can't lack that level of talent and pass the test.
Another reason that this should seem absurd and obviously wrong. It's trivially simple to pass the test without the "level of talent" being targeted. It is just a matter of repetition. There are entire websites dedicated to beating this well known hiring step. Anyone can learn recursion and caching and work a bunch of leetcode / hackerrank problems and get a job at a place who implements these interview practices. Whether it takes you 2 weeks to prep or 6 months to prep is not revealed to the interviewer.
An IQ test wouldn't tell you about programming ability though or clarity of writing. I don't think an IQ test would be an inherently bad idea, but if this test is similar and tests programming and communication, it just seems better and therefore like it couldn't be replaced by an IQ test.
If anyone could pass these tests via hard work then we would have millions of Indian and Chinese people able to pass them, but we don't. Instead the ratio of people who can pass them from there is roughly the same as everywhere else.
Neat! It's either awesome that half of Vietnamese high school students could pass a Google interview, and a strong endorsement and credit to their system, or it's not so. Either way, I don't see how this is an argument against the programming test model of interview.
I really hope it's satire, but based on those tweets shared in the other comment, I don't think it is.
The problem is that the views and practices here are just within the range of views I've actually seen interviewers and hiring managers express. And that's more damning of our industry's hiring practices than the person who wrote this.
I've run into enough folks that actually think this way that a priori, knowing nothing about what Slava usually writes, this doesn't come across as satire. I could see how certain folks think they're "building an extraordinary team at a hard technology startup" and would use this as an interviewing handbook, no sense of irony.
That's true, but for context, there's a pretty big population of people on this forum that are trying to fight the "tech-bro" stereotype and this article plays perfectly into the stereotyped view that SV tech companies are filled with insecure, personalityless men. I'm guessing that's why this comment section is filled with so much hate for the author if anyone wonders.
> Thus, Strauss agrees with the Socrates of the Phaedrus, where the Greek indicates that, insofar as writing does not respond when questioned, good writing provokes questions in the reader—questions that orient the reader towards an understanding of problems the author thought about with utmost seriousness.
So my take is it's a kind of litmus test: anyone who expresses genuine outrage at this or treats it purely as satire isn't qualified to interview anybody, because they haven't put any real commitment to wrestling with the (possibly impossible) challenge of successfully, reliably, economically identifying good hires.
Like I probably only want to work with people who read this and say, "Hmm. Interesting, but what about -"
Satire or not, I have see an example of what the article is describing. Not Tic Tac Toe, but a more complex problem involving octrees. Essentially, given a scene(hard-coded), it would have to output all octree nodes (specifying white, black or gray if they were unnocupied, completely occupied, partially occupied, respectively). That was a full end of semester assignment. As in, you had the entire semester to do it.
We were 'pairing', he was 'driving'. Well, he wrote the whole thing in one sitting, in ANSI C. Even added couple of extra shapes to the program (that we thought were required, but in fact weren't). The bottleneck was his typing speed.
I was able to follow everything, although I remember thinking that a couple of spots along the way had more clever solutions, compared to what I would do, were I the one typing.
I did not notice any mistakes whatsoever. None. Turns out, neither did the compiler. He wrote the whole damn thing, one sitting, in a single afternoon, and only hit compile once. It worked. First try. Hundreds of lines of codes with pretty high complexity.
I later ran Valgrind on it. It didn't even have memory leaks.
I had never seen anything like it, and haven't seen since. I'm pretty sure we'll have a new algorithm or data structure named after him at some point.
Would I hire him for a startup? Only if it was based on complex algorithms, otherwise he would probably get bored.
Even if you do find a genius (and you never even have the chance to interview guys like him, as they can essentially pick their job), you may still need to get a Carmack, not a Dijkstra
I'm relieved that I'm not alone in feeling bad about these pieces of advice and horrified of the possibility of interviewing or working in a place where people think like this.
The author acknowledges right in the article that he would fail this interview, but expects the readers to trust him on this one... oh you just follow these steps and the plane will keep flying.. I tried it myself and crashed, but maybe it will work for you..
If it's satire, then it's not obvious enough, if it's not, then it's just bad advice.
When he says he wouldn't pass himself, he does not mean that the interview doesn't work. What he means is that the standard he is setting is too high for him to pass but not for exceptional candidates.
A lot of people are pointing out that this article is satirical in nature. English is not my first language. Even though I don't have any issues in reading English books, I was not able to see the "satirical" aspects. Up until now, I've generally been able to distinguish between satire / sarcasm / irony etc.
If anyone can shed some light on how to "see" the satirical aspects, I'd be really grateful. :)
I don't think it's satire. People are presumably reacting that way because what Slava's saying is outside their acceptable range, so they need an alternate interpretation. This is missing his point, which is to write things that break out of acceptable range—or rather, not to not write things because they're out of acceptable range.
On the other hand, it's a merciful way of resolving the cognitive dissonance, since the alternative would be "what an asshole" and that would make for a worse thread.
Those that assume this is satire, if it is, it's bad satire.
The author tosses in references and citations that if this were satire, are affirmations of how much smarter the author is than their audience, since they know the absurdity (apparently if it's satire) of these claims, but the gullible audience thinks they're plausible.
I don't think this is satire, but maybe I'm not intelligent enough to see through it.
Board states, yes, but in a game of tic-tac-toe, placement order matters as well. There are a total of 9! (362880) possible (but not necessarily valid) game states. With such a low number, it's not hard to brute force the correct answer. Someone more well-versed in combinatorics could probably do it in a more efficient way.
I don’t think it is valid to consider two identical boards, arrived at via different sequences of moves, to be different “states”. They did take separate paths to get there, but their state trees have now converged and they are de-facto the same state.
And these are the kind of questions that would (hopefully) come up in the interview!
Two issues with this methodology. First as he states there is a difference between talent and IQ, this does not seem to tease out the difference.
You can imagine someone with alot of experience but slower thinking speed doing just as well as someone with less experience and faster thinking speed.
Second this is also relative to the problem being asked. With the tic tac toe one, you can also think about someone who maybe has been on leetcode alot and has no actual work experience and is generally slow, to be fast on this particular problem since they happen to have studied that before or have exposure to it.
You can also imagine the opposite someone who is really fast at thinking and has experience, but they haven't looked at binary trees in 10 years so they may be slow thinking about a particular subject.
> Interviews are filled with posturing, political correctness, and socially convenient falsehoods that have everything to do with large company recruiting constraints and nothing to do with you.
Doing the right thing is infinitely better than doing the wrong thing, no matter how fast you are at doing it.
Technical excellence is the key to success in surprisingly few businesses. This hurts for engineers to learn, but it's true. By the time you need technical excellence, you usually have more options to get it. But it's rarely the deciding factor compared with doing simple things in the right direction.
If this is triggering Poe’s Law, then what is the opposite end of the belief spectrum? My brother runs a company that will hire literally anybody. The interview is orientation. 80% can’t do the job and quit.
Side note: 10 minutes of pure fast typing sounds disastrously long for a simple brute-force algorithm question.
This is more high pressure garbage that is both biased and misses a lot of the other important parts of being an engineer. I would rather have on my team a mediocre developer with stellar communication skills than the other way around. Timing someone on an observed programming assignment is a dumb way to assess performance, and it is a great way to miss a lot of really good candidates.
Building real applications with real world complexity requires teams, and while being clever and curious is essential, so is being able to ask where something is or ask for help effectively is equally, if not more important.
It's harder to interview for those kinds of things, and instead hiring managers have elected to make the interview process some kind of trial by ordeal, and we are all worse, and the products we make are worse as a result..
Ok, took me 20 minutes, and I forgot to count the draws–I only counted the games when there is a winner.
But I think his math doesn't work out. The solution is only about ~100 lines. If typing speed was the bottleneck, it would only take 2 or 3 minutes from start to finish.
> The most talented candidates will think about it for a few seconds, then write the program as fast as they can type (and they'll type fast). You can almost sense their frustration because they think way faster than the keyboard allows them to interface with the computer. Typing speed is their bottleneck. (...) The whole process will take about 10 minutes from start to finish.
Saying typing speed is the bottleneck may be a bit of an exaggeration, but if you look at the leaderboard for Advent of Code, the leaders are incredibly fast.
(My result: 5 minutes, but I'm normally a lot slower than those leaders. I'd just already written a tictactoe program and only needed the count-games function.)
Jim Keller famously says "99% of what I believe is wrong" in an interview https://lexfridman.com/jim-keller/ (although I am not smart enough to read much into that quote).
There is some deeper truths behind the gloss, otherwise the article would just be trash, but I haven’t had enough experience conducting interviews to know where the truth is hiding.
> Then, give them a problem. The one I gave for years is to write a program that plays every possible tic-tac-toe game, and then prints the number of valid games.
There must be people he interviewed, they should be able to answer your question.
I'd have to question picking any single programming problem like the one mentioned is a great proxy for IQ. Could it be that there's a class of problems or conceptual framework (in this case, maybe combinations/permutations or tree traversal), where a person might have solved a dozen problems like this in the past, and this is just another slightly different one?
There seem to be a big element of luck when interviewing and being interviewed to hire the right person. That’s in addition to property evaluating and answering the questions which also lends into luck. That’s why other elements should be way more weighted such as previous work, referrals, recommendations.
I'll be honest I cannot tell if this is satire or not, I would even say that it isn't.
There are some "good" nuggets in there, and by good I mean that while they may be unpleasant, they may still be true. The part about "Theatrics" makes a lot of sense to me.
It's really only the "Talent" part that I refuse to admit is genuine. This whole typing speed thing, I really hope nobody thinks like that.
I don't think this is satire. It just delights in being realistic. This is how FAANG (and Stripe) interviews work. You play the game to get paid (and contribute to the company). Some people might disagree with the IQ comments (but former presidents of Harvard and many in SV like to thwart people's assumptions here as a form of loose, survivorship bias, imho).
IMHO, he's being honest. Interviewing and capitalism is super imperfect currently (life isn't clean like logic). It's part of the game. One goal is to change the game.
I thought his sections on Judgement and Personality were pretty good / show experience. Though for some of this there are more targeted ways that FAANG does this (Amazon leadership principles, Googleyness, Netflix freedom and responsibility etc.) - but I agree with him the signal is mostly around those who prepare for it (as it is with just about all tests; the SAT, etc.).
Even if it's satire — which I think it should be more clear when you write about such a delicate topic — I think it represents quite well the levels of "assholiness" the software industry is reaching these days.
I think they can get away today with so many silly hoops because the supply of engineers is quite high these days, and I guess, the FAANG industry standard is the "fair" way to identify the best candidates.
Engineers now are putting up with all this because there is not much choice than to spend more time training for this kind of interviews.
No matter how much do they talk about talent, judgment, IQ or whatnot, the fact is the more you train, the better you get at this kind of interviews, independently of how capable in your day to day job you are as an engineer. Proof that training works is the whole emerging cottage industry around interview preparation that is thriving these days.
Perhaps this is a bubble that will burst at some point, but I'm afraid the bar of ridiculousness will keep raising. If any of the FAANG companies started requiring to complete a marathon in less than 5 hours, they will still find great candidates that will train hard to achieve it.
The rationalization after fact would be something like the ability to complete a marathon correlates well with the ability to deliver a project or whatever, but they are just selecting for engineers that will do whatever it takes to get the position, no matter how silly the requirement is.
I agree that it doesn't obviously read as satire, though I am not very familiar with the author. The users determined that this is satire seem more familiar with his previous work.
We do 30 minute blocks with your boss, one director + one developer, one with the CTO and then the wrap up with the internal recruiter.
The other departments do roughly the same (finance does comptroller + one of your coworkers at the same level instead of director + developer)
Edit: We also make sure to allow the candidate to get some water, pee breaks and take them out for lunch (if it happens it to be during the lunch time)
It depends on the company. Startups and teams with autonomy are more informal.
The main things they care about are:
1. Can you do the job (and do it well)?
2. Are you really interested?
3. Are you batshit crazy, or can you work with the tiny/close team?
Large companies put you through the grinder. My most unpleasant interview experiences were with some very big names (but not all are like this).
I’m not a good fit for small companies, I don’t have the desire to study for FAANGs anymore (a week of studying isn’t enough), and I’ve personally had great luck with medium sized ones.
The official rationale is that it reduces bias. A single 1-hour interview has a low signal to noise ratio. Two interviews is better. At four interviews a fairly reliable signal emerges. Add a fifth interview so that you can train/calibrate new interviewers, take an hour for lunch, and you're at 6 hours.
6+ hour interviews are common for senior-level+ engineering roles at big orgs. They fly you in for a day of interviewing. Some orgs will even break this up to a 2-3 day thing to even it out.
If you're a bay company that's taking a $250k+/yr risk on someone you typically rake them over the coals (for better or worse).
Sure, I'm aware of this and have been flown in to various locales to be grilled in interviews that started before it was really light out and finished after dusk (late in the year to be fair). My point isn't that companies do this, it's that the giveaway was that interviews must be like this that gave it away.
On a personal note I'll add it's even more exhausting when you schedule these types of interviews at different companies back to back on consecutive days so you don't have to keep flying in for each company.
There should be a major distinction between programmers and Engineers.
As an engineer that made the transition to programming, many interviews I was told Engineering isn't programming. I had to take an entry level programming job (and pay).
And the opposite is True too. Programmers are not Engineers. In Engineering you are expected to prove everything and there generally is only 1 correct solution(given cost, quality, and time specifications). That's generally the scientific method approach.
In programming, outside of algorithm efficiency, you are taught to listen to Authority and follow best practices or standard practices. This isn't science, it's art.
Programming pays better and is in higher demand, so I'm not hating on programming. But these are distinctly different professions and ways of thinking.
I have a deep respect for engineers and as a programmer, I think you are absolutely right. I don’t like the term “engineer.” It doesn’t fit, and I hate that we have adopted it, I think mainly because it makes people in the business side take programming more seriously. Like we have to justify to people that our job is actually difficult and requires some level of skill. It doesn’t convey the level of creativity, programmers use on a regular basis.
I see it slightly differently. As an engineer, most of my time is spent trying to understand the scientific principles behind some observed failure so that some tiny tweak can be applied to fix it. There is design, but very analytical and minimalist. As a programmer, I’m mostly interacting with people based on their ideas and desires. There’s also a lot more trade knowledge about software tools and languages, but very little about first principles. The ideas about how to design something seem arbitrarily fashionable rather than optimized based on constraints.
> First, I cannot myself pass this interview. Last time I tried, I got the correct answer after about forty minutes or so. I could get it down with practice, but it doesn't matter-- I think slower than I type. That's a no hire. The point of the interview is to hire extremely talented engineers, not engineers as talented as me.
The humor is so thick in that paragraph that you could cut it with a knife.
I don't read any humor in it at all. What's supposed to be funny about it?
I can imagine someone wanting to hire an engineer more talented than they are and so asking questions that they themselves wouldn't be able to answer. That just seems logical rather than ironic.
I feel like it would bias your selection towards a fast moving hacker rather than a more methodical person, but that may be exactly what you're looking for in this context.
The author isn't just saying that they themselves couldn't write a Tic-Tac-Toe simulator the first time they were exposed to the problem. The author is saying that after watching candidates work this problem "for years", apparently not even a high level algorithmic idea stuck with them. This is... surprising.
If this was meant as satire or humor, this will be the first time I've read anything on the internet that was meant as satire or humor and been personally unable to detect it. If it was meant humorously (I don't think it was, even after re-reading), it's a bad attempt at humor.
To me that seems obviously not satire. It's even a cliché to say that one should hire people smarter than oneself, not be the smartest person in the room, etc.
The old manager adage is "Hire people smarter than yourself", I don't see how this can be satire, everything he says is very reasonable if you believe that these interviews are a good way to filter candidates. Satire would try to make some jabs at it but there are no jabs against the interviews in this article.
Am I the only who stopped reading when the author outlined their question?
How many games of tic tac toe are “valid”? What does valid mean? As usual the problem with hiring software isn’t measuring “talent” “iq” or “personality”. These things are arbitrary, it’s communicating business needs to an engineer and having the engineer build the system based on the needs of said business.
As always the issue is communication, and the author makes it clear despite him thinking long and hard he’s not able to effectively communicate what the interviewer actually wants.
Edit...
Upon reading through the comments and seeing everyone having a different answer, simply proves my point.
I don't totally agree with the article, but I think it's totally fine to give an ambiguous question in an interview, to see how the candidate handles it. Almost every real world project will involve ambiguous instructions.
Obviously you don't want to penalize the candidate for not reading your mind and guessing the definition you had in mind. But I do think it's fair to give extra credit to candidates who notice the ambiguity and ask for clarification. Done well, an ambiguous question can be an opportunity for a candidate to demonstrate a strength, rather than a trick to unfairly penalize someone.
He did say the interviewee can ask questions freely. My question would've been whether to count symmetrical positions as distinct, but otherwise I'm not seeing any particular ambiguity.
I get that this is putatively "satirical," but the section about IQ is really inexcusable: look at the citations. Two wikipedia articles, one seemingly reputable blog, and.... gwern.net, the ravings of a racist crank whose shtick involves routinely cites racist psychologists to make false arguments about whether IQ is "impossible to improve."
In general idea that IQ has biological significance, and linking to anyone who approvingly cites The Bell Curve's most controversial parts, are both dangerous. More to the point - it's not dangerous because it's "taboo facts," it's dangerous because it's misinformation which has been discredited by decades of science.
If Slava is being "ironic" or "satirical" with this garbage then he needs to make it more clear, or just delete the entire section.
> write a program that plays every possible tic-tac-toe game
Last time I interviewed in the Valley, some mofo actually asked me how many TicTacToe games end in 5 moves ? 9.5%. Here's my MonteCarlo solution if you can't be bothered with this garbage. Sorry that repo is 7 years old. Never got a chance to use it again.
Makes me so glad I quit the field & went into stats. My classmate just graduated with his stats phd & I asked him how the job interview went. Apparently they asked him if he was a frequentist or a Bayesian. So he said he was a Fisherian, and they had a good laugh. After a few preliminaries on p-values, credible intervals vs confidence intervals, 2 way anova, standard proofs on Cochran's theorem, Cramer-Rao & how to simulate multi-variate gaussian, everybody went for lunch. He was out the door under an hour with a job offer.
> 2 way anova, standard proofs on Cochran's theorem, Cramer-Rao & how to simulate multi-variate gaussian
I don't see how regurgitating standard proofs is far and away different from regurgitating (or solving) algorithmic coding problems. "Implement a priority queue" is to software engineering what "prove Cochrans' theorem" is to stats (arguably the second is more difficult/less reasonable).
Sadly, many of the implied alternatives to coding interviews are becoming a regression toward credentialism: People who have already secured good jobs, impressive internships, or prestigious university degrees would much rather coast on those credentials than be challenged for a few hours during an interview. This feels good if you’ve already made it or you grew up with the right connections, but it’s far from ideal you’re a promising developer who doesn’t yet have the right credentials on your resume to unlock the job. To many people, the coding interviews is a chance to break the traditional mold and land a job that would have been out of reach otherwise.