I stared at it for what felt like minutes and then said, if I looked in your search history would I see you looking this up on stackoverflow?
The guy said "yes" and I said I would make it work by asking you to send me the link to the stackoverflow answer.
He laughed and said "you got me".
Same company different interviewer asked me to explain the "pros and cons of Java vs Rails."
I turned the job down.
There isn't a right answer, you basically just start shallow and probe until either you or the candidate can't go any deeper. This question could lead you into exploration around the JVM, concurrency, memory management, typed vs un-typed, choosing libraries, toolchains, dependency management, strategies for persisting state(i.e ORMs), back-of-the-envelope performance, etc
I'm pretty sure I could get a full 45 minutes out of that question.
Java vs. Rails is a bad question as it's comparing two things that aren't equivalent. One is a language, the other a web-application framework.
"Well, for starters, Rails is a framework while Java is a language. Granted, Rails is THE framework on Ruby, so I can tell you about the differences between Java-based webdev, vs Ruby/Rails. With R/R, you have convention over configuration, which means blahblahblah and is good for getting started but at times you run into blahblahblah. And community blahblahblah, etc. Java, OTOH, has stating typing, which gives you guarantees blahblahblah but there's no standard web framework like on Rails so you'll spend a lot of time researching not-so-great choices, etcetc...."
I could also, as the grandparent suggests, talk about that particular topic for a long while. But as an interviewee it would suggest to me that they possibly are just spewing out words they heard around the watercooler. Either that, or it's a 'trick' question, due to the fact that they are not comparable.
I expect chefs could talk all day on the question: "What are the pros and cons of flour vs. forks?" That doesn't make it a good interview question.
The point of such an interview question is to foster dialogue.
So there is no place in this world for a senior programmer who isn't particularly familiar with those two languages?
Why are they so special?
It doesn’t require an in-depth knowledge of either, but this question would serve as a great way to get a basic level of an applicants knowledge.
I honestly think that talking about some tools someone never needed is a terrible way to get the feel for a basic level of that applicant's knowledge. And it really doesn't tell you anything about them as a programmer. I think even fizzbuzz and linked list reversal would be more telling, and that's not much.
Going by the translator analogy, I feel like it's asking say an Asian translator (who might command a few eastern languages with decades of solid experience) who never studied English and French to talk about English and French. Because in your world everyone skilled translator knows enough about English and French to at least have a useful conversation about them. It smells very elitistic. And pointless.
I would consider it a necessary job requirement for a senior developer to know about common technologies in the field they are working in. If that meant working on web applications, I would expect them to know what both Rails and Java are. I don't need them to be able to program in them; I need them to know what they are, why they exist, some of the key facts about them, where they might most commonly be used, that sort of thing.
Over-specialisation to the extent that developers are unaware of technologies outside of their direct knowledge is a warning sign about a developer's skillset, to me. How can I trust that a developer is picking the correct tool for a job if they aren't aware of what tools are available?
My imaginary senior web developer knows more than a dozen languages. It is almost a given that one of these languages is technically sound and a reasonable choice for a given task, and that developer should be able to pick that language just fine. However, if there are other compelling reasons to look into other languages (for example because company wants cheap coders and think there's a large pool of cheap talent skilled in language X, or because the company needs to leverage the ecosystem surrounding language Y), that same senior developer is more than capable of doing a bit of research and evaluating these languages.
You almost sound like one of these interviewers who think that a programmer who can't implement a red-black tree off the top of his head and state its worst case insertion complexity couldn't possibly be capable of looking it up and evaluating the relative merits of different data structures -- and thus cannot be trusted to any position where he might have to pick an algorithm. Nevermind if he could implement two dozen other data structures in his sleep.
Over-specialization? Seriously? A programmer who's used more than a dozen programming languages is over-specialized because he can't discuss the two you want to talk about? The stench of elitism is only getting stronger here. You're assuming the worst of a programmer because his world doesn't revolve around the same things yours does, and you are assuming that despite their experience they probably will not try to or cannot educate themselves further.
Unfortunately I see this sort of prejudiced attitude everywhere... "I think this is important so if you don't know this, you can't possibly be any good." Such statements reveal a rather narrow world view.
It's possible to be aware that certain tools exist and have an idea about what they're used for, and yet know little enough about them to butt into a discussion and give any sort of fruitful, educated opinion about them.
Yes, that's probably true. If that were the case, I would expect an answer like "I don't know anything those beyond the fact that it's a cross-platform language that runs on a virtual machine and Rails is a framework for Ruby. But I do know a lot of Python and Go, which have some similar characteristics. We could talk about the general difference between compiled and interpreted languages?"
These questions serve as entry points. I'm not expecting a developer who doesn't use these tools to have an in-depth knowledge of how everything works, but if the question kills the interview to the point where someone is unable to move forward then that is a warning sign.
You almost sound like one of these interviewers who think that a programmer who can't implement a red-black tree off the top of his head...
No. I absolutely would not expect that, and those questions are stupid. I would expect a senior developer to know what a tree is, and maybe what a red-black tree is. I would expect them to know what operations can commonly be done on a tree, or what use-cases they might have. I would expect them to be able to, say, contrast a tree with a hash table. If a developer said, however, "I have never used a tree or a hash table and have no idea what you are talking about" then I would also consider that to be a warning sign. Do you see the similarity?
Over-specialization? Seriously? A programmer who's used more than a dozen programming languages is over-specialized because he can't discuss the two you want to talk about? The stench of elitism is only getting stronger here. You're assuming the worst of a programmer because his world doesn't revolve around the same things yours does, and you are assuming that despite their experience they probably will not try to or cannot educate themselves further.
You are misrepresenting my view; see above for more details. I do not think it is elitist to expect people in senior roles to know about common tools used in that role. I would not expect them to be specialists, but I would expect them to understand the state of the field and be able to discuss that. I mean, we're not talking about obscure technologies here!
Unfortunately I see this sort of prejudiced attitude everywhere... "I think this is important so if you don't know this, you can't possibly be any good." Such statements reveal a rather narrow world view.
That is an unfortunate misrepresentation of my view, so I'm sorry if I've failed to communicate it effectively. I guess from what you've said above that you think I'd place much more weight on the technical content of that answer than I would in practice.
All these threads about interviewing over the years have definitely left me with a bitter taste, mainly due to all the people who think they're clever and have figured out the most important thing, that one thing by which all applicants' worth can be tested.. if they just know about this one little thing. And in doing so, they effectively trivialize the software industry as a whole, failing to realize just how wide and deep it is, completely oblivious to how different developers with up to two decades of experience even in the same subfield may have walked rather different yet fruitful paths.
Again, I'm sorry.
He also wanted the pros and cons of them verses each other based on his wording.
My original answer was "the obvious pro is a death match is always interesting"
He got the job, so there you go I guess.
My interviewing has boiled down to 30 minutes of shooting the shit to see what the person is like and to make them more comfortable and then a pair session with a ticket I haven't had experience with so we are both on approximately the same level.
My goal in the session is to see how they interact with an unfamiliar project and what kind of questions they ask and their discovery process of learning our business domain.
1. The shoot the shit session is guaranteed to be driven by your personal biases.
2. Working on a different ticket with every candidate ensures you can't possibly compare one candidate's performance to another's.
I also send the same screening test depending on role so I do have a baseline, although I'm much more interested in our interaction during the pair session.
I didn't get the job, but I'm glad. His reaction was a red flag as far as I'm concerned.
I love the compare/contrast style of questions between languages. I try to not phrase it as "pros and cons" however. What I'm looking to see is the breadth of their knowledge and what insights they've picked up from hard earned experience. Also, showing the ability to think critically about different tools demonstrates that they understand when a hammer is appropriate vs a screwdriver.
I don't care what they think is better or worse, I want them to show me that they've traveled the world, so to speak.
If you need something up and running fast, but don't care about performance per instance, such as a marketing landing page use RoR.
If you need something very performant per instance and are willing to pay for it, such as a enterprise application or large web site, use Java.
If that's not an acceptable answer, the interviewer doesn't know what he's doing.
I agree with everything you said.
For this reason, I wish they would allow redundant questions to be asked, if the phrasing is even slightly different. But the rule-mongering mods always remove it if they can.
Should duplicates be deleted?
In general, no: most duplicates stay around. Having multiple copies of the same question with different wording is useful as search fodder, because people looking for an answer may use different wording too.
It's not due to some Google-fu though. People ask questions (on SO) in the terms they think about problems. But documentation is written according to the structure of the thing being documented.
So SO ends up being a more direct mapping from problem to solution. Google simply captures what's there (in both cases). It's certainly good to consult the authoritative docs in many cases.
besides, google is a knowledge base index, if you want to search a documentarion website you can both both inform google via site: or use the docs own search system which most have as of today
For simplified example, if you want to find out what flag -wtf does in some unix utility, the man page would be useful; but if you'd want to find out which flags should be used to achieve WTF then you'd often need to read the whole documentation to find out - stackoverflow serves as a "reverse index" in this case.
When you don't exactly, SO is a good pointer to the part of the documentation you didn't know.
I'm sitting at a bar in Turkey. I want to know how to say, "I want a beer," in Turkish. I know how to order a beer, drink the beer, and pay for the beer; I just want to know how to do it in my current circumstance.
How many times have you read docs and thought "well that's relatively straightforward" just to end up googling an error message 10 minutes later and end up on SO.
"Where do you see yourself in 5 years?"
"I don't know. Are you reading these questions from a textbook? FYI they're not very effective if you want to find someone who will do the job."
"What's your greatest weakness?"
"Trick questions in job interviews."
This is obviously not good advice; I have just reached a point in my life where I will not be made to dance to the whims of the interviewer, despite which the job would likely go to one of the employee's friends rather than it being given to me anyway.
One of the few merits of this approach is it tests if you can be frank with them or not. If they get offended at your lack of sucking up, then you probably wouldn't want that job anyway, because I find that if you suck up during the interview, you will find yourself struggling to maintain that ideal forever if you do get the job. It's better to be upfront about what things you actually care about.
Still, the approach has its advantages. "Listen, I know you'd rather get back to doing whatever you were doing before this. Let me tell you why I'm a good candidate .... ". And then go on having "developer-to-developer" conversation, ask them about their work, talk about your own. Obviously only do this on the canned questions, not the actual technical questions, unless you've already aced a few tech questions and are getting bored. Unless it's a hugely bureaucratic organization, this will probably receive a good review.
Another thing I've noticed is that it allows the interviewer to lower their guard as well, so you can usually get some more real "what's great, what sucks about the job", and I've even scored on "what's the pay cap of this position" multiple times.
 Also, the interview process should tell you a little bit about what kinds of candidates make it past the interview, to become employees.
No. It is good advice. It's very good advice if you want to find a place where you fit well. Unless you're starving and just need a job, any job, right now.
Allow me to explain: After some ups and downs in my career I resolved to be true to myself. In my case that means calling bullshit on ridiculous things. You know what happened? I found out really quickly where I would have been miserable, and I was totally surprised by some incredible groups that wanted someone just like me, they just didn't know it.
Actual responses from me in interviews that found me a great fit:
Q: <contrived hypothetical completely unanswerable>
A: I don't know that sounds like a contrived question. I'd be asking why you have that problem in the first place.
Q: "people who do what you do are often vague and noncommittal"
A: that's because most people who do what I do haven't bothered to read the foundational texts, thus they rely upon tropes and ceremony. This field is full of muppets.
Q: what's your greatest weakness
A: sometimes I'm just a little too awesome smirk that says "are we really playing this game?"
Q: are you <blah blah> certified?
A: no I've purposefully avoided <blah blah> certification.I think it's heavy handed bullshit and I don't want to be associated with it or asked to do it
No joke these were real responses that got me great jobs and client engagements. They also quickly helped me discover people I wouldn't enjoy working with.
Strong fences make good neighbors, yada yada.
I think it's critically important to explore, at the outset, what the boundaries of your relationship will be with any employer or business partner.
So for me, that means I lay down boundaries that I'm not interested in playing along with ridiculous ceremonies just because that's the way things have always been done. I think the way most interviews are conducted are ridiculous. Thus I push back and see what happens. You surely have your boundaries too. I feel your should respect them.
And did you ultimately receive the offer?
He asked me seriously and I said something to the effect of "I eat at least one donut a day for the past five years despite saying 'this is the year I get beach abs' so if you want an honest answer I think you've got it"
You know, in my 20+ years of experience, there isn't a single moment I could go back to where I could even fathom what I was going to be doing five years after that.
Of course, 5 years later life is completely different, that question is useless in my opinion, especially when the company itself is no older than 5 years or barely has more than one-year runway.
I think this interview style sends a signal, a lot of requirements that a programmer has to implement at work are made up ad-hoc/in an arbitrary manner. the message is that one is not supposed to question requirements too much and 'just do it'. (at least the filter is good enough to filter out a person who questions things)
On the job questions come with a context, or if not with someone who can offer context, or at least some guidance. (In the worst case you can ask your boss WTF to do. And maybe you'll get the answer, I don't know, and it's not important, it's up to you, etc.)
But never does the question what should this thing we're building right now will do in 5 years comes up without the answer, huh, good questions, but we have absolutely no idea, so we need more data and a shorter time-frame, and let's call up Gartner, Gordon Moore, Gardner and Tetlock, and maybe a few fortune tellers while we're at it too!
The problem is not the question, it's a fine question to start a discussion in the right context. (Maybe even during an interview. But it's not a "Q&A with tweetable answers" type question.)
Indeed, you (almost) never need to reverse linked lists in practice - but you often need to chase references of one kind or another (database, pointers, etc.) and do some manipulations on them that would result in a different list. If you have 20 data points, it doesn't matter what you do, but if you have 100M or 10B points, it makes a great deal of difference whether you do O(n), O(n log n) or O(n^2) or O(n!).
I think this is a good question, because it is an abstract version of the kind of problems that any non trivial code has to address. It is easy to describe, and easy to solve if you know what you are doing. I don't use it anymore because it's too common to be useful.
And yet, through the years I've gotten the feedback that it is "far detached from real work", "tests how good I did in school rather than how good I am" and various other comments -- and almost always from candidates who did poorly; I've never gotten this feedback from anyone I would consider competent.
 I try to get feedback through whoever referred the candidate, whether it's the friend-of-acquaintance, or the recruiter. I don't, in general, trust direct feedback from someone I turned down (or hired, for that matter).
This is precisely the problem the OP is talking about. When's the last time you ever did anything as low level as reversing a linked list? It's a solved problem, and something an employee would NEVER do in your company, let alone search for the optimal solution in 10 stressful minutes while you breathe down his neck.
> And yet, through the years I've gotten the feedback that it is "far detached from real work", "tests how good I did in school rather than how good I am" and various other comments -- and almost always from candidates who did poorly; I've never gotten this feedback from anyone I would consider competent.
Major red flag. I don't mean to be rude, but this sentence comes off as very arrogant.
> I don't, in general, trust direct feedback from someone I turned down (or hired, for that matter).
Why? Because they're in a lower caste? If you can't handle direct feedback, you're a terrible manager.
I think that misses the point. A linked list is a simple data structure that needs little explanation, and yet uses indirection in a way that can get you into trouble. As such, it is a great toy problem to asses a person's ability to work with linked data. Reversing a list, is similarly, very simple to explain, and a quite rudimentary operation.
So if I'm trying to asses basic ability to manipulate data in code, it seems much simpler than trying to give them a real-world linked data structure and have them do a real-world manipulation on it.
The real problem is the assumption that this is a 'solved problem', i.e. that a) there is an one-true-way algorithm out there, and b) that is what the interviewer wants you to know (i.e. know in the sense of knowing a fact). That is true of too many interviewers, and that is the problem.
I can imagine setting that as a question in an interview. And I wouldn't be interested in 'the algorithm', I'd want to see that someone could figure it out easily. If someone came in and clearly had just learnt it, it would tell me nothing. I would immediately ask them something else to take them out of their rote prep. But if someone came in and couldn't figure it out, why would I think they'd be any better on 'unsolved problems'? In terms of being able to manipulate data, it isn't much beyond Fizzbuzz.
The 'it's a solved problem' attitude smacks of leftpad thinking to me.
These "solved problems" are the result of countless hours of research by computer science experts with masters and Ph.D degrees. For those of you who studied computer science in college, you likely had lectures where the teacher or TA walked you through the process of resolving each of these algorithms.
Unless you are hiring for companies like Netflix, Cloudflare, Google, etc. where you basically need to invent new fields of research in computer science, expecting a programmer to "come up" with the same solution in 10 minutes a team of CS experts solved after many hours of research might be a little unrealistic.
We're talking about reversing a linked list, right?
This is the second time I've had this kind of surreal conversation on HN (the first was summing a list of integers), and I confess I am completely baffled. We are talking about reversing a linked list.
I absolutely expect somebody with a even minimal programming competence to be able to perform simple manipulations on interconnected data. And I can't imagine a much simpler manipulation, taking into account the connections, than that.
Knowing the difference between a problem that you can solve intuitively, And one that requires hours of PhD research, is also a good skill to have.
Really, those kinds of questions such as algorithm optimization only test your ability to come up with creative solutions in a stressful and time constrained situation. And even then you're not testing that because even the best of us will experience temporary brain paralysis in stressful situations, so you're actually just testing for luck.
And if that's what the working environment is like, fine so long as the candidate knows it's always like that and accepts those conditions (I wouldn't). But if that's NOT what the working environment is like, why are you even testing for it?
Now of course, I'm making a specific point about a specific question. I'm not making a point about all 'algorithm whiteboard' interviews. Maybe you misunderstood the point, and thought I was. But I agree, many of them are utterly bonkers. Nobody is going to intuitively derive Floyd Warshall's algorithm with a whiteboard in 10 minutes. And coding trivia interviews are always pointless. If you can learn the right answers to 'pass' my interview, I'm not doing a very good job, imho.
But for very simple things like basic manipulation of the the simplest possible data structure involving a reference, I absolutely think a minimally competent programmer should be able to demonstrate their ability. Can you reverse it? Can you rotate it? Can you split it into buckets? Can you twizzle pairs? Can you insert in the middle? If you can't do those things with a cons-cell, in O(N), I can't imagine how you would cope with basic manipulations on more complex data.
It's a problem we face (or rather, faced and solved ) unlike reversing a linked list (or implementing a Red/Black or AVL tree or sorting data).
 Lua (very easy to integrate with C and C++) and LPeg (for parsing text).
Oh, and the parser code is C, so it integrates perfectly.
And for the record, flex and bison might be faster than Lua and LPeg, but so far, it hasn't been an issue.
This has been fixed long ago, you can definitely get reentrant parsers if you need to. Also, I don't know the SIP grammar, but fwiw, I'm able to write an extremely clean recursive descent parser in C++ (1); for simple grammars, it's often an overkill to go the flex/bison route. Also by "writing code to parse messages in C++" someone familiar with boost might be thinking about using boost::spirit. That's not necessarily a bad choice.
(1) Just looked - it's in fact too complex for that; but the point is, the right tools are dependent on the skillset of the person who answers, which may be different from your own/ your team's. Just because you find writing parser by hand daunting, doesn't mean that I do (I taught compiler design at the university, worked on a commercial C++ compiler, wrote many parsers).
Your interview question is good - but the key to it is the "why" part, not the answer of "how would you do it". And it's simply a different sort of question - your question tests the mindset + domain knowledge; The "reverse a linked list" question aims at filtering-out the developers that are not worth loosing time with. A developer that can't reverse a linked list is completely useless in a C++ project (in fact, his contribution is likely to be negative - doing more damage than adding value).
Hmm. Scratch Cloudflare from that list. We are not 'inventing new fields of research'. We are looking for smart people of all sorts of backgrounds to come work on our stack.
It perplexes me. What do you gain by not doing that?
A simple task on a linked list takes two minutes to explain and five minutes to solve. The kind of task someone might do for me day-to-day have a whole bunch of context, and require knowledge or explanation of architecture and data configuration. I don't want to spend more than a couple of minutes explaining the problem, just to test whether they can program their way out of a paper bag.
>The kind of task someone might do for me day-to-day have a whole bunch of context, and require knowledge or explanation of architecture and data configuration.
Did you think I wouldn't have to face this problem too?
If you can't decouple one relevant chunk of code from your software sufficiently to be able to give it with a minimum of context to an interviewee then I'd have serious questions about the level of coupling in your codebase and, by extension, your programming ability.
Yes you would. You would not faced that problem on the first day of job and you would be expected to take time to learn during first days/weeks of the job (depending on complexity).
Also, once you will have that context and understand architecture and data configuration, the task will boil down to "reverse the damm list" or "write one epic sql query" or alternatively "read this epic sql query and refactor it away for christ sake".
It's okay, we're very unlikely to work together. Feel free to keep doing it your way, and I'm not going to worry about a random HNer impugning my coding ability based on a comment about interviewing style.
Also, if you're only giving real-world work tasks, do you screen at all for basic competence first? I ask because I suspect we're talking about different things.
Also 2 (edit), does your work only involve one kind of task? How do you assess the breadth of their basic understanding without using simple problems? Are your programmers doing very repetitive stuff?
I've done both with and without. I honestly found it hard to make judgments about what effect what I was doing was having on the hiring funnel though.
If you're talking about that rather than later stage hiring then yes, I agree it's a different kettle of fish.
Then use it in the test! If you're judging people on how well they pick up proprietary software, give to them and make them do something with it.
My tests usually involve giving the candidate exposure to some software/module or other which they were not previously familiar and I judge them on how well they can work with it, applying general programming knowledge they will actually use day to do in the process.
(or in other words, total time taken to do n tasks is mn + b, where m is the efficiency when using your stack, and b is the time taken to originally learn your stack. b is relatively large, so with n=1, that's a very different thing than with n=100.)
It tests how long it takes somebody to learn a small part of it, certainly.
Would that not be relevant for a job where you intend for them to learn your stack?
"not how well they can solve problems once they're using your stack."
A properly structured test could (and, clearly, both of our cases, should) accommodate both - to begin with, the ability to find their bearings in code with which they are not familiar and subsequently to solve problems within that context.
I usually do tests like this that last 45 minutes.
>or in other words, total time taken to do n tasks is mn + b, where m is the efficiency when using your stack, and b is the time taken to originally learn your stack. b is relatively large, so with n=1, that's a very different thing than with n=100.
For clarity, I work at google and I'd argue that for all but the most trivial problems, it would take even a good candidate more than 45 minutes to get their bearings. Most new hires do multiple days worth of codelabs before sitting in their desks.
Sure, you can limit scope, to things with 1-2 files where everything is handled for you and no data transfer between systems, building, or commiting required, but then we're getting into a class of problems where your limited to relatively simple, logical issues with very well defined APIs, a class of problems that data structures fit very, very well.
It's an easy problem for sure, and I yet I could imagine many a competent programmer to lock up for reasons that have nothing to do with the problem and everything to do with the situation and sense of pressure they might experience during it.
It's like saying standup comedy prepares you for 3AM calls.
> It's a solved problem, and something an employee would NEVER do in your company, let alone search for the optimal solution in 10 stressful minutes while you breathe down his neck.
Actually, just a couple of weeks ago we had something almost equivalent, that involved chasing "pointers" through databases and files. It was for less than 1M cases, but still - any O(n^2) solution would have been infeasible (and an nlogn solution about 20 times slower than an O(n) solution; in this case, it was a one-off, and the difference would have been between 1hr of runtime and 1day of runtime).
> When's the last time you ever did anything as low level as reversing a linked list?
Please re-read what I wrote. I wrote, in fact, "you (almost) never need to reverse linked lists in practice". I also explained that it is an abstract version of a kind of thing that DOES happen often.
> Major red flag. I don't mean to be rude, but this sentence comes off as very arrogant.
Perhaps. For all you know, I'm a dog, but I'll say that I'm soon entering my 4th decade in the industry, I'm well respected, I've hired (and fired) tens of people through the years (and interviewed hundreds if not thousands); and I'm well respected among my peers for having very accurate evaluations of people's abilities. I do not aim to be politically correct.
> Why? Because they're in a lower caste? If you can't handle direct feedback, you're a terrible manager.
No, because I've just hired them, or just didn't hire them; either way, I think it is irresponsible to trust the statement of a person I don't actually know and who may either hold a grudge or want to please me right now. I meant to say "direct immediate feedback", which is perhaps better wording.
After I know a person, I ask them (and usually get) the most direct and raw feedback I can get.
What's wrong with 1h vs 1d for a one-off thing? Your effective delivery time would be close enough to equivalent after all the administrative BS and customer expectations, and a naive approach piss simple to verify.
If it was going to mean the difference between 1h and multiple days, here's what I'd do:
- Fire up my trusty browser.
- Navigate to google.com
- Type in "reverse a linked list"
- Evaluate all of the solutions I find and pick the best efficiency / verifiability match to solve the problem in a reasonable amount of time.
Are you going to test that in an interview question? No, you're going to effectively "whiteboard" me, asking me to reinvent the wheel on the spot. And I'm going to resent you for it.
> but I'll say that I'm soon entering my 4th decade in the industry, I'm well respected, I've hired (and fired) tens of people through the years (and interviewed hundreds if not thousands); and I'm well respected among my peers for having very accurate evaluations of people's abilities. I do not aim to be politically correct.
And I'm in my third decade in the industry, all the respect, blah, blah, half a million lines of open source in the wild. So what? You can make the same mistakes over and over in any industry and never think to question if you're doing it wrong. This is ESPECIALLY problematic when it comes to people skills.
> No, because I've just hired them, or just didn't hire them; either way, I think it is irresponsible to trust the statement of a person I don't actually know and who may either hold a grudge or want to please me right now.
It's the PERFECT time to ask them, because it's fresh in their mind. A skilled interrogator can get information despite the subject's frame of mind. I'm actually surprised that you assume everyone you accept or reject is either going to be a sycophant or hold a grudge.
You WANT them to talk while the frustrations they encountered are still fresh in their minds. That's the most raw feedback you'll ever get.
You really do need to start treating them like human beings.
Wanna know how I evaluated potential game devs in person? I asked them what dev environment they were most comfortable in, gave them a computer hooked up to the internet and a picture of a tweetie bird, and said "Make the bird jump. Use google to look shit up." Worked amazingly well.
And now we're getting into absurd arguments. End of interview. Thank you for your time, but I don't think this will be a good fit.
Unfortunately, the real world business problem that needs to be solved is not "reverse a linked list", but rather "discover the relationship between the widgez and the widgix that the customer is after". That relationship, once you understand the problem domain, can be discovered by (doing something quite but not exactly like) reversing a linked list. And unfortunately for you, a google search for "discover the relationship between the widgez and the widgix" yields nothing. If you can't (do something similar to) reverse a linked list, you can't deliver the solution.
And it's fine you are going to resent me. I do not want to employ anyone who thinks "reversing a linked list" is an inappropriate question in a technical interview.
> What's wrong with 1h vs 1d for a one-off thing?
I was just giving this for completeness - in this case, it doesn't matter. But if it is 1h vs multiple days, or it happens often (say, every 3 hours), then picking the correct solution -- which your method will NOT generate -- is actually important.
> It's the PERFECT time to ask them, because it's fresh in their mind. A skilled interrogator can get information despite the subject's frame of mind. I'm actually surprised that you assume everyone you accept or reject is either going to be a sycophant or hold a grudge.
I ask again, have you hired people? because the "skilled interrogator" mindset makes me think you haven't. I've just negotiated with a person, whom I've had less than 2 hours to know. If I don't want to strike a deal with them, I want to be fair and tell that to them as soon as possible, at which point the vast majority of them just want to move on (and I happily oblige the very few who do want to continue the discussion). If I do want to strike a deal with them, we start the salary negotiation phase, at which point (assuming they are interested) they have a vested interest in me offering them as much as possible; which some people express by trying to please. These things are often subconscious.
I do not assume anyone is a sycophant or holds a grudge. But I do assume, at that point in time, that they are not exactly objective; and I cannot, in fact, calibrate their response since I don't really know them.
> Wanna know how I evaluated potential game devs in person? I asked them what dev environment they were most comfortable in, gave them a computer hooked up to the internet and a picture of a tweetie bird, and said "Make the bird jump. Use google to look shit up." Worked amazingly well.
Sometimes, you can do that. Often, you can't. Especially if the person has no experience in the field they are interviewing in.
"Here's a stock exchange API, write a bot that makes markets" is super simple for someone from the field, and would take hours (perhaps days) of grokking for someone who isn't. And making a bird jump would not necessarily tell me someone is apt for writing trading software.
How does testing a candidate on how to reverse a linked list help evaluate his ability to understand that the implementation is an appropriate solution for a given problem?
What reversing a linked list does tell me is, once the solution IS arrived it, that this person can implement it. And I repeat again, I treat this as an abstract version; real world problems often have solutions with many corners and are seldom if ever "Eureka! i just have to reverse the list".
Another real world problem I just recall is analogous in many ways to a "git merge" which requires tracing through a huge DAG. In these kinds of things, even if you realize that it's a git-merge, it is often not possible to generate a git repository just so you can run git-merge and observe the result.
There are two independent skills: abstracting a problem, and solving an abstract problem. I test for both but the linked list question only addresses the latter.
The problem (heh) with this is that, unless one is actively doing research on improving an existing technology or method, all abstract problems that might come up are already solved.
Every solution is one search in google away (including your git example) so there's no point to it. Paraphrasing someone else: if the time spent in coding a problem is expressed as mx + c, m is the time spent in figuring out what the problem is and c is integrating a solution. That c tends to be insignificantly small.
Frankly, as someone holding a degree, if you were to interview me and asked me to reverse a linked list, I'd do it, laugh in your face, and walk out.
On that point, interestingly, in your earlier examples of people you got feedback from, you didn't mention the most significant one, the one more likely to point out the problems in your process and that would give you actually actionable feedback: the people who rejected the job offer.
Didn't happen many times, I did ask for feedback, but rarely got any; mostly along "your interview was fair, but I prefer a different offer I received". Which is what I would expect - unlike what some people in this thread expect, the candidate is maximizing their revenue and future benefits; saying anything nontrivial after rejecting the offer is not helping towards either, and potentially harmful.
> Frankly, as someone holding a degree, if you were to interview me and asked me to reverse a linked list, I'd do it, laugh in your face, and walk out.
That's good, actually. Means we have a culture mismatch, identified early. Win win.
I had already noticed a very strong reality distortion field in your other comments but this one is just weird. I have been a candidate, most people in this thread have, exactly why their saying what they'd do is not a reflection of what a candidate might do? For instance, I cannot conceive why saying "anything nontrivial" matters in regards to revenue and future benefits. I've done it before (probably not "laughing in their face" but, then again, I have yet to meet an interviewer with such a clear misplaced arrogance) and so far, rejecting a company and giving feedback has not hurt me in any way; they have no impact on my future because I rejected them.
For that matter, how did this even address what I said? I see not how it isn't a non-sequitur, other than you believing that you have a strong enough pull in the industry that you could badmouth me into not getting a job if I rejected your offer and told you it was because your interview was terrible. Of course, that is not the case.
> That's good, actually. Means we have a culture mismatch, identified early. Win win.
Yeah, no, that wouldn't have been deduced from that interview; one could get a culture mismatch by exposing the applicant to the actual company during the process. What I said would only bring to light that your interview process is terrible and doesn't actually account for people who aren't desperate for getting the job (which, in itself, explains why you think everyone you hire is a sycophant, they probably have to).
Of course, I understand your inability to see that it might be a problem with you specifically; no, it must be that the interviewee has a problem with "the culture" of the company.
: For instance, there was no majority saying that the creator of homebrew "was in the wrong" regarding his tweet. At most, there was an even split, although there was far more complaining about interview processes like yours.
It is obvious our different experiences give us different views of the world. I would just add that I have seen this happen, mostly in business settings. A lot of people take things personally.
I don't particularly take things personally myself; it's all business or math to me. But when I was younger, I used to give brutally honest feedback about everything, and there were a couple of investment opportunities that it cost me (and I cannot say that it created others). I learned to play the politics game as I matured. I hate the game, yes, but I needed to start playing to break a glass ceiling.
> Yeah, no, that wouldn't have been deduced from that interview;
No, it would have surfaced immediately. I try to hire people whose attitude is more of the "yes, I can do anything that's needed. But why?" rather than "I laugh in your general direction since I think you are wrong".
> with you specifically;
Me and google. And facebook. And hundreds of other companies. You are entitled to your opinion that e.g. Google's culture is the problem.
Which is not to say my company is as successful as google. Just that I share the idea that culture fit is important. I recommend reading "Peopleware" if you haven't yet -- they have some empirical anecdata for that.
Also, I'm amused at how my "I prefer to ignore noisy data" comment (which is all my comment said, explained several times) was taken to mean "I think everyone's a sycophant or crybaby". I think it reflects on the readers' ability to consider that I'm calibrating my stats differently than they expect, or rather lack of it. And I calibrate it that way based on decades of interviewing -- I didn't start that way.
Wow... Just wow... You've let a big part of your personality slip out with that one.
> And making a bird jump would not necessarily tell me someone is apt for writing trading software.
And now we're getting into absurd arguments.
The thing about such things is I know I can do it. But maybe it'll take 5 minutes, or maybe an hour. Maybe I'd attempt to do it differently given an unfamiliar scenario for which to be doing this in the first place; maybe I've never done this specific task in this specific language, and I'll take some extra minutes figuring out how pointers or references work in the language I'm using in the test. Maybe I've recently enjoyed some momentum on high level concepts (i.e. I produced software that fulfills a business need) and it'll take me a while to refocus my attention on low level implementations.
Engaging one's brain carries some very variable overheads. If you just give me 5 minutes and watch me write every single line of code and comment about it while I'm doing it, then it's just not going to happen.
That said, work environment often has some kind of time constraint. If one can't solve any of the (simple, abstract, representative) problems that I pose within a 2hour time frame, it is possible that they are not a good match. And unfortunately, I rarely have the leisure to hire someone for a month to prove otherwise.
[from a comment of yours below]
> You're right about that particular exercise. I kind of used it to make a more general point.
That may be true for others, but my interview questions are all of this mold: abstract, with a short and simple (but not trick-required-here) solution, close to (an abstract version of) a real world "business problem".
I agree that technical interview questions (much like many aspects of life) put the wrong kind of stress on some and puts them at a disadvantage. But, and I always ask this, what alternative do you suggest?
 The answers so far have been essentially either "no idea", "let me prove to you on my terms" (which, when broken down, is rarely economical for either side or at all possible), or "just trust me on this"; I find none of them satisfying.
In many ways, writing on the Internet is much like the interviews we are discussing, especially in a place such as this, which has so many highly opinionated experts. I get why you got hostile responses to your post, but I also get what the reason for writing that post was. In person, the exact same words would not have lead to such a high-strung feeling as one gets when posting about anything controversial on the Internet.
On the Internet, if you go out on a limb, someone will always cut the branch.
Plus, if you consider "you" (or an imaginary candidate you project yourself upon) risk not being able to answer it in 10 minutes because of such the high pressure it would put on you, then it basically means that you did not even try to answer it in your head, or that you were not able to.
But this is the moral equivalent of asking a person who pretends to be a mathematician to do a 3 by 3 figures multiplication. It won't show much if they answer correctly (at least demonstrate a correct method to do it, never mind potential mistakes), but you can easily dismiss them if they fail.
Now programming is a little bit different, because I admit there can be some positions where people can somehow contribute despite being incapable of lots of basic things. It depends on several factors, though, and even then I'm not sure the end result would ever be very solid. But I understand that in some contexts, such people would not be hired...
Of course I think more relevant questions are obviously better.
I need occasional semi ad-hoc datastructures and manipulations like that. The requirement is never "reverse the list", but it is often something like "this api here needs input in opposite order and data happen to be in list". I would not see reverse the list as something specially unusual.
It's a topic covered in a sophomore-level data structures course over just a few days. If a college sophomore can pick it up that quickly (and "sophomoric" is a word for a reason), it should be no problem for a seasoned programmer to (re-)learn it in a couple of hours.
That's a problematic attitude, in my view. It reads to me that you think you have all the answers already, or at the very least that you view yourself as being superior to people you interview. That lack of humility is troublesome for a number of reasons.
As far as your interview question, do you normally not interview candidates to develop software? Your question is appropriate for a grunt who's entire responsibility is to thoughtlessly code up somebody else's design. It's not appropriate for someone expected to contribute thoughtfully to the engineering of a software product.
I would correct that (if I could still edit) to "direct immediate feedback". What I meant was, if someone I just turned down for a job says "it's stupid", I don't consider that useful data; similarly, if someone I just hired says "This is the best interview question ever", I don't consider that useful data either.
I do ask for, and usually get, direct and raw feedback from people I work with (both those who report to me and those I report to).
> As far as your interview question, do you normally not interview candidates to develop software? Your question is appropriate for a grunt who's entire responsibility is to thoughtlessly code up somebody else's design. It's not appropriate for someone expected to contribute thoughtfully to the engineering of a software product.
Can you elaborate on what you consider "contributing to the engineering of a software product"? and more generally what "software engineering" entails?
I mentioned as explicitly as I can that I do not expect anyone to reverse linked lists. However, it is an abstract question that is not unlike the kind of things that arise in practice, e.g. figure out inverse relationships in data. And gives a good ground for discussion about things like requirements (time, space, correctness, verifiability, boundary conditions, erroneous inputs, etc) -- inside a well defined and (supposedly) familiar context; The same problem in a business setting would require a lot more definitions and would be frighteningly foreign for some people (to digest within an interview).
How do you interview?
Actually, not in a space appropriate for an HN comment. Here's a very brief summary: I think in its current state there is very little actual engineering in this industry. It seems to me to consist of a large number of folks who approach every problem like a CSX01 project. It could move toward a more mature engineering footing if there were less focus on CS trivia and more on broader, deeper understanding of systems and relationships between them.
To be blunt, your "reverse a linked list" question is something I consider an example of the Law of the Instrument (https://en.wikipedia.org/wiki/Law_of_the_instrument): every problem is a CS nail to hit with a CS hammer. It's great for exploring the items you note at a very low, narrowly-scoped, and shallow level. It's not so good for gauging actual problem solving ability or engineering ability.
> How do you interview?
Interviewing is a crap shoot, and my methods are at least as flawed as any others'.
Phone screens with a shared editor are the place I ask questions like yours, except I'm not looking for a correct implementation, complete solution or discussion about it: I'm looking to see if they at least have some vague notion of what writing source code entails (even if it's just because they read about it and memorized it).
At the in-person interview I tend to do two things: discuss in depth a project in the person's past and a higher level discussion about something pertinent to their work at the company. For me the latter has varied from details for some feature to be implemented in a real time system to motivating a data exploration approach in the context of developing a data science product.
> It's not so good for gauging actual problem solving ability or engineering ability.
Also agree. It is not the only question I ask. However, I only want to hire people who, when the need arises, can actually use a hammer. I am perplexed that people find that offensive.
> At the in-person interview I tend to do two things: discuss in depth a project in the person's past and a higher level discussion about something pertinent to their work at the company. For me the latter has varied from details for some feature to be implemented in a real time system to motivating a data exploration approach in the context of developing a data science product.
Agree. However, I also do some technical parts in person/
And there are thousands of other things that you (almost) never have to do, yet you do, throught the course of your career. When they arise you're not an expert in them, but after some careful research you become one, or become enough of one to say which solutions are better than others for the problem you have. And you'll be able to implement it, support it, and explain it to the people with whom you work.
These types of interviews don't see the forrest for the trees. The problems that you have had won't always be like the problems that you will have. What's important is the ability to solve problems you've never seen before (a graphql request took down the database cluster), not recite known but not widely remembered tasks (balance a b-tree in-place).
Getting to the point where you have to start to worry about Big O is the hard part, not the other way around. And Jesus, I really hope we work in places where if you're unsure about Big O you can ask someone and they'll help you, not judge you.
Edited to add: What these types of interviews say to me is the companies who use them have no interest in their employees growth and education.
Most definitely! And that's exactly why I no longer use this question (as I wrote in my original post) -- by now, everyone has seen it.
> Getting to the point where you have to start to worry about Big O is the hard part, not the other way around. And Jesus, I really hope we work in places where if you're unsure about Big O you can ask someone and they'll help you, not judge you.
I disagree. I deal with lots of data (just ordered another 100TB storage the other day; I do that every couple of months these days). Toy solutions that work well for small 100GB datasets often don't extend. Without a good grasp of complexity, it is easy to build solutions that don't scale properly, or cost $100K more in hardware/AWS than they should. I've seen it happen a lot of times. As they say, "a year in the lab can save you a week in the library!". If you don't know that you should go to the library, you're likely to lose that year.
> What these types of interviews say to me is the companies who use them have no interest in their employees growth and education.
Some of us are bootstrapped in a cut-throat market and can't afford to plan employee retention 10 years into the future; And neither are most employees interested in that.
This matters? Any other question stinks just the same, like my example of balance a b-tree in-place.
> I disagree...
What is it you desagree with? "Getting to the point where you have to start to worry about Big O is the hard part, not the other way around." or "I really hope we work in places where if you're unsure about Big O you can ask someone and they'll help you, not judge you."
> Without a good grasp of complexity...
I don't see how the interview style you seem to advocate finds people who are good at understanding complexity. This comes from experience, the opportunities to have worked on unfamiliar projects, not from reading a how to pass a job interview book.
> Some of us are bootstrapped in a cut-throat market
Some of us work in toxic environments that are hostile to women and people of color but ¯\_(ツ)_/¯, right?
> can't afford to plan employee retention 10 years into the future; And neither are most employees interested in that.
I would wager that everyone is interested in learning something new or deepening an existing skill in whatver job they take no matter the time period. The alternative would be to, which you seem to advocate, take a job, stand still in it for a few years, then wehn you're bored or feel like you're becoming obsolete, read some books on nights and weekends on whatever new trends people ask about in interviews, take a new job, stand still in it for a few years, and so on.
"Getting to the point where you have to start to worry about Big O is the hard part, not the other way around."
Or maybe I misunderstood what you meant. I think that unless you are doing mostly "do this and then do that" style programming (e.g. CRUD), you have to be aware of big-O, or you're going to hit a lot of walls. In things I do, anyway.
> I don't see how the interview style you seem to advocate finds people who are good at understanding complexity.
Complexity here being the well defined CS term about space and time, of which big-O is the customary notation: How your resources (time, space, etc) grow with the problem size. I suppose you meant complexity as some measure of cohesion/details/size - in which case, you are correct - this kind of questions do not address it (others in my interview do, though I admit it is much harder to judge one's view of other-complexity in an interview).
> Some of us work in toxic environments that are hostile to women and people of color but ¯\_(ツ)_/¯, right?
Some of us make unrelated remarks, but ¯\_(ツ)_/¯, right? What exactly were you trying to say here?
> I would wager that everyone is interested in learning something new or deepening an existing skill in whatver job they take no matter the time period.
Then you would lose this wager. There are so many people who just want to clock in, do their job, clock out and get paid. Yes, even in technology. And you know what? that's fine; They are not lesser people because of that - they likely care more about hobbies or other things, and would happily quit their job and have fun if they won the lottery.
> The alternative would be to, which you seem to advocate, take a job, stand still in it for a few years, then wehn you're bored or feel like you're becoming obsolete, read some books on nights and weekends on whatever new trends people ask about in interviews, take a new job, stand still in it for a few years, and so on.
Not at all -- but there's a huge difference between "being interested in something new" and "train someone in a new field they are not proficient in" especially when there is no commitment (nor could there be) on their side. The latter costs money to the employer, which some can afford and some cannot -- either way, it is not a basic human right.
Of course you do, along with many other things - many other things that you won't know you need to be aware of because you haven't come across them yet or because they haven't been invented yet. What I said is that it's important to know how to figure these things out, not be able to recite them from memory.
> What exactly were you trying to say here?
You said "Some of us are bootstrapped in a cut-throat market and can't afford to plan employee retention 10 years into the future" To me this is a defeatist attitude. Our jobs are fundamentally about solving problems, and this, like my example, has a solution.
> Then you would lose this wager.
You can't have it both ways. You seemed to say that your preferred interview style selects the best of the brightest, but now it selects people who just want to clock in and out?
Unfortunately, if resources are limited, you have to compromise. Google's resources are limitless, mine isn't. if your solution is "close up shop and go work for someone who has more resources" then I'm not the defeatist. What's your solution for constrained resources?
> You can't have it both ways
I do try to select for the best and the brightest. I pointed out that some people like to clock (your wording implied everyone likes to learn new things on their job). The best and the brightest who are not Jeff Dean tend to change jobs every few years even if they work for Google. And no, I cannot afford to employ Jeff Dean, I admit. Can you?
Which resources are constrained?
Our examples are 1) (yours) cultures that are hostile to employee growth and education, and 2) (mine) hostile to women and people of color. These are solvable problems, even ones as awful as Uber as we're seeing play out. As for what we can do, we can start to hold our industry accountable. When we hear about some unicorn at some absurd valuation, rather than simply applaud we need to ask them about their culture, how they treat their employees and customers, do they have a healthy work/life balance, what exactly are they doing to foster diversity, what's their impact on the environment, does their product actually make the world a better place, are they in service of the surveillance state, and so on. We need to redefine what it means to be successful in our industry.
> And no, I cannot afford to employ Jeff Dean, I admit. Can you?
How did we get here? I said "What these types of interviews say to me is the companies who use them have no interest in their employees growth and education." As I read the thread that follows from this point, it seems like you agree which is at odds with your defense of these types of interviews that started our discussion.
You assert that, but I have no idea why you would think that. That's why I was perplexed at your statement. Can you elaborate? Non of the examples you give liek work/life/diversity/environment has featured into the discussion, nor the definition of "success in the industry"
> I said "What these types of interviews say to me is the companies who use them have no interest in their employees growth and education."
And I ask again, is, why would you assert that? It seems trivial to you, but I'm completely at loss. All I said was that I cannot, at hiring, offer a 10-year growth plan for a person. It does not mean I do not expect, encourage and help them to grow (quite the contrary).
Why I would think what? That this is true or that you think so too? As for the latter, because you said so...
* "Some of us are bootstrapped in a cut-throat market and can't afford to plan employee retention 10 years into the future; And neither are most employees interested in that."
* "All I said was that I cannot, at hiring, offer a 10-year growth plan for a person."
* "There are so many people who just want to clock in, do their job, clock out and get paid."
* "The latter costs money to the employer, which some can afford and some cannot"
All of which seems to say to me that a person better come with all of the skills that they could ever possibly need now and in the future because they're not about to get any support from their employer or teammates. Just sit down, shut up, and mash on that keyboard. What you seem to advocate as growth and education is something that a person should do own their own time and at their own expense. People who aren't willing to attend meetups, read research papers, work on side projects, etc. after hours are just a bunch of clock punchers who go through life without contributing much. This sounds pretty horrible to me.
"10 years" and "the costs" are just rhetorical attempts to set the bar so high to say that it's just not possible or too unreasonable to care about lifting up your teammates. This is easily provable false simply because of the companies that do this already no matter the tenure of their employees and at very little cost.
As for the former, I say there are far more important skills that an interview should focus on rather than if someone can recite the pros and cons of different sorting algorithms and their Big O costs because this is easily teachable. But when you don't care about growth and education, or just simply teamwork and collaboration, you want to know if candidates already have the answers to the problems they'll encounter, which, of course, is totally ridiculous to me.
Would you please share where you're working so we can avoid applying?
because I've just hired them, or just didn't hire them; either way, I think it is irresponsible to trust the statement of a person I don't actually know and who may either hold a grudge or want to please me right now. I meant to say "direct immediate feedback", which is perhaps better wording.
You may not have gotten it personally, but some very smart people have given precisely that feedback regarding those sorts of questions.
Being smart and being a good fit are not equivalent. Linus Pauling was definitely smart - and most definitely wrong about quite a few things (most famous was his statement that "there are no quasi crystals, only quasi scientists", a derogatory remark about Dan Shechtman who later got a nobel prize for those quadicrystals)
Put it this way: if I decided someone was incompetent as a programmer based on some criteria, and then I later found out that that person wrote Homebrew, I would take a pretty hard look at the criteria, and how strongly I believed in its predictions.
I do not disqualify people who cannot reverse a list (as long as they demonstrate critical thinking skills and reasonable capability). I do disqualify people who get up in arms like Howell did, and I dare say this tweet does show Howell to be a bad match for many teams, regardless of his technical skills; He is likely not a competent team player. (As much as can be inferred from the tweetstorm that happened 2 years ago, and as much as I can remember it).
All I'm talking about is the predictive power of the "please rejigger this data structure" interview question, and how much faith people place in it. I mean, if someone looks at the Howell thing and says, "well, any interview can have false negatives, but questions like this are still good predictors of competence", I'd certainly have no argument with that.
But a lot of people looked at it and said, "Nope, if he failed a question like that then he's not a competent programmer, regardless of what he's built". When someone says that, they're implicitly claiming that "passed an interview question" is a better predictor of programmer competence than "built and maintained one of the world's more popular developer tools". Doesn't that strike you as a pretty extreme amount of faith to put in an interview question?
The responses I remember (memory bias possible) were along the lines of "well, he might have built that but is not a good match for google if he failed a question like that".
I have actually calibrated my interview in the past on friends, colleagues, employees and students, and I know that it has predictive power for the kind of work I do (which does NOT include, e.g. CRUD or the simple if-then-query logic which is often referred to as "business logic").
With respect to Max Howell - I will, indeed, leave speculation out, but will say that the attitude he displayed on twitter would disqualify him from a lot of places, regardless of his technical merits. (And as for the real reason he was disqualified - it was pure speculation on his side).
Personally, I strongly suspect that the predictive power of interview questions depends ~90% on the knowledge and experience of the interviewer, and the specific questions aren't that important. I reckon the disconnect here is that you and JBlow are judging things as the guy had failed the interview question as you would have administered it. And it's possible that that's roughly the case, but it's also possible that he failed the interview as administered by a bad interviewer - who, say, cut him off after the first trivial error rather than suggesting he check his answer for bugs, or whatever.
As such, I take no issue with data structure questions per se, I just find it hard to buy this idea that they're massively predictive - compared to, say, having shipped lots of good code for a long time.
> With respect to Max Howell - I will, indeed, leave speculation out..
This would read better if it wasn't followed by speculating where the guy's qualified to work (based on memories of a tweet from three years ago!) and then speculating what he does and doesn't know about an interview he was in.
The process at Google (of which I'm not intimately familiar with, I admit, but I have read a lot about) is such that if you are rejected, you are very rarely told why (I know of one case in about 30, and it was an offer that was rescinded based on some bureaucratic reasons). It might have been the reason, for sure; but it is extremely unlikely that this is what he was told, thus I infer speculation on his side. By accounts from Googlers I know (and some who commented on those threads at the time), it should be considered truism, not speculation, that he was not told the reason.
Also, I'm not speculating about where the guy is qualified to work. I was referring to several places I know for sure (some high profile, mostly in finance) where such a rant on Twitter is enough to disqualify you (and get you fired if you're already employed). I don't know Yelp specifically, but it seems like one of those places
Nobody's suggesting HR sent him a letter - presumably he formed his conclusion based on what was said in the interviews. It may be that he did so unreasonably, as a defense mechanism, and it may be that whatever was said left little doubt why he wouldn't be getting an offer. All we know is that he felt there was enough information to form a conclusion, and the principle of charity compels us to assume that such may be the case.
Honestly, I'm not trying to pick an argument or anything here. I just think your confidence judging this guy's character and hirability and so on is way out of whack considering that it's based on a three year-old recollection of a tweet.
For a hypothetical example:
I've spent a day for your interview. I aced it. I was told there is somebody better. I was then asked to waste more time in form or feedback.
Same goes for references. In fact, from a real life example:
My company needed an entry level technical writer. I referred them this kid with a fresh comp sci degree, eager for a job. Reviewing code: yay, he thought he made it. Got turned down because they wanted only a rockstar entry level, technical writer. Then chose to not get one.
So, given everything, because I'm a polite person, I'd rather not answer than tell them they are delusional and rude.
My replies to dodge the question would be: That's not a question, that's a command.
I've found that having common questions used across many interviews allows for more objective direct comparisons. I like this one in particular because even if they solve it quickly, you can move on to questions about cycles in the input.
What do you use now?
Another favourite of mine is basically "implement a very sparse 100Mx100M matrix". Thought process much more important than result. (important things to consider: access time, storage size, static vs. dynamic, use cases, adversarial coordinate selection).
I actually consider the "cycles in the input" question a bit of a trick question -- I calibrated the test on people I knew were good, and no one who didn't know it already was able to come up with "the trick" in reasonable-settings-for-interview-time, and not many were able to come up with any O(1)space O(n)time solution.
The discussion with those who don't know the solution is useful, of course.
Agreed, the discussion on these questions is the whole point, not whether you can come up with a "correct" solution (and I tell candidates this up front).
I generally draw a small example input with a cycle, and ask what their list-reversing solution will do with it, which I feel makes it much less of a trick question. "Can you work out what the code you just wrote will do with inputs that don't meet the assumptions you made?" seems like a fair thing to ask anyone who will be required to write code that handles untrusted data.
Interviewing bad people at good companies goes more like this:
Q: Can you explain the difference between a noun and an adverb?
A: I've worked at the UN for 20 years! I'm an accredited translator! I translated for Putin and Obama!
But I think a lot of the grief around whiteboard interviews comes from all the additional crap that's grown up around that, including:
1. That ideally a candidate should code up solutions on a whiteboard, while standing. That it's unnecessary, or even bad, to try to replicate a working situation by having the candidate seated at a computer.
2. That there's a correlation between difficulty of question and quality of developer. So a candidate who can implement both fizzbuzz and quicksort at whiteboard from memory would be a better developer at your company than one that can do fizzbuzz, but not quicksort.
3. That the ideal programming question has some sort of "trick". So the ideal flow when a candidate is answering a question starts with the candidate implementing a naive solution. After which the interviewer says "it's possible to do better". Then the candidate ponders for, ideally 30-60 seconds, whereupon they realize the trick and code up the optimal solution.
My ideal interview process recognizes the need for a developer to demonstrate the ability to code while at the same time recognizing the above interview style for the cargo cult exercise that it is.
EDIT : Add to that, perfect time complexity.
I had a question recently, which was so tough, that forget the allotted time, I have asked my friends about how to solve it and they seem to have no clue either.
An array represents a binary tree like this :
so apart from the last level, any missing child nodes are represented as *. You need to find the set of unconnected nodes whose sum is maximum. Child-parent relationship means they are connected.
node.next = list.head;
list.head = node;
Performance complexity is spoken to general cases and averages unless indicated otherwise.
Singly linked list or doubly linked lists are still considered linked lists. Skip lists, even though they technically are lists of linked items, are not generally considered a "linked list" data structure.
The half-trollish point being that nothing is "obvious". Indeed, list insertion is O(1) in popular programming languages s.a python or JS.
Case in point: I interviewed a guy who nailed even the most obscure stuff, but he couldn't finish a project. He was great at small, complex problems, but he couldn't ship. We needed someone who can ship.
Q: Explain the difference between they're, their, and there.
Q: Principle or Principal?
This part got a laugh out of me.
Otherwise, the analogy feels really stretched and at times feels straight up incorrect. For instance, I've never sat through a technical interview administered by non-technical staff. I've likewise never been quizzed on the history of computation.
I agree that the programming interview in the US can be overly algo-and-whiteboard happy, but I think this critique is unfair and possibly even outdated (my most recent round of interviews involved more live coding, and less whiteboarding, than when I last interviewed 4 years ago)
Base 2 problems tend to require clever recursion and dynamic programming to solve, while the natural logarithm often requires calculus.
But then again, I'm not a mathematician - just what I took away from a broad undergrad.
I think I disregarded HR screens because the questions (IME, at least) are often so trivial the experience of the interviewer doesn't really matter. There have been a few times when I've disagreed with a screeners pre-defined answer, but none of those cases resulted in a rejection or a halt to the interview process.
I had to smile when one interviewer described his company to me as very process oriented, but without the overhead.
> Weird analogy. Companies don't ask candidates the history of binary search trees, computer architecture, or anything like that.
A better analogy would be if they gave this translator a particularly challenging piece of text to translate -- for example, one that didn't have a clear right answer and the candidate had to discuss different tradeoffs.
But... then that doesn't seem like quite so silly of an interview process.
There are absolutely valid criticisms of whiteboard interviews, but most criticisms made are either based on terrible implementations of whiteboard interviews or based on stuff that's just incorrect. (Yes, it's totally fair to criticize a company who conducted a flawed whiteboard interview. But that criticism doesn't apply to the system as a whole. That same company could mess up whatever your favorite interview style is, too.)
> By the way: I don't actually know how translators are interviewed. But one of my best friends interviewed to be a journalist with some major New York newspapers (WSJ, etc).
She was already a journalist before this, so they had lots of public writing samples for her (analogy: GitHub code samples).
Did they just hire her based on this? Nope!
She had to do a live writing test (analogy: whiteboard coding interviews). She also had to do a pitch session to talk about different potential stories she could theoretically write about (analogy: design/architecture interviews). Plus some behavioral interviews.
Why not just look at her writing samples? Unlike for coders (which might not have public portfolios representing a significant portion of their work), basically all of her work product was actually public. So why not just hire from that?
Well, because all they see is the final output. They don't know what direction she was given, how long it took her, how much editing/collaboration was involved, etc. A crappy writer in a particular environment can churn out good work -- because someone else is doing a lot of the work. Looking at the final result is actually not a great measure of someone's skills.
Coding interviews aren't that special.
What you describe is what interviewers think they're engaged in when they ask CS trivia in interview settings. They are not, and in fact are much closer to what the article describes than the picture you paint.
The point the author is making is that interviews often question knowledge that is so far removed from actual work (aka the Arab influence on modern day Spanish), that it is kind of a joke.
I call this mind wanking. Interesting and may be fun. But not actually relevant.
Instead of asking far removed questions, I interview with actual current hard engineering problems that I or my team is faced with. That way you learn something, they know what actual work would look like. No time wasted. Win win.
What makes you think that? My experience is that Google at least would never ask a question about red-black trees. I doubt Amazon would either. These big companies stick to questions that don't require a lot of specific knowledge because they know that a question about red-black trees mostly tests how recently you've studied red-black trees. I've done a bunch of whiteboard interviewing, and I would be shocked to be asked a question about red-black trees at a competently run company.
"Cracking the Coding Interview" explicitly mentions red-black trees as a topic that you're unlikely to see in an interview (for obvious reasons).
I think the places you see these sorts of questions asked are smaller companies that are imitating the big players without really understanding how to do it right.
By then you have wasted a good candidate's time, who doesn't want to interview at your company anymore. Go hire the folks who memorized RB tree.
If it was before, then I think it is fine. It basically test whether you are able to learn something like red-black trees when needed. I would consider such question a good one.
I don't say this to personally attack you, but Google and other top tech companies have made it clear that years of experience no longer holds the same weight as other industries, perhaps because other industries are more tightly regulated with what its members can do. Otherwise reference checks would handle most of the technical interview ("Did this guy actually build X, Y, and Z at your company?" "Yes." "Great!"). You can spend 10 years in a bad position and Google can't know every company to decide whether it should hire from your company or not.
This has been one of software development's strengths: literally bootstrapping yourself from "Hello World" to 120k+. It's hacks and cheat codes compared to becoming a doctor or physical engineer (the ones that need a license). It means a disadvantage upbringing can be overcome with time and tenacity. It's a romantic notion, but because of this, you have to evaluate the candidate or risk a bad hire. And bad hires, as we all know from the same echo chamber that gave us teach-yourself-boostrapping developers, can utterly ruin your business and you never want to be a manager known for making a bad hire.
Otherwise, we'd all take an exam, refresh it every number of years, and be free to disregard most of the technical interview. Getting well-regarded regulation and certification is no easy task, and I suspect there's not movement behind this because you can't kill people with software as easily as you can with a surgeon's scapel or a bad bridge.
The industry is actively fighting against glue-things-together developers by asking serious CS questions. It hasn't accepted that we have to branch the "Developer" position into more well-defined roles and add specialties (Security, AI, UI/UX). It hasn't accepted that a lot of work is actually glue-things-together.
They're appropriate questions for interviews in some cases. But very few. Even at the companies you note.
Where do these get asked.
Nobody actually looks at your code and think "Oh wow, that code is clean, it works well, it's loosely coupled and high cohesion - We should hire this guy".
Often they just want you to be able to solve lots of tedious algorithmic problems REALLY QUICKLY. The horrible thing is that even if you CAN solve these problems, you might not be able to solve them all within the allocated time (unless you've practised a lot recently).
More to the point, at zero times in my career have I needed to do fundamental algorithm implementation. If I did, I'd question what people were doing by reinventing the standard library.
And even if I did end up needing a particular algorithmy solution to something, I'd be reading the relevant parts of TAOCP, looking at prior art, discussing it with other people in a team environment...
None of which is implementing red-black trees on a whiteboard. Really. It's completely orthogonal to software development.
It is important when doing lots of interviews to have a question that you know well and can be used to benchmark across your interviews. Something relevant to the job is an added benefit.
After that, I started asking for answers to problems that I really had in the day to day. Some of them actually had real complexity (like doing binary search to overcome a buggy HTTP request items pagination library).
I think the main problem with data-structures and algorithms questions is that, the person that originally envisions a question, understands the "soul" of the question and how to vouch the candidate as she solves the question. However, when someone else takes that question to apply it, he only sees whether the answer is right or wrong, without looking at the resolution process.
In addition, I think that when someone asks a binary tree traversal problem to a candidate, the interviewer starts fretting over trivial edge cases, function signatures and coding style etc.
Whereas, if you would have asked a distilled version of that pagination problem, I would be so excited to have a chance to solve that problem and we could focus on my ability to solve that problem instead of my coding style on a whiteboard.
In reality, the sad state is, in my opinion, confluence of the following forces:
1) HR people more often than not have barely any clue about the topic. They must, unfortunately, play this charade because if they knew the topic they wouldn't be working in HR.
2) The technical people who prepare the questions typically believe they are too busy to spend time thinking about the problem and instead decide to settle on any test. The assumption is that a good candidate will be able to navigate any kind of test better that a bad candidate. In view of the technical person, this is just a screening, the real purpose being to elliminate as many phonies as possible.
3) The candidates more often than not have highly exaggerated view of their abilities. Unfortunately, high demand means that the market reaches for lower and lower quality of "resource" leading to comical situations where a large portion of the workforce (especially in countries like India) is developing software by shuffling around keywords until the code compiles which entitles them to call themselves Senior Engineers. Real senior people have no problem finding a job to the frustration of others who find the situation "unfair" and the entire process "rigged".
One more note on the process: while the cost of failed interview for candidate is quite low, the cost of making a mistake is very substantial for the company.
And also, http://www.jasonbock.net/jb/News/Item/7c334037d1a9437d9fa650...
> Carpenter: Gosh, I really don't know -- was I supposed to be counting the walnut?
> Interviewer: Would you say you're an entry level walnut guy or a walnut guru?
Oh, hey, I just had that interview. It's called the California State Systems Software Specialist examination.
Almost 60 questions like:
"Create processes (e.g., install, configure, maintain, secure, backup/recover, etc.) to ensure that technical
staff are consistent with vendor documentation, application requirements, and departmental standards."
"Consult with internal/external entities regarding services provided by systems software teams and answer questions/inquiries in technical areas such as connectivity with departmental systems, data exchange, security, etc."
For each of which, you must answer:
a. Performed the task within the last 2 years
b. Performed the task within the last 4 years
c. Performed the tasks more than 5 years ago
d. Not performed
Years of experience
a. More than three years experience performing this task
b. More than one year to three years experience performing this task
c. Less than one year performing this task
Level at which the task was performed
a. Supervised or served as an expert on task
b. Performed task as a lead
c. Worked independently on task
d. Worked under close direction/supervision on task
e. Assisted another person on task
f. Not performed
If you don't happen to have been assigned the proper tasks in recent years to score highly enough on the test (by whatever algorithm they use), then you are not eligible to even apply for any Software Engineer role in the state, and can't retake the test for 6 months.
Having said at my first job our in house wood shop where capable of museum grade work in any wood you name but we where no 1 or 2 in the world for the areas we worked in
"Could you translate '¿Donde es el baño?'"
"I'm sorry, I don't do well on tests. I thought we were going to talk about my past projects on my résumé. In fact, I'm quite offended by your question. Good day, sir!"
"Wrong, sorry. My sheet here says 'Where is the bath?'"
Those companies go out of their way to reduce as much bias as possible. In the long run we are all better off if we rely on data instead of anecdotes and fancy resumes.
Google: 90% of our engineers use the software you wrote (Homebrew), but...
Haha, what feedback?
lol... I actually won a trans-comp when I was in (middle?) school: I was attending a catholic school and we were taught latin and went to some translation competitions: you had to translate a chunk of text as fast and accurate as you can. It was fun :)
...we do Agile. Everything is
in a flat structure, except
when it comes to salary and
I really should have left at that point: >1 hour lost there...