Interviewing people is hard. Most people suck at it. I've interviewed over 100 people in my life, and I've been interviewed about ten times. I know I still suck at it.
The code test is one of the more annoying facets of interviews. On the hiring side, it's an efficient way to screen and vet candidates for basic skills. It's relatively low cost to the employer to have a recruiter issue the test.
If you come up with your own in-house code test, it still requires time to evaluate and grade the submissions. There's also a fairly large cost to creating an in-house code test (I know from experience). You have to balance complexity so that it requires enough skill to pass, but isn't so onerous that it puts off too many candidates.
Using something like LeetCode removes all the cost from the employer side. Someone else maintains it for you, and you get a canned score at the end. It also depersonalizes it, and provides an opportunity for people to game the system by memorization. The same applies to using canned whiteboard/algorithm questions.
I think most of this is a colossal waste of time. It turns away people who don't want to spend hours (or days) on each interview. Imagine applying for even a handful of jobs where each one expects you to do a take-home test that can take a full day. Who has time for that? There's the "if they really want to work for us, they'll take the time to do it" argument, but let's face it, most people are not that passionate about where they work until they actually work there.
As a candidate, I am loath to spend my time on these. As someone who's been doing this for 30 years, I'm also a bit insulted by having to do the monkey dance over and over again. At this point in my career, I don't bother applying to places like this. My resume speaks for itself.
Of course it makes sense to vet the technical skills of candidates, but you can learn a lot more from 15 minutes of questions than you can from a code test. If you present a candidate with some abstract questions and a couple of targeted detailed technical questions, you can tell if they've got the right problem-solving mind and if they've had experience with the technology you're working with. It's also a much more personal experience.
At the time of this writing, I am working with someone who has 10+ years of experience, including former positions as a lead developer. I was involved in the interview process and made the recommendation to hire, as they were able to answer a lot of domain-specific questions very well and had a good knowledge of the ecosystem. Fast-forward three months, and I am spending hours per day sending changes back on code reviews for features that would have taken me 30 minutes to implement. Every review has at least one conditional that always evaluates to true. Yesterday I found a try/catch block that always throws inside of the try block. The crown jewel was the following for-loop (psuedocode):
for i=0; i<=3; i++:
try:
doSomething();
catch:
if i<3:
continue;
else if i==3:
break;
And the best part is because of political reasons and difficulty hiring a replacement, we can't fire this person. I now include coding exercises in my interviews. I typically focus on reviewing code with bugs and refactoring bad code instead of forcing them to whiteboard.
That said, this does seem like another reiteration of the "senior engineer couldn't write fizz buzz" story, though this particular example is amusing.
I can write fizz buzz. Hell, I've interviewed enough that I can print all permutations of a string (or all permutations of a set and/or subsets of that set). I used to be able to implement merge sort in 45 minutes at the whiteboard, but I can't now, and I never want to be able to do this at 45 minutes at the whiteboard again. I could certainly detect a cycle in a linked list, and I could do DFS and BFS on a binary tree. I might be able to reason through how to find the last common ancestor of two nodes, or find all matching subtrees in a binary tree if I could hack around for a while, but at the whiteboard, in 45 min? Nah. So, no-hire.
It's time to stop pretending these tests are about weeding out people who lack very basic coding skills. Tech interviews are one of the reasons I'm not enthusiastic about recommending a career as a software developer.
The original story here wasn't about code written in an interview, but code written on the job by someone who wouldn't have been hired if they'd had to write code in the interview.
But, 45 minutes for mergesort? I mean, it's not fizzbuzz, but this took me seven minutes, and it passed the test on the first run (well, not counting the test runs that verified the test was invoking an empty function and correctly reporting that None wasn't the correctly sorted list). Does it have some kind of subtle bug I'm missing?
def mergesort(items):
return (items if len(items) < 2 else
list(merge(mergesort(items[:len(items)//2]),
mergesort(items[len(items)//2:]))))
def merge(sa, sb):
ia = ib = 0
while ia < len(sa) or ib < len(sb):
if ib == len(sb) or ia < len(sa) and sa[ia] <= sb[ib]:
yield sa[ia]
ia += 1
else:
yield sb[ib]
ib += 1
def ok(a, b):
assert a == b, (a, b)
ok(mergesort([3, 5, 3, 1, 10, 11, 12, 0, 1]),
[0, 1, 1, 3, 3, 5, 10, 11, 12])
If I were doing it on a whiteboard I'd probably want to use something like http://canonical.org/~kragen/sw/dev3/paperalgo rather than Python, but I can't imagine it would take 45 minutes. Do you mean in assembly language or something? In minimal space (none of the logarithmic number of copies required by this version)?
I agree that these are not very basic coding skills, but they are kind of basic coding skills, I think? Like, it's covered in the first part of an undergraduate CS curriculum, isn't it?
Super late responding, but I enjoyed your comment and code.
Unfortunately... yes to both your questions. Yes, it is certainly covered in basic undergraduate "Data Structures and Algorithms."
And yes, I would have trouble writing what you just wrote in 45 minutes at a whiteboard, unless I studied up in advance. I know the algorithm in the back of my mind, and I'd be able to hack away at it. I don't mean in assembly, or in minimal space, I just mean a reasonably efficient implementation of quick sort in a high level language like python ruby or java. Again, I could do this with a bit of study, though I'm no longer inclined to do that study. I can't think of a reason, other than interview exams, for me to front load "into to data structures" into short term memory for a midterm exam one. more. time.
Maybe it does say something about my skill and mind set as a developer, that it would take much longer for me to do this than it would for you. I'm ok with that, as long as we're clear on what "basic" coding means.
I maintain that it is an immensely different thing to say you're weeding out people who don't have basic coding skills when the test is "fizzbuzz" vs "implement merge sort".
Well, I tend to spend a lot of my spare time noodling on what I think of as fundamental CS problems (what's the optimum radix for a min-heap? 3, I think, not 2), and I've spent a fair amount of time programming in languages like C where implementing mergesort from scratch is actually a practical thing to do. And I have a prejudice in favor of knowledge with a significant shelf life, like how mergesort works, rather than, say, how to work around the bugs in the latest version of React, which is also practically necessary for getting stuff done. So I might have a bias in favor of that kind of thing.
I certainly agree that mergesort is a lot harder than fizzbuzz, more than an order of magnitude. Fizzbuzz is well under a minute.
I'm pleased that you enjoyed my comment and code! I was reluctant to make it because I feared you might interpret it as an attack.
That's all true, but it's not really relevant to what I was saying. We don't do whiteboard interviews; that was some code I found during a code review. My original comment was in response to the assertion "my resume speaks for itself", and was simply meant to serve as a counter example. A lot of people seem to have interpreted it as "interviews should involve complex whiteboard programming" and/or "you should always use redux for your back office CRUD apps".
Would a better interview test be to ask the candidate to review code rather than write it themselves? That is, give them a code sample, describe what it aims to do, and ask:
1) Does this code accomplish the goal?
2) How would you improve this code to make it more readable? More efficient?
Personally, I think I'd rather work with people that know how to review code and provide feedback/commentary, since that usually means they can write good code themselves. It should lead to a better team dynamic.
Many years ago, I was given a piece of code and asked to explain what the code did. I got the job because I starting asking questions about what the code was expected to do and what sort of evnironment the code was running in. Even though it was only three or four lines, without first understanding the contextual questions, the code itself could have been doing a variety of tasks. Since I did become the maintainer of the code base, it did turn out that there were quite a few non-obvious effects occuring in that piece of code.
There are plenty of programmers who would have jumped right in and explained what the code was doing (surface appearance) and missed the simple fact that there was nowhere enough information present to actually know what that code was actually doing.
So even with you two questions, which are a start, getting them to indicate they understand the context/environmental problems can be a more challenging process.
I think both the ability to write, and (read+understand+effectively explain and give an opinion on existing code) are important, and the latter is severely under-appreciated in interviews
> This person is so convinced that they are a good developer that they refuse to take direction or ask questions
While I agree a coding exercise is valuable - this is the actual problem. The way I test for that humility is by asking "tell me about a time you personally did a poor job." I then follow up by asking for a time they identified a failure of a colleague. I look to see if they're empathetical, open to being wrong, and declaring they're always trying to improve.
A difficulty with this kind of question is that the how-to-beat-the-coding-interview books and sites, which many students and professionals read, already prep them for how to answer.
People get step-by-step strategy for this kind of question, what the interviewer is looking for, and what notes to hit. And sometimes prep them for the exact question, so that they can have a prefab answer ready.
Then we might be selecting for more like what used to be MBA students, or at least people who know how to play along, and say the right corporate going-through-the-motions things.
Some organizations will want to select for test-prep specifically, as a good predictor for the kind of employee they want, but I like to think there's other organizations that decidedly don't want that.
I don’t think this is a great approach. You’re selecting for good talkers. There are many great engineers and coworkers who don’t do well on these types of verbal psychological games.
Folks are given all sorts of interview advice about leading to claim things like “my greatest weakness is that I work too hard”.
One way to test for humility is have them do a programming challenge that isn't too difficult (it's not about whether they remember the algorithm for merge sorts), but has a few common pitfalls and arbitrary choices, and see how they react when you ask questions about their choices or make suggestions to them. Can they explain a choice? Can they follow a suggestion (or correctly and politely argue that it's unnecessary)?
I think a bigger problem is that they probably weren't lying or fabricating their resume. They probably weren't trying to trick you in some way, they probably just have actually programmed like this for the last 10 years! In a lot of work environments especially in the midwest, it isn't surprising to encounter people like this that don't know basic best practices.
The standards for what a company wants from their programmers can vary wildly depending on the location. This speaks to a broader problem of not having consistency across the industry and various locales.
As someone from the Midwest I can assure you that we don't have a larger percentage of crappy coders than the coasts and tech hubs do. We spend just as much time fixing the code from your terrible developers as you do from ours.
It's really a lack of training provided by companies and that most 4 year degrees don't actually prepare people to write good maintainable code so new developers have to luck into finding a mentor that can teach them these things.
I didn't mean it to be insulting or to imply that there are not talented people in the Midwest. What I mean is that if you look to places like Louisville, KY for just one example, or Dayton, OH and look at the type of pay that is being offered there, it is vastly below the coasts. Even adjusted for cost of living and other expenses, many cities across the midwest pay below what is a reasonable market value, and naturally they only get people who will accept those below-market rates.
When I said 'the midwest' in my original post I didn't mean places like Chicago, St.Louis, or Indianapolis that pay industry standards adjusted for cost-of-living. It isn't out of the realm of possibility though for someone to get 10+ years experience in a place like Lexington and have them still not know basic things EVEN after having been trained.
Again this is just my anecdotal experience, I don't mean to insult anyone from these places. This is about brain-drain and salaries, not about anything intrinsic about a place or their people.
We all know small businesses don't care about code quality. Ditto that best practices take a long time to trickle down from ivory towers to business, academia, to bootcamps.
Companies pushing the status quo tend to cluster. Poster probably shouldn't have picked on the midwest, it could be anywhere outside "cutting-edge" software centers.
This pushing of best practices you speak of is clustering especially in the geographic area where the norm is frequent job-hopping (before you fully reap what your practices sow), and where many also celebrate "move fast, and break things" during each employment stopover?
It's true that many devs aren't trained to write maintainable code, but there needs to be some responsibility on the part of the individual dev too. If there's no training, they need to be made aware of resources like Code Complete and Clean Code, so they can take some initiative and work through those books.
Some people have 10 years of experience, others have 1 year of experience 10 times.
This is a very hard thing to really make a part of your thinking process, in particular because nobody wants to come to terms with the fact that they might be in the latter group.
Every body writes something like this. Every body. Just because some devs don't write this in a single block of code, and scatter them across program logic doesn't make them any better.
I've had developers balk at other developer's code for writing a series of like 100 if-else's until I show them, they do it themselves only that they scatter it. You could say this thing about nearly every OO programmer out there.
Everybody writes code that mutates state in a way that produces the kind of code you mentioned as an example.
It's only that some developers think they can never write bad code, while thinking other's almost always do.
This is one reason why some shops hire only OCaml, or Haskell or Clojure devs because bad devs tend to use those languages less. So you buy into a great community by default.
ok Im stumped :) Would be interested to read some clarification on this.
the try always throws an exception, and if i<3 resumes the code. But on the last iteration through it breaks so will not execute code after the exception is thrown. The first 3 times it will execute code after the exception is thrown. Does continue return to the beginning of the for loop or resume execution where the exception was thrown?
If continue returns to the beginning of the for loop then I could see that the code is pointless. But if continue resumes execution inside of dosomething the line after the exception was thrown, then it could actually be useful.
Is your issue that the code does actually work, but conceptually it is horrendous to implement the branching in that way, or is it that the code doesnt work at all, or is that it actually does nothing?
When I see things like this I integrate them into our interview process.
Sorry, I think there's a couple of confusing things there. I mentioned a try block that always throws, but that was a different bit of code. In the example I posted of the for-loop the try/catch is unrelated to the previous, so we can assume it doesn't throw every time. The only WTF is why are we counting `i` and using `continue` and `break` to control a for loop.
The part in the except: does exactly what the for loop already does, it's completely superfluous. It also has nothing to do with handling the thrown exception.
Yeah it's pretty weird. Was trying to think of something that would need this exact behavior, assuming it's legit...maybe parsing some piece of text where you expect a fourth part only if the third was parsed successfully and you don't care about the outcome of the previous two
¯\_(ツ)_/¯
Scenarios like this scare the bejubus outta me, it's like a business logic landmine waiting for a refactor tractor to drive over
I do the same for a “senior” developer with 20 years more experience than me and a bafflingly appalling understanding of how to write basic code.
Bless you sir, you are keeping the code legible and clean for everyone else on your team and doing a great service for your company — even if they don’t appreciate it.
Is this person doing (valuable) work that would even require hiring a replacement? If not, fire them. If so, lobby to secure a backfill and then fire them.
The crux of the problem is that we've interviewed dozens of applicants and none of them have been able to pass the sniff test. We are setting the bar pretty low, but the average skill level of the people we've had access to is shocking. There just isn't a supply of good developers for companies that aren't FAANGs or unicorns. And before anyone says we aren't paying enough, or we are being unfair, the reason these interviewees are not passing is because they can't answer basic questions about the languages and frameworks they've been using for years. Things that you would know if you spent five minutes reading the first few pages of the official documentation. It's baffling. If I could even get a talented junior that I thought was capable of leveling up to a senior position in a few years I would be ecstatic.
Edit: for example, when interviewing for a react/redux position, one of the first questions I ask is "if you need to make a network request in an app using redux, where do you put it?" at least 3/4 interviewees answer "in the reducer". From page 3 of the "Basic Tutorial" in the redux documentation:
Things you should never do inside a reducer:
Mutate its arguments;
Perform side effects like API calls and routing transitions;
Call non-pure functions, e.g. Date.now() or Math.random().
>There just isn't a supply of good developers for companies that aren't FAANGs or unicorns.
...at the salary you are offering.
I see many people hiring adjusting for cost of living but not adjusting for the place being less desirable to live. Also, given the reputation of FAANG and similar, working for them is viewed as something of a future investment. That is a benefit your company likely cannot offer. So even if everything else was on par with a FAANG in terms of pay, benefits, and working conditions, ideally you would have to pay a slight bit more to make up for the difference. Granted, the market is never that rationally acting in any individual case.
Having seen numerous IT managers having trouble hiring, the most common shared issue is too little pay.
No, that's why I said "before anyone says we're not paying enough...". There are team members making $2xxK/year here, and we're not in a coastal city. The problem we have is that almost every person that comes in here bombs out in the first five minutes of the interview.
If that's a competitive salary for your area (you would know better than me, but that range in the middle of the country certainly seems competitive), then have a look at your recruiting sourcing funnel.
If you're only sourcing the same 75% of active job seekers who are bombing most of their other interviews [thus leaving them available to come in and bomb yours as well], start advertising your positions elsewhere or trying to recruit passive job seekers/encourage employees to scour their own networks.
If you don't change something, you're likely going to keep getting more of the same.
You're correct. The problem is that the positions we have open are for front end developers, and for a variety of reasons (mostly that the scale of complexity on the front end has increased so rapidly that the industry still has a lot of people in it that were writing mostly HTML and CSS a few years back) the supply of competent developers is far lower than the demand, and anyone worth their salt is already happily employed. Highly skilled front end developers can basically write their own check these days, and if you manage to find one in the brief amount of time they're in-between jobs it's pure luck.
Thanks. Unfortunately we can't afford to pay front end developers $300k/year. And what we are paying is more than competitive. It's not an issue of pay, it's an issue of availability and not having the resources to stalk and poach developers from other companies. We also have a generous internal referral bonus, and various recruiters sourcing candidates. Is there something else I'm missing?
Edit: I don't think you are understanding what I'm saying. We are offering higher than average pay for a job with average requirements, and we haven't been able to find average candidates. And I'm well aware of average, upper, and lower bounds to salaries here.
If you have budget for two poor front end devs, IMO you’re way better off paying 1.5x and getting one good one, if comp is the sticking point for landing a good front-end dev.
“We can’t afford that!” “But you can waste money on two crappy devs?!”
(If you only have budget for one, the cash flow doesn’t work, of course.)
Yes, probably. Why do you need a supply of new front end developers? Is your workplace exceedingly lethal?
Have you considered that paying $2XX/year is a signal? It attracts a certain type of candidate and you can't extrapolate the talent level of the entire market just based on your experience with those that would try to land a $2XX/year job. Where I live, trading firms can pay that much and more, with potential bonuses that would make your eyes pop out of your head. Yet many extremely talented people I know would never submit a resume: "No thanks I like having a life." Many people I know that used to work in that environment think of it as an insurance policy: "at least I know that if I ever need money I can go back."
> We are offering higher than average pay for a job with average requirements
A lot of people will pass you by and assume that something is really wrong at your company to need to offer so much money for an average job.
If there is both a pool of qualified candidates that don't think the pay is high enough and thus don't apply and a pool of candidates that think the pay is so high there must be some catch and don't apply, they could both be true. You have to decide which pool to target.
At least one of those two pools exists, because if not it means there are no qualified developers looking for a job, in which case perhaps the issue is with the definition of qualified.
Also, there is the possibility they are getting great applicants that are filtered out before it reaches the individual currently here. It may be possible HR or some automated system is filtering them in such a fashion that good candidates are lost. Maybe HR is demanding 7 years experience in a 3 year old technology.
What we can reasonably assume is that the hiring manager is not finding qualified talent and qualified talent does exist. The idea that qualified candidates are only applying to FAANG or unicorns seems far less likely than something about the position is the reason they aren't getting anyone qualified.
If you're paying $2xxK/year and allow remote just post your job link here. I, and all my social circles, are fairly successful devs in a coastal city and would jump on any offer that high as long as we didn't have to move to the midwest
Edit: I mean this as in I know several people including myself who apply for 2xxk/year and remote. Post or dm me and I'll see what people I could send your way
Could you not hire someone smart who doesn't already know react/redux and teach it to them? Or give them a couple of weeks to do some tutorials?
I knew almost zero about web dev (I had been a decent C++/Java/Scala backend programmer) when I was hired to work in a web dev role -- I spent the first two weeks doing RoR tutorials on the internet and was just as productive as anyone else a couple of months after that.
This is usually not a consideration and has not really been there for decades. What many forget is that people who are speciallists in the area thay are recruiting for are not necessarily the best candidates for the actually position.
Sometimes (many times) if you can get someone who is more of a generalist and is capable of diving in to gain the necessary skills is far more valuable. They are quite often the first to be discarded from the interview process.
Many of the psoitions I have participated in required handling new tasks that were not within the broad scope of what the position was expected to handle. Yet, having the attitude of "let us see what is needed", meant that I was bale to be functionally more effective in that position, including being the "go to" person for those who needed problems to be solved that their normal IT groups would not handle.
I have found too many "code guru's" to not be capable of thinking outside of the box whaen it is necessary. You don't have to be a "brilliant coder" but more of a person who can develop an understanding of the "problem space" of those having the problems that require solutions.
This is what we've resigned ourselves to, but it's very expensive. We're building crud apps, not doing cutting edge research, so I assumed we would be able to find someone that is already up to speed on the language and frameworks. You also have to factor in the cost of senior developer time spent coaching, the lost productivity, and the code quality suffering in the meantime.
You might be shooting yourself in the foot by having not picking an ecosystem that better supports your business case. No one I know who's good at React+Redux would pick a CRUD app as the domain they'd like to exercise those skills in.
The app in question is used by retail employees at hundreds of locations to manage inventory. It runs on a tablet, has to work offline, and has a very dynamic UI that has to keep thousands of complex relational entities in local state, and perform aggregate functions on them in real time. It's still a CRUD app, but it's very much the use case for a SPA.
How quickly do they answer that to get that so wrong? I freeze like crazy with interview questions. Even you just putting the question there, my brain froze and I couldn’t remember the word “action” for a couple of seconds.
They usually answer confidently right away. It's one thing if you're nervous in interviews, it's another if someone asks you a novice-level question about your area of expertise and you give an answer that is objectively wrong.
Judging by your posts in this thread, have you considered the problem might be that you (and many other companies) hire people based on their ability to answer theoretical questions (which also happens to be while under a lot of pressure), rather than their ability to code? I'd like to think there are plenty of candidates that have open source projects that showcase their coding skills, or that have made impressive side-projects.
It's not really a theoretical question though. It's a very concrete example of the most common operation you would need to perform in a front end developer role. And none of the applicants we've seen have side projects or code on github. If you know of a method to ascertain someone's ability to code in a professional environment in a one hour interview, I'm all ears!
Honestly you don't seem to be a well put together environment and it feels like you expect to hire experts in some random framework instead of good software engineers. I'm not surprised you're only getting code monkeys who have used your framework in passing or have done basic pixel dev applying that don't meet your bar.
I'm getting tired of replying to troll-ish posts, but:
> Honestly you don't seem to be a well put together environment
That is a pretty bold and unfriendly statement to make based on very little information.
> you expect to hire experts in some random framework
We're hiring front-end developers to work on a react and redux application, which is kind of a de-facto standard for single page applications these days. We have an app with a highly dynamic UI that requires immediate user feedback and must work offline. I don't think we're misapplying the technology.
> I'm not surprised you're only getting code monkeys who have used your framework in passing
Again, we're interviewing people with several years of professional experience using react and redux listed on their resumes. We're also not tailoring the interview process to only react and redux specific questions, but if someone claims to have been paid to write react apps for three years, I'm going to ask them some questions about writing react apps.
Even so, you shouldn't underestimate people's ability to have a meltdown during the pressure of an interview.. I won't comment on the React question, since I've never used it and don't know what a 'reducer' is.. but even the truly bizarre for-loop you posted is likely to cause confusion to some people during the interview. If I saw it while I felt the pressure of an interview, then I'd possibly feel confused and would need a couple of minutes to understand what was happening. This would no doubt reflect poorly on me, which is unfortunate because when I don't feel the pressure of the interview then I can instantly understand (and laugh at) the code.
I believe you'd have much better results if you ascertained the applicants technical ability before the interview, and instead spent the interview getting to know the person, to see if (s)he had a positive and friendly personality. If none of your applicants have side-projects or code on GitHub then perhaps you're searching the wrong places, and/or failing to properly promote the position (especially if some of your developers are earning 2xxK). I'm sure if you posted the position in HN's 'Who's Hiring' and included a 100+k salary, then you'd get flooded by talented developers that have open source code/impressive side-projects. And if somebody with an impressive resume doesn't have that, then you can just request a small take-home assignment that's related to the work you do. If you'd like to avoid getting flooded with resumes', then you could also just reach out to the developers you find interesting on GitHub.. considering your salary range then I imagine you'd have a very high interest rate.
This question is really tricky, though: depending on your frontend stack, the answer could be in an event handler prop (e.g. onClick). In mapDispatchToProps, in a thunk, in a saga, etc.
Correct. Any one of those answers would be acceptable :) (except mapDispatchToProps, that will fire the API request every time an action is dispatched anywhere in your app). Also, the context of the question is very specifically "within redux", and all I'm looking for is an understanding that redux is synchronous by default and asynchronous logic has to be performed in a middleware. If your answer is "put it in literally the only place you aren't allowed to put it", you lose some credibility. Also, so many people in this thread are getting hung up on the fact that we're trying to find people that understand a very common framework that we use, and that the candidates claim to have professional experience with. I wouldn't expect a junior to understand the details of redux, but we need senior people that don't require a year of hand-holding to be productive. If I am hiring a plumber to fix my toilet, and they say they have three years of experience fixing toilets, and they don't know how a toilet works... am I at fault for having unrealistic expectations?
Plumbers with 3 years of experience fixing toilets are usually still in their Apprenticeship programs. Around year 4 or 5 they'll graduate to Journeyman at which point they're authorized to work without the supervision of a more senior plumber. You hired a plumber to fix your toilet, but you have a waterless composting toilet and your plumber's knowledge is still at the level of "water and poo goes down the pipes".
Your hiring woes will get easier to handle once you start treating the Competent Sr React+Redux Developer role as rare. React is certainly popular tech, but solid knowledge of the ecosystem is still far from common. All the places I've done react at had ~10 frontend capable devs, but only 1 or 2 people who really knew what was going on.
I mean in the functions in the object returned by mapDispatchToProps, something like `onClick: v => fetchJson(‘/foo’).then(w => dispatch(fooAction(w))`
No, usually with sagas, your action creator stays synchronous and you yield your actions from a generator function that contains all the asynchronous bits.
It’s a beautiful system that brings a lot of the benefits of a Free-monad style effect system.
I work with lots of really, really good developers who struggle a bit with JS. Most of it is because they come from a totally different background, and the other part is Javascript in general just makes things difficult.
This is just false: there are a bunch of sharp edges in JavaScript, but once you know them, the language itself is one of the more pleasant languages to use: this can be a bit of a problem, since the languages itself doesn’t force a particular structure on your code like Java or Haskell do, but on the other hand, JavaScript is an amazing language for experimenting with new programming paradigms and application architectures.
What structure do you believe Haskell forces on your code?
My experience with Haskell is that it's the most expressive and flexible environment for exploring different ways of structuring code that I've ever used. I've seen everything from extremely-imperative code carefully managing memory allocations and munging bits, to very abstract high-level libraries that declare abstract computations that can be interpreted or executed in a variety of ways.
I'm very confused and surprised to see someone assert that Haskell forces a particular structure on your code.
Haskell doesn't force anything, but it does make some pretty strong suggestions. Haskell suggests immutable data, pure functions, and algebraic thinking. You can make your data mutable, do everything in the `IO` or `State` monad, etc., because Haskell doesn't force structure on your code -- but the suggestion is still there, whispering in the back of your mind.
(This is what I love about Haskell, BTW, so please don't read this as a criticism.)
This is what I was thinking: if I just want to sit down and design myself an object system, Haskell’s type system gets in the way of the necessary parts. (Unless you want to limit the interaction between your object system and haskell’s types in annoying ways). I’ve implemented CLOS-style mutimethods as a handful of functions basically translated from the Common Lisp in the art of the meta-object protocol. And the multimethods work transparently with any ja type.
This isn’t a criticism of Haskell: there are plenty of benefits to forcing declarative IO and limiting the language (roughly) to features that don’t break type-inference, but those advantages of Haskell make it much less enjoyable to experiment with programming paradigms in.
Forgive me because I have used a similar idom when dealing with io.
I do something like:
for i=0; i<3; i++:
try:
doSomeIoWork()
catch:
if i<3:
# maybe some network/disk issues retry?
sleep SOME_TIME * i;
continue;
else:
# tried too many times now quit
raise
what was wrong with this code? I don't deal with exceptions a lot so not familiar with the idioms here. Is it because the catch is not doing anything at i<3? And then at i==3 it just breaks from the for? So rendering the whole thing useless ?
We've had several sit-downs. The problem with a low-skilled senior developer is you'll never convince them that they are writing junior-level code. It would shatter their world view.
The catch block merely does exactly what the loop would do at the end end anyway; it's no different than:
catch:
pass
(That is, if the pseudocode isn't making a Rubyish distinction between exception handling and control flow [rescue vs. catch] it's just an extremely verbose way of silently swallowing exceptions, and if it is making a Rubyish distinction it's an extremely verbose way of doing nothing and the whole try/catch construct is unnecessary.)
For loop syntax is unnecessarily complex when you first encounter it. A simple if then control flow is much easier to understand and is how it’s done in assembly anyway. Unfortunately he used both. Maybe an opportunity to teach him about goto and labels...
When you first encounter it sure. After you've been writing code for more than a few months I would expect you to have mastered it. If you don't understand the fundamental behavior of a for loop after ten years of professional experience, something is very wrong.
I don't agree. I think there is no reason to get over pithy about code style. Question one is "does it work," but question two is "can others read and maintain it." By adding in extra logic and lines that do absolutely nothing, the next reader of the code has to slow down and think, "ok, person A did this for a reason, what is their reason..." and disassemble the bad code only to realize that they just waited their time doing so. Readable and maintainable need to be considered for any code that will be maintained or read by others.
The introduction of such tools serve not to prevent bad code to land, but to force people and code into a process.
If somebody code is routinely denied to land on code base because of some of the issues catch by automatic tools, it is reasonable to assume that the quality of the code and the thought given to the code will improve.
Is this person routinely writing code so bad that passes all the automatic tests?
By the way, such code ends up having a quite high cyclomatic complexity.
> Is this person routinely writing code so bad that passes all the automatic tests?
Yes, a linter or a test cannot know if you are writing silly, unnecessary code, or re-implementing functionality that already exists elsewhere, or making multiple changes downstream when you could make it once upstream. Only a human can see those things, which is why we have code reviews. And for posterity, yes we are using CI, pre-commit linters, unit tests, integration tests, end-to-end tests, and have formal branching, merging, review process, and coding standards. There are no cowboys here, and nothing that you are suggesting will prevent someone from writing low quality code that requires an excessive amount of time to protect the codebase from.
If I ever hire an engineer for my business -- I hope it'd be someone I've already worked with and know -- but if it had to be someone random, I'd pursue this method:
1) Solicit Resumes
2) Pick N candidates
3) Phone screen to make sure they aren't complete assholes
4) Stack rank & pop top candidate
5) Pay them to develop a feature on contract
6) If satisfied, repeat step 5 until ready to hire; else repeat step 4.
I think it'd be important from the start to let candidates know they might need to do a week or more of work (for pay) before getting a final hire. Some people would be put off by this, sure.
But I suspect the people put-off would be people desperately seeking full-time employment. I think it might attract a lot of talented people that don't want to go through a 4-week long ceremony of an interview process that takes up days of time with no pay, though.
This seems good on the surface, but I can't imagine top tier talent at a FANG or equivalent ever leaving their job to do this.
I feel this filter will only leave behind those who are currently contractors that are interested in maybe going full time, and the unemployed/employed who are desperate enough to go along with such a scheme. Foregoing the typical crappy interview process is one thing, but working, even for money, without any certainty as the end outcome would certainly turn me off.
To be honest, this setup seems like something that would be hatched by a scummy company to constantly string you along with a carrot that will never come. I am not implying that you are that type, but there are enough horror stories out there that I would avoid ever getting myself put into that position.
Must be looking to get out for some reason. Doing a small side-project shouldn't be a big impediment.
> I would avoid ever getting myself put into that position.
Extreme risk-averse behavior can cause problems on the other side. Easy enough to give a small-project a try and cut out early when red-flags are actually encountered.
They may be looking to get out, but you are essentially asking them to leave their current job so they can spend a month (or whatever) on a paid interview at your company. You are not only giving up your previous job, you are essentially giving up your ability to interview elsewhere as well. If this doesn't work out, either because you don't like me, or I don't like you, your business, the tech stack, whatever, I am now in a really shitty position, I am giving up what little leverage an employee has in the labor market- the ability to look for a job and say no without worrying about paying his bills.
If you are doing well at a company, I am not sure how you make this seem like a good value proposition, especially in this tech job market. Another way to put it, with a personal spin- I don't work at an actual FANG, but a company that arguably pays better, and their would have to be a very large upside at the other end of this process to make it worth going through. Things would have to be very bad for me to give me up my deep in the six figure job to spend a month interviewing at some place unless it had some legendary track record.
This is all theoretical though and not really worth arguing over. I'd like to see OP try this out and report back the results, good or bad. I could be wrong, but I would be willing to wager that you get a lot more resumes just out of a bootcamp as opposed to people with proven track records currently at prestigious companies.
Not sure what the other guy implied, but that’s not what I had in mind. Rather a side project type of thing. Little risk for anyone. No one has to quit.
Step 5 sounds awesome and I totally understand the desire to do it, but it will screen out a lot of top-tier candidates. A really good candidate is probably already employed. They probably don’t want to quit without another job lined up and they probably don’t want to work double hours for a week. If they’re job hunting then they likely have other offers you have to compete with, and are they going to chose the “maybe we’ll hire you if we like your work” over “sign here and you start in two weeks”?
Desperate people won’t be put off by this. If they’re desperate then they’ll have time to do your contract and the money will be attractive. They can keep job hunting in the meantime so there’s no downside.
That’s not to say this can’t produce someone good. I got my first full time programming job like this and I like to think I’m not completely useless.
And alas, I don’t have any answers for how to do it better....
There's no perfect way to hire, don't let it be the enemy of better.
For example, I like this one and I'm gainfully employed. Much rather work on a real side project than useless, unpaid coding challenges any day.
For me, the first has a 98% chance of success, while the second has a 2% chance.
Just to provide an alternate perspective here, this seems to tilt the incentives towards people that are not working already. I wonder how many folks have time to take a week or more off from their current job to participate in this exercise. I understand this is paid but its still a huge risk that candidates would be taking.
I've had interviews start very positively with such an approach of questions, but then "just to be thorough" I have them code something simple on the whiteboard and they completely fail.
Some of them couldn't even get nested for loops to work correctly.
If you're hiring someone to write code for you, you need to have them write code in the interview. There's no substitute for it.
Obviously, you can improve the experience - let them bring their own laptop and code on it, no problem.
Years ago, I definitely was one of those people who completely bombed a code-on-a-whiteboard problem. It was exactly that - a nested for loop problem. It was one of the first real interviews as I was in a new geographic area and previously I had moved into jobs by being recruited by my professional network. This was before I started specifically studying the patterns used for engineering interviews.
Thing is, I still had a good 3-4 years of professional experience with stellar reviews, some hackathon wins, on top of having coded throughout my entire life prior. But when I was up on that whiteboard, I pretty much froze for that entire segment. Whiteboards are just a different skill and if you don't regularly code on them at work, you better buy one and practice at home because using an editor does _not_ translate to coding on a whiteboard.
I'm glad you mention this: "Obviously, you can improve the experience - let them bring their own laptop and code on it, no problem", because I did better in interviews that followed which allowed for coding in my usual working environment. Heck, I even passed a technical screen before this company brought me on-site where I live-coded some elevator problem they asked me to implement. After bombing the whiteboard, I wouldn't blame them if they thought I had paid someone else to solve the tech screen.
This disproportionally filters for those who can think/code and perfom while under pressure/observation... i.e. the "porn stars" of software development, if you will.
Similar to how not everyone is able to perform well while having sex-on-demand in front of a camera -- these same people are more than likely able to have sex when not under pressure and observation, and probably are quite adequate while doing it.
My personal experience in software development spans decades, I'm quite competent and I have definitely "choked" at the whiteboard, just like you're describing -- for the same reasons I don't do well speaking in public, I'm not comfortable doing it, and I'm just not cut-out for it... I don't think it's a good measure of actual problem solving and/or software development competency.
By the same logic you might say that we shouldn't interview at all because it selects for people that are good at being interviewed. Obviously this is true but I think it would be pretty crazy to not interview people.
Frankly I think most people suck at being interviewed initially, but you can definitely improve with practice. The same as public speaking and interviewing candidates.
Some people are even good on a take-home test, good at chatting in an interview, good at coding at their desk, but completely unable to code in an interview. And you can fail an interview at some places if you are one of them.
To be fair, the reddit post was about a DevOps applicant being asked a single dynamic programming questions for the interview.
It's as if you asked an auto repair mechanic about designing involute tooth profiles on fishbone pattern gears in a transmission. Sure, if they could do that, then yes they'll be a great technician, but it has nothing to do with brake jobs and oil changes.
EDIT: Per the responses in the reddit thread, the question was: "So basically it was a variation of the exact change problem (I realized afterwards). You had to validate a string of size n. 1 chunk of 8 characters could have 1 format, 1 chunk of 16 characters another, and 1 chunk of 24 characters another. You essentially had to determine if the string was valid. So in other words, find a combination of these sizes in which the string could be validated. I had 30 minutes."
We don’t know what expectations were set in the interview for the solution space, but anyone with (Unix) ops experience should have the thought “grep can do that” pop into their head. And people with a formal CS education should be able to come up with the dynamic programming solution.
So, whichever side of dev-ops you lean to, there’s a reasonable answer that should be within your field of expertise. And given Palantir’s line of business (the company involved), I wouldn’t be surprised if dissecting a byte string with limited information about it’s structure comes up from time to time in their actual work.
Nested loops are difficult to code up on a whiteboard, much easier to do on a computer with a code editor. It’s just difficult to manage the real estate, while fixing edit errors on a whiteboard is super messy (I’m left handed and constantly smudge also), easily throwing me out of my code think.
Who the heck writes code on a whiteboard for production anyways?
This doesn't Test if they can develop software; this tests if they can diffuse a bomb.
Also there's no point in bringing someone in for an interview before they've shown any coding experience. It's a colossal waste of time for everyone, including the candidate
We give an invite for a phone screen. It's just a chat so both candidate and interviewer can make sure that the job on offer and experience of the candidate match up. It's also a nice time for candidates to get a decent idea of what our company does so they can bow out if it's not really what they're looking for, which is good for everyone.
Then we send a tech test (take home). It's a simple challenge, with a lot of freedom. We give a guide time of ~90 minutes, but also let the candidates know that they can do more or less if they want. We ask the candidates to submit a written response as well, so they can explain if they're happy/unhappy with the test and their submission. Didn't have time to refactor as you wanted? Didn't write as many tests as you would usually do? That's fine, just let us know generally what you'd have done so we can take it into account when looking at it.
Then it's onto the face-to-face interviews etc.
The tech test segment is really enlightening. A lot of the time we can figure out a lot about the candidates ability to create software systems and communication skills just from the test. Most of the time, we spend a non-inconsequential amount of time looking at the test submissions. If a candidate was good enough to write it, we should take the time to really see what they did and review it. Sometimes we get really poor tests back, even from candidates we were enthusiastic about. Other times we get pleasantly surprised, with candidates who phone screened poorly smashing the test and giving a wonderful submission. I think it's immensely important to see what a candidate does with actually writing code. I also think it's important to make sure we see that prior to inviting them in, as they need to take time off of work, get to our offices and prepare for it. And even when we really want to progress with a candidate, the tech test gives us something to talk about to get to know the candidate better, and hopefully have a conversation that allows them to ask us questions as well.
It's a great point we hear often, so I incorporated it into our last round of hiring. Some still couldn't do it with an IDE, their language of choice and access to Google.
I must have written a couple dozen sorts, but only in high-school or when preparing for interviews. It's the one thing that is very well supported in most programming languages. C++ has stable/unstable sorts, partial sorts, whatever you want.
Accepting things like bubble sort also makes the exercise doubly pointless.
Since people still fail it, even with all might of Google at their command, even with bubblesort allowed, it's anything but pointless.
And if you are unable to sort an array it's a strong signal that command in all other areas of CS is lacking. Does C++ has a standard library for RSTP? Do you think a person who can't sort an array after 5 years of school would be able to fix a bug in a switch' spanning tree implementation?
You're looking at this the wrong way IMO. One should not give exercises and try to eliminate candidates, instead they should prepare a set of exercises so that at the end, if the candidate solves them correctly they can confidently say "this candidate knows X, Y and Z", so they should do a good job in our project.
In your case, at the end you've only figured out that your candidate knows how to sort an array, which even high-school students can do. You can confidently say that your candidate has a level somewhere between student and Donald Knuth, a pretty useless piece of information.
Is there any connection between sorting arrays and spanning trees? If I'm not mistaken, the latter involves graphs.
Problem is however that a good crop of candidates doesn't know how to sort an array, am not sure why am finding myself explaining it for the 2nd time. So all I can say is they are below high school student in your neck of the woods, even if they have Masters in relevant discipline. And if you can't sort an array with all help of Wikipedia and Stackoverflow, you would not be able to walk a graph, let alone do anything useful with it.
So I'm not quite sure what is your beef with it exactly. Do you think I should have used a more complicated task? Like, Dijkstras' on a graph? Are you sure that you wouldn't find that objectionable too, given that sorting an array is apparently asking too much?
This was the only coding test but we also either talked through opensource code candidates brought in or discussed some parts of one our real project here.
Let’s say someone didn’t know any sort algorithms since he has been using one provided by the language. Would you have been willing to explain on a high level how the algorithm works? I wouldn’t be opposed to any algorithmic type question if you explain the requirements on a high level and let them do it within an IDE.
But, you are hiring for a software developer, who should be able to translate business problems that you face everyday into a working system. Don’t you have any real world scenarios that you could use to test on?
I would be willing to explain sorting to someone without formal CS training, but not to one with Masters degree in the discipline. They had 5 years to figure it out.
Seeing that the Bubble Sort was something I did in 6th grade in the 80s as part of an AppleSoft BASIC programming class - congratulations, the interviewer has now proven he can hire a programmer with the skillset of a middle schooler....
It depends on the position. If I’m looking for a developer for yet another software as a service CRUD app, I may not need anyone that understands “CS fundamentals”. Heck it’s been 20+ years since I graduated from college with my CS degree and over a decade since I’ve had to do any low level programming.
Does C++ has a standard library for RSTP?
Maybe not, but I know that there are very few new problems under the sun that most people face and someone has solved the problem before. I also know how to use Google.
Yes, I am saying exactly that. This is a userspace implementation not making use of switch hardware fabric. You can probably hack hardware forwarding into it (haven't looked into this, our implementation predates it by 2 years). Or you can read an RSTP spec and implement it. None of it is secret, like none of CS in general is.
Both approaches however would be beyond the skills of a programmer who can't sort an array. And really it's not a high bar, I don't understand this obsession with anti-intellecualism. Everyone likes plumbing analogies, well I wouldn't trust a plumber who doesn't know that water generally tends to flow downwards - it's this level of CS knowledge we are talking about here.
I’m saying that you don’t need to memorize an algorithm. For one job I had to maintain a proprietary compiler/VM/IDE that never saw the light of day outside of the company. I needed to solve an arbitrarily complex algebraic expression in a string. I had no idea how to do it. I found both algorithms I needed with a quick google search and implemented them in C - one was converting the expression to RPN and the other was implementing the “Shunting Yard Algorithm”. Would I have also been dinged because I didn’t know those two algorithms off the top of my head?
I partially agree, but a bubble sort is slightly harder than FizzBuzz (?) and even if you give them the algorithm, it might be a little more fair since knowing how to do a nested for loop is less esoteric than the mod operator.
Now if someone did a BogoSort, I would hire them just for their ingenuity and sense of humor.
Would using a built in library be acceptable? Actually in C knowing how to use qsort() does require having a little knowledge of how function pointers work. If they did, I would ask them what could be some drawbacks of using the built in qsort function (function call overhead).
On the other hand, would you accept a bubble sort as a solution? Or the most degenerate case- a bogosort ;) ?
Bubble sort, bogosort, sleepsort, anything that solves the assignment (gets an array sorted) of course. And no, a library call would not count towards implementation, but it's not too much to ask from candidates with masters degrees.
I went through a similar interview once - bring your own laptop, and code-up a bowling scorer which spanned two sets of interviewees. Was not the worst interview experience I've had.
I think it's a holdover from hiring more junior candidates: after a few years interviewing and sitting on HCs at Google, I hired for a tiny startup that had tons of trouble sourcing candidates. My interviews at the latter place eventually devolved into basic IQ tests. Simple coding problems are pretty ideal for this, since they also lightly test necessary domain skills.
The problem is that this doesn't scale with talent and/or experience: I certainly wouldn't be better at coding interviews than I was out of college. People who aren't great at hiring take an approach that's valid at the low level and extrapolate it well beyond the bounds where it makes sense.
On the other hand, this only works if the pedigree is recognizable and reliable. I can be pretty confident that I don't need to test basic coding literacy of someone who was an engineer at Google for four years, but I've interviewed people with ten years experience and great references who were clearly dumb as a brick during the interview. In one case, we hired the guy over my objection and spent the next year fighting the fires he'd set whenever he touched anything (the founder eventually found some relatively unimportant, isolated niche for him to keep busy in).
It's a really hard problem, and most complaints I've come across are generalizing from their skewed view of what the candidate pool is like (usually assuming that most candidates are roughly as talented as they are, when the reality is that many candidate pools consist of some of the dumbest people you're likely to ever encounter)
> My interviews at the latter place eventually devolved into basic IQ tests... The problem is that this doesn't scale with talent and/or experience.
Industrial psychology has consistently found that IQ is a better predictor of job performance than experience.
The reason being is that intelligence is defined as the ability to acquire knowledge and skills. Intelligent people literally learn faster than others. The skill acquired by a low IQ in five years is often surpassed by a high IQ person with a few months of experience.
You cited a publication called "Intelligence Is the Best Predictor of Job Performance", but funnily enough there's also one called "Intelligence Is Not the Best Predictor of Job Performance", which one year later claims that the former one is flawed.
There's also this: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4557354/, which says "Supporters of IQ testing have been quick to point to correlations between IQ and job performance as evidence of test validity. A closer look at the data and results, however, suggests a rather murkier picture."
> The skill acquired by a low IQ in five years is often surpassed by a high IQ person with a few months of experience.
This looks wrong to me, most things take more than a few months to learn.
I'm probably a stronger believer in this than most, which is why it was easy for me to follow the evidence for our narrow case where it led. But you only need a weaker form of this to explain the experience I had recruiting at that company.
All it takes to be a good engineer at the lower levels is a sufficiently high IQ/creative problem-solving ability. Put someone like that under the mentorship of someone who's learned good engineering habits and (barring severe behavioral problems), you'll have a pretty great junior engineer in a few months.
Otoh,Experience and other habits and knowledge do become more relevant at higher levels, only because the responsibilities of the job shift and grow, such that you can do a lot more damage in the "learn by exploration" period of becoming good at something by just being really intelligent and learning on the job.
I currently work with a bunch of physics and math and CS PhDs, people who are likely as intelligent as I am or more, but I'm a lot more equipped for our job because I'm applying that intelligence in the context of understanding how engineering systems work, at both a technical and personal level. There are tons of engineering habits that are second nature to me that never occur to them; if they shadowed me closely and I dedicated time to explaining the reason I do every little thing I do, then they'd pick up those habits in the same way junior engineers do.
That is to say, as one would expect, the relative supremacy of IQ over other skills like conscientiousness (or yes, experience) diminishes at the higher levels. Everyone I work with would be considered "really smart" by any reasonable measure, but it's not purely the smartest who are the most effective anymore, the way it was when we were entry level.
> Industrial psychology has consistently found that IQ is a better predictor of job performance than experience.
If hiring managers are aware of this, then it might be a good idea to apply to Mensa. If you're able to get in, maybe putting "Mensa" down on your resume isn't that bad of a thing. (I've always felt it would make one come off as a douche, but maybe that's not so true.)
My opinion of Mensa was formed by someone who told me about how, when she was a hiring manager, she once had an applicant who made a point of belonging to Mensa and didn't seem to have anything else she was proud of or had accomplished.
That doesn't mean I am necessarily on board with the fashionable view that IQ is meaningless, but I think to the extent there are people whose IQ is disconnected from their abilities/achievements, it's going to be the ones who are preoccupied with it. So I would think belonging to a high-IQ society would send the wrong message.
Hiring managers should also be aware that in the United States, Supreme Court precedent makes the use of general intelligence tests for hiring legally fraught because they can produce racially-biased outcomes.
Definitely comes off as douchy. I wouldn't discount someone for it but it would definitely make me wary of them (probably a know-it-all who doesn't listen to other people and always knows best).
"the reality is that many candidate pools consist of some of the dumbest people you're likely to ever encounter"
I think people genuinely experience this but are unwilling to consider that their pool is not the random sample they expected. Something subtle is filtering out (nearly) all the non-dumb people.
Have you ever seen a personal ad with a long list of characteristics that are unwanted?
Everything you say sounds extremely reasonable... but still, there are plenty of stories (mostly online, so they might not actually be true) of people hiring a person that aces (mostly non-technical) interviews only to show absolutely 0 actual technical skills on the job.
In my experience (also interviewing hundreds of people) this is totally an urban (internet) legend. Not once has someone been able to talk intelligently and at length about a project they built and the underlying technology choices and then been unable to deliver. Once you gain experience building things yourself, it's pretty easy to suss out whether someone is any good just by talking to them. No coding tests needed.
Agreed. The best way to interview is to simply have a joint discussion about their past projects (from resume). There is no value in whiteboard puzzles. It is easy to gauge who is competent and who didn't do or didn't understand the work they claim on the resume.
In over 25 years of interviewing candidates, this approach has never failed me. I've never asked for whiteboard coding. Whiteboarding architectural concepts, sure.
The only caveat is the interviewer needs to have substantial experience in the same field to spot the wheat from the chaff in a casual conversation. If the interviewer is very junior, I can see how this might not work. Solution: don't have your most junior people run the interviews.
I've interviewed hundreds of candidates over more than 20 years. I've lost count of the number of times that a candidate has talked-the-talk so well that I'm thinking "This guy/girl is amazing. They're a shoe-in. Better do a little coding just to make sure"... and then they literally couldn't even get close to solving something like fizz-buzz.
I agree asking highly specialized, deep CS-algorithm type problems is usually a terrible indicator. I typically start with a very rudimentary question that could go off in various directions and to various depths as time/progress allows. I let candidates choose the programming language, and to use either whiteboard, pen & paper, or laptop with choice of IDE.
I've seen candidates who claim to have years of "senior" experience struggle to get the basic syntax of their "language of choice" right - to the point where a competent hobbyist who spent a weekend doing a couple tutorials would wipe the floor with them.
There's just something about coding that makes it impossible to truly evaluate a candidate (for a developer role) unless you see them actually code.
I've not had this experience. How are you screening your pipeline? I have a hard time believing that someone who can tell you why they designed a particular REST api the way they did, describe the schema and talk about what performance issues they encountered and how they dealt with them and can intelligently discuss the language they're using but can't code fizzbuzz.
My guess is that either your interview is too much fluff, and not enough technical detail, or you're intimidating the candidates and they get lost.
It makes zero sense that someone can have a deep technical discussion but they're not able to write fizz buzz (which by the way is a crap exercise).
Sure, if one is asking general questions about agile, TDD, SOLID and other fluff, anyone can answer those.
At one of my previous gigs, we managed to hire someone who couldn't write more than basic control structures. I have absolutely no clue how it happened as I wasn't involved with the interviewing process at the time. The people who were involved blamed the recruiting firm for not doing adequate technical screening, but that doesn't explain how this person passed the on-site interview they performed... At any rate, we let them go after 3 days.
I have no way to prove to veracity of my story, so feel free to add it to the "urban myth" pile. But it was extremely bizarre to experience.
Acing an algorithm type interview has little to do with answering questions like can you translate business problems into working systems? Do you know whether you should be writing this code at all or use a third party solution? Do you know how not to gold plate a solution and ship software? Can you solve a performance issue with a process that may not involve code but involve another part of the stack?
We have ~4 hours to judge a candidate. Like it or not, we are going to have to choose :)
The complex questions, if I can get answers are awesome. They are also the ones that are hard to objectively describe. Based on the number of blogs read, candidate charisma, my own work pressures - there is a big chance a senior enough person can wave their hands through those. Hence the tilt towards the surety of coding.
Also TBF, in my 1000s of interviews, we have always had a couple of rounds (Bar raisers / System Design / Architecture ... whatever) which try to get the fuzzy input. We get a sense, but that's not in depth as you might imagine. OTOH we have a clear cut answer in the problem solving world that gets us a bit more closer to the hire/nohire decision.
I’m on the opposite end of the spectrum. I work for small companies mostly and I’ve conducted interviews for small companies. We have a limited budget for a limited number of positions. One developer can be working on a project that can individually move the needle. We need to hire “engineers” in the truest sense of the word not “coders”.
For instance, currently my title is “senior software engineer”. But I am the “Directly Responsible Individual” for a project that one of our clients is paying for. It will be added to our core product.
This means that I’m expected to interact with the client and BA to come up with the SOW, work with the product manager/UX person to design the interface, do some of the web work, write the backend using one of our approved languages (C#, Javascript, Python) , design the database schema, do all of the CI/CD setup and to know the AWS fiddly bits and know which services are appropriate for the asynchronous processing requirements.
I’m no special snowflake, this is life in small companies since my first job 20 years ago.
I don’t interview candidates for algorithms. I interview them to know whether they can hit the ground running on as much of our (industry standard) technology stack as possible, will they make good architectural decisions, can they ship a feature across the finish line, how fast they pick up new frameworks and will they embarrass us with customers and senior management.
LeetCode ability does us no good. I want to know whether you can translate real world business requirements into a working system - including knowing which questions to ask when customers/product managers come to you with “XY Problems”.
I don't know your scale for small, but my current org is tiny compared to AWS (my previous).
So yes - we have to do all those things. I am directly responsible for ALL of data analytics infrastructure in snapchat. Pretty much most senior engineers have a similar scope.
We might be talking over one another though: I am not saying any of that is useless: I am saying they're hard to measure. We still do it though. We also do the coding round as something that gives a more definitive datapoint.
My definition of small is less than 50 people in total including salespeople, managers, implementations managers, and a few QA people.
It’s really not that hard to ask a few technical questions based on what they purport to know, the soft skill behavioral questions, some design questions, and ask them about previous projects. The hands on part is the part where they fix the code to make the unit tests pass.
I wasn’t asked a single technical coding question for my current job. I was given real world questions about issues they are having with process and how I would go about (re)architecting some processes.
Now that I think about, my job before that when I was interviewing for the dev lead position I was only asked about process, system design, and a lot of high level theory - both jobs were hands on coding positions. Both of the managers were highly technical with current programming experience.
But, I’m not completely disagreeing with you, if you are solving hard problems(tm) at scale you need people who understand algorithms and you may not be able to use off the shelf solutions - I would say Snapchat qualifies for “at scale”.
But, for you typical yet another software as a service CRUD app or your typical dark matter corporate developer - and those positions are a lot more numerous than HN seems to realize - the complications aren’t in the complexity of the code, it’s in translating the business needs to code and shipping software that the company needs.
As strange as it sounds, the “coding” side of software engineering has gotten a lot simpler 20 years into my career than it was when I started (C, C++, and a little x86 assembly). The “engineering” side has gotten a lot more complex and requires me to know a lot more technology.
I don’t think it’s because I am better “coder” now either. When I got my first job, I had already been a hobbyist for a decade doing assembly.
No, I was defending the use of FizzBuzz as a “Bozo filter”. But FizzBuzz is so commonplace that it is meaningless.
I have been able to weed out candidates by asking them to do an isOdd() function as part of a pair programming exercise. It was still the Bozo filter before I gave them harder problems. Similar in complexity. But even that, I’ve done with an IDE with a skeletal class and failing corresponding unit tests.
In my experience, I've seen close to 50% of the people that can talk their way through an interview and put up a good game wind up being absolutely useless at actually writing code once you hire them and set them to work. There is a huge disconnect between being able to understand the big-picture hand-wavey ideas of how software fits together and being able to turn that into nuts-and-bolts working code.
Something as simple as having somebody write a little web application that takes a request, calls an external service, parses the response, and returns a transformed response, filters out the bozos wonderfully.
An alternative hypothesis I have is that some organizations inadvertently skew their pool somehow, and they really do get a lot of these, but other places don't.
The problem with non-coding interview's is its very dependent on the interviewer to determine their capacity, and _many_ people are bad at that. I think if you are good at it, its pretty easy to tell (like < 5 minutes) if someone can do the job. But I've seen very poor candidates get past this screen many times, even when screened by people I otherwise trust. So I know its not a common skill.
The problem is, your resume doesn't speak for itself, or, more accurately, resumes in general don't speak for themselves. It's not because people lie on their resumes, but because there's a pattern in this industry of people failing laterally and upwards.
It takes much longer than people assume to get a dysfunctional team member off the payroll. In most shops, new hires spend months in a (reasonable!) reduced-expectation state of on-boarding and ramping up, during which their performance can't readily be measured. It then takes several engineering cycles to figure out that they're not working out on a particular project, after which the normal response is often to try moving them to another project. You can be more than 6 months in with an employee before there's a confident assessment in engineering management that someone isn't working out. In many shops, at that point the next step is to begin the process of "managing them out" (often with multiple stages of PIPs), which includes a fig-leaf premise that performance can be fixed and the team member retained.
And that's if things are carefully managed. Lots of teams, even good teams, aren't carefully managed, and it can take more than a year to detect and propagate the signal that someone isn't working out.
The net result is that people can accumulate impressive-looking resumes without ever effectively delivering for any team, and because resumes get more impressive over time, the roles people get offered improve as well, compounding the effect.
I'm an evangelist for evidence-based hiring techniques and a fierce critic of standard interviewing practices, which I agree are dehumanizing and unreliable. But we started our hiring practice[1] not because we wanted to include more people, though that was a happy result of it, but rather because we'd been burned repeatedly by good resumes and strong interviewing skills.
At this point in my career, for a technical/IC role (really, any role), I'd be alarmed if I wasn't given practical tests to verify I can actually back up my talk. Their absence would make me suspicious (though perhaps not dispositively so) about the technical ability of the team I was considering joining.
For a technical/IC role (really, any role), I'd be alarmed if I wasn't given practical tests to verify I can actually back up my talk
Agreed, and let me offer an anecdotal, real-world example. Before I was in this racket, I used to be a land surveyor. Because it's both technical and licensed (and btw, I'm not going to make an argument for licensure, just mentioning it as a qualifier), you had to take a written test and pass a rigorous interview before you were accepted as an apprentice land surveyor. The test involved trigonometry, plane geometry, algebra, property law, and basic statistics. The sit down involved attempts to surface red flags all the way down to testing if the candidate was afraid of dogs.
And after passing that, then you were hired on as an apprentice which meant that all your work had to be reviewed and signed off on by a licensed PLS while you learned on the job and could eventually take the PLS exam yourself.
Too often in software engineering, a candidate can jump through the hoops and get hired but even if their role is junior, they're essentially on their own except for code reviews. There's very little mentoring, because senior people are neck deep in work and nobody's got time for that. The company has to be committed to mentoring and carve out space and money for it in order for it to happen.
This lays out the actual problem with programming "education" these days. Most people are not doing "computer science", they're doing a subject called "software engineering" when there's no actual engineering discipline involved. Programmers have "title inflation".
We would be much better off if the craft of programming was treated like other trades, requiring training and formal apprenticeships, with the necessary mentoring and signoff.
Programming is to CS as plumbing is to fluid dynamics. Knowing how to write a bubble sort is an irrelevant skill.
Knowing how to use the standard library in the environment is much more relevant.
How do you assess juniors? I feel like the desired quality there is how coachable they are and their capacity to develop said technical chops, but that feels even murkier to suss out, especially in the time-limited recruitment/interview process.
> most people are not that passionate about where they work until they actually work there
I do like coding a lot, but these home tests can never excite me. I have a family. I can't do it after office. In office I want to work honestly on my job. So frustrating!!
> The code test is one of the more annoying facets of interviews. On the hiring side, it's an efficient way to screen and vet candidates for basic skills. It's relatively low cost to the employer to have a recruiter issue the test.
I agree with you in principle for a programming job, but from the post, this was a dev ops job, and the candidate was asked a dynamic programming question. There was no link to the JD, but I suspect it had little by the way requiring a CS degree.
Resumes lie. You can’t imagine the number of times I’ve interviewed someone with $years of experience who were actually no more than “expert beginners”.
Older developers come in two forms. Those who aggressive learn, have matured beyond “just developers” and those who are set in their ways. I’m 45.
The coding is to get a sense of how they write code, how rusty they are, how many good habits they've picked up relative to their experience, how do they think about validation/testing of it, how the go about building it (ad-hoc vs top down), their problem solving skills when faced with an unknown problem and so on.
There will always be unknown unknowns. We are just trying to cover the known variations and if he has any unknown knowns[1].
When you need a doctor or lawyer or pilot, because they are certified, you at least know that they all have at least a bare minimum amount of competence. You may not know if they are elite ninja skilled, but you know they know the basics. Literally anyone can say they know how to program, and there is nothing on the resume that can confirm or deny this. I’ve interviewed Senior Software Engineers with years of experience who couldn’t complete a FizzBuzz-like task. Software engineering unfortunately does not have its own “bar exam” or board certification, so it has to be “Monkey Dance” over and over no matter how much experience you have. It’s frustrating both as an interviewer and as an interviewee, but until there exists a verifiable signal of competence for one’s resume, basic skills need to be tested over and over and over.
Sure, if they actually have any. You’d need a way to verify the code was actually written by them. You’d also need to be careful you’re not overly biasing towards “people who [have time to / are allowed to] publish on GitHub”. Plenty of good developers don’t/can’t use GitHub.
You don't really need to test basic skills over and over again.. you could just ask them what part of the code they wrote, if they don't have any open source code then ask them to send you a take-home code assignment they completed for another job interview, or have them complete one you make for them.
You're right that there are plenty of developers that have no experience with GitHub, but I'm sure there are far more who would struggle with simple theoretical questions and programming quizzes during the interview, because it's difficult to think clearly when you're feeling a lot of pressure.
I'm sorry but this is such BS. There are what, a dozen other engineering industries out there? EE, Civil, Mechanical, Aeronautical etc. and in not a single one of them do they do this monkey dance. It would be absurd to ask a Aeronautical Engineer to do something like derive the Navier-Stokes Equation on a whiteboard and its equally absurd to play the same game with SWE's, you all have just been bamboozled and strong-armed by employers into accepting it as a valid practice. I doubt you would be willing to say that those other industries are failures or hire low quality engineers because they don't torture their employees with essentially pointless trivia games.
In fact one might say that the Aeronautics industry is in far better shape than the Software industry, a failure in one of their systems results in people dying but I can go to any major tech companies websites and experience a bug on any given day.
> It would be absurd to ask a Aeronautical Engineer to do something like derive the Navier-Stokes Equation on a whiteboard
A thorough interview in physical engineering disciplines will often expect you to demonstrate/validate something about your technical skill in-person, unless you are being interviewed because people already have validated knowledge about your capabilities (e.g., hiring a known individual from a competitor that is well-respected among peers).
To your point, a physical engineering interview might be more about "spot checks" than "assume the person is lying so ask them to complete a bevy of undergraduate final exam questions". In the software world, maybe that would be akin to starting with FizzBuzz and finding other ways to get solid proof the person can implement, not just talk. To me, if you have open source work that can be read by an interviewer, I would find it disrespectful for that interviewer to act as though the work doesn't exist and they should verify from scratch you can actually write working code.
It's comparatively a lot harder to just have a conversation with a software person and be certain they have the technical competence they claim vs. other technical realms. I tend to think it is the nature of the subject, as well as the salaries drawing in more charlatans than in physical engineering realms.
> In fact one might say that the Aeronautics industry is in far better shape than the Software industry, a failure in one of their systems results in people dying but I can go to any major tech companies websites and experience a bug on any given day.
This probably has more to do with Aeronautics actually practicing engineering as compared to today's software development practices. Also software development as a field is far less mature than traditional engineering disciplines. That's not meant to be pejorative - it's just where humanity is.
>A thorough interview in physical engineering disciplines will often expect you to demonstrate/validate something about your technical skill in-person, unless you are being interviewed because people already have validated knowledge about your capabilities (e.g., hiring a known individual from a competitor that is well-respected among peers).
This to me is one of the worst things about the Tech industry. You could be a senior engineer at Google and to move to a new company you will have to do the same algorithm interviews that New Grads are doing. It's a bad look for the industry. Like oh I see on your resume that you have worked for Facebook, Google and Amazon, well we better put you though an algorithm interview because you could have just been a false positive at all 3 companies. It massively devalues SWEs in an industry that we already know colludes to depress wages.
>This probably has more to do with Aeronautics actually practicing engineering as compared to today's software development practices. Also software development as a field is far less mature than traditional engineering disciplines. That's not meant to be pejorative - it's just where humanity is.
Not disagreeing but my point was that they have a far better performance record despite not doing the bizarre interviews that the Tech industry favors.
> EE, Civil, Mechanical, Aeronautical etc. and in not a single one of them do they do this monkey dance.
EEs definitely go through basic, advanced, and domain-specific circuit design questions in their interviews. And compared to LEETCODE they are much more difficult to prepare for too.
There is a reason why SWE is a fallback option for many other engineering disciplines -- resources like leetcode and low barrier of entry to learning make it much more accessible.
In a few months I'm going to be sniffing around for embedded software engineering positions (c on microcontrollers, DSP, maybe even some PCB or FPGA design).
I'm still buying a copy of cracking the coding interview and doing leetcode in the weeks leading up to applying for precisely this reason. It's dumb but I'm not going to give up an otherwise promising opportunity in protest, although in the interview I might politely ask (after solving the problem) what relevance it has to the job, just to make the point. ("I thought I was a applying to do X, and this doesn't seem to fit the job description. Does the job involve a lot of work like this?")
> although in the interview I might politely ask (after solving the problem) what relevance it has to the job, just to make the point.
I highly advise you don't do this, because I don't consider it polite. Everyone tries to be objective in final review, so don't jab them with something snarky they'll remember about you while discussing your performance.
It's an exercise, it's artificial. Can you convert an abstract idea into code. Yes there are bad questions and bad interviewers, that doesn't mean wanting to see you write code is bad. You don't want Microsoft's new "real world, from the job" style questions [1] because I've had a few of those and they're misguided. Way too much internal domain knowledge is missing and I spend too long trying to get it out of them.
That said, DP in a phone screen for devops is a little tough.
I wouldn't ask it snarkily. I've been hired for jobs before and then asked to work on stuff that, while related, was nowhere in the job description. Got hired to do Java/C++ development, was pushed into being part-time sysadmin for our primary test environment because my team had spare capacity and our dev-ops team didn't. There were times I went 6 months without touching a line of Java or C++. Another guy I worked with said he got hired for real-time c development and then got pushed into Java development shortly after arriving.
Some managers embrace the notion that all software engineers are basically the same and you can shuffle them around outside their area of expertise and then complain when efficiency goes down. Now granted there are probably more strategic ways to ask about that then the direct method I originally posted, but if I'm interviewing with one of such managers, I want to know.
Fair enough; phrase it "What would my day-to-day work look like?" "What percentage of time coding/designing/testing/meetings?" The current interview process could be better but right now these coding questions are just a proxy for ability/aptitude.
I just want you to have the best shot at this job. Be as critical about the process as you want after you get it.
There's literally no way to not ask that snarkily. Don't do it.
Also, it reeks of "justice driven opinion having" which is scary beyond reason for an employer, e.g. "They didn't ask for this feature the right way so I'm not going to deliver it/deliver it in a way that makes them unhappy."
I know it's the internet, but you seem to projecting a certain tone onto my statements that wouldn't be there. I don't know where you got "justice driven opinion having" from, it's mission-driven opinion having. They need to know I can code my way out of a shoe-box, and in return I need to know they can lead a team of engineers through a Chuck-E-Cheese ball pit.
Someone who gets defensive about an honest, neutral-tone question about their interview process is likely going to be defensive about more important decisions, which means they take feedback poorly and are likely a poor leader. This is assuming I'm talking to a hiring manager and not an HR rep.
I don't apply to missions I'm not willing to work overtime for. If I'm going to make sacrifices for whatever my employment does, I require that my leadership shares my commitment and is fundamentally competent such that they don't needlessly waste my sacrifices. In any interview I'll be probing for that as respectfully as I can, just as they'll be probing me for technical aptitude and whatever character profile they're looking for.
But that's not what you're accomplishing by asking that question, you're taking a moral position beyond your role on a topic you're not an expert in. What you're demonstrating is your lack of perspective and a hubris that can poison a team, turning it into a snake put where people "neutrally" question every decision.
The mere fact that you think you can neutrally ask a question like that is a red flag.
Or I'm just politely asking for confirmation of what I'm interviewing for, as I'm participating in said interview. You sound like that proverbial manager who refused to hire someone because at the lunch interview the candidate put salt on his fries before trying them, and the hiring manager extrapolated that to mean he doesn't properly analyze situations before acting.
As for "role" and "expert", you seem to think that in an interview my job is to be subordinate. Quite the contrary, at present I'm not unemployed and my bills are paid. I'm looking for an upgrade, not a necessity. So in an interview I'm still deciding if the company is worthy of my subordination/my making them more money. I may not be leadership, but I can pick which leader to support. Leaders who are oversensitive to basic, polite questioning during a job interview clearly don't want/need help that badly.
Now once a subordinate, obviously there are times/places for questions and times/places when they are not helpful. But if you consider questioning decisions in general "poison" for your team, and you fear your team turning into a "snake pit" if someone starts asking questions, then that points to a severe lack of trust and/or poor communication between you and your subordinates. Either of which would turn me away even if you offered to double my salary.
It's not honest. You're not asking the question to better understand the situation, you're asking the question to make a rhetorical point. You're not expressing your opinion, you're hiding your opinion, and setting up a "trap" for the employer to possibly fall into if they answer "incorrectly". You think you know better.
Asking questions doesn't create a snake pit, not being straightforward with your opinions does. Asking leading questions that force people to not trust your motivations will certainly create a toxic work environment.
Related, I don't think you actually believe I was genuinely putting forward the idea that you shouldn't ask questions in your interview. I have some not-great theories about why you chose to do that, but this probably isn't worth the time it'd take to figure it out with you...
It is honest, it's just not 100% blunt. I'm saying a professional version of "Now that I've proven my resume isn't a complete lie, you know these problems are irrelevant and non-representative, right?" and gauging their reaction with a plausible out that I was just confirming basic information if I need it. I can even throw in that anecdote about my previous job where I was hired for one thing and pushed into another. Hell if there's another engineer in the room we might even bond over it. If I was 100% blunt I'd be guilty of the very snark you say is impossible not to have and likely get less information about them thanks to pushing too hard. I have to pass their "is he completely full of shit?" test, why shouldn't they have to pass mine? Ideally we both pass our respective tests and both sides become much less guarded as the interview goes on.
As for "knowing better", in the past I occasionally have known better than some of my bosses, as born out in money lost (once to the tune of millions of dollars) doing it their way until they (miraculously and of their own accord, of course) came around to doing it the way some of my co-workers and I had been promoting. Was a minority of cases, but often enough that I don't assume competence or honesty from strange managers without at least some indirect verification.
As for not asking any questions in interviews, I didn't mean to insinuate you meant that exactly. But from your statements it appears you would have issues with non-technical non-job-specific questions, because I'm an engineer and such questions would be "taking a moral position beyond your role on a topic you're not an expert in.". Which to me sounds like "Engineers shouldn't judge businesses beyond their immediate role because they don't have business degrees". To which I'd say I don't need to be an "expert" to judge interviewing techniques any more than I need to be a licensed auto-mechanic to change my oil and brake pads. Likewise I've learned the hard way to examine a business's financial incentives/disincentives, business model and direct competition more closely than the job descriptions, because those factors will determine the nature of the job I'm applying for a lot more than any given mission statement or brief description in an interview. So I may ask questions about those topics as well, despite being "beyond my role".
Since you value direct honesty about intentions: this conversation is beneficial to me in forcing me to explain/defend these perspectives in greater detail than I usually have the opportunity/need to. So for whatever it's worth I thank you for that. It's clear we both come from very different experiences in approaching this scenario, and I'm sure your experience bears out your arguments just as mine does my arguments. I doubt we'll come to an agreement here, but part of me wonders how I'd do in an interview with you if I wasn't aware it was you and you weren't aware it was me, from this thread. Probably never going to find out, but would be interesting.
I can't honestly said I read this. The fact that you're writing this much to try and justify something so simple and unimportant should be a red flag to you.
Interviews go both ways. If you have a problem simply saying something like "nah, these are just screening problems, it's hard to come up with good ones" then it sounds like there's deeper problems here. Like, the company having issues recognizing even obvious flaws in its own processes.
The problem with tech interviews is that it’s super rare to find the right combination of elite technical skills and emotional intelligence required to properly assess a candidate. Those people are usually busy.
What you end up with is someone who is either a mid-level engineer and almost totally green to recruiting or an HR drone with zero technical skills who is liable to believe a bunch of bullshit. Both tend to ask stupid interview questions they learned from a book because they’re out of their depth in some way.
This is not a situation unique to software either; it affects nearly every professional specialization (especially ones where the pay gap between an HR role and a functional role is as big as it is in tech). You’re right, it makes zero sense to put your interviewer on the defensive — being likable is almost always more important than being good.
It is all about phrasing. If you ask it like "I thought this was a X position, doing Y isn't something I expected to be related. I can do it, but I was hoping you could clarify if the expectation was the ability to do X, Y, or both?" you will likely have a far better outcome than if you come off as challenging them.
For embedded programming you are more likely to be asked questions that require bit manipulation, shifting, masking, etc. Maybe low-level C, mapping structs to memory locations, etc. Maybe how you might go about implementing hardware interrupt handlers, or how you could put a microcontroller into low-power sleep and then wake up, etc.
It is common for embedded software engineers to come from electrical engineering, and they are maybe less likely to ask CS-type questions.
I interviewed for a position (and got the job) about four years ago for a C#/Angular senior developer and I was suppose to be “speaking my architectural experience into the team”. At the time, the only way I could document 10+ years of development experience was by putting a job I had worked at for a decade doing cross platform C and C++ with a little bit of assembly.
Out of nowhere, after I thought the interview was over, one guy kept drilling me on that type of low level stuff that had nothing to do with the job just to prove how smart he was. I remembered all of it thanks to a decade of dealing with it and too much time on comp.lang.c but it was still off putting.
I wasn't there, so I can't know how the interview went, but are you sure the motivation was "just to prove how smart he was" ?
If I'm already 100% sure I'll recommend a hire, it's not rare for me to go off-script and dig into random stuff in the candidate's experience, first of all because I find it interesting or fun to talk about, and second because having deep knowledge about anything is positive signal, regardless of whether it's useful for the job.
I have definitely been in this scenario and asked people who had relevant background to work out low-level performance details of their code. I hope they didn't think I was "drilling" them.
My (would be) manager who was also technical (he has since self demoted to an IC role after his kids graduated college) was already satisfied. It was one of the other devs who was sitting in.
I think he asked me something about the difference between undefined and implementation defined behavior and the an example of each. This was in 2015. I had the same question when it was relevant for an interview in 1999.
I’ve geeked out with another hiring manager at an interview about some old 8 bit coding I did but that felt different.
If I was hiring for this sort of position, I'd want them to talk about the difference between hard and soft real time. I'd want to know whether or not they were familiar with embedded software development standards and what sort of things are restricted from "normal" environments when you are working in the embedded environment.
The last thing I would want to see is lines of code on a whiteboard. I'd want to see how they'd structure their code, not whether or not they remember C/C++/Rust/language-de-jour syntax.
All of the proposals for questions that I've seen in this thread are for basic programming skills. Not whether or not the person knows the environment we are hiring for.
Hopefully. But I've heard from people who got asked leetcode style stuff in addition to those kinds of questions because it was company policy for anyone interviewing to be a "software engineer".
Although embedded is possibly one of the few software positions where these questions might be somewhat relevant- aside from embedded who is writing a sort by hand anymore; although even in embedded it isn't that common of a problem. I just find the questions especially egregious for an average software dev working on something like a web app, since in that case hand-coding something like a sorting algorithm or BST is almost always the wrong thing to do.
Oh I completely agree, but like it or not such questions have become the unofficial Software Engineering SAT. Part of being an engineer is working with/around other peoples' bullshit. If the furthest that extends is a dumb interview question or two, that's a dream job. :)
I'm a Web developer, and I did hand-code the Dijkstra path-finding algorithm recently. Granted, it's not common, and I didn't do it from memory, but at least I knew what to Google.
I needed to represent the shortest path between nodes on a map, so I needed the internal data from the algorithm, thus the need to whip out my own version.
Don't bother with books about how to get a job. You're wasting your time.
Instead, spend more time with books about hiring. How to run an interview, what to look for, ect.
The reality is that companies mostly want to hire someone who can do the job and behave in a team. (There are some exceptions, but you probably don't want to work there.)
At the end of the day, doing lots of leetcode exercises doesn't prepare you on how to convince someone in an interview that you can do the job. A few leetcode questions might help build your confidence, but that's about it. Really, all you need to do is play along and evaluate the people who are interviewing you.
BTW, what I usually tell candidates goes something like this: "Look, we only have an hour, so these questions are contrived. We don't write code like this on the job. The best way to succeed is to play along and answer the question." And then, as I get into the questions, I sometimes add things like, "This isn't a JSON versus XML question" and "If you can use async-await, we wouldn't have a question. I'm trying to evaluate..."
You are still asking the leetcode question, and no matter how fairly you explain it the candidate who practised those questions will probably do better.
I doubt it. Leetcode questions tend to be deliberately technically challenging, and focused on algorithms.
I don't ask technically challenging questions. I'm more focused on comprehension in a technical discussion, and a basic comprehension of concepts that are critical for the product that I work on.
actually, i would think leetcode interview problems would be more relevant for embdedded development jobs. in embedded, you're less likely to have access to libraries that implement advanced data structures, sorting and searching, string manipulations, etc. so you'll need to do more of it by hand.
To a point, but like most things in software my experience is you look up someone who's already figured it out and copy them :) Granted you have to understand the code a little more and be able to apply it to whatever system/hardware constraints you're working with.
But just as higher level Software Engineers rarely have a need to implement recursion/graph traversal/etc, in embedded its even rarer due to the relative expense of such operations. I'd say knowledge of signal processing and how to power/performance/memory-optimize code would in general be much more valuable.
That’s my impression too. I would expect more handcrafted code on embedded systems and you need more awareness of computational complexity and memory needs.
It's not as important as you would expect for a couple of reasons:
1. The size of the data is often much smaller - the savings from using an nlogn time algorithm vs a n^2 algorithm are much smaller when n=10 versus n=10000. There's a good chance that the constant time setup will dominate for those smaller datasets.
2. To a certain extent, embedded systems don't need to be as fast as possible - they just need to be "good enough" to meet their deadlines so that the rest of the system functions as expected. It'd be nice to be able to compute 1000 CRCs per millisecond instead of only 10, but if it already takes 1 millisecond to transmit the CRC over the serial bus, it's not going to make the system substantially better at its job.
So yes, there is a lot of handcrafted code, but quite often it's just implementing your standard naive brute-force algorithm with minimal code size, minimal memory overhead (preferably without dynamic allocation), and well-understood worst-case latency.
This. If I'm in embedded land, especially hard real time, I don't care how quick something is, as long as it complies with the real time requirements.
Programmers will endlessly prematurely optimize. I'd much prefer to see simple code that can be understood and optimized where needed, rather than squirrely code that runs fast but is unmaintainable.
As much the current interview process is broken, I honestly believe this is the best attitude.
In a way, it's nothing more than studying for some goal you think it will improve your current situation, and the good news that it is pretty much a solved problem, with plenty of information online that can be used to practice.
Just take into account that for an embedded position you might actually be asked more domain-specific questions.
I guess if you don't like the company's interviewing process you don't have to work there. Unemployment rate for software engineers is pretty low these days so just go with somewhere else. I'll keep working at places that ask programming questions because basically all the highest-paying places do that these days. And as snobby as it sounds, I want to keep working with other people who can pass algorithms teasers
Also, yeah probably no dev-ops position really requires knowledge of DP. But they generally do require programming knowledge and probably an algorithm here and there so it's not entirely inappropriate
Even Amazon doesn't ask about algorithms for DevOps. When I interviewed with them for a DevOps position, they asked me to write a Python script to parse logs files. They also asked about various linux commands that you would use to work with those same logs. Pretty relevant. But the interviewer never once asked about algorithms or anything like that.
My current company gave me a take-home test to build out a solution that deploys a code package. Best interview question/task I've ever had. It was hard, but I got to show off my real skills.
Companies that ask leet coding questions for DevOps probably have no idea what they're looking for and probably don't know what they're doing.
Hm. I spoke with Amazon (AWS) recently (haven't interviewed yet), and they made the express point that if I were to accept an interview that I should brush up on my algos and HackerRank questions because they loved HackerRank and would surely use a series of those-style problems throughout the interview process.
My last in-depth interview was a take-home/collaborative project (paid) for a smaller company here in Canada. It was a great experience. So good on you.
I think you are talking about something different. The parent poster was referring to a Devops position. Were you looking at a software engineer position?
I wasn't really actively looking for anything—they contacted me about a range of positions that centred around special projects that ranged in subject matter/disciplines.
It's probably per-team, someone on here mentioned that's how Google works at least.
Someone I know did an AWS interview round during business school, and they do this thing called "bar raising" (IIRC) that involves one of the interviewers literally doing something unpleasant to see how you react (e.g. picking their teeth in the camera, cutting you off when you speak). Very odd.
That is not what bar raising is intended to be. There's always a bar raiser in an Amazon interview loop who is not affiliated with the hiring team. They ask the same kinds of questions as the rest of the interviewers in the same way, but they're not influenced by how much the team needs to make the hire, which in theory prevents the team from lowering the bar just to get a warm body in.
I'm not saying your friend didn't get a bad interviewer. It's just an interesting demonstration of how risky it is to generalize from anecdotes.
It was a generally accepted practice based on talking to the folks who interviewed with Amazon during my wife's time at Kellogg that one interview would be conducted by someone (an interviewer who had done 100+ interviews) who would be unpleasant in some way.
I'm guessing the way Amazon interview for business roles is somehow different than how they interview for specific team positions. As I understand it, MBA hires aren't filling specific gaps on teams, since the hiring process is so long and involves an internship. There wouldn't be a "team" to lower the bar for, in other words.
That is not what "bar-raising" round is. It's a difficult question (often algorithmic) that 50% of the employees wouldn't be able to solve. Its designed to raise the bar of employees.
>Companies that ask leet coding questions for DevOps probably have no idea what they're looking for and probably don't know what they're doing.
I think it's a more a case of cargo culting rather than not necessarily knowing what they are doing, although that might also be the case in some instances.
If you’re cargo culting do you really know what you’re doing though? I’d think the two situations are mutually exclusive. Or were cargo cults actually effective at getting supplies dropped from the sky?
What i meant is that they are cargo culting in their interviewing method but this does not necessarily mean that they don't know what they are doing day to day.
Palantir is pretty well known for giving algorithms problems so it would have likely been solved by a cursory google "palantir software engineering interview"
I guess there is a disconnect where some people don't think dev-ops people are software engineers and others do. I fall in the latter camp and have never worked with devops people who weren't also software engineers
To me, there is a pretty large gap between "interview questions to test the basics of coding" and "software engineer interview questions". Searching for the latter to find the former is like search for copywriter interviews to study for questions on basic English proficiency.
Wouldn't "the basics of coding" be something like knowing how to read/write a short python/bash script to automate a task? Algorithms questions ala leetcode are far higher level than the basics.
Can you or anyone cite where you have read that the unemployment rate for software engineers is low? I'd love to see it but I'm not able to find much trying to search for a citation on this point. I'm not sure that it is actually low, but this seems to be a common parlance amongst recruiters.
I am preparing myself for a week of onsite interviews as we speak.
Of course I hate them, but what other choice do I have?
The biggest problem is that the questions are so random and span almost anything.
Will I get asked:
a pthread question?
implement mergesort or quicksort?
implement a Read/Write lock?
implement a smart pointer?
a bit manipulation question?
topological search?
dynamic programming?
graph question?
backtracking question?
What the default timeout for TCP/IP is?
a Java internals question?
best practices about Cassandra?
N-ary tree search?
linked list?
b-tree?
Paxos vs Raft?
What is the access time for an SSD vs spinning disk?
How does Hadoop work?
Design Netflix
various random programming questions, as evidenced by Leetcode's 1000+ questions
It's impossible to know everything and yet this what people expect and ask. I've been asked those questions above over the last 6 years. It's utterly insane the expectations that interviewers have.
I personally also interview candidates, I've done hundreds of interviews throughout my career. These days, I do about one interview a week, and often 2.
My coding question is a simple recursion question, with a for loop and some basic knowledge of data structures. It's not hard, but I look for perfect code. If you can't code a bug-free 15 line function in 45 mins, that's my red flag. I don't think that's unreasonable. As long as the person asks good questions, we work together well, and they can code in a sane way, to me that's a pass. Interestingly, 70% of the people fail the interview because they can't code properly, even after the phone screen. I think because people are expecting a hard leetcode algorithm question, they can't think properly when someone asks them a simple question.
> The biggest problem is that the questions are so random and span almost anything.
Exactly. I have no problem coding up a solution if I have an hour to do some research. Either give the topics ahead of time or ask it as a homework assignment.
If the job requires hurry-up coding solutions to whatever crap falls on your desk without any design, research or discussion of alternative, the quality of their product and codebase probably reflect this.
> It's utterly insane the expectations that interviewers have.
Not really though. They always manage to fill the position.
Interview’s see the system as unfair because they think their interview happens in a vacuum. You get x questions answer all correctly and your awarded with a job. So naturally you complain when you feel half the questions are too advanced or that more brain understanding is enough. In reality the interviews are just a sorting, and when everyone can answer the relevant questions you start asking harder ones because “we picked a candidate at random from those who got the questions right” is just not a satisfying endstate.
> In reality the interviews are just a sorting, and when everyone can answer the relevant questions you start asking harder ones because “we picked a candidate at random from those who got the questions right” is just not a satisfying endstate.
That's just silly. If they can answer the relevant questions, they're qualified period.
Generally people don't just want to know that they got a qualified candidate, they want to know that they got the best candidate available. And so having a more sensitive differentiator than 'passes basic qualifications' is important to most people.
The reason that people still ask algorithmic questions is because they work for avoiding false positives. False positives are a lot more damaging to a company compared to false negatives. While algorithmic questions can get many false negatives, they don't get much false positives.
What does it mean for a software developer? Spend a few hours a week doing leetcode questions. They're really not that hard.
The one whiteboarding code review I had was to pseudocode a merge sort on the board. I knew the question he was going to ask because I had two coworkers who had gone through the process previously, one had worked with the company and the other declined the offer.
I already had an offer by then but I did the interview for grins and giggles and didn’t try to remember what a merge sort was. I actually wanted to test the culture of the company. If they ask me to write a merge sort and I don’t know the algorithm, will they describe it on a high level or will they look at me derisively? That would tell me a lot about the company.
Well anyway, the guy was nice enough, I did the mergesort on the board, got an offer but decided if their interview for “senior engineers” consisted of the ability to do a merge sort instead of deeper architectural questions, that wasn’t the company I wanted to work for.
When I conduct interviews, I have a simplified non working (but compiling) class that is a sample of the types of problems we solve everyday with a set of failing unit tests. We pair program and they have to see if the tests pass.
Or the software company can spend a few hours actually building a real interview process instead of making ridiculous interview projects. I have no problem proving that I can code (whiteboarding, pairing, etc), but I refuse to do stupid college-esque programming assignments on my own time unless they're paying me for it.
How does answering tough algorithmic questions avoid false positives for roles that don't require algorithmic skills? I can breeze through an interview like that but I'd be a terrible hire for a role requiring skills in Kubernetes, Docker, AWS, Terraform.
> False positives are a lot more damaging to a company compared to false negatives.
This is accepted blindly as dogma in our industry but in my experience it's only the case if the company is dysfunctional and management is too weak to let poor performers go quickly. Being an effective leader means being able to hire and fire quickly. If you're open and honest about it, you don't have to do as much as much intense vetting up front which costs your team's time and filters out potentially great hires.
I disagree. We had a candidate talk the talk when we were experimenting with different interview methods. They came in and used a lot of team time to start learning the system and stack. And continued to use the team's time, costing real productivity. They lasted a quarter or so. Not only did the team lose out on productive time, they lost morale by having to support a team member who was not getting up to speed and had to have their work redone and an couple members lost more morale when the newer person was let go. It was a net loss to bring in the new poor performer. I've seen it play out similar more than enough to be convinced that the adage is accurate.
Depending on the country that might not be that easy (in France you have basically 4 months to fire somebody and then you are screwed). Also, you have to include cost of on-boarding.
California is an at-will work state, which means that either the employee or the company may terminate employment at any time, with or without cause or prior notice.
However, CA has very strong worker protection laws, where the worker can incur a lot of legal costs for the employer by accusing them of violating those laws. For this reason, employers are reluctant to fire and more likely to offer a severance package to leave, in exchange for a general liability release. The company has to have all of its paperwork in order to terminate your employment or it'll be open to lawsuit. Anyone over 40, all women, all non-white people, all non-straight, and all people with medical issues or handicaps are a protected category in CA, and in firing them, the company has to show that those reasons were not involved. Put another way, it's really simple to fire a white, under 40, straight guy, but you have to be more careful with everyone else.
The net effect it has is that firings are uncommon, and people hang around until the next round of layoffs, when a reason will be found to lay them off.
Very unusual in the United States outside of Director/Executive level roles. While there’s a “tradition” of giving two weeks notice before leaving it is almost never required. Leaving without notice might get you placed on a no-rehire list and have your former colleagues think you are a jerk, but beyond that no repercussions.
The whole "false positives" narrative is an example of the "saying something enough makes it true" fallacy.
I've interviewed dozens of people, and you can easily root out false positives by asking about their prior experience. Simply probing about things like design patterns to challenges with Javacript front-end coding is enough. If the candidate can go into explicit detail about past projects including bugs they fixed and limitations they came across in a Angular 6 project (as an example), then that's usually a good sign they know what they're talking about.
Surely urgent care nurses can spend time prepping for dental assistant interview questions, but they usually don’t because ER interviewers don’t go on irrelevant tangents.
Suggesting that they do would not address a glitch in the interviewing process but rather try to accommodate it at the cost of the interviewees’ time.
Can you explain that? Wouldn't being skilled at algorithms be a false positive at times for dev-ops skills in OP's scenario? Maybe the algorithmic knowledge is simply rote and practicing, as opposed to skills and knowledge in dev-ops specifically.
> False positives are a lot more damaging to a company compared to false negatives.
People say this, and then they whine about how they can't find good candidates, can't find diverse candidates, etc.
They could, of course, study interviewing, treat it with the same level of seriousness you might apply to other skills, and actually learn how to evaluate people with a pretty good level of accuracy so you can avoid both problems.
There is a shortage of qualified software engineers. The whiteboard interview has spawned out of necessity. There is no good formal examination like an MCAT that companies can rely on.
This math doesn't check out: a false negative allows a follow-up false positive. And without that effect, you also have that each one costs thousands of dollars.
But I don’t think I would want to work for a company full of people hired via that process. I’ve met too many smart people(tm) who spend more time arguing about how many angels can dance on the head of pin than they spend figuring out how they can ship software faster/better/profitably.
I don't understand why you feel it's necessary to share your personal justifications (rationalizations), other than you're trying to normalize your own mediocrity. Which, by definition, already is normal.
In the real world, where your job as a software engineer is to translate business requirements into working systems, algorithms are of little value when solving problems can involve a combination of knowing how to ask the right questions, knowing the underlying architecture, knowing the industry/ecosystem well enough to know when to build versus when to find someone else to do the “undifferentiated heavy lifting”, etc.
But if in your own words “you studied for an interview”, it’s not exactly showing that you have a special skillset if all someone has to do is study for a month.
If you’re an experienced software engineer and companies aren’t beating down your door and you’re still randomly applying for jobs and jumping through the same hoops as someone just graduating from college - you’re doing it wrong.
Using your simple logic, the company could ask you to walk on all fours and eat grass and you'd put in the effort, if you want the job.
We're irritated with the current interview practices, because the gauntlet that many of us have to run is not only wasteful, but it's also harming the profession. If one has to jump through the same old hoops their entire professional life only to be thrown out at 40, what's the point of building a career in software engineering?
Because you are applying for a job where the interview requires you to answer that sort of questions. Maybe you don't want that job so much, and that's okay. Personally, I would refuse to do take-home projects, and that's also okay.
Getting past the interview isn't the only step to being fine. I also want to work at a company where my co-workers know what they're doing. When the hiring process uses an irrelevant filter, you end up filtering out qualified people and letting in unqualified people.
I worked at a place with a demanding interview process for about 3 years and the people I worked with were excellent. Nobody is making you do anything you don't want to do. So drop the attitude and the childish entitlement.
I'm not sure what you think my attitude and entitlement are. I can handle the interview process fine without studying. It plays to my strengths. I've only worked at places with demanding interview processes.
My concern is about the success of the company I work at. My coworkers tend to be excellent when the bar is a tough algorithms interview, but they can't hire enough people. I'm a particularly demanding interviewer. I've rejected candidates who failed miserably, then later learned from my excellent coworkers that the candidate I rejected was excellent at their job.
I've also referred an excellent coworker from a previous job, only to have them fail the interview process. My current company would have benefited greatly from that hire, but instead rejected a talented person for a role they were having trouble filling.
If a company is having trouble hiring, it needs to at least get these excellent people that are being filtered out. Lowering the bar to let these excellent people in also lets in much less qualified people. Instead, the filter needs to be changed to let in people based on how qualified they are rather than how good they are at algorithm puzzles.
Successful companies need more than just people who are good at algorithms like me and studious people like you.
Good DevOps engineers are software engineers that can also do infrastructure work. They should be able to code. Unfortunately, many are "cloud sysadmins", for lack of a better term.
I know that everybody say this, but my experience in real world tells me differently. DevOps is a skill, beside being anything else.
Good DevOps engineers are above everything else good developers. Those kind of developers like to control every aspect of the final product so then they enter system administration, networking, deployments, metrics etc.
On the other hand there is DevOps culture and teams and that is something else and can be reasoned about separately.
Exactly. A devops team has a spectrum of people. Some who only do dev, some who only do ops, with plenty who can do both occupying the space in the middle. There is room on a devops team for people who can't code, and there is room for people who can't be trusted to sudo to root.
Not every person on the devops team needs to be a perfect 50/50 split of dev and ops.
The whole point is to make your team heterogeneous*, so in terms of skills you need to staff for all your needs.
"We'll use python on GCP with PostgreSQL as a backing database"
Well, then you need an expert in GCP, Python and postgresql. This can be as many people as it takes to have those skills covered.
As for responsibility, well, that goes for where your skill is. If you're the resident postgresql expert then you're expected to be responsible for supporting people with it.
It's likely you'll need to worry about patching the OS and stuff like that, but I don't think you need a dedicated person to be responsible. The point of DevOps is mostly to bring down the silo'ing, not to eliminate a particular role.
But it certainly appears employers are salivating over the idea of hiring less specialists, or simply rebranding sysadmin to "devops".
I'd rather say the point is the opposite; devops teams are (even moreso than agile dev teams) heterogeneous (or “cross-functional” or “multi-disciplinary”).
There's basically a continuum where assembly line analysis / dev / test in separate teams (and security, operations are separate higher level organizations with the technical area, and business is separate from technology) is one extreme, then agile dev teams unifying the analysis/dev/test functions, then devops, then devsecops, then maybe the other extreme is some thing that goes beyond devsecops where also domain expertise (and not just the system analysis skills that interface with domain experts to elicit requirements) are an organic part of the team.
But each step along the line does increase the expected range of skills of an individual team member, because you need also to avoid any team member being a single point of failure and you have to keep teams in manageable sizes (e.g., 5-9).
>DevOps are teams
Despite having worked on a DevOps team in the past, I would disagree, in favor of "DevOps are companies". A DevOps team is just another silo, while the DevOps approach should be implemented at a company level to be most effective.
Can you do Python scripting? I’m a software engineer but also the resident “AWS Architect”, there are plenty of times I need to throw together a script to automate something or glue things together.
I've yet to see a developer who knows simultaneously the least privilege principle, a minimal object for a deploy process and what is a Linux container (it's 2019 after all) without having a sysadmin background.
You don't need a cloud to implement DevOps culture to development process, yet you need people who know their shit when they do Ops work (i.e. Poor Yorick incident in Gitlab).
When interviewing people I usually push until I get the candidate to answer "I don't know" on SOMETHING, just to see if they can do it. Not everyone can, and a lot of people will happily come up with a technical word salad to hide their perfectly understandable ignorance on some obscure technical problem I just came up with. That way you get to hear how they ask technical questions. Asking questions is a valuable (and largely separate) skill too.
It's important to have someone who is able to stop and ask for help instead of going further down a blind alley. I've found it filters out a lot of the know it all types who never mange to get anything done.
I doubt that was the intent of the OP's interviewer, but you never know.
That's a trick question. Trick questions are bad because they test interviewing skill and not programming skill.
It's a trick question because you want the interviewee to answer something different (I don't know) than what you're asking them to answer. Confident applicants might feel up to saying they don't know, but nervous people who are very self conscious in an interview setting may respond completely differently than they would if they were asked the same question on the job by the coffee machine. Their mind would be racing "oh no, I'll look so bad" and they'd go into full panic mode.
Don't ask trick questions. It's smug and useless. Plus you'll be missing out on great smart colleagues who happen to stress out about job interviews.
You make a fair point, but an interview is not solely about programming skill. It's also about being a co-worker. And the question "what does this person do in X situation", where X is "they don't know the answer to something they need to know'" is highly relevant. So there may be a better way to go about it, but the parent's desire is reasonable.
The deal is when you’re interviewing me we’re _not_ co-workers. You hold all of the cards and decide my fate.
A big part of interviewing candidates well is breaking down that dynamic. There are many ways of doing it - I get up on the whiteboard first and write out my problem, and stay there until they’ve explained themselves and I need to get back to my notes so I don’t forget stuff. I phrase questions like “I need help implementing this function...” or “how are we going to solve this edge case”.
I don’t think either side of a tech interview is easy. The problem is only one side is evaluated.
I'm not sure I agree here. Granted I work for quite a small company, so it's pretty different than working at a company with a significant reputation to help candidates make decisions. But when I interview candidates I'm very aware that they are evaluating me whilst I'm evaluating them.
I actually think this is a good thing. It incentivizes me to make the interview a more pleasant experience, and perform it in such a way that it feels collaborative, like a real work situation, and not as standoffish as some interviews can be. That said, we still do ask hard technical questions solved on whiteboards or in code, and setting them up in such a way that the candidates feel comfortable asking questions on things they don't know / are unclear about is definitely challenging.
To be honest, something I think we (as in the software community at large) lose sight of some times is how unique and challenging performing interviews is. We see stories like this pretty frequently, and we're used to the idea that being interviewed is a fairly distinct skill, orthogonal to what we usually perform in our day to day work life. Something that's talked about a lot less frequently (understandably, more of us are getting interviewed than performing them) is that interviewing others is also a separate skill, and one that really isn't practiced or honed enough in my opinion.
> The deal is when you’re interviewing me we’re _not_ co-workers.
Not yet, but the purpose of the interview is to figure out whether we should be. (And I disagree that it is one sided; this whole topic is about an interviewee's evaluation of the process.)
I tell all of my friends that it's your job to interview the interviewer as well. The company is about to get 8 hours of your day for the next few years. figure out how you can figure out if it's the right place for you.
Obviously, if you are in dire straights and you need to take what you can get, sure, but the questions are still relevant and will prepare you for whatever poo you might find yourself stepping into.
It's not quite the same as a trick question (which I agree you should avoid). He's not looking for the interviewee to say "I don't know" to any particular question. The correct answer is a perfectly acceptable answer too. The unacceptable answer is to make up bullshit, which is precisely what he's trying to filter out.
Except that some interviewers say that when they hear "I don't know" it's a sign that the interviewee gives up easily and extrapolates that they will give up easily when they encounter problems while working. The point is that as an interviewee you're supposed to "know" an unfair amount of knowledge and have incredible spontaneous algorithmic skills to boot (the interviewer picks 1 question and has the answer while the interviewee anticipates and is expected to competently answer the set of all possible questions). What ends up happening is everyone crams for interviews by rote memorizing common system design and coding questions. From the interviewee's side it's a "damned if you do, damned if you don't" scenario when OP asks his trick question. This is similar to hazing where initiates are unsure of the correct way to conduct themselves because they are being judged by some vague, perhaps intentionally withheld ideal, standard, or cultural archetype. I'd imagine disenfranchised groups would disproportionately 'fail' this question, as they are already insecure of their belonging in tech culture
In the process you are pushing the candidate to the breaking point and eliminating any confidence they may have had in their abilities. They will end up demoralized and most likely fail the following interviews if they're the nervous sort. If they're smart, they'll bid you goodbye and you end up with a black eye reputation.
Any interview (of mine) is a conversation - they're not something you pass or fail - it's just a way for a potential employer and employee to get to know one another so they can make better decisions. Nobody won, lost, passed or failed. The interview is just there to provide more information to determine if the potential employer and employee would fit well together.
If I were in an interview that was presented as a zero sum game I would walk out too.
The interview is there to provide more information about a possible relationship. Winning an interview is like winning a date (assuming you want a relationship with the other person.).
Granted, this is tech, so finding a paying job isn't too terrible. But with a lot of other industries [0] it takes a long time and a lot of effort to get employment. I'd say for most US workers, even in today's market, not getting the job is a loss.
[0] which is not really super relevant to this discussion, I know.
> nervous people who are very self conscious in an interview setting may respond completely differently than they would if they were asked the same question on the job by the coffee machine.
This part is crucial. If an interviewee is pushed to the brink but doesn't say, "I don't know," that does not imply they will not say, "I don't know," once they've secured a position.
It’s not a trick question. Saying I don’t know is a perfectly acceptable answer. The follow up should be questions to clarify or reduce complexity. I interview a lot and this is something I expect out of candidates.
Maybe you could give the critical replies to your comment some consideration. There seem to be many interviewers who abuse their temporary power over candidates by freestyling a psychological excercise, and brag about it in threads like this.
In a way forcing the other party to give up with an "I don't know" is humiliating, and graciously allowing this answer doesn't make things better. You'd put yourself above the other person.
Imagine you'd interview someone you truly respect, a luminary of Open Source walks in. How would you go about the interview then?
I think a great interviewer can make the interview an enlightening and wholesome experience for both parties - while easily identifying the exact qualifications and the fit of the candidate.
I thought I had given consideration. I do not regard "I don't know" as giving up. Giving up would be not finding the answer eventually. I certainly don't find myself "under" some other person if they know the answer to a particular technical question.
As for interviewing a luminary of open source what would be the difference? I imagine they (much like everyone else) don't know lots of things and are probably comfortable saying so. Their answers to how they would go about finding an answer would probably be wonderful.
You can call it whatever you want, but I think if a candidate would be humiliated by saying "I don't know" in an interview, that's a strong negative signal. Saying "IDK" and reading the docs, googling it, or asking for help is at least half of the job.
"I don't know" is something that developers encounter weekly, if not daily. There's nothing humiliating about acknowledging that there are things one doesn't know, whether in our daily jobs or in an interview.
Saying "I don't know" like this in an interview scenario is nearly impossible for most candidates and only serves to humiliate rather than prove their abilities.
If you want to see them ask questions, give a half explanation of the problem at hand instead and guide them to ask more detailed technical questions.
Be an ally, not an adversary, and you will find talent better than if you push.
> Saying "I don't know" like this in an interview scenario is nearly impossible for most candidates and only serves to humiliate rather than prove their abilities.
I don't think this is true only in interviews though. I have worked with people who can't seem to admit they don't know something and they're difficult to work with.
It's completely fine if you can't do something. What's not fine is that you fail to communicate it to those around you and create false expectations.
That being the case, I think it's fair to score a candidate negatively if they aren't able to say they don't know something.
It’s really not - as someone whose done 100s of interviews for a brand name tech company I don’t see this as a problem. The goal being for candidate to indicate their lack of knowledge but work through it. As an engineer you will encounter problems you don’t know how to solve. Pretending you know and bullshitting is a red flag.
There are hundreds of ways to do this without intentionally tilting the candidate into failure later. As an interviewer you have absolute authority in the room -- don't abuse it and destroy the candidate.
I think you are misunderstanding my point by indicating that interviewer's shouldn't abuse their power.
To be specific, I do a lot of things up front in an interview to help the candidate. The most important is informing them that they should treat technical exercises as a team exercise where we are teammates working together. Furthermore, I declare that there is no penalty for my assistance during the interview. The idea is for us to work together to solve a problem, my interest is in how they think and work through the problem. Furthermore, I explicitly tell them for discussion on prior experience and technical facets of interview that they are welcome to indicate they do not have an answer to my question or are unsure what's next.
All of this combined creates a very pleasant experience which is an important part of my goal as an interviewer. I'm not trying to embarrass people, though I am trying to make a informed hiring decision which is a very expensive endeavor for my company. Therefore, beyond letting candidate enjoy themselves, my main focus is to ensure that I have sufficient confidence to hire or be willing to walk away due to lack of conclusive data.
You are proving the OPs point: they are looking for people where "I don't know" is acceptable in collaborating with others, not for people where saying "I don't know" is considered a failure or humiliating. If someone is humiliated being asked a question they don't know the answer to, then they aren't a culture fit.
As Jeff Bezos says, the older he gets, the more he realizes he doesn't know.
Most people have no issue with saying "I don't know." I've worked with people that can't say "I don't know" - they are incredibly toxic.
> Saying "I don't know" like this in an interview scenario is nearly impossible for most candidates and only serves to humiliate rather than prove their abilities.
with hacker news so utterly convinced that employers have literal powers of life and death over their employees, when do you expect the potential employee to regain the ability to tell their employer that they don't know? after they've quit their previous job and moved across the country? a week after that? six months? maybe after they've got some kids?
It’s not a trick question. I am expecting them to answer the questions honestly. If they know the answer, go ahead and answer correctly. If not, don’t pretend.
I don’t want to hire someone who isn’t going to admit the limits of their abilities, for whatever reason. Could be smugness or lack of confidence, either way it’s a bad signal.
its a trick question, period. The question is formulated with the expectation that it cant be answered, even if there is a small chance to answer it right.
its not a bad signal. to be honest its the smartest thing to do (like guessing on the GRE), especially since most people ask questions they expect the right answer too.
the only bad signal I see is in you. I would wager to say, in most situations in life, don't play tricks, just ask the question you really want answered.
As a lot of other commenters are pointing out: there's an adversarial and a non-adversarial way to do this.
When I'm interviewing someone I will often preface groups of questions with "have you worked with {x}?" or "are you comfortable answering questions about {x}?" before asking the actual questions covering that area. A simple intro question like that goes a long way towards keeping an interview friendly. The interviewee can say "no" to a small-sounding question without feeling like they're embarrassing themselves.
For example, they may be comfortable with Python but not super-comfortable with the GIL/multiprocessing/multithreading. If I ask "have you worked with Python's multiprocessing or multithreading before?" that comes across as much less adversarial than asking them to describe Python's GIL and forcing them to say "I don't know."
Doing an intro question also allows them to give a partway answer like "I've touched it before, but I wouldn't call myself an expert". If they say something like that I can re-focus my questions to another area, or just ask higher-level questions, rather than hitting them over the head with a deep question about something they've just told me they're not an expert in.
You do want people to be honest about their own limitations, but you don't want your interviewee to think that your organization is driven by macho posturing about being King Nerd.
This will easily pitch you into an adversarial position and give you and your company a black eye. Remember: interviews are bidirectional. If I was pushed to the breaking point like this in your interview, I'd walk out without saying a word.
You're only going to be selecting candidates who can handle pressure -- not skill or ability with this.
You're already treating it as adversarial if you're thinking of it in terms of your "breaking points" and "walking out without saying a word."
Back in the day I had an interview for a business position where the interviewer's personal printer mysteriously started printing documents part way through. I'd just read "What Does Somebody Have To Do To Get A Job Around Here?" which listed that among other tactics a company might use to see how you handled distractions. I've suspected it was on purpose ever since. Even were it intentional though, it would be more of a gimmicky ploy than an adversarial thing to do.
An interviewer should be able to modulate the difficulty of a question without making you feel uncomfortable.
Also the job isn't done in a vacuum - being able to remain calm when you don't know the answer is an ability all its own. Interviewers shouldn't be creating artificially stressful situations but you've also got to be able to manage your own stress response!
My favorite interviews are the ones that have pushed me to the edge of my knowledge and had me derive things that were completely new to me. I love interviews that are genuinely enlightening, which is the opposite of regurgitating Leetcode.
Truly, the best interviews I've had have also been the hardest. Different strokes for different folks.
Doing this during a job interview is not going to give you the results you expect. For every person like you who wants to hear "I don't know" there are 10 people who will pass on you for not knowing something. People's expectations are set by the worst interviewers they experience, not the best (and honestly, trying to trap people isn't the "best" anyway).
So far it's worked very well for me. Since I've started using it (admittedly my sample size is small) I've found some very effective programmers, and automatically filtered out a lot of egotistical types. Software is usually a collaborative effort and this technique helps in that regard.
As another commenter mentioned, you're probably doing more damage than you realize. A lot of more experienced people are going to recognize this as toxic behavior and word will get out that your company isn't healthy. You may not even notice, but this kind of thing will keep the best people out of your pipeline if you continue to do it.
> Since I've started using it (admittedly my sample size is small) I've found some very effective programmers,
This is not the only way to find very effective programmers. You should level up your own ability to discovery this in someone in a non antagonistic and more straightforward manner.
I've done well over 200+ interviews myself, and been on hiring committees where I have had chances to review other interviews. This kind of behavior usually trashes the following ones and aets the candidate up for failure.
To be fair I have "fessed up" to not knowing about a subject that was mentioned on my CV where I had used the tech a little bit 5+ years ago. Been offered the job both times. Though I could see if you hadn't had that experience it wouldn't be the answer that you would want to give.
I don't know about others, but I expect people to have some fluff on their resumes. I don't blame them because recruiters and HR types love fluff, so fluff gets your foot in the door. Recruiters might skip over a person who has MongoDB on their resume instead of Elasticsearch because they don't know they are similar technologies. So it's reasonable that a candidate hold the recruiters hands.
If candidates come out and say such when I ask them questions about it, I'll just move on. I won't hold it against them unless it happens excessively.
I once told an interview I didn't know something and they told me that I should never say I don't know something and just try to make something up.
I don't really agree with him and just kept doing it though and things seems to be fine.. If anything I would say that interview was successful for both parties because we probably wouldn't get along and that is totally fine.
I was told something similar by my high school history teacher Mr. Kolhaise, but I think it was phrased in a more helpful manner and it's always stuck with me.
He said "Don't just tell me what you don't know, tell me what you do know" or in other words, don't just say "I don't know" but say something like "I don't know, but based on what I do know this is what I think the answer would be and this is how I would figure it out".
It really helps that everyone forgets what people say before the "but" and you can help get across the idea that while you may not know everything you know how to figure things out.
When interviewing people I usually push until I get the candidate to answer "I don't know" on SOMETHING,
"just because I can". To me this technique comes off as condescending and manipulative, and smells like a power play basically. Not to mention - the better candidates will pick up on it, and your company will eventually get a reputation it doesn't want to have.
And really there's no need for it. Time and time again, I find that just by having a normal technical conversation with someone, on virtually any topic that they claim to have expertise in -- they will show their strengths and weakness (and propensity for BS, if any) very, very quickly.
You seem like a person that has never had to deal with someone that is incapable of saying "I don't know." It is incredibly frustrating to project manage a team like that. It is tantamount to lying.
Sure I have - and like I said - it's easy to read this behavior from simply having a normal (but let's just say slightly inquisitive) conversation with someone. In fact even in recent memory I can think of multiple occasions where this behavior has emerged, simply when trying to get to the bottom of routine technical matter.
Not just because I can - interviews are always time limited, and technical conversations always (pleasantly) meander across many topics, usually the strengths of the people talking. It's easy to for someone to spend an hour talking about their strengths - interviews usually need some sort of direction to get useful information.
And it would be manipulative if there were some answer I was going for that wasn't true - I'm trying to elicit an honest response.
I do something similar, I ask the candidate what they do when three different people come to them with urgent needs that must be done immediately and there is no way that they can do them in the time given.
The answer I’m looking for is “I ask for help.” Or “I talk to my manager.” They don’t fail if they don’t answer it that way, but it’s a good starting point for a conversation about why they didn’t say that.
For me, interviewing is more about getting to know someone and how they think than checking off various skill boxes.
They can, it depends on the role. Not referring to you but I have never understood why DevOps at many companies can’t program and are just hired to copy paste files.
“DevOps is a set of software development practices that combines software development and information technology operations to shorten the systems development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives.”
Most of the devops (side question, is SRE technically devops?) people I work with are programmers who know a lot about test+build frameworks, CICD, cloud resource management, etc.
I don't think it would be appropriate to call someone copy-pasting templates, manually adding VMs, etc. "devops"
In my previous job we had hired a windows system administrator with no coding/scripting skills whatsoever with title as DevOps engineer, he didn't know anything about what developers do and CI & CD practices. So he ended up just managing updates (manually) on vms and restoring backups . But i see this more of a failure of management rather than the guys fault. Management heard a buzzword, and hired based on what they have hired before.
> Most of the devops (side question, is SRE technically devops?)
In my experience a “good” SRE, devops engineer, or software engineer is difficult to distinguish in skills. It’s more about responsibilities. This is especially true of the folks I’ve met from FANG.
Specialists still exist in DevOps, and it was never supposed to be a singular role. It's a major holistic cultural change that's intended to integrate dev, ops, and other specialties into a shared sense of ownership of the entire software lifecycle instead of sitting in silos and blaming each other for systemic problems. Turning it into a single role without all the major cultural changes is just yet another hunt for heros and silver bullets.
This is a natural consequence of an increasing supply of qualified labor. Do you need a college degree to serve coffee? No. Does it have anything at all to do with serving coffee? No. Yet, it's now something sought after among new baristas, who generally earn a hair more than minimum wage. [1] Why? Because there enough people who have a degree that be willing to do this job at these wages to make it a requirement.
The consequence of 'get [anybody with a pulse] into coding' is to help increase the supply of people with some coding ability. Out of curiosity I just searched for 'QA job listing' and found stuff like this [2]. A 'senior QA engineer' requires, among other things, a masters in computer science alongside a willingness for unanticipated travel/relocation for "long and short term assignments at client sites." Starting salary? $73k. That's probably an H-1B angle-shoot (H-1B employment requires not displacing American workers, and if nobody applies you're obviously not displacing anybody!) but even then they still have to be able to argue that their request was not entirely unreasonable.
That depends on what is expected of the DevOps position I feel.
If you're going to need to dig into the source code of infrastructure pieces to find/fix bugs or write your own infrastructure pieces then a SE interview makes sense. If you're doing things at scale or using newer technologies then these come up often enough and resolving them can have significant business consequences (not getting into if choosing that path was a good business decision or not, separate topic).
If you're depending on open source software then you also need to be able to fix the code in question. And do so in a way which the project would accept. Otherwise, your bug report may linger until the end of time.
a lot of SW out there are just plumbing code. Rarely you see dynamic programming/recursion algo. Most use standard library as the implementation is solid and kinks have been ironed out.
Sure but that's a discussion on the value of leetcode itself which I've not argued one way or another. I'm simply saying that IF a company (many do) assumes leetcode is a way of gauging SE talent then it'd make sense to use the same criteria for gauging SE talent in DevOps.
>It's pathetic employers keep demanding more skill and free work for the same pay.
In my experience DevOps pays better than SE (and much better than sys admin) because it requires a diverse set of skills. DevOps in that context is not just a new term for sys admin.
I have yet to see DevOps roles that require "diverse set of skills" beyond sysadmin skills, very basic knowledge of at least one language such as Java, Go, or Python (occasionally only PowerShell if its a Microsoft shop) and either Terraform or Cloudformation for IAC. The pay delta between the two roles is less than you think (excluding SF/BA market, but that market is always an outlier).
DevOps is sysadmin with more scripting, more infrastructure as code, and more expectation that you're an underpaid SRE. It also seems like DevOps role demand more unpaid on call responsibilities, where this is usually not the case with traditional sysadmin roles (or it's spread across a much larger team, negating the pain). YMMV.
Disclaimer: 20+ years in tech, the majority in ops roles.
Doesn’t sound like that’s what happened. Seems like they have standard questions and phone in the interview process by trying to apply it to every “tech” position they hire for.
I don't see anything to indicate one way or the other myself.
There are also companies which want DevOps to know SE simply because they believe that such people create better infrastructure. Better in this case means more automation, code versus todo lists, understanding the SE users, etc.
It doesn't matter whether it's hard or not, it's at best a mediocre tool to determine programming skills. Also, it leads to a spiral of becoming worse:
1. being able to solve LeetCode tasks doesn't mean you're a good developer
2. not being able to solve these tasks doesn't mean you're a bad developer. So we have a tool that has a certain (IMHO high) rate of false positives and false negatives.
3. people start realising that LeetCode is becoming a go-to tool for recruiters, so instead of studying to become better developers, they study to become better at solving LC exercises, which increases the rate of false positives
4. with developers becoming better at LC, the bar is set higher and higher (because people spend time learning LC), so people who do not study LC exercises become far behind those who do, which also increases rate of false negatives
5. repeat points 3. and 4. until mastering LC exercises becomes more important than being a good engineer
This is a ridiculous trend which in my opinion only emphasises how screwed up is hiring in our industry.
Honestly, I pass on companies that do leetcode, I'm simply not going to do it.
In my day to day I am working on mission critical applications, creating web applications that rely on legacy databases, writing SQL SPs. I'm decent considering my short career, i try to improve on every previous project. However, when I go home I'm not wasting my time grinding leetcode, even though I was top of every class in college, algorithms, data structures, theory of computation and so on. I'm competent and pick up new stuff very easily. I tell recruiters I will pass on an interview if they are doing leetcode/hackerrank. I'm okay to solve some coding problems that they have day to day, talk about architecture/design, talk about what I'm good at and where I want to improve. I also wish other people would stop grinding leetcode and that everyone was realistic about the job at hand. Most developers won't be writing the most efficient code, but start easy and refactor if certain part of the application becomes a bottleneck.
You have to consider the perspective of a company or a recruiter. Their goal is not to simply find somebody who is capable, it's to take the hundreds if not thousands of prospective applicants they have and then cull that down to the tiny handful that they advance along for the openings that they have available. You're not looking for reasons to accept people, but for reasons to reject them. It's just the nature of employing for positions when with a large pool of interested and qualified labor.
Should movements such as 'get [anybody with a pulse] into coding' succeed, you can expect to see an ever increasing absurdity in requirements. The reason is not to try to be elite or whatever, but because the reality is you have to get rid of of > 90% of applications, for larger companies it may be > 99%. And so as the number of applications increases, you get an ever larger quantity of people with at least the minimum requirements and so you are forced to be increasingly selective.
Does passing some leetcode task mean you're a good developer? No, of course not. But if you take 100 passable developers and you're looking for the best, are they more or less likely than average to be able to pass such a test? It's the same reason that even though a college degree has absolutely 0 relevance to serving coffee, you're increasingly seeing companies "preferring" college degrees for their barely above minimum wage baristas. [1] If you take 100 potential baristas is that best of the best of those 100 more or less likely to have a college degree? Even though it has nothing to do with the actual job, it's just a decent way to filter 100 down to 1.
I feel this perspective gets lost a bit. Big companies don't roll through 1000 applications with 50 open spots and try to hire the best 50 candidates. They want to find maybe people in the top quartile of that, while spending the minimal amount of time on costs to find the adequate fits, and the cost of a false positive is way higher than the cost of a false negative.
Is the process great? Absolutely not, and I'm sure that these places miss out on qualified candidates because of the process and hire some people who gamed the process. Things like degrees and LeetCode tests and other screening methods are just ways to reduce cost and risk to get that "good enough" threshold.
It is no self evident fact that false negatives are less costly than false positives.
Understaffed projects might lead to failures, people quitting, expensive consultant aid etc.
Hireing a bad programmer and two good and ending up giving the poor one easy tasks or letting him go might be cheaper than hireing two good more slowly.
In general I feel the recruitement process is so random anyway that employers should just get it done and pick someone after throwing darts at a billboard with resumés for picking some to interview.
This is (kind of) fine if you’re a well known company with thousands of applicants, but my experience so far has been that finding qualified candidates in the first place has always been the hard bit. I can’t imagine throwing arbitrary hurdles in the way of getting good people into interviews and talking to them!
> 2. not being able to solve these tasks doesn't mean you're a bad developer.
Half of those questions are based on sophomore-level data structures and algorithms classes, the same concepts that operating systems, compilers, and most libraries/frameworks are based on. I would certainly hope that an experienced developer could effortlessly relearn basic concepts taught to college sophomores long enough to pass an interview.
Wasting time on training a skill they are not realistically going to benefit from. People can only spend their time once so learning other skills will suffer.
If the job requires algorithm wizardry, I see the benefits of LeetCode-style interviews but many jobs are not like that.
Company use leetcode style because is not useless. it make the interview process more streamlined, scalable and predictable.
For the me interviewee, it also make the interview process more streamlined, scalable and predictable. I practice once and I can use it for many other company, thus save my time.
There is no point in streamlining or scaling a process that leads to wrong outcomes.
At the margin, companies will lose out on qualified candidates that have better things to do with their time than practice algorithms and hire unqualified candidates that got lucky with their Leetcode practice.
I think it's instructive to consider who the employer is too. Leetcode works well for Google. They pay top notch compensation and have a long line of people who want to work there. They can put horrible hoops up and enough people would still want to jump through them.
Most companies have cargo-culted the programming interview, even when they don't have the same amount of demand from candidates.
I find things like Leetcode very hard and my memory on it does not keep at all. I have to relearn all the relevant things every single time I start studying for interviews.
Don't assume something is easy or straightforward for everyone because it was for you.
I agree. I was pursuing an SRE position at Google but their interview prep material was a lot of nitty-gritty "What does it mean when this syscall returns this integer?" questions that I didn't consider worth memorizing. In addition to the syscall questions, linux internals question, the bit of leetcode sprinkled in, there was also the practical exam of diagnosing a problem in a live system - I felt pretty confident in that however since it was my everyday for the last 3 years.
I switched tracks to the normal software engineer with only-leetcode interview questions. Spent 3 weeks studying and last week I went in and had an easy time because it was exactly what I studied for. Also applying to a few other dev shops was effortless because they asked the same stuff.
> Leetcode is not even that hard, tons of material and key to the answer is on the internet.
Doesn't that support the argument a bit? Its "not that hard" -- so even someone who _doesn't_ have the requisite experience could learn it? So its just another hoop to jump through, _before_ they validate whether you are a good fit for the job in the first place. That's what people are complaining about.
>so even someone who _doesn't_ have the requisite experience could learn it?
I consider it a plus. Every company use different technology and different way to do things. Not to mention it keep changing all the time. With a standardize test, even though I don't have the experience with the technology they use, i can still have have a shot.
I get where you are coming from and agree with the sentiment, but disagree on the specifics.
> Every company use different technology and different way to do things
To a small extent, but mostly untrue. If you are an expert UI programmer, you should be able to move between disparate UI technologies relatively easily.
> With a standardize test
People aren't upset with the standardized test part. They are upset it (frequently) tests things that are not relevant. Hard things at that. There are an extensive number of practical, language / framework agnostic questions you can ask about UI or back-end programming that will easily tease out quality candidates. Instead they ask you merge sort. Hence people complain.
Isn’t the point of DevOps that you’re a software engineer who happens to work on infrastructure automation code? If they wanted someone who only knows how to operate systems they would have called it “sysadmin.”
Well it sounds like a phone interview. One goal of the phone interview is to not waste people’s time with onsite if it’s determined the candidate couldn’t pass onsite.
If their onsite has more leetcode coding questions, then the interview served its purpose.
Agree that LeetCode style questions aren't the best way to interview candidates but with "infrastructure as code" ruling DevOps shouldn't your Ops people also be expected to code? Does not seem to be an unreasonable assumption.
What does concern about some DevOps interviews is the sheer level of knowledge some companies expect up front. I brushed on everything from the Linux kernel, ops techniques, distributed systems and programming to prepare for an Ops interview.
Yeah, they should be able to code. However a DP question is a bad one to ask, for it is not representative of the kinds of problems they solve day to day. This is like asking a programmer about deep security questions related to ssl or the details of the latest kernel bug or spectre. It is good to know those for a programmer but it is a poor signal for the job at hand. Light on my front porch might illuminate my backyard, but nobody chooses a bulb on that basis. I would use another light for my backyard.
I once was in the interviewer seat in a similar situation.
We (my company) were interviewing a DBA. He listed C# on his resume. We develop C#, so I was brought in to evaluate his C#.
He flat out refused to answer my questions.
(In general, if you don't want to work in XYZ, or don't want to be asked XYZ in an interview, don't list it on your resume. That's why I don't list Installshield and Visual Source Safe, even though I've done them and can do them.)
The issue I have with this interview style is that there’s no data to suggest that this results in selecting for engineers that can do the job best. Furthermore, it tends to leave rather little time for the candidate to ask many questions to the interviewers about things that may matter more such as “what does your code review process look like?” and “how are tests done?” Far too many times I’ve gone through these kinds of interviews and wound up in a horrendous codebase anyway, so fat good hiring for all these people that can do programming puzzles alright in an interview has done for an end result. The better indicator I’ve seen for that is a matter of culture and testing for that is really hard either direction.
I’m pretty happy with how my company does it now. A code sample on a fairly simple problem designed to determine how their pull requests would look as well as how much direction they need. On-site is a series of design discussions with algorithmic questions about data structures to use and the tools / services that make sense. Primarily meant to hire senior engineers rather than juniors, but that’s our objective now.
> So basically it was a variation of the exact change problem (I realized afterwards). You had to validate a string of size n. 1 chunk of 8 characters could have 1 format, 1 chunk of 16 characters another, and 1 chunk of 24 characters another. You essentially had to determine if the string was valid. So in other words, find a combination of these sizes in which the string could be validated. I had 30 minutes
> It can be a string of size n. It needs to form a valid combo of 8, 16, and 24 char strings - each of which have an expected format. Result is T or F
This sounds like a pretty basic parsing problem to me, not dynamic programming, and likely to be a regular language. There’s a standard tool for this sort of thing, a “regular expression”. Not recognizing this much is a huge red flag for any kind of systems/integration role, like DevOps.
The answer here is just matching against this regex, with “fmt1”, “fmt2”, and “fmt3” replaced by regular expressions for the different packet formats:
The problem is that is bound by your regex implementation and how it handled overlaps the search strings :). Fwiw many places will just go the next step of - well implement the regex parsing then!
But this is a dev perspective though. Am assuming a scripting answer might be good if the candidate knows about the actual regex behavior for devops.
I'm not sure they were going for the regex answer - it seems like they expected the DP answer since they're using multiples of 8, which allow you to do it as a DP problem quite easily.
Best interviewer I ever had for a development role gave me a sheet detailing a (terrible) high level implementation of a ticketing system, gave me an hour to look at it and then asked me what I thought about it and what I'd do differently. It got us talking for a long while and both of us learned a lot about how the other thought and worked. I still think about it, and found it brilliant. Downside is you can't have an HR clerk do the interview but I think that's actually a positive thing in the long run.
I am a recent graduate of a well known bootcamp in SF, and impressed there are a decent number of graduates from last 2-3 cohorts that have landed jobs at Google. The only trick is knowing how to land an interview in the first place. If you can do that, succeeding at the technical interview can be easily taught/learned.
I think it's great that the interview process is so learnable. Why wouldn't you be happy knowing pretty much what to expect going in when for most roles that isn't the case at all?
1) if the employer thinks DevOps is all about knowing how docker and k8s work, it's not a DevOps role (which doesn't actually exist by the way). a lot of employers think DevOps means "sysadmin in the cloud that can follow an sdlc", so you have to ask lots of questions to find out just wtf they're actually looking for.
2) if they ask stupid questions and generally don't seem invested in finding the right candidate, it was a crap job and you dodged a bullet.
I got into the habit of answering those questions orally, and then directly asking when was the last time it has been used by the person interviewing.
In some cases the interviewer didn't quite like it, but most of the times it actually led into interesting discussions (finding questions is hard, what they are trying to gather as info ...).
I'm based in Europe though, so YMMV
Ha, I'm up front: "this is a synthetic question, which has the purpose of x,y,z". Synthetic questions don't rely on specific tools, or specific problem framings, it can be thought of in isolation.
E.g., I usually put a database question on interviews, for the purpose of seeing what and how much the candidate knows about databases.
I also like doing a design question regarding handling larger data and doing back of the envelope complexity analyses of the solutions they come up with. I.e., what I expect a colleague to be able to do when we're working at our desks.
It's frustrating for the interviewer, because they usually need to photograph your working out to allow others to independently check it later, and insisting on verbally solving a problem denies that.
I'm not afraid to hire people who challenge and ask great questions. I ask that question all the time. The interviewer should be involved with improving the hiring process, not following the cargo cult of "hiring process". Obviously if that question comes up more often, they might start asking "why are we doing X?" and "does X provide better outcomes?"
Amen! Since I've been involved in hiring and recruitment, I stopped seeing interviews as a validation of invalidation of my worth and rather more as a discussion where we are both trying to see if what we have to offer matches what the other needs.
Based on this, most of the interviews are not interviews anymore, but open discussions.
I expect a person in a devops role to not just write code, but write code with unit tests, integration tests, documentation, deployment automation, to the style guide etc...
I also expect them to work well on a team, especially in design reviews and code reviews.
The main difference between a software engineer and a devops engineer isn’t writing code, it’s the kind of code they write.
> The main difference between a software engineer and a devops engineer isn’t writing code, it’s the kind of code they write.
The other main difference is on your team. At my company we would expect that from a software engineer working on the platform, creating tools or shared libraries. A DevOps Engineer at my company would handle the orchastration of shared tools, (i.e. where is Jenkins hosted (VMs or Docker), how it scales, upgrades from Nexus to Artifactory, setting Dynatrace on prem with hooks into multiple cloud providers, the list goes on forever.)
Let's not assume all companies are the same, when probably half of DevOps roles are not about programming in an OO language and you can't easily or usefully unit test something that is like a configuration for mount locations across cloud provider
> It really feels like there’s no connection to the reality of the role whatsoever anymore.
There's not going to be any reality to your role once you start either. Your devops role might suddenly become an IC role on their line of business app. It might become a support role. You might be asked to skill up on a stack you've never even considered without any notice or slack before you're expected to produce results in that stack.
The tech landscape changes too often these days. I have a friend of mine who was hired for his Magento experience but soon found himself learning Java after Adobe bought Magento. The things department heads think they want are rarely what they still want by the time you get hired.
It's ridiculous but there really isn't anything anybody can do about it right now. Tech is eating its own tail, the pace is accelerating, soon there won't be any snake left.
> It's ridiculous but there really isn't anything anybody can do about it right now.
This is patently false. If your role changes drastically, and you don't want to stay in it, quit. But it does a disservice to workers to say, "You're trapped and this is just the way it is". It is not "just the way it is" unless you tolerate it.
Always Be Interviewing. Always Be Ready To Walk. You are a mercenary, and your job is to command top dollar for your skills while building your network so you always have your next opportunity at the ready.
Sure, walk away. How sure are you that the next role is going to be better? Another friend of mine changed jobs six times in two years. Now he's having trouble getting taken seriously.
You're not. You have no guarantees. Is staying in a volatile role where you're not sure what your job will be next week better? You are in control of your own destiny. Don't let someone else be.
I might be able to shed some light on one possible way this ends up happening because I was on a team that did exactly this.
The team/manager probably doesn't know what they want or does and has the budget for only a single hire. Their expectations are completely disconnected from industry/reality because they just don't know any better and probably lucked out getting into their position. They want a strong developer because the team needs one (or several), but they desperately need ops to keep their project afloat. So, sure, OP has all the right ops keywords on their resume, but can they also be that developer we really need?!
I butted heads constantly with my manager who had that mentality because of things like that. We rejected great DevOps candidates because we wanted them to be developers and good developers because we wanted them to know ops.
Isn't this a good thing though? It shows that the competency of candidates has increased so that harder tests are needed. It used to be that 99% of programmers failed FizzBuzz: http://wiki.c2.com/?FizzBuzzTest
From the thread of reactions on reddit, I got this gem:"And finally, the realistic problems we have at work typically take 6 months to explain to new people."
My guess is that whoever was organizing the interviews couldn't find a devops person to do the interview. They found a regular dev to do it instead, and that interviewer was inexperienced and asked one of the only questions they knew.
Moral: Take interview training and make yourself available to do interviews, especially if you're in a relatively niche role. Make this attitude the norm in the industry.
I've always wondered what the attrition rate is for these types of interviews. I know I'd probably fail them and I've been writing software professionally for a while. And this is with me regularly practicing these types of problems because you simply just can't know. Often solving them takes the entire interview time in itself.
What does this have to do at all with the job at hand?
Nothing whatsoever.
It really feels like there’s no connection to the reality of the role whatsoever anymore.
You got that right. Basically what's going on here is that the people running these interviews don't really know what they're doing -- and are secretly terrified of being "found out". So in a desperate attempt to compensate, they lunge for whatever technique they vaguely heard about being used at, you know, "leet" companies. And (as others have pointed out) at (apparently) very little cost or risk to themselves.[1]
Because that way... even though basically don't know why they're asking that question of you... and not only that, most likely would not be able to answer an analogously difficult (and unrelated to their own problem domain) question themselves! --
if you do manage to make it through, they'll at least know that you're "leet".
[1] Except of course, the opportunity cost of false negatives, and the reputation risk (and damage to recruitment prospects) that inevitably ensues once your company becomes widely known for mindlessly cargo-culting outdated and discredited interview techniques. Which they would know about, if they knew that they were doing. But then again, apparently they don't.
If companies don't have strong enough incentives to change their interview processes, doesn't that kind of imply that it is working for them, at least to a certain extent? At least it doesn't seem to be the case that they are hiring so many bad candidates they are forced to change.
Actually, yeah, I think that the current hiring process is pretty good at avoiding false positives. But at the expense of a huge false negative rate. I'm seeing a lot of roles in SF sit open for months (even > 1 year!) - plenty of companies just aren't hiring anybody. The best I can tell is that a lot of engineers looking for jobs are young and relatively inexperienced?
Is the problem that leetcode questions were used or that coding was a hard requirement for a devops position? I do screeners for my team and being able to write code is a hard requirement for us. If you can't reason through and write the equivalent of fizz-buzz it's a problem.
> I'm not trying to defend all current interviewing practices, but Leetcode-style interviews became popular because they're designed to find people who are good at programming without requiring overly specialized knowledge about specific tools that are popular right now but may not be in a few years. The idea is to find someone who can reliably learn what they need to know for the job rather than finding someone who's an expert in X and Y which you currently happen to be using.
I use synthetic / algorithmic questions for this purpose. I don't need tool specialists, I need people with wider skills.
I also like concocting fizzbuzz variants as a high pass filter. Some people fail them...
Heh. I got the full HackerRank/LeetCode/whiteboard shebang for several interviews at well-known companies, for Release Engineer positions (which basically fall into the realm of what hype today calls DevOps).
I think I got lucky. I landed a job at the first company I applied too about 6 months before graduating. I was asked only two technical questions neither required a white board. The first was about how to deal with a temperature sensor which was giving erratic readings. The other was about sorting arrays. Since then I have participated in about 5 interviews of other hopeful candidates. My company's approach to the interviewing process seems to be rather straight forward:
* Does the interviewee have a basic understanding of what we do?
* Does it seem as though they will fit well here?
* How do they deal with failure?
* Are they excited about working for us?
and the one thing that seems to be most important:
* If they don't know how to do something are they willing to put in the time to learn?
It seems to have served us well. I am always surprised when I read stories like this where candidates are turned away for not being able to build a quad-tree on a white board in 5 minuets whilst hanging upside down by their toes and repeating the alphabet backwards...
I find it much easier to just study those problems when I interview versus complaining on the internet. The bar is damn low for most leetcode problems and refreshing is not that hard.
What bothers me about all these is how irrelevant they all are to actual day to day programming on a project. Like okay cool you know how to do the Fibonacci exercise. What now?
Do you know how to write SQL queries? Have you ever mapped an ORM to a data store? What about rate limiting an API? Ever had to setup Single Sign On? Have you ever scaled anything from 100 users to 100,000 users? How do you handle job running? Concurrency? How would you debug your server locking up due to 100% CPU usage? Ever configured a dev environment on local with xdebug? Breakpoints? Command line tools? Ever normalized data between multiple third party APIs? What does normalization even mean to you? What about unit testing? Mocking data pre-test and cleaning up after test? How do you make it fast? What's the autoloader? How do you properly setup composer? Tabs or spaces? Why? Windows, OSX, or Linux? Why? Favorite IDE? Why? Are their opinions so strong they can't work with others? On and on it goes.
In the past 6 months at my job I've dealt with all of the above issues and more. I would still probably fail most of the exercise in your Github repo. Mainly because all of them are irrelevant to the actual work and honestly I can't be bothered to sit here and memorize various math algorithms that have quite literally nothing to do with the work.
PHP comes with with 13 different sort functions. None of them test you for how to write a bubble sort. I'd rather a candidate know when to use one over another instead of knowing how to write just one or two of them from scratch. Here's the thing though. I've written bubble sorts before. In college. When learning C & Java. In no way shape or form do I ever practice them or actually remember them at a moments notice. I look them up like a normal person if I ever actually need them.
When I interview a candidate I don't use any of these nonsense exercises. I have one small code exercise at the end which makes use of recursion. The rest of the interview is an open ended series of questions on systems and just basic "shooting the shit" style questions. The nuance of how you answer the questions tells me everything I need to know about a candidate.
On the flip side if I'm the candidate and someone asks me one of these I know they actually suck at interviews and my immediate instinct is to walk out. I try anyway to be polite and not burn any bridges and most of the time I'll come up with a correct solution but again my initial instinct is "I don't actually want to work here anymore".
So what do you do instead? Be creative.
Have fill-in exercises. Write an abstract class and an interface and have them build you a class that implements both.
Write some code with obvious mistakes. Have them fix it.
Build a full from scratch login system. Have a user already in the database with a hash already there. Make them fill in the authenticate function. Watch them not use password_hash and ask them why. Make them use Google.
No exercise should take more then 15 minutes. The majority of the interview should be you digging deeper into their previous roles. Have them tell you success stories or cool hacks they've put together. You'll learn way more about them both as a person and as a developer.
I feel like interviews are more of a legal IQ test than anything else. If you are smart and spend a couple weeks grinding LeetCode, you should be able to solve most of those types of problems pretty quickly, so those types of interviews filter out those who either aren't willing to spend time on LeetCode to get the job, or those who aren't smart enough to quickly match up the interview problem with the ones they have seen before. And of course it also filters out people who get really nervous during interviews, or who just happen to get a problem they haven't seen before, but if you are a big tech company you can generally afford false-negatives. IQ is a pretty good predictor of job performance, so from the employers perspective it makes some sense imo, and it also avoids a lot of room for bias accusations.
Ignoring all the false assumptions you make about IQ, the couple of weeks I could have spent grinding LeetCode I instead spent applying and interviewing at other companies. Maybe I could have had 3 rather than 2 offers by the time I made my decision if I didn't ignore Company X, but I also might not have, as my focus on LC would have taken away focus from preparing for interviewing with companies who know how to hire engineers.
In my experience, a company using LC usually means that, if hired, 95% of your coworkers will be young 20-somethings fresh out of college. Generally, that is who has the time and lack of self-worth to put themselves through this process.
But the often-mocked LC interview wasn't as prevalent as I thought it would be. As a senior+ engineer I generally saw some system design whiteboarding and some take-homes, with a lot of engineer-run companies proudly proclaiming during the interview process that they don't interview that way. So I get it, LC is the standardized test for college grads entering the real world who have no professional experience to demonstrate. And that's fair, you gotta vet junior engineers somehow, I guess.
The one company that asked me straight-from-the-book LC-style puzzlers was an established SF-based startup that opened an office locally. Luckily it was a puzzler I had already watched someone solve on youtube and had memorized so thanks for testing nothing about my technical ability I guess.
At the same time big tech companies are always complaining about not being able to find qualified engineers.
A portion of those people who were filtered out because they get nervous during interviews or got stumped by a riddle or esoteric question, could have been entirely qualified to be a good or great engineer.
Big tech companies should be designing interviews that don't find just people who are awesome at interviews and riddles.
The code test is one of the more annoying facets of interviews. On the hiring side, it's an efficient way to screen and vet candidates for basic skills. It's relatively low cost to the employer to have a recruiter issue the test.
If you come up with your own in-house code test, it still requires time to evaluate and grade the submissions. There's also a fairly large cost to creating an in-house code test (I know from experience). You have to balance complexity so that it requires enough skill to pass, but isn't so onerous that it puts off too many candidates.
Using something like LeetCode removes all the cost from the employer side. Someone else maintains it for you, and you get a canned score at the end. It also depersonalizes it, and provides an opportunity for people to game the system by memorization. The same applies to using canned whiteboard/algorithm questions.
I think most of this is a colossal waste of time. It turns away people who don't want to spend hours (or days) on each interview. Imagine applying for even a handful of jobs where each one expects you to do a take-home test that can take a full day. Who has time for that? There's the "if they really want to work for us, they'll take the time to do it" argument, but let's face it, most people are not that passionate about where they work until they actually work there.
As a candidate, I am loath to spend my time on these. As someone who's been doing this for 30 years, I'm also a bit insulted by having to do the monkey dance over and over again. At this point in my career, I don't bother applying to places like this. My resume speaks for itself.
Of course it makes sense to vet the technical skills of candidates, but you can learn a lot more from 15 minutes of questions than you can from a code test. If you present a candidate with some abstract questions and a couple of targeted detailed technical questions, you can tell if they've got the right problem-solving mind and if they've had experience with the technology you're working with. It's also a much more personal experience.