Hacker News new | comments | show | ask | jobs | submit login
Things I Learned from a Job Hunt for a Senior Engineering Role (fuzzyblog.io)
772 points by fuzzygroup 7 months ago | hide | past | web | favorite | 741 comments



To take a more nuanced view, I think there is an important distinction, frequently lost, between "can't code" -- which is all too common in practice -- and "can't easily code a stream-of-consciousness solution to a synthetic problem unrelated to anything I've ever built". Or its close cousin "can't easily code a toy solution to your toy problem since I've only worked on massively scalable versions of the same problem". Something I keep in mind when interviewing candidates is that there is that good understanding of the general design of algorithms does not imply that they should be able to throw together an implementation of an arbitrary algorithm with minimal effort unless they've needed to do it in the recent past. Expecting people to spending many days practicing the implementation of arbitrary and often irrelevant algorithms is unreasonably in my opinion. I am more interested in figuring out if they could given adequate time.

A valuable practice, which surprisingly few tech companies do, is to ask candidates about diverse and orthogonal problem domains, or alternatively, allow the candidate to pick from a diverse set of problem domains. In the former case, you often find that candidates have difficulty writing even simple solutions for some problems but on others they instantly and fluidly can code up a good solution. Because they've had to do it in recent memory and still have the "muscle memory" from doing it previously. You often see bipolar results across the problem set this way.

For senior engineers with deep domain expertise in an area, there is an additional trap in that they use more sophisticated and often very different algorithms in their day to day lives than are applicable to the toy problem domains. Graph algorithms are a good example of this, and they are popular in interviews. The representations most engineers know (e.g. adjacency list or matrix) don't scale but the extremely high scale algorithms operate on a different set of principles that aren't trivial to code and aren't relevant to non-parallel cases; coding up an adjacency list graph traversal algorithm is going to be very unnatural to a software engineer used to doing the same on trillions of edges in real-time. It would be crazy not to hire an engineer on this basis but I've seen it happen, ironically because their expertise caused them to show poorly on the coding exercise.

Interviewing for technical skills is intrinsically difficult but I think that as an industry we are much worse at it than we could be. For all the claims of companies that they only hire the "top 10%" or whatever of engineers, the interview process is often optimized to the benefit of the median engineer.


Most engineers would not hire themselves. That has been apparent to me for awhile now. I’m not sure why they expect people to be to be better than they were when they were hired. I don’t expect engineers to be better than me. I have but one qualification. Can they do the job? Are they strong enough that I can guide them into the position I need them at if it is required.

So much focus has been put on 10x this and high performing that. Companies waste lots of engineering talent and company money looking for 10x when 10x is mostly situational in nature. Set the scene. In one scene the engineers are not producing. Change the scene and now they are 10x.

Instead of looking for 10x, that time would be better spent making incoming and current engineers better at their jobs and creating 10x situations.


> Companies waste lots of engineering talent and company money looking for 10x when 10x is mostly situational in nature. ... In one scene the engineers are not producing. Change the scene and now they are 10x

I can relate to this. I have pretty good resume - good schools, advanced degree, impressive sounding projects, a long list of publications. My track-record suggests I am at least a 3x engineer.

So when I get into a coding interview, I'm expected to perform as such... but I usually come off as a 0.3x engineer. Like really bad. Forget syntax bad. Forget algorithms bad. I've bombed so bad in a technical interview, the interviewer questioned if I even knew basic trigonometry. They had spent 10 hours interviewing me prior and felt good, but got me in person and suddenly thought I was an idiot.

In each of my jobs, I've had a considerable ramp-up period. I'm talking 6 months or more of feeling completely lost and unproductive. I get in my head and it makes me perform worse.

If I'm not fired in that first period, I'm lucky. Only after about a year on the job I can perform at the level my resume would suggest.


Forget syntax bad. Forget algorithms bad.

I regularly look things like that up. I am a builder and a problem solver, not a reference manual.


I love to flip that around in an interview before it even comes up, one of the 3 things I start out any interview with is:

"I'm your Google, anything you'd need to look up just ask me and I'll give you the answer".

Great way to take the pressure off plus it gives you a good idea on how well they ask questions/are comfortable leaning on others. It's a huge plus if I can get a reasonable candidate to admit that there might be parts of the problem at hand they don't understand.


This is fantastic. Often I am wondering should we start test how good engineers are in asking questions for search engines. I am not going to write the perfect compilable code for them, I have a tools to tell me if I forgot the semicolon in line 31. I don't need to remember standard library 100% and recall all functions' signatures - I have a docs for that. But I should be able to ask meaningful questions that get me closer to the answer and I should do it efficiently. We have tools now, we should start using it instead of memorizing things.


I love this! Thanks for sharing!


"I'm not a reference manual." I'm going try to remember that to use in an interview some time when I don't know something off the top of my head, and they get in my face for it.


You probably wouldn’t want to work anywhere where people get in your face for not knowing something off the top of your head. The three phrases I wish that I heard more frequently at work are “I don’t know”, “I’m not sure” and “Do you have any ideas?”


Well said.

I've written code tests where I'm expected to read the code, find the issues and fix them on paper. But when they take it too far beyond the spirit of what I believe to be a reasonable test of understanding and fixing code, then I write that I'd just for example compile the program and look at the output.

The companies that are most into questions like these seem to be the classic (failing agile) body shops with ridiculous turnover of their talent, which causes them to have to interview even more, which causes in turn for their tests to become even more abstract and worthless.


I have a long text file that I copy and paste out of that has pretty much everything I ever do from read/write from databases to displaying data 90% of everything I do is a variation of the same thing.


Forgetting syntax shouldn't be a problem. That's what manuals are for. Of course, if you are applying for a Java programmer job and you don't know the first thing about how Java program looks like, it's a no-no. But if you forgot some detail about how to write some obscure construct - that's what manuals and SO are for.

6 months to get into things is kinda long though. Unless it's super complex things, it should be less. Are you sure you are not exaggerating?


Even after hiring, it's not uncommon to be asked "Do you really know . . .", especially after forgetting a detail in discussions.


6 months? That's terrible.

Most engineers can be productive in a month. The 3x engineers take half that.


There's a difference between "feeling unproductive" and actually being unproductive. The person you're responding to said the former, not the latter. And in my experience it's nearly universal that when you ask excellent and very senior engineers "how long did it take you to feel ramped up?" they answer "I still don't!" despite having been there for months/years.


Had few jobs in low level system codebases in the millions of locs. I've had 9 months probation periods.


Nine months is insulting unless you guarantee me a VERY generous severance if things don't work out.


So you weren't productive during that entire time?


There’s a reason people refer to it as “ramping up”, as you become familiar with the environment and code base you become more and more productive until more or less plateauing a while later. It’s not really useful to categorise it as binary productive or not productive.


So you are telling me you never qualified at a workplace with longer than a month probation period?

Do you see how that putting-words-in-others-mouths looks? It's rude and does not contribute too any kind of productive discussion.


[flagged]


I'm wondering if you find it ironic that you feel attacked in a thread you started with an attack on me?


Depends entirely on the system you are working with.


>I don’t expect engineers to be better than me.

I respect your opinion but I find this strategy very strange. (Or maybe I don't exactly understand what you're saying.)

In my view, it would be a dream scenario if the next 10 programmers I hire were all superior to me. If I was the worst programmer on the team, that would be an ideal outcome. Yes, I've been programming for 30+ years but I'm also self-aware of my limitations. This gives me the ability to recognize better programmers and learn from them. If I'm not hiring interns, I do expect (or at least hope) that they are better than me.

Am I competent enough to write a "do/while" loop and knock out FizzBuzz in 5 minutes?!? Sure, but that doesn't mean there aren't better programmers than me. I would consider it a great business skill if I'm able to consistently attract superior programmers -- either to work with me or for me.

Larry Page attracted programmers better than him (e.g. Jeff Dean). Yes, Larry earned a masters degree in computer science but his java programming (the rudimentary 1996 crawler) wasn't considered very good. His employees rewrote the whole thing in C++ and made it scale for billions of pages. Bill Gates hired superior programmers. Mark Zuckerberg hired superior programmers. Etc. etc.


You actually agree with each other. You're saying "it would be a dream scenario if the next 10 programmers I hire were all superior to me"; it's an ideal outcome, even if unlikely. The parent's "I don’t expect engineers to be better than me" merely points out the unlikeliness, you point out the desirability.


When I was at one company (later acquired by Google), one of the developers created a model to show just how unlikely it was that every new hire would continue to raise the average level of competence for the team as a whole (as our hiring traditions espoused). After years of hiring very good programmers, it just wasn't going to be possible to hire many more who could pass that test.


Could you elaborate on the hiring process? Was it calibrated to hire people more competent than X% of the current engineers?


>You actually agree with each other.

It doesn't seem like it. The next 3 sentences from the thegayngler:

> I have but one qualification. Can they do the job? Are they strong enough that I can guide them into the position I need them at if it is required.

In my mind, programmers superior to me have abilities that would be beyond my knowledge to "guide" them. They can come up with programming solutions I could never think of. It's very realistic that "the qualification to do the job" essentially means they are a better programmer than me.


> In my mind, programmers superior to me have abilities that would be beyond my knowledge to "guide" them.

Presumably, in your mind, "programming" skill is along one spectrum and one focus, as well? I think that's part of what's being talked about here. There are many, many aspects to software engineering/programming (depending on whether you want to classify them differently), and many roads that could have been taken to get to the point someone is at. I see no reason to think that someone that is better or worse than me in one aspect will also be so in every one of the other aspects of the job.


>There are many, many aspects to software engineering/programming [...] I see no reason to think that someone that is better or worse than me in one aspect will also be so in every one of the other aspects of the job.

Let me try to restate that in more general terms and you can tell me if I interpreted it correctly: Since programming skill is multi-dimensional / multi-faceted, and it is unlikely for a programmer to be superior in _all_ dimensions compared to another, it is therefore not possible to conclude if one programmer is superior to another.

Is that a fair rewording?

My response is... sure, programming skill can be looked at as multi-dimensional. But so can other real-world comparisons. We prioritize the dimensions that are more important and collapse all aspects into a composite score.

Multidimensions such as choosing a new job: Company X is a 5-minute commute with 4 weeks vacation but Company Y has the newest technology and has lucrative stock options. Yes, it already goes without saying that there's more than 1 aspect of "desirable working conditions" at each company and no single company can claim a superior score on _all_ aspects. We can still weigh all aspects and eventually pick one (and only one) which is superior.

If I hired a "Jeff Dean" type of programmer, it would be an upgrade to my team and my company. Could I search for an "aspect" that I'm better at than him? Well, looking over his expertise[1], it looks like he doesn't have much frontend GUI programming experience. So yes, I guess I could claim superior skill on writing event handlers in C# language for buttons. That's not important though for my assessment. My point is I don't need for programmers to be superior to me in every aspect for me to determine that they are superior overall. (Same as determining which company is "superior" to work for.)

If I was founding a startup and I was the "ceiling" of programming skill instead of the minimum "baseline", my company would fail. I think I'm pretty decent at programming but I have no Dunning-Kruger delusions that I can "train" anybody that can pass FizzBuzz into a "Jeff Dean". I know of no 6-week bootcamp or even a 6-month corporate training program that can claim that feat. What I can do is recognize the programming skills in others that are (overall) superior to mine and hopefully convince the candidates to come work with me.

[1] https://scholar.google.com/citations?user=NMS69lQAAAAJ&hl=en


> Is that a fair rewording?

Well... no. But I think that's my fault, for the most part. I read the part of your original comment that said 'In my mind, programmers superior to me have abilities that would be beyond my knowledge to "guide" them.' as implying all their abilities as being beyond you, but it doesn't specifically say that.

I wasn't trying to imply that it's foolish to try to ascertain if one programmer is generally better than another (as long as some common domains along which to measure are involved), just to note that it's possible to hire someone that's far beyond you in some areas, and behind you in others, with the intention of specifically helping them build in the areas they are lacking (which have application to the job). The result should be a programmer that is, generally, as good or better than you in most areas, making them a more obviously superior programmer, even if that wasn't entirely obvious at the time of hiring.

In short, you can hire for the things that are important that you can't easily improve while allowing deficiencies in the areas that are less important which you feel you can help them improve in.

As a strategy, it has it's pros and cons. As a pro, it's probably easier to find candidates since your criteria is less stringent. As a con, you might find the candidate you hired has problems coming up to speed in those areas they are deficient in (which may be why they were deficient in the first place, instead of lack of experience).

I didn't really express the point very well originally, and just stopped at the opening, probing question.


Of course you want the best you can get. I’m arguing 10x is constantly influx for any number of reasons. So it’s unlikely you’ll get one. If you have a process for making engineers into the best then you don’t need to look for the unlikely 10x because you can make them into the 10x high performers you need.

Ive seen companies literally turn good engineers into underperforming engineers...stifling them with process, tech debt, politics, etc... Then they get angry some people can perform and others can’t under the circumstances and conditions the management created.

I’ve watched perfectly good engineers get fired or forced out for not being miracle workers only for people to later realize there was nothing wrong with the engineer in the first place.

You don’t know who is going to be the best until you work with them. I would advise people to pick people they will like even under the worst of conditions and train them to the way you want them to be.


I have observed exactly the same thing. It's truly mindblowing.

I also find it remarkable how quick people are to blame the engineer, rather than considering the whole picture (process, tech debt, politics, etc). It's a complicated equation with a large number of variables.


I recall one of my friends, when I was at Google, sending me a screenshot of code written by Jeff Dean with an obvious bug. Jeff is, of course, extraordinary, but I wonder how many companies might have passed on him if he submitted that code in an interview setting.


Absolutely. There exists no single human being that does not make any mistakes. The trick is to find the people that make as few as possible, know how to track them down, and that can figure out how to hot-fix them as needed.


This anecdote made my day. Thank you for helping assuage my Imposter Syndrome.


I believe he is referring to the case of the interviewer being a peer rather than a manager.

In my experience the manager cares about fit and attracting you to the team, whereas the peers are all looking for reasons NOT to hire you.


Java in 1996 was version 1! Doesn't necessarily say much about Larry'd code-foo :)

Jokes aside; I agree with the advice of hiring people smarter than you in the domains where you won't be paying attention should the company actually take off.

If you're looking to be a founder/CEO level type, doesn't seem to matter what your skill in Java and OOP are, yes? No need to be a water walker.

That's the "genius" of management; they sell an idea and get someone else to do the work.


I noticed that quite a few programmers have the following attitude: If there is something (a programming language, framework, library, paradigm, design pattern, tool...) that they know and use, then they believe that familiarity with it is essential to call yourself "senior". On the other hand, if they do not use it, because they do not know it, they believe that it is useless and anyone studying it is just wasting their time. They may not be aware that this is the algorithm they are using to evaluate others.

Imagine a programming ecosystem with e.g. 5 approximately equally popular frameworks. Suppose that someone with this attitude is familiar with frameworks F1 and F2, but never used F3, F4, F5. Put such person in charge of an interview, where equally skilled candidates (i.e. knowing 2 of the 5 frameworks) apply. They will only classify 10% of them as "senior-level" (those knowing F1+F2), and additional 60% as "somewhere between junior and senior level" (those knowing one of F1,F2; and one of F3,F4,F5). So they would hire literally themselves, but probably not someone on the same level.


Most engineers would not hire themselves.

IIRC a Google recruiter once sent a set of packets to a hiring committee and all of them got a "no hire". Then the recruiter revealed the packets were lightly-anonymized versions of the actual packets of the hiring committee members, from when they'd all originally interviewed.


Agreed. I wouldn't even mind so much if they picked "better for the job". So often, it seems like the criterion is "better at doing things that never actually come up in the job".


It has to do with companies looking for people to be off the shelf cogs/resources instead of having a plan for training & growth.


Agreed. They say here’s your $250 training budget and leave you to it...thinking they really did something.


> Most engineers would not hire themselves... I’m not sure why they expect people to be to be better than they were when they were hired.

I wonder how much of that comes from being exposed to all of the consequences any poor decisions they made shortly after they were first hired.


>>creating 10x situations.

This is so correct.

Productivity follows when people are working for a worthy cause. In fact the core of productivity isn't lists, management techniques or whatever, it is a thing called 'constancy of purpose'

There exist no such a thing called 10x burger flipper.


"A players hire A players and B players hire C players" - Steve Jobs (Maybe?)

If you don't encourage a culture where everyone is asked to hire people better than themselves, it can easily degrade where each progressive hire is slightly worse.


It also entirely depends on your career path. Mine didn't involve programming outside of shell for most of it, and involved tons of deep expertise on a variety of topics.

I was shocked doing an interview in my last round and got so many test questions. I bombed so hard because it wasn't what I was doing.

The real key to me is, we treat all of these jobs as more or less the same, when in reality they are all vastly different. There is no singular senior engineer skill set, but a variety.


"There is no singular senior engineer skill set, but a variety." Amen!

I wrote a whole blog post on that: http://www.mooreds.com/wordpress/archives/2812

I just wish companies would be more upfront about what they really needed, as opposed to the wishlist.


"There is no singular senior engineer skill set, but a variety." That is extremely well said. Thank you.


Also, expectations of coding tests should be upfront. E.g. we're looking for the most optimal solution, not the one that is most convient.

I was declined for a position at a trading shop for a C# backend role because I used linq to do something. I didn't have unlimited time for the tests, so I went with expedient over performant, for which they did not care. Fuck that, set clear expectations for interview "tests".


Were you able to talk to them and ask questions during the test?


If you tell me "here, do this, you have 2 hours" I won't even think about the possibility that you want anything that isn't quick to write.

If your priorities aren't aligned with the environment you set up, you are the one that must say so. On a real world project that means once in a while I'll take a project back for small fixes, instead of wasting unending hours thinking about every possible problem up-front... if you expect something different, well, that's not optimal.


You also need to ask about resource time and money budgets here some little start up is not going to have the resources a tier one TLA will have in its black data centre.


> For all the claims of companies that they only hire the "top 10%" or whatever of engineers, the interview process is often optimized to the benefit of the median engineer.

This really seems out of necessity. There are going to be a lot more qualified median engineers rather than superstar coders, so it makes sense to optimize your hiring pipeline to assess these folks.

In my experience, most of the "top 10%" hires are either through interpersonal contacts or a fast-tracked interview process for a specific candidate.


I can just imagine a room full of top 10% engineers/architects...No we need to do it like this...Well, no did you think of this obscure use case... what about performance? I think...should we...fast forward a year later and the company went belly up because everyone was trying to be best and piss on each other to prove who was top...top.

At least when middle managers do it they're only interested in who gets credit for the work so they leave the average tech guys alone long enough to get the work done.


OTOH, I've had the good fortune of working in a team of top 10% engineers/architects (I'm not top 10%, btw) who worked harmoniously, and it was one of the best times I ever had in my career. Learned a lot, upped my level trying to keep up, helped create a good product, etc.

In that case, the key was that clumps of them had already worked together in earlier jobs, so the incompatibilities had been pruned.


I've worked at a company that generally hired extremely good engineers - much better than average and they were very successful because (1) doing that was actually important to the very hard problems they were solving, and (2) every other company in the world wasn't trying to take the same approach to hiring. Today, everyone thinks they should hire like google, even though they're building a mid-sized web or phone app.


That's not a problem, the problem is they want to hire like Google yet they do not provide equivalent level of compensation.


How did they interview these engineers to hire them in the first place?


Really hard puzzles that you had to submit a solution to if you wanted an interview. This was very rare in those days. The problems were original and clever, not pulled from a book or website. Also, the company was in Cambridge, MA and had very close ties to MIT so they could find people like that.

If you got an interview, and did well you got another challenging programming problem to solve on-site. You were given a computer with the programming environment of your choice and as much time as you wanted, as well as someone to ask questions from.

I think it was also important that the company solved hard graph problems in Lisp, with a lot of macro-driven code. If you were a Lisp programmer, you generally wanted to work there.


That room sounds like every room at Netflix, except it works just fine.


Top 10% engineers aren't just really good at tech, they're really good at working productively in team environments and delivering products.


Someone being smart does not mean they are going to be an architecture astronaut or that they will be unable to cooperate with other people (especially with people who are roughly as smart).


People with top competency rarely go out of their way to display it. Are you sure you are correctly assessing those people?


Also depends on what they're counting in their percentiles.

You can pretty easily hire the top 1% of applicants, and still get only mediocre results. E.g. at one job, when looking for a senior-level eng position, clearly stated in the description, we easily weeded out a couple hundred applicants who had effectively zero experience whatsoever. No prior programming-related jobs, internships, side projects, open source, nothing.

Which is a waste of time for everyone involved. Hence the heavy reliance on connections :(


Maybe companies only hire the "top 10" of their applicants and assume that this means they only hire the top 10 of engineers.

In reality, the applicant pool is heavily biased towards very bad engineers.


Hmm, maybe we should 'equalize' the playing field then: Give everyone something from left field that pretty much all people bomb, that'll see how quick they learn.

Exp: You must code in an unfamiliar language (Fortran 1988, for example) and do something simple in it. At least then you'll know that people are all starting from scratch. Time it, make sure it doesn't last more than 2 hours, see how laughably slow we all are, pick those that get the furthest.


I've worked somewhere where we used this approach for one of the 5 interview sections.

Even being upfront about our low expectations and providing very friendly and helpful people to pair and guide, it lead to record numbers flipping the table and stomping off premises before the completion of the interview.

I don't use this approach now, as I don't think its a good use of time (and I think an interview should be a positive experience even for those that fail), but it did provide, I believe, a fairly strong signal of the attitude with which a candidate approached learning new things (but not really anything else).


Wow, alright, this is exactly what I was thinking. The 'novel coding' test was just one test, that it was forewarned to be laughably hard, and that you all were friendly about it. I really do want to know more about how this all went as I thought this would be a really good way to interview coders, but unfortunately I am totally wrong. I want to know why my thoughts are totally off-base.

Really, please, tell us more!


> I thought this would be a really good way to interview coders, but unfortunately I am totally wrong.

It's not that you're wrong, it absolutely is a really good way of finding out how someone reacts to being asked to learn something new in a stressful situation. And that is a valuable thing to know - I love working with people who get excited about an opportunity to learn something.

It's just that you have to be prepared to give so much help and the question has to be so easy because it's all completely new that you can't test competence in any meaningful way. Which means that at the end of the 90 minutes or whatever, you know exactly one thing more - their attitude to learning under pressure, but you have relatively little time with them, and you probably need to learn multiple things per session.


Maybe this cockamamie way of doing interviews is the correct one then? Each company tries their own thing, and if you get flustered and bomb it, we'll both parties know it not a good fit. Some companies value the ability to learn and be cool under fire, some like GPAs, some like HW tests. The sorting occurs and is not 'fun' but works in it's own way.


I like to pose technical problems that can scale to the candidate's level of experience. If I sense I'm hitting the candidate's comfort level, I can back off. Otherwise, I can take the problem even deeper with more technical questions. Either way the candidate still has a positive experience.


This is my approach as well. Ask really easy questions and get more complicated as they do well. I also try to increase difficulty in line with the strengths espoused on their resumes. If they are fantastic with databases, I might make sure the problem can be solved well with a database oriented solution.


> Even being upfront about our low expectations and providing very friendly and helpful people to pair and guide, it lead to record numbers flipping the table and stomping off premises before the completion of the interview.

You also have to be careful when in the interview you throw this. A bad hit like this can throw a candidate off for the rest of the interview even if they are quite acceptable.


"Give everyone something from left field that pretty much all people bomb, that'll see how quick they learn."

The Kobayashi Maru test.


I hadn't thought of it that way, but yes, I think the KM counts


That would still not level the playing field for several reasons:

1) Some people may find Fortran easier to understand than others even if they haven't touched it before. This was the case when my University decided to use Haskell* as the first language to teach students in order to level the playing field. It did help somewhat, it leveled the field for those with little and those without any prior programming knowledge. But to those who had already come to terms with recursion etc it was a very tiny stumbling block.

2) Some people need to dive deeper and gain a greater understanding of things before they feel at ease using them, but once they have reached that level, they will often outperform most other people.

* This was 20+ yrs ago and Haskell was not so well known back then.


I like that 2) - i feel i have to dive at least one "level" below what i actually need else I feel like a fraud.

nice to see it's more common


I’d probably pick the ones who interrupt the coding exercise, tell me how ridiculous it is, and tell me about the ways they’d solve it in plain English / pseudo code.


> You must code in an unfamiliar language (Fortran 1988, for example)

Oh wow, that would definitely get me sweating. All the FORTRAN i learned was 77!


The joke is, that there is no FORTRAN 88, unless you count some prerelease version of FORTRAN 90. ;)

https://en.wikipedia.org/wiki/Fortran


I love this idea. Testing how well someone can learn new skills can be just as important as the skills they've already learned. However, the problem is that as soon as it becomes known that Google is giving tests in Fortran 1988, applicants will study Fotran 1988 and thereby reduce the efficacy of the test. It will also have the sad side effect of incentivizing people to waste time studying Fortran 1988 rather than building skills useful to their job. It seems to be a good interview needs to satisfy the following properties:

* Predictive of job performance

* Either (1) difficult to optimize against or (2) easy to optimize against but in a way that doesn't reduce predictive value

* Low cost to give

* Low cost to take


> However, the problem is that as soon as it becomes known that Google is giving tests in Fortran 1988, applicants will study Fotran 1988 and thereby reduce the efficacy of the test.

How about "Google is giving tests in these 25 languages"? They could even choose languages that are purposefully dissimilar from any language that you've claimed familiarity with.


Umm. Not if they only stay 2 years at your company. I think that's about average outside of ones with golden handcuffs.


Battle Code Royale


Co-coding with 3 other candidates. I love it. With the 'suicide' option of declaring all variables global.


What are the graph representations that are used at larger scales? They're not adjacency lists at bottom? Can you point toward some resources to read up on?


Would it be better to have the interviewer provide take home context a few days before the interview that will be related to the coding problems during the interview? Somewhere between the take home coding problem and the whimsical toy problem solving by during the interview?


I really don't like take-home assignments. It's a one-sided time investment. If I'm asked to do one I say that I'm only prepared to put in 45 minutes on it. Now that I've done this a few times and subsequently been rejected after submitting, I'm leaning towards rejecting assignments altogether. It doesn't really serve me well in job searching but I feel strongly about it. It's a bad investment on my time and the assignments never asses things I'm actually good at.


Same. Now that I am on the other side I send out a short piece of code and ask the person to point out all the errors in it. There are about 20. It takes about 5-10 minutes of the candidates time and I have had a lot of success with it.


Really wish more interviews were like this. My favorite onsite interview was when the interviewer printed out a class from the actual system I'd be working on (as I verified for myself later), asked me to figure out what it did, any errors I found, and a few ideas to improve or refactor it. Then he took my code sample and asked me to describe what it did as well.

After that he said "Okay, I know you know enough to handle the job, now I'm curious how much you really know. " (This is key, I feel. He specified I wasn't going to get dinged for not knowing the answers, so I didn't feel the pressure in the questions he asked afterwards, when I usually feel intense pressure, as I know I've been passed over for getting a single question wrong in multiple interviews in the past.) He then asked me some pretty deep questions about low level memory and other things, and if I said I didn't know he turned into teacher mode and taught me the concept.

I actually walked out of that interview having learned something new and useful. I then got to chat with the President very casually, and I received a job offer a few days later.

The interviewer new his stuff, too. He'd been programming arcade games in assembly for decades. His games have sold millions of dollars, and two of the series still get made today.

Ever since, I figured if a freaking legend could be satisfied after reviewing some code and a code sample, the crap I've had to endure everywhere else is completely and totally unnecessary, and it's just frustrated me to no end.


I had a very similar experience many years ago. It was an start-up. The interviewer was the ex-CTO of a widely used open source project/product, and led the engineering team in the start-up. Although I regarded myself a good engineer, I was pretty nervous.

After several questions that he was sure I knew the regular stuffs and was qualified for the job, he said he was going to ask some really hard questions to see how much I really knew and what was my thought process. The questions had no simple answers. Each of them required knowledge and experiences from a few technical fields. I didn't know much then. He guided me how to solve the problems, discussed the pros and cons of each approach. Then he told me how they did it, how other products in the same category did it, etc.

The half-hour interview was prolonged to one and a half. At the end, he said, "You have a very intuition on what are the right directions to go when facing unknown complex issues", and gave me an offer. It is my best interview experience. (I didn't join the start-up due to other reasons.)

The technologies have been advanced in a blazing fast speed i the last 10 plus years. Technical interviews, on the contrary, regress in a similar speed.


That's a unique approach or at least one that I haven't encountered. I like it.


Not a take home assignment, just prep material. Like saying, we will ask you about X, Y, and Z, here are some pointers. I would just worry the night before anyways, might as well put my anxiety at ease by prepping and building confidence.


Oh I see. Prep material would be great.


exactly. If the job and the pay were exceptional, sure. But today it's standard practice for average companies.

One company I interviewed with gave me a program that had to be written in Scala, with the full knowledge that I would have to learn Scala to do it. Please.


> ...the assignments never asses things I'm actually good at.

I actually really like homework assignment approach when done correctly, both as an interviewer and as an interviewee (though seanmcdirmid's suggestion of prep material, rather than a homework assignment, is quite sound), but this is a separate issue, independent of how skills are assessed.

It's really about the company/team having a preconceived notion of what skills they are looking for vs. looking to see if a candidate has a distinctive skillset that could help round-out the team. How do you, as the candidate, market yourself and the things you are good at?

Obviously, the resume is a good place to do such marketing (and I'm sure you already are) so if teams are turning down applications based on the homework without looking at the other skills the candidate has to offer, then that might be an issue.


Take-home assignments are a great way to filter out women, the sick, and the poor, but make it look totally above board.

"The IT industry is committed to increasing the number of women engineers. But if they're too busy caring for their children to spend six hours doing our silly test, that's their fault for being women. We are forced to hire 20-something white males who have nothing better to do with their time."


There are us guys who are involved in toddler care also (like me, since my wife is working instead). Anecdotally, my wife failed in all her in person interviews until she got one with a substantial take home assignment. She got an offer for that job easily, above what she was looking for.

For her at least, this worked out better for her. She does design, which is even more difficult to evaluate in 30 minute time slots.

Not that I’m arguing this is the way to go, I was just wondering if interviewers could give some hints on what they would cover I the interview, so we could prep like studying for a test.


> Take-home assignments are a great way to filter out women, the sick, and the poor, but make it look totally above board.

I hope your not impugning the motives of those who issue take-home assignments. Existing practices for hiring are objectively bad at selecting qualified candidates, and most people are just trying to do better. There is some evidence that take-home assignments are an improvement above the norm.

Also, consider that such assignments can serve as a "blind-audition" and can help eliminate bias from the interviewing process. And there are lots of ways in which loading everything into the interview unfairly penalizes certain types of otherwise qualified candidates. What's left, referrals? That's definitely biased, even if it also tends to have a lower false positive rate.


Motives are mostly irrelevant, if the effect is the same.

An easy switch would be to convert take homes to onsite work. One of the more recent interviews I had consisted of putting me in a room with some data and a stub of code and asked me to fill in the stub over a few hours. Every so often, they’d come by and ask if there were any problems. (More frequently at first, and then later less frequently.)

It’s basically the same, but it’s timeboxed and well defined with enough back and forth, to not feel like they’re just throwing a problem over the wall to you.


> Motives are mostly irrelevant

They are relevant enough that the prior poster felt the need to smear them.

> if the effect is the same

The claimed effect is, at this point, purely hypothetical and indeed runs contrary to my experience.

> An easy switch would be to convert take homes to onsite work.

Generally speaking, an onsite task like you describe seems like a reasonable choice for a company to make. However, this strikes me as worse for the affected groups that reaperducer was concerned about:

- It requires scheduling an even larger fixed block of time outside one's normal job/life responsibilities. A take-home assignment at least allows for flexibility for when you work on it. It strikes me that time-poor candidates would appreciate the flexibility.

- It would no longer be a "blind audition". When I've reviewed take-home assignments, I don't know anything about the candidate except that the code. It strikes me as a good way to resist unconscious bias from seeping in.


I think you’re reading malevolence where none is intended. Simply pointing unintended consequences is not smearing. While it is true that the effect is merely claimed, as I know of no relevant study, your anecdote equally meaningless.

The the on-site solution remains a blind audition if the people doing the evaluation are not the people administering the test. This is exactly the case for a take home assignment, only now it is timeboxed and with access relatively timely help if needed.

The purpose of the blind audition is not to hide the identity of person being interviewed, as much as it is to enforce an objective standard and remove bias. If the rubric is, “Accept all that got automatic evaluation score of at least X”, then that achieves a similar goal.

As far as time-poor people, never underestimate the desire of having a clearly defined block of time. Of course, onsite or not can always be optional as well.


> I think you’re reading malevolence where none is intended. Simply pointing unintended consequences is not smearing.

Pointing out (potential) unintended consequences is fine, even welcome.

Putting sexist remarks in the mouths of others is a smear, even if it is done merely flippantly.

> While it is true that the effect is merely claimed, as I know of no relevant study, your anecdote equally meaningless.

Anecdote is more meaningful than mere speculation. Even without statistically valid data, examples that are contrary to our expectations can bring to light factors that we had not previously considered.

> The the on-site solution remains a blind audition if the people doing the evaluation are not the people administering the test...The purpose of the blind audition is not to hide the identity of person being interviewed, as much as it is to enforce an objective standard and remove bias. If the rubric is, “Accept all that got automatic evaluation score of at least X”, then that achieves a similar goal.

A fair point.

> This is exactly the case for a take home assignment, only now it is timeboxed and with access relatively timely help if needed.

The point about timely help is definitely a point in favor of on-site work tests. But working in a timebox can also be stressful. A take-home assignment without artificial deadlines (i.e. don't schedule any interviews until after the assignment was received and approved) can eliminate that stress.

> As far as time-poor people, never underestimate the desire of having a clearly defined block of time. Of course, onsite or not can always be optional as well.

I think this hits on an important point, and one that I think would benefit from research and expertise I don't have. I'd hazard a guess that it depends upon why one is time poor.

If the candidate is intelligent and capable, but somewhat disorderly, then they would probably benefit from a structured, on-site task.

If the candidate is more organized but simply has a difficult and rigid schedule (which is actually something I would expect true, for example, of a working mother) then a take-home assignment might be better.

I don't know that either one is superior in all cases. Honestly, I think the best option would be to give the candidate a choice.


Blind audition is BS, because you're never ever comparing apples to apples, and frequently because the problem set is little like how you actually work. If the end goal is to find someone who can do the work well, be a pleasure to work with, and contribute to your organization or company in a meaningful way, we have not optimized for that. We've optimized to hire kids out of college who are experienced taking tests, or for people who have worked on the exact same stack you've worked on.

You will never be unbiased in a hiring process, because the bar you set is not universal, and always be biased in some way. Being "as blind as possible" isn't the right metric, and we shouldn't hire based on it.


>I hope your not impugning the motives of those who issue take-home assignments.

Yes, that's exactly what I'm doing.


It's highly personal preference I do much better on take home vs live code on the board in front of N people you never met before.


Geez I could not agree more. Not only has it, overall, been a bad investment of time (for myself and others I know around me in the field) but so many companies are unclear in what they're looking for in that test to begin with.

I wrote a thing about it and I got someone who responded to me with a rather scathing rebuttal. I don't know, it's very conflicting.

Post in case you're interested: https://dev.to/spirodonfl/should-i-accept-coding-challenges-...


Wow, some real dickheads in the replies there.


It's polarizing. Everyone has their own viewpoint and it feels like that's the viewpoint they cling to and so it becomes personal when you go against it. Something like that, anyways.

Edit: better wording


I feel like take home assignments waste less of my time than an interview that doesn't properly assess my abilities... They're also fantastic for junior devs who can prove themselves without extensive working history.


They’re also unfair because different candidates have different schedules. One may even be on vacation that day.


I'd expect that interviewee to request reschedule then. Isn't that the job of the recruiter to manage the interview process and ensure that requirements like this are being met?


Only if they care enough. Given the volume of applicants, they're incentivized to sift out applicants that don't adhere to the process.


What about a completely untimed take-home project?


I usually like sending take home assignment for graduates.And having a discussion about the approach taken, and do a peer programming session to add features.

However, don't see how well take home can work for senior engineers. In fact, its unfair and delays interview process to expect take home assignments from working folks.


I have a theory of why most of these changes in the job hunt came about: people became afraid of firing. I further theorize that this is an indirect consequence of primarily technical folks filling management roles.

How does a fear of firing impact hiring procedures? If you are afraid of firing, you're afraid that you won't be able to get rid of a toxic individual; that a single individual will act as a poison to the morale of your entire workforce. Thus, you want to make damned sure you don't make a bad hire in the first place - even when it means you leave a position open for a really long time.

This fear of firing has gotten so bad that FAANG-alike companies directly espouse how bad the impact of a bad hire is, how they would rather give up on 99 good candidates than hire one bad candidate.

But that's bullshit.

That's letting the fear of interpersonal interactions drive business decisions. The impact a bad hire can have on a work force is pretty minimal when caught and addressed quickly; it can even provide an overall boost to morale to know that the company is willing to address real problems in an adult way. Even a perfect initial hire can turn toxic after a few years: what will you do then, if not fire them?

Do you have 10 positions and 10 potential candidates? Hire all 10 folks, and fire the one bad person; your company will be better off for the decision. Much better off than it would be if you instead overwork your existing team because you haven't found your unicorn hires yet.


I agree that it's related to firing, but I don't think it's fear of interpersonal drama.

The simple explanation is that it's really expensive to fire someone without cause. It takes months because the company has to produce a paper trail proving they fired the person for a valid reason to protect themselves from litigation. That's months of paying an individual who may be incompetent or toxic, months of paying people who put together the paper trail, all the morale damage the individual causes, the damage that might arise from firing someone, even if it's someone no one really liked.

All of those things are costly, probably much more costly than not hiring the right candidate. At least so the reasoning goes.


The simple explanation is that many employers believe that. In fact I don't think it's that hard.

1. As another commenter pointed out, most jobs are in "at-will" states.

2. Many companies require arbitration instead of litigation to settle any disputes as a requirement of employment.

3. It's just as expensive for the fired employee, probably more so, since if you litigate, you're unlikely to ever work again.


Every state in the US is an "at-will" state. That is a US based vs non-US based employment distinction.


As much as I hate it as an employee, "at will" employment means that cost simply does not exist for 48 of the 50 US states (including all states where FAANG have employees).

The cost is also not that high in non-at-will states, since there's typically a 3-6 month "evaluation" period where no paper trail is required.


Not entirely true. Just because you're in an "at will" state doesn't mean that someone can't allege that they've been fired for belonging to a protected class. That's why companies have HR departments, PIPs, and all that other nonsense -- to protect the company when someone tries to turn a justifiable firing into a payday.


The courts are really not that friendly to the protected class in cases like that.

There’s a lot of political bodies focused on playing up the idea that women and minorities are this big bad wolf suing legitimate businesses left and right over nothing, and winning. But it’s a largely fabricated narrative created by people trying to turn the leftist identitity politics arguments into a quagmire by “both sides!”ing the situation.

In reality there’s a substantial burden of proof for someone to win a wrongful termination suit.


> Do you have 10 positions and 10 potential candidates? Hire all 10 folks, and fire the one bad person; your company will be better off for the decision. Much better off than it would be if you instead overwork your existing team because you haven't found your unicorn hires yet.

Even if you hire two do-nothings and a negative-performing person (consumes half an FTE's day in questions), you still end up with 6.5 productive people. And frankly, a 65% productive workforce is pretty damn good. I've seen lots of companies where <65% of people are productive.

Plus, some of the do-nothing people can be alright with some direction. Sometimes people don't get anything done because they just don't know what to do. I've been on so many teams where the manager said, "we need a person to do X" and they end up with some random person that doesn't really know how to do X. The solution that I've found is to find something they can do and have them work on that. Most projects have no shortage of work that's more time-consuming than difficult.

For negative performing people, I usually give them something kind of busy-work that doesn't involve the rest of the team. They might not be getting anything done, but they also (mostly) aren't weighing down the team by asking them questions all the time.


"For negative performing people, I usually give them something kind of busy-work that doesn't involve the rest of the team."

Why waste time? Why not just fire them?


Because you can get sued for firing them. Sued for multiples of their annual salary.


Then that's the heart of the problem, and it should be fixed.


How do you create laws that both

a) Protect a vulnerable class of people from being fired (say over 40 people)

and not

b) Require 30 forms when firring someone in that class for perfectly valid reasons

Seems really really hard to hit both. You need to err on one side or another.. which way do you go?


Document repeated negative performance or toxic behavior and then act on it.

There are plenty of people in non-vulnerable classes who regularly do indefensible things because they know their companies won't act on it.


Untie a person's ability to survive from their employment status. If we had universal healthcare and basic income then the necessity of most of the employment laws would cease to exist.


Not so simple. Is their suit likely to be won? Are the unemployed able to pay expensive lawyer fees?


In my case, I do not have the authority to fire someone.


Don't know why you're getting downvoted. Much of the language I get from bigco people about why they are so selective is exactly this; it's almost a confession that once in, they are very bad at managing performance.


The downvotes are probably because I've implicitly called out individuals who believe in - have perhaps even published materials encouraging - the "missing out on 99 good hires is better than making one bad hire" philosophy.

I've also explicitly called out the FAANG hiring practices, which are frequently held up as the best practices in the industry by those doing the hiring. After all, who wants to be even partially responsible for hiring "that guy" when you can not or will not fire them later?

But, here's my point: I've been part of the process that hired "that guy". I was also consulted on the decision to fire "that guy" a month later. I was wholly on board with the action in both cases; and in the process we got two other great employees that we never would have hired if we were afraid of taking chances.


Thanks for the details. I too have been part of that committee. I've also given the green light on people I could barely understand in free-flowing conversation due to language gaps based on the assurances of lower-level people that they are good and can get better. Like you, "I regret nothing". :-)

To me it paints a terrible picture. My suspicion is that the dangerous bad hires are not the utterly incompetent people who will flail in place until removed. The dangerous ones are going to energetically do all sorts of peripheral, irrelevant work, try to insinuate themselves into a lot of 'process' type setups, and move around the company in a way that stops anyone from ever noticing that they can't actually do anything. I suspect a lot of BigCos have very weak immune systems against such mobile 'attacks'.


It's in no way a fear of firing someone that's little easy. The fear is the weeks/months of issues before someone is fired. Further, many people that are part of the process will be working with this person, but not be able to fire them.


Is there also a fear of three people doing the work of four for weeks/months since there's the possibility someone could be a bad hire?

Is there not also a fear that an overworked person will become toxic?


> Is there also a fear of three people doing the work of four for weeks/months since there's the possibility someone could be a bad hire?

Apparently not at my employer. My team was 10 people 4 years ago, and we're down to 6 now, without much dip in workload. We had someone leave about a month ago, and we're still fighting to get a job rec to replace that person.


Downvotes might be from CMV-norm people[1] thinking "that's bullshit" is rude/inflammatory and unfit for HN.

[1] https://changemyview.net/2018/03/28/thats-bullshit-rude-enou...


Dont forget that in many European countries it is not easy to fire people. That is why the 6 months probation period takes place. However the amount of bad hires I have seen is immense. There are so many people coming into the industry because its hip or because its easy money with knowledge from an online course that I could not hire them.

They miss basics of OO programming etc. And then what? you take him in, which has costs time, money, hardware to let him go within a month? On the other side I think you can weed these out quite fast with 1 personal interview.


Totally agree.

Looking back, often no talent has been better than poor talent (or those that actually decrease group productivity). Huge opportunity cost with bad hires, and even more challenging to move them out to somewhere that they would be happier/more effective/less disruptive.


Supposedly risk should be probability times cost. But I think that people have a remarkably difficult time processing a risk that's based on a very high cost and very low probability. (The limiting case of an infinite cost and infinitesimal probability is how I describe Pascal's Wager).

In those circumstances, it's easy to imagine the mental math: Firing someone could cost many times their salary, but a screening service costs 50 bucks per candidate. So the chance of weeding out one bad candidate pays for the service, right? Where's the fallacy?

I think the fallacy is that the service doesn't actually improve the hiring process, and the ROI is zero.


I have a theory of why most of these changes in the job hunt came about: people became afraid of firing.

That in itself isn't unreasonable. While much of the US has at-will employment, most of the rest of the world does not, and hiring the wrong person can be extremely expensive.

However, I suspect with current hiring trends we should not attribute to malice that which can be sufficiently explained by incompetence.


Firing a person on a work visa is a seriously dick move, especially if they are from china or India, where the waiting lines for a green card is 3-10+ years.


If they're literally incompetent or toxic (i.e. "bad"), it's obviously better to find that out during the interview process and not hire them in the first place. The 10 "potential candidates" don't include anyone that the interview process detected as bad.

Interviews are imperfect, and damaging candidates will slip through the cracks. Are you seriously suggesting that a business should decide not to fire someone based on their immigration status, rather than their performance and integration with the team?


You can give them plenty of signals that it's time to look for a new job before you fire them.

Plus now there is a 60 day grace period for H-1Bs to try and find a new job.


That one bad person, before he gets fired, would destroy the productivity of 5 others; And his firing would make 3 others doubt the employer.


Number 2: No One Believes Anyone Can Actually Code

It's surprising to see the number of people who interview for lead technical roles that cannot code, or whose work is exceptionally sloppy. Incompetence is more commonplace than the author believes, even at the highest level.


Seriously. I've sat on the hiring side. I've seen impressive resumes. I've had reasonable discussions with people. And then I give them a very, very trivial coding exercise (a take home, they're free to Google, do it on their own computer, in their own IDE, in their professed preferred language), in a time frame that while constrained is still plenty...and the result is -terrible-.

I can try and come up with reasons why that might be the case, why a simple OO modeling problem + a couple of trivial algorithmic problems (like, take an ascii string, return me a dict mapping characters to character counts) would trip someone up...but that isn't sufficient justification for me to want to continue to an onsite.

And that's the people who get past the verbal screen. Plenty drop out at that point. "I see you spent two years as a Senior DBA. Can you tell me what the acryonym ACID stands for?" "No, I'm not familiar with that" "Okay. Well, the 'C' stands for consistency. Can you tell me what one means when we talk about data being consistent in the database?" "It means when you write something it stays written (or some other made up twaddle)" "I see."

That said, I agree that coding tests that are completely unrelated to the job, and are geared toward college grads rather than long term developers (i.e., "Implement a (data structure)" or "Remember/discover an algorithm that was a PhD thesis 30 years ago to solve a contrived problem in a theoretically optimal way" rather than "Solve a real problem of a kind similar to what we'd expect you to do here") are dumb.


I once got knocked out of the running by a whiteboard-coding interview question that went something like "How would you find all triples from a list of a million integers, where the first two numbers add up to the third?" I said, "Hmmm that sounds like an O(N^3) problem." Interviewer: "Can you think of any way to do it in smaller big-O?" Me: "Not off the top of my head, no."

For some reason that company insisted on only doing interviews at 7:00 AM, and my brain doesn't come fully online until after 9:00 anyway. I felt annoyed later when the answer came to me that afternoon, too late: put the list in a hashtable, add all pairs of numbers for O(N^2) complexity, and check if each answer occurs in the hashtable.

Much later after that, I realized that all the other red flags I'd picked up on while interviewing there added up to an impression that their corporate culture was seriously f'ed up, and that I probably dodged a bullet by getting passed over quickly in the process.


>"Not off the top of my head, no."

They were probably looking for the answer of, "Lets talk through it and figure out a better answer". Coders are often under a mistaken impression that interviewers care about your answer. They don't. They care about how you approach it, how you think through it, what you do when challenged, etc.

So from their perspective, they asked you to try harder on a problem, and you just said, "No." And they dropped you for it.

FWIW, I had similar questions in some interviews where we hashed through similar problems, and together collaborated down to a "No" answer... but we spent 15 minutes exploring the problem and seeing how well a couple coders can work through it before saying no. And I got the offer despite an otherwise rusty performance.


> Coders are often under a mistaken impression that interviewers care about your answer. They don't. They care about how you approach it, how you think through it, what you do when challenged, etc.

This isn't always true. Some interviews are really about solving the problems as fast as possible. And lots of interviewers are looking for exactly the answer they have on hand, and will think you're doing it wrong if you come up with a different, but equally valid (or better!) solution.


In which case, I don't think that's a place I'd like to work. Most often the solution an engineer comes up with is _not_ ideal and will be improved either because they come up with a better one later before they commit it or because they go through code review and someone else sees a better way. If you don't leave room for people to improve their solutions you'll just end up with the first, crappy one that comes to mind.


> They care about how you approach it, how you think through it, what you do when challenged, etc.

I was asked to sketch the proof for the irrationality of sqrt(2) in a quant programming interview. I kind of froze, and explained that though I had learned it, I could not recall.

He prompted me with "Well, what does it mean to be irrational?", and with that little hint I did the rest. Though ultimately I did not end up working there, it was very satisfying to have answered it.


A friend of mine didn't understand how he could have done better than me in a "write a quick sort" interview (we were still students) when I knew the algo and he never heard of it (and obviously struggled more).

Well, in one case (me), all the information the interviewer had was "this guy happens to know how to implement a quick sort by heart. K.".

In the other case (my friend), he got "this guy is able to understand and implement an algo he never saw before in a reasonable time", which is much more impressive


Errr in the set of numbers that complete the rationals wrt to standard metric? I guess you have to show it’s not rational so you probably did a proof of upper/lower converging bounds always having nonzero difference for 1/n increments or something? I would definitely not get this in a pressure interview situation.


Here's the proof from Baby Rudin:

> We now show that the equation (1) p^2=2 is not satisfied by any rational p. If there were such a p, we could write p=m/n where m and n are integers that are not both even. Let us assume this is done. Then (1) implies (2) m^2=2n^2. This shows m^2 is even. Hence m is even (if m were odd, m^2 would be odd), and so m^2 is divisible by 4. It follows that the right side of (2) is divisible by 4, so that n^2 is even, which implies that n is even. > The assumption that (1) holds leads to the conclusion that both m and n are even, contrary to our choice of m and n. Hence (1) is impossible for rational p.


From what I remember, it's simpler than that:

Assume sqrt(2) is rational and has the reduced form (x/y). Thus we are assuming that x and y are integers and that gcd(x,y) is 1. You square it and get x^2/y^2 which are somehow simultaneously coprime integers and also reducible to 2/1...


I think here they just meant, show that no rational number is the square root of 2. Not rational = not expressible as a ratio. You assume that sqrt(2) = p/q for some integers p and q, then derive a contradiction.


> Coders are often under a mistaken impression that interviewers care about your answer. They don't.

Huge asterisk that as long as you arrive to the correct solution with minimal help. I've had plenty of algo/ds interviews and it seems like needing help to see the trick in the question pretty much means you're out.


> Coders are often under a mistaken impression that interviewers care about your answer. They don't. They care about how you approach it, how you think through it, what you do when challenged, etc.

In many of the places that do claim this, even if you solve the problem "correctly" you still get more brownie points than someone who got a less efficient answer or didn't get an answer. Meaning that they /do/ actually care. I do get that some places are smarter with regards to this than others but it's depressingly common in my experience. "You didn't get the O(n) solution that has weird edge cases and was published as a paper in the 70s? Too bad, because some other person did, because they remember the solution from some programming interview book!"


Then I'd argue they need to make that more clear. The typical interview setting does not suggest it's an "exploratory" environment, it usually feels more like taking an exam.

At my last company, we would sit a candidate down at a pairing station. We'd offer them about 5 problems to choose from, and together we'd pair program on the problem for 2 hours. Using the internet, the IDE, anything they want was totally fair game. No system is perfect, but this was by far the best one I've ever experienced.


Had the same thing happen to me. Simple problem: find out if two strings are anagrams of each other.

My immediate solution: Sort the two strings and then compare them:

  (defn anagrams? [x y] (= (sort x) (sort y))) 
or some such. I was fortunate in that they didn't make me implement the sort (because it's been a long time for me) :)

"Ok, what's the efficiency of that solution?"

"Well, assuming the library sort functions I'm using are sane, I'll take a guess and say O(n log n)"

"Can you come up with a more efficient solution."

Off the top of my head, on the spot, I couldn't. Later, during the plane ride home, the obvious occurred to me: Don't sort the strings, just scan through each string once and build a hash table, keeping track of the number of occurrences of each letter in each string. Then compare the occurrences for each string. Not as elegant to express in code, but faster.

As it turns out, they later declined to hire me, giving me an excuse that made it clear they weren't really serious about hiring anyone for the position (as is the case at least half the time, it seems).

And thus ended my latest round of failed interviews. I cancelled the last one I had scheduled (probably another fake) and decided to take a break for a while (I mean, I do have a job right now, so it isn't urgent).


You need not to sort the strings. Create a vector with indices he ascii codes, incrementing for the first string and decrementing the count for the second, and keep a count of the number of chars, if you get a negative number exit false, else if the count of chars is zero then each one is an anagram of the other. (n+m+128) operations n and m are the length of both strings and 128 for creating the vector


Interviewer: "OK, how about with a unicode string?"


That sounds a lot like doing a radix sort of the strings, then comparing the run length encoded, sorted strings ;p


on a tangent, one way could be assign a number code to each alphabet. Add the numbers that occur in the strings. IF the sum matches, they are anagrams.


I don't see how the sum would be unique to a particular combination of letters.


Works if you assign prime numbers to each letter and multiply instead. So a=2, b=3, etc.


To get decent anagram lengths and complexity, implement the numbers as a dict of repetitions of primes, and implement the multiplication by summing the repetitions. ;-)


In which case you can just compare the dicts without performing the multiplication (which happens to be the costliest part for arbitrary-precision integers).


Exactly.


I briefly mused about just summing the ASCII codes for each letter in the strings. But quickly discarded the idea for this reason :)


“ad” = “bc”

You’re right.


What if a=1, b=10, c=100 - etc? Assuming the strings were English words...


You would have to make sure the sum of any combination of all characters was unique. For example, if the code was the character number, a=1, b=2, etc, both "abc" and "bbb" would have the same sum.

So I think something silly like: character_code = len(string)*len(alphabet)^character_index should work.


You need to multiply the numbers, (and they must be prime) not add them.


a=1, b=2, c=3, ...

ac = 1 + 3 = 4 bb = 2 + 2 = 4


I guess the numbers should be primes... 4+1=3+2


10 = 5 + 5 = 7 + 3


that's right... even prime isn't enough


Prime is ok if it’s multiplied (there is one and only one prime factorization of a number).... but that can get to absurdly large numbers.

Still, consider the word ‘abe’ with a = 2, b = 3, c = 5, d = 7 and e = 11.

abe would then be 2^1 * 3^1 * 5^0 * 7^0 * 11^1. ‘abba’ would be 2^2 * 3^2. Each anagram would have a distinct value.

See also Gödel numbering and FRACTRAN.


That’s right... stupid me, it’s the base of public key cryptography (doh)


def anagram?(a,b) a.reverse == b end


Or you could do top 100 questions on leetcode or hackerrank and you would have solved the question in a minute. It's kinda sad that you could remember the top 100 solutions and clear interviews in almost all big tech companies.


I recently interviewed at a big tech company (phone interview). I spent quite some time practicing on leetcode (completed at least 150 problems). During the interview, it took me a few minutes of thinking before completing the assignments with what I think was the expected solution. We discussed the complexity and a few possible variations. The interview sounded satisfied and I really had the feeling that I had nailed these assignments.

I wasn't rejected but I've been asked to retake a phone interview. Apparently, the interviewer had very good feelings about me, but found that I was "out of practice" as far as coding goes.

I wonder if my age is an issue (40+) or if they have a really high level of expectation. Are most other candidates super fast at solving these kinds of problems. I'm a bit disappointed because I'm not sure that I've got much room for improvement at that stage.


found that I was "out of practice" as far as coding goes.

They want 23 year olds who just did their finals. You know, real stallions.


And who don’t care that they’re going to get paid less than minimum wage, and work over 120 hours per week.

You know, real pegacorns.


I wonder if my age is an issue (40+)

The answer to that is almost always yes.


I really wonder how age is taken into account. If it were really an issue, they could have rejected my application from the start and not bother with an interview.

Maybe the interviewer was biased and led to think I was "out of practice" because he knew I was on the older side. But I'll give them the benefice of the doubt. I believe it's a competitive position and I simply wasn't in the right percentile to qualify.

Now, would my former self 20 years ago be more competitive for this kind of interviews? slightly faster perhaps, but I don't feel I'm at a disadvantage compared to younger candidates.


As someone that does lots of coding phone interviews I can say that yes, time is a factor. But it's relative, ie I'm comparing you to the time it took other candidates to solve the same problem. After all, we have to evaluate you, over the phone, in the course of less than an hour. If 2 candidates arrive to the optimal solution, the one that did it much faster is the better candidate.

It sounds to me like you did pretty well so don't give up.


> the one that did it much faster is the better candidate.

From my perspective that is only about 3% of what I expect from a software engineer. Writing readable, easy to maintain code that is well tested, collaborating with product and other developers for how a feature should work, influencing technical designs, giving good feedback are things I value much higher than how quickly someone can write a snippet of code that works.


> If 2 candidates arrive to the optimal solution, the one that did it much faster is the better candidate.

This is _a_ metric but I wouldn't bet too much on it. I know lots of people that come to optimal solutions to "algorithm type" _much_ faster than I do. I'm pretty slow at that type of stuff. But... its just such a small part of what makes someone a good programmer. Building out a medium to large application requires balancing a lot of trade offs and figuring out how to keep things simple. I know seasoned, quality algorithm solvers who honestly just repeatedly churn out garbage applications. And they can tank the productivity of an entire team of developers in their wake. I don't know how you test for that, but I can promise phone screening for problem solution time isn't it.


If 2 candidates arrive to the optimal solution, the one that did it much faster is the better candidate.

How did you arrive at that conclusion? Is it based on data or does it "feel" right based on an easily-measured metric?


the one that did it much faster is the better candidate

Translated: "the one that did it much faster is a more recent graduate and exploitable for 100-hour weeks at below-par pay, hire that one".

Congratulations, now you know how to hire at a tech company!


its kinda sad you think professionals should memorize useless tricks that dont generalize to the profession in order to be considered for a job.

The skill takes a long time to practice, and quickly evaporates once you stop doing it. Yet none of our work is anything like that. a more real world situation is reading documentation or SO for the function concat_ws that will turn an array column to a concatenated string.

however, the system that is, and one that you seem to think is ok, is one that discriminates people on many levels. Its a skill you do not get good at by working, so you must practice this in your free time. Now you just discriminated against men and women with families, people from less fortunate backgrounds, and others who otherwise do not have the time in the day to dedicate to a skill that is only useful to coding interviews.

as far as a comment above, I was a DBA for 10 years and can do pretty complicated queries off the top of my head, know how to optimize my indexes, worked with both structured and unstructured data in the multi tb size, ect ect, but I have no recollection on what ACID stands for. I dont really care. Sure, you may hire someone who memorizes useless crap, but then they have no idea why the IO has gone through the roof when inserting IDs out of order on a clustered index.


I didn't take the parent's post the same way. The situation is more that if you do leetcode or hackerrank then you pass the interview. So once again the interview process has failed; instead of identifying good candidates you identify candidates that do hackerrank in their free time.


thanks for the clarification, i read it as "its sad they cant do it, whats wrong with them?" kinda way, but what you said is probably what he/she meant


Exactly, I as well have been doing technical work for my whole life, and sitting here right now, I could more easily explain to you how to optimize the planner cache, and what hints are required to get a given result than I could explain to you which joins do what.

I just look up joins when I need them, and it's straight forward, but ask me in an interview and I sound like an idiot.


I don't read anything about OP's post as saying they think people should memorize or that this practice is okay, as evidenced by their final sentence that it's sad that you could do so and pass many technical interviews.


I would say that if you know the answers to 100 top coding questions (and understand them, a good interviewer will poke you to determine that) then you are a pretty good coder and you should be hired. Seems working as intended :)

For many tech companies where they get a lot more candidate applications than they have positions for, the goal of the interview process isn't to avoid losing good candidates, it's to not hire bad candidates.


That's like saying that if you can retype The Great Gatsby, then you're a great writer.

So much more goes into software engineering than "Can solve tiny example problems in an examination system"


Like I said, a good interviewer ensures you understand the solutions of those problems (the problem might even be formulated in a different form than the exact form you learned) not just typing out text verbatim. If you do understand the solutions of top 100 coding questions, I think you have a good start. Part of solving the problem in an interview will also be being able to code, so it's also testing your coding ability in a given programming language.

True that software engineering is more than that, but those other things largely depend on in-house company processes, tools and policies so you would learn them afterwards.


Yes, because nothing says top notch developer more than "I can memorize stuff and regurgitate it."


Interviewer: "Can you think of any way to do it in smaller big-O?" Me: "I would first google it"

It annoys me that the good answer if you actually encounter the problem during the job is never accepted during an interview.


I'm as critical of how we interview in this industry as anybody, but I've never found this particular criticism compelling or charitable. It's implicit that you know the correct first answer is to seek prior art. Making that explicit is fine but a bit pedantic. The follow-up question is: "ok great, now say that you can't find any satisfying prior art for this on Google, how would you reason through your own solution?".

They aren't looking for people who don't know to start by doing some research, they're looking for people who won't be completely stuck when that research doesn't turn up the answer.


Maybe but I once had an interview in which I was asked how to find how similar two text strings were (for search). I answered that I would use one of the algorithms from Apache commons text or within Lucerne which implement one or several of the appropriate distance algorithms. He told me he had written his own. I asked why he would do that when these algorithms were written by people who did intense research and the implementations in these libraries have been tested by more eyes than his could ever be. He said “what if I want it to run in Ruby?”...I was interviewing but this is a guy I would not hire. He was wasting time for his own amusement. Never assume people know to do the research on existing solutions. Many developers would rather work on their own toy solutions while avoiding the actual unsolved problems in their domain


Sure, your mileage may vary and there are definitely bad apples, but my claim is that the vast majority of interviewers aren't looking for people who like reinventing wheels, but are rather trying to suss out the extent to which you can reason through a problem. So I think that's the charitable assumption to make.


Are you serious? For any algorithmic question, if googling does not turn up a great answer, you would need to be a genius if you could come up with one on the spot.


We had a CS class at UNH, a state university, where we learned how to express the complexity of an algorithm and then how to refactor algorithms to be less complex, when possible. Although we do not do this type of work for most of our jobs, it is not a stretch to think that half of people with a CS degree might have at one point known how to do this on the spot.


Yes, that’s exactly the kind of thing companies like Google are looking for.


Sure, but I would seriously question someone's Google fu, if it came to this.


My job often entails reasoning through a solution to a specific problem which is dissimilar enough from the general formulation of some problem that there isn't an obvious way to Google for an exact answer. It may take a genius to devise the best algorithm to solve some general problem, but it does not take a genius to reason through a decent solution to a specific problem using general knowledge and experience.


Except they don't just want a solution, they want a solution with certain performance characteristics. And they want it to be the solution with the best performance.

Which means what they're really saying is not "we want problem solvers". What they're really saying is "the bare minimum for entry level work here is being able to, in 30 minutes and on the spot, out-perform top-tier theoretical CS researchers".

Which in turn really boils down to "be a recent CS graduate who memorized this algorithm in advance so as to 'derive' it later on command".


That's reasonable, but at the same time, unless it's a top-secret clearance job, don't threaten me with doing web development on an air-gapped machine. We both know it's absurd, just don't do it.


> ok great, now say that you can't find any satisfying prior art for this on Google, how would you reason through your own solution?

So then I try and give them a O(n^3) solution (or something).

Of course this isn't accepted as there's a better O(n^2) solution, which can always be found by googling, and I fail the interview.

The "turtle and the hare" problem of finding a loop in a single linked list in O(n) time and O(1) memory is the perfect example. It's easy if you know the answer (or can google) but basically impossible if you don't.

What did the interviewer do when I said I couldn't do better than a hashmap? He laughed and said "not many can".


I’m not familiar with this one but have been thinking about it for a few minutes and think I have an answer (granted I think it will depend on language and OS whether it would work)

The gist of it is: walk the list and after each step, multiply the previous node’s pointer by negative one.

If the current node’s pointer is negative you’ve found a loop (because you colored it by inverting the pointer in a previous step).

Clean up by starting at the beginning and multiplying all pointers by negative one until you reach the last one you inverted. Then return your answer (true).

If you reach a null pointer there is no loop because you’ve gotten to the end of the list. Clean up by going through again and multiplying all the nodes by negative one. Then return your answer (false).

Your worst case running time is 2n (if there is no loop) which reduces to O(n) and you use O(1) memory because you’re coloring the list by using the already existing pointers in the list.


That actually sounds like a very reasonable senario to not do so well in.

The problem that I have with those problems is, they always assume it's easy to answer after seeing the solution. But if you're working on them for the first time, it's a terrible environment. The minute that they stop asking you, as the canidate, are on a timeline to finish. (Also, to finish as well)

The answer to that item would be: (with scala)

> datasource.window(3,1).filter((a,b,c)=> a+b == c)

I mean if you knock it out of the park and know the internals of window.. then asking the bar raiser question is appropriate as a curiousity.

There are other problems as well: Those are the ones where they have an expected answer, response, and reiterate. I've seen that with tree problems. That's where they would low-key ask for the recursive version and then get you to say stackoverflow exception.


My personal opinion on what a senior dev interview should be is: open with a fizzbuzz-level 'bozo filter' coding question, then step away from the whiteboard and spend the rest of the interview speaking to each other like normal, functional adults.


I actually implemented fizzbuzz in Scala recently.. It's a lot easier to do it in a language that has good pattern matching. Doing that in c++/Java is pretty annoying


So is it actually O(n^2)? Because that is a solution that can be implemented.


That all depends on the sliding function and if it stores in a buffer. (If it does this is o n)


>For some reason that company insisted on only doing interviews at 7:00 AM, and my brain doesn't come fully online until after 9:00 anyway.

Reminds me of my final round at Amazon, which they always do in Seattle. I woke before 4 AM west coast time having flown out the night before from the East, and 12 hours later was still coding on a white board. After writing (to my surprise) a correct merge sort, they asked my to write a program to do basic math with, IIRC, binary numbers. I was like, um. I'm out.


Three sum is what google uses for their video 'how to do a google code interview': https://www.youtube.com/watch?v=XKu_SEDAykw


This doesn't change the crux of your story--which is that Senior DBAs who aren't familiar with ACID are probably not good hires--but that said, "consistency" is actually a somewhat ill-defined guarantee [1].

And if you throw in distributed databases, there is yet another understanding of "consistency" (vis-a-vis the CAP theorem), which really means "linearizability".

Sorry, didn't mean to be super-pedantic but consistency as a concept may seem simple to the uninitiated but is easy to get tripped up on, especially for folks who are aware there is a deeper layer of understanding associated with the concept, but can't articulate it right off the bat.

All this is to say filtering people truly is a hard thing. The extremes are easy to tell apart, but people who hover around the average are much less differentiated.

[1] https://en.wikipedia.org/wiki/Consistency_(database_systems)...


It filters out certain groups of people, but you will also lose out on good hires that don't focus on your given question. ACID has had almost no impact on my career. Only evaluating lies of ACID from certain db vendors is the only time I think I can remember it coming up meaningfully.


Yeah, I would have been okay with pretty much anything close, and would have given massive bonus points if they could give me a couple of relevant definitions. I wasn't looking for textbook, just for a reasonable discussion.

And I have similar anecdotes related to candidates who claimed years of distributed systems architecture experience, and were yet unfamiliar with CAP (and when explained, could not tell me even at a high level whether, in the event of a partition, their system was CP or AP, let alone the specifics of how that actually exhibited itself).


"That guy didn't have 20 years of experience with db2... He had 2 years of experience, 10 times."

One of my mentors said that about an interview he conducted a while back. I found a lot of truth in that line. I run my tech interviews by starting very green and let the candidate dictate how fast I ramp it up. I've gotten pushback from managers before that you can't start with basic questions, but I've equally gotten positive feedback from candidates (even one's who've been failed the interview). In the end, programmers want to see an algorithm that allows them to judge an interviewee. I don't believe one exists.


This is a brilliant approach because everyone has more context, meaning answers can be more pointed and exact. When the questions start small and build incrementally the next topic of conversation becomes much more natural. The candidate knows that they can use a technique because it's already been discussed and the interviewer is has more control over the flow because there are fewer tangential topics.


I don't claim to have solved the problem of interviews, but I do think I and the candidate get more out of it when we leave it a bit more free-form. I'll cover architecture of our system, API design stuff, debugging, testing, security, problems we're currently having, etc. I'm looking for passion so if the candidate starts riffing on any area, I'll let the conversation flow into that area and go as deep as we can. I usually play dumb as we talk, asking the candidate to explain why and ask what does that mean if they throw an acronym or design pattern into the conversation. A plus of that Socratic method is that if we get into an area that I don't know much about, the candidate doesn't know the difference :)


I did DB work forever never hearing ACID. I initially heard of it certainly from slashdot or hackernews, and it only ever ended up coming up due to not DB work, but DB analysis of tools.

Once you start working on a particular database, and are working on all of the specific details of that db, how often does ACID come into things, really? It just doesn't.


Hence the followup question about consistency. I purposely gave a useless and very inaccurate answer as an example (and one I am paraphrasing from an actual candidate).

If you legitimately did the work without picking up -any- of the terminology, -and- respond confidently to questions you should know you don't know the answer to (rather than admitting ignorance and asking for follow-up questions, or declaring assumptions, i.e., "Well, I would assume it has to do with the behavior that if a constraint is violated or something similar in a transaction, the transaction will be rolled back instead of committing"), I don't want you. Because it means you both did not have formal training in, AND didn't seek out information or knowledge in the domain you were expected to be -senior level- in.

And while atomicity and durability are just characteristics of a relational DB (and so adminning one doesn't really require you to understand them), isolation and consistency both have definite relevance, because the isolation level is configurable, and that affects the consistency guarantees of the system. I expect someone saying they were a -senior level DBA- to be able to talk intelligently about those behaviors, and yes, to understand the words, because just reading the docs would have introduced the words.


The responding confidently about things you don't know is obviously a huge red flag for anyone regardless of position.

Otherwise, I think we simply disagree in some respect, although I think it's over emphasized due to the topic. I think both perspectives have a level of truth to them, and have their flaws. It really depends on the candidate, and what they really know. Perhaps my particular circumstances are sufficiently odd enough that they aren't useful in a broader context. Who knows.


I’ll be honest, I don’t know a lot about databases, but Atomiticy comes up a fair bit in the web apps I write, as does Consistency in terms of designing data models that can’t go wrong, or trying to encode domain structure into data models to use the Consistency guarantees that Postgres gives us. Isolation is pretty important in general, and easy to reason about from a web app perspective, with requests being isolated from each other at the database level, I rely on that all the time, and Durability doesn’t come up a load, but that’s because I haven’t had to do any sort of disaster recovery.

I don’t think it’s essential to know this, but I’d be surprised if a web dev interviewed with us and didn’t know at least the basic version I’ve given above.


I've done tons of disaster recovery, and no one ever called it ACID. It's never come up. The only time the type of discussion specific to the terms of ACID comes up is with other database administrators.

Just my experience, perhaps I live in another world.


> The only time the type of discussion specific to the terms of ACID comes up is with other database administrators

I think this is partially what I'm referring to. I use these terms in conversation with others when designing systems because they are useful in describing very specific properties of databases.


It's possibly OK (very strange but OK) to not know what ACID stands for. It's def. not ok to not know about Atomicity, Consistency, Isolation, Durability and more importantly why you need them if you are a DBA.


It's weird, I've heard plenty of stories about this kind of thing but when I sat on the hiring side and interviewed for intermediate roles (couldn't even afford senior) I didn't come across anybody who was stumped and simply couldn't code.

There were people who were bad at (possibly some because they were under pressure), it but nobody who couldn't do it at all.

I wasn't giving out a trivial question either.


I've given coding interviews for 20 years, and I'd estimate 9/10 candidates have been competent whiteboard coders. I don't have any explanation for the extreme discrepancy between my experience, and the "hardly anyone can fizzbuzz" folks. Of course this colors how I treat people. I'm much more inclined to interview seniors conversationally and judge competence by the way they talk about work they have done. I have a hard time imagining people investing the time to become conversationally fluent in a domain, as some kind of long con.


Beyond fizzbuzz, whiteboard coding is rough. Better to put someone in front of an actual computer. Space is constrained, hard to make corrections, some people (such as myself) have crappy handwriting...


It's such a weird hiring market. 1. There are a lot of imposters out there applying for programming jobs who can't program, and 2. There are a lot of very talented programmers out there who are being rejected by overly picky companies. Both can be true, and I'd argue that both are true. I don't know what the solution is. Current interviewing methods don't seem to be solving the problem.

I'd suggest a widely-accepted professional certification could help a lot, like doctors and lawyers have with the medical board exam or the bar exam. Easier said than done, but what we have today, where the candidate pool is overflowing with impostors with great resumes is not working.


That might solve half of the problem. But then we need certifications for employers - something that says, "Yes, I have a real job opening; I'm not just wasting your time." It needs to have real penalties if an employer violates it, too.


I'm not sure this is working, since employers not only search for general software developers but maybe also developers with domain-specific knowledge. It would be very time consuming develop a certification for every domain.


Hey, I said it was needed. I never said it would work, or that it was workable. But if you're going to certify workers, so that they don't waste the employer's time with interviews that are going nowhere, well, we need something to do the same in the other direction...


> I'd suggest a widely-accepted professional certification could help a lot, like doctors and lawyers have with the medical board exam or the bar exam.

"i passed the bar exam!" and nothing else doesn't get you a legal job you want.

and your medical boards, if i recall correctly, just mean you're qualified to go be a slave, er, uh, resident, some place for a few years.


On the other hand, passing actuarial exams does get you in on the ground floor of that career.


certifications are plentiful in the infosec sector. They absolutely do not solve the hiring problem.


Maybe because they are plentiful? I'm not familiar with infosec but if the bar for getting certified is "I can take a class and pay $XX to get this certificate" then of course they're worthless.


I agree, but in theory that can mean that we haven't found good certifications yet.


There are very few certifications that are worth the paper they are printed on.

Find me good certifications that are actually meaningful and relevant to my career, and I’ll do the study work and get them.

I have yet to find any.


And don't forget people being asked to do coding exercises for non-development jobs :|

(I've had that happen several places)


One problem with viewing the certification thing as a solution is people can then question - how do you know if you got the person who graduated at the top of their class versus someone who barely graduated? How are lawyers/doctors vetted beyond their certifications?


Make the bar for passing high enough such that even a person who scores the lowest passing score has demonstrated that he/she can at least code. A standard exam doesn't solve all problems with hiring, but it could at least help solve the very first "can this person even code at all?" screen that weeds out the total phonies.


Oftentimes this is the "CS degree from a good school" stick. But we don't have an easy way to check those claims, nor to evaluate what constitutes a 'good school'.


Id say recertification is an option as time passes. It keeps people up to date and on their toes.


Maybe just luck? I interviewed someone for a senior position last week and that candidate didn't know what an array was. That was probably the worst case I've seen but by no means the first time I've seen someone struggle with the very basics.


I think it's a matter of scale (how many you interview) and how good the people in front of you filtering candidates are.

Go long enough interviewing hundreds, you'll see a few no matter what. Have a crappy enough filter in front and you'll see them right away.


Same here. I have given an interview to exactly one person who would fall into this category, though most have been bad coders.


I remember interviewing a candidate that didnt even know what a binary tree was (context was big-O complexity of algorithms). That interview was really bad. Left the candidate in tears (which was not intended, but I think of their realization they werent going to get the job), and me annoyed at the waste of my time (should have been caught in phone screen instead of on site).


Was it necessary to know on the top of your head without Googling what a binary tree is for the position the candidate applied for? I've been a developer for 12 years and I've never had the use for that. Computer science is a VERY large field and being good at everything is impossible.

Asking the right questions at interviews are crucial for finding the right people. If a candidate for a programming job knows binary trees but doesn't listen to other peoples views I would say he's less worth to us than a listener that easily learns new concepts but is currently not familiar with binary trees.

BTW, leaving a candidate in tears is not professional recruiting. Please let someone with more people skills accompany you to your interviews, you might leave people with scars that takes years to heal.


I've been a developer for 30 years, and I haven't ever had a need for it, either - but I still know what one is. I expect that you do, too.


I came across it when studying computer science 20 years ago, yes. Heard about it after that but never directly needed the knowledge.

I'm sure it's used under the hood in a lot of code I write and have written but so is XOR, manual memory management and a bunch of other lower level implementations that I don't need to spend time on when developing on a higher abstraction level.

Not sure why you would expect all programmers to know about binary trees specifically.


Binary trees are the simplest kind of nontrivial tree, and trees are used extensively in programming. Most computer problems are solved using trees of one kind or another.


If you've never needed to know it, what is its value for finding a good candidate? A lot of these questions are basically testing whether you did a CS degree. If you're self taught, you might be just as good a coder, but have never had reason to learn what a binary tree is or how to implement quicksort, or whatever similar puzzle.


It's a nice way to filter out people who can code and basically do the job, but didn't have the financial resources to get a CS education, and weren't lucky enough when researching to see that his is an interview question. Although at least they'll know for next time.


Why on earth did you think that was a good, relevant question to ask? Did you ask about how to implement merge sort? I learned about it in college in the early 90s. Haven't implemented it directly since.

I'll bet, even with this feedback, you'll continue to ask that question, just because.


How often would you estimate either of binary trees or big-O were discussed in one year of that job?


At least, for sure I know I wouldn't hire you as a recruiter.


Most programmers never need to interact with binary trees directly, and I would hazard a guess that if I asked ~10 of my compatriots about big-O notation, maybe 2 of them would understand what I was talking about.


Do you really think that text-book question about ACID determines whether the applicant is a good DBA? I know something about databases but I couldn't remember all the letters.


Hence the follow-up (which I'd ask regardless, as we work our way to more and more hands on stuff). It's okay if someone has never heard of the acronym before. And an answer of "Uh, I remember the C is consistency...and the D is durabilty. But otherwise no" is as good, to me, as remembering the entire thing; it shows you've run across it before, that you've at least read up or received some sort of training with DBs.

I mean, look at this thread; would it be better to start with "Here are some example tables, please write the SQL that will return me (etc)" and turn it into a coding exercise? Because I honestly don't care that much about that from anyone espousing a senior level of knowledge.


I understand that's a problem, but if I have an entire github of highly starred and heavily developed projects with reasonable commit histories... don't make me do your do damned homework or implement a toy BST. I have better things to do with my personal time than toy problems because you can't be bothered to open my github.

The worst part is that usually these toy problems are justified with "but you can post this to your Github for others to see!"


I'm not going to speak to the homework type of problem (because honestly I think you should be paid for that sort of thing), but I always ask people programming questions in on-site interviews, not strictly because I want them to prove they can program (although that's one useful side-effect), but because I want to observe the candidate's problem-solving process. I can't deduce anything like that from your Github or a homework problem.


And that is entirely fine. I'm all for group white boarding exercises, or pair programming. I think there's a lot of value to be had in making sure the other person can communicate in a technical setting effectively, and explain why they make choices.

Likewise, if someone wants me to walk them through a project I have with explanations of what choices I made and why, I'm happy to do that. But please don't waste my time asking me to implement DFS, or write another twitter API client.


If the GH code is there, works, and is up to coding standards, why do you care about how the sausage was made? I'd look to see if the code is well-designed, organized, original, tested, etc. It's generally easy to see if the author of code knows how to decompose a problem into pieces. So..saying "we're looking for thought-process not the actual code" feels a bit disingenuous tbh.

Now: If you care about being able to collaborate on a tough problem with somebody, work on a problem neither of you has seen before together :)


Alas, I really do care about the thought process. If a person arrives at a great decision for bad reasons, I may not want to hire them. More importantly, though, I'm interested in when candidates pick not-optimal choices for good reasons. So much of software development is about tradeoffs. Seeing and hearing how they tackle those tells me much more about how they'll do than the specific code in question.

As an aside, the reason not to work on novel-to-you problems with job-seekers is that it's very hard to fairly compare candidates after. I strongly prefer pairing on the same problem with all the candidates in a batch. Otherwise it's hard to tell if a bad result is due to a bad problem or an unacceptable candidate.


> Alas, I really do care about the thought process.

Sure - but you're not likely to really see the "real" thought process you're hiring for by asking a stranger a riddle you already know the ansewr to in a high-pressure situation.

Real problem-solving and thinking is done in a huge number of ways that sometimes isn't conducive to strangers, pressure, or whiteboarding.

> I'm interested in when candidates pick not-optimal choices for good reasons

Right - which is why starting with something they've written on GH (which is something they've thought about and are probably passionate about) is a great jumping-off point for such a convo.

> As an aside, the reason not to work on novel-to-you problems with job-seekers is that it's very hard to fairly compare candidates after.

I see your good-intention here, but it's nearly impossible to compare two candidates without all kinds of biases (implicit and explicit) coming into play.

> I strongly prefer pairing on the same problem with all the candidates in a batch. Otherwise it's hard to tell if a bad result is due to a bad problem or an unacceptable candidate.

I think this may be another way of trying to compare candidates to eachother (which is perilous).

Good candidates can do well with bad questions, and bad candidates can rarely do better than okay with good questions. There are lots of good/proven questions you haven't solved - ask your colleagues.


If someone has a good GitHub profile I won't ask the candidate to do much or any coding exercises, because it's much better to just discuss the code they already wrote. I'll put it up on the screen and get them to talk me through it. I'll ask about anything that looks unusual, but also about anything that looks particularly elegant. Even if they wrote it a while ago they should still be able to talk about it.


> If the GH code is there, works, and is up to coding standards, why do you care about how the sausage was made?

Perhaps because, like many developers on modern teams which eschew the older role distinction, the incumbent in the position being hired for will need to act in the role of a classic system analyst in defining specific requirements give a fuzzy business problem as well as the role of a grunt coder.

Also, the presence, functionality, and quality of code on GH does not establish it's provenance.


> position being hired for will need to act in the role of a classic system analyst

Sure but that's a different skill from solving DS/algos, so don't conflate the two. Some people know their DS/algos super well but need to solve them solo, some people need to google around a bit for inspiration or take a walk if they get stuck.

Instead, separate it out. Give a specific "system analyst"-style question that's distinct from coding. A question you don't already know the answer to or that could be taken in a thousand different directions that you've definitely never thought of before.


I want to know how resourceful the candidate is, how quickly they make mental leaps and whether they'll play nicely with others in the team under the context of the work we're trying to get done. If the choices are a) find out first-hand by (gasp) asking for a demonstration of skill while I watch or b) try to decipher these things by shaking a tuning rod at their GH repository--then I guess we'll have to agree to disagree :)


Sure - my point is that asking them to solve a problem on their own that you already know the answer to isn't a realistic environment of "playing nicely" either and is only going to show you how they deal under pressure, not how well they collaborate with somebody who genuinely wants to find the answer and can build off of.

If you need proof they can "actually code" and use DS/algos etc, use their GH if it exists. And if you want to see that they can work on a team to define and deliver a messy problem with other people, do that with them on a messy problem you've never solved before. (It could eventually be a DS/algo problem; my point is it's a group effort and neither of you knows the solution; you're looking explicitly for how the interaction goes, not if "the candidate" got the "right" answer.)


So I'm supposed to spend my time researching your GH instead of just letting you spend 5 minutes proving it? There isn't enough time in a day to research every candidate's code they wrote on their own time (and personally I'd rather not be judged by mine). And, if people lie on their resumes already, what makes me trust their GH? It's like saying here's an essay I wrote, you don't need to talk to me in person, I"ll just stay home and you can decide whether you want to hire me.


Yes. It's a lot of work on both sides to interview and be interviewed.

If you don't take a candidate's prior art into consideration, you're wasting everyone's time. You're also eliminating candidates who may excel in ways that aren't solving riddle-problems out-loud in front of new people when their livelihood is on the line.

Start with the GH and if you have doubts fall back to portions of old model.

> So I'm supposed to spend my time researching your GH instead of just letting you spend 5 minutes proving it?

I've never seen a worthwhile programming q take only 5 mins to answer. And the candidate is supposed to waste an hour of her time proving to you what you could see in 5 minutes on GH?

> There isn't enough time in a day to research every candidate's code they wrote on their own time

Yet there is enough time to spend with whiteboarding problems that prove the same things the candidate has already proved on their GH?

> (and personally I'd rather not be judged by mine).

Sure - only use the candidate's GH if they prominently put it on their profile and/or own an "intended to be used/seen" public repo.

> And, if people lie on their resumes already, what makes me trust their GH?

Not a replacement for conversation! It's a way of indicating they can code without solving an arbitrary riddle in a high-pressure situation. If you're not convinced they wrote or understand the code you're looking at, ask them about it or ask them how they might change it slightly.


This is fair, it's just not realistic for a lot of places. In the places where I've worked and interviewed people, this was not my main responsibility. Typically I'd get a resume that morning and have to prioritize looking at it along with whatever else I had going on that day.


On the one hand, I agree entirely. An interview should not make people jump through hoops if they can answer their questions via existing material.

On the other hand, fakers have caught on to the "GitHub is my resume" thing. I have had applicants who have basically fraudulent Github projects, or who have taken group projects on which they did basically nothing and claimed them as their own. So a Github page now requires an expert to evaluate it.


Surely getting them to talk you through the code would solve that problem.

If its is their own code it shouldn't be a problem.

If it isn't their code and they can still talk you through it, even better - they have managed to understand code that someone else wrote which is probably an even more valuable skill.


You realize that your third second contradicts your first, right?

And no, having somebody who's an energetic fraud on the team is poisonous. Having somebody who's so good at it that they're hard to catch is worse, not better.


If people are genuinely a fraud they won't be able to talk about the code in a sensible manner.

I didn't write the monstrosity that I am working on just now, but I could explain how it works and I know where to look to fix bugs. I don't know why some crappy design decisions were made. Does that make me a worse programmer than if I had written it myself? Is harder to understand someone else's code than stuff you have written yourself. There are far more jobs working on existing code bases than on new new projects.

(My guess is that you have never worked with anyone who has claimed ownership of someone else's code, have you?)


> but if I have an entire github of highly starred and heavily developed projects with reasonable commit histories... don't make me do your do damned homework or implement a toy BST.

Unfortunately, the current common argument from the hiring side is that GitHub profiles are not sufficient proof of skill since code can be copied (not fair) and GitHub projects bias toward people with extra free time (reasonable, but not enough to discount GitHubs completely IMO)


Right. Don't "require" a GH, but use it if it's there!

Yes, some of the code may not be original, but unless the repo is a fork you can pretty easily see if the code is organized in a coherent way that shows the committer knew how to decompose a problem well. If they solved /every/ piece of the problem themselves, it may be a sign of a different problem (and also copy/pasting without citing source is also telling!)


I feel like this is a myth, what do you estimate the figure is? I think under 10%, maybe under 5%. I have significant experience interviewing senior, junior, and mid-range candidates. 99% of my candidates can code, as in iterate over collections, write case statements, and call functions. I've only had one junior candidate who couldn't code at all. Sloppiness is rampant, but sloppy code that gets things done is what my company wants (not me personally).


We set our in-house recruiter up with a coderpad question that screens candidates with a simple question:

"Write a function that counts the number of vowels in a string"

Candidates are allowed to run it multiple times and just have to produce a correct result within 10 minutes. It's not a trick question -- the test case in place makes sure you pay attention to case.

Success rate for mid to senior devs? Only 60%.


I applied for an analytics position that unexpectedly had me take a Python coding test like this (I know a bit of PHP and Java but no Python) and I was able to google everything I needed to pass the test in the time limit. Apparently I got one of the higher scores too. Got the job. It has not required me to write a single line of Python, lol.


Googling stuff to copy is one of the most important skills for programmers. I was appalled when a middle aged senior programmer at my first internship told me this, thinking he was lazy and unethical... (people are paying you after all!)... how naive I was!


This isn't bad. I've used one similar:

"Write a function that reverses the words in a (string) sentence"

i.e. (x) => x.split(' ').reverse().join(' ');


Just a little trivia;

The fastest way to reverse a string is in JS is:

  function reverse(s) { 
    var o = []; 
    for (var i = 0, len = s.length; i <= len; i++) 
      o.push(s.charAt(len - i)); 
     return o.join('');  
  }


In C#, the string object has some convenience in it:

  return inputString.Count( "eaoiuAIEOU".Contains );
Or if you want to solve two problems:

  static Regex VowelMatcher = new Regex( "[aeiou]",
    RegexOptions.Compiled | RegexOptions.IgnoreCase );
  ...
  return VowelMatcher.Matches( inputString ).Count;
And if you want to be clever and snarky:

  return 2; // number of vowels in "a string"


The Count and Contains methods are not on string, they're both extension methods on IEnumerable<T>, which string gets for free by implementing IEnumerable<char>.


That's surprising - ill have to give that a go later this evening as I have been thinking about going back to development form a consultant non coding role.

Took me less than 10 seconds to come up with an approach that would work oh just though of a more efficient one but that would use a regex :-)


What's your process like after that screening measure? Because to me, that's honestly trivial, and a lot nicer than dealing with an extended (8+ hours) homework problem as an introduction to a company's hiring process.


A ~4 hour on-site (or google hangout) technical interview and discussion. The first 30-45 minutes is usually us selling you on the company, followed by 2-3 hours of technical stuff. We do throw a few more coding questions at you (ones that are technically trivial, but made more difficult with interview jitters etc), but go beyond that and ask how would you design an API, how would you think about the technical requirements about business problem etc. After that, Q&A, and more selling you on the company.


That sounds like a nice process. I don't know anything about your company, but I'm thankful that you keep it sane and reasonable for prospective developers.


Screw it, everyone else is posting code examples:

    def vc(s):
        return len([c for c in s if c in ['a','e','i','o','u']])


Probably s.tolower() or whatever it is in Python.


def vc(s): return len([c for c in s.lower() if c in ['a','e','i','o','u']])


Is Y (Greek I) a vowel?


For this problem, we explicitly say no in the description.


As an experiment, I was able to do this in under 3 minutes with ruby. I'd say I'm an average programmer.


It's absolutely not a myth. It does depend on how you're getting your candidates, but assuming you cast a net a bit wider than "only people you know personally", then you run into these cases.

Our standard fizzbuzz question was something like: print the multiplication table. This is a loop inside a loop. I think every programmer should be able to do this given 10-20 minutes. Around 30%-40% couldn't. Like literally, didn't know how to do this in languages they supposedly work in. I'm sorry, I make lots of allowances for stress, I calm candidates, I tell them syntax doesn't matter, do it in pseoudo-code for all I care. But if after 10 minutes you can't print a multiplication table on the screen, I'm going to pass.


I feel like sometimes it's out of the applicants hands if they come by way of a recruiter / head hunter. Sometimes non-junior devs who aren't quite to senior developer status yet have no control over the recruiters mistaking your pay at previous jobs and the pay you could make next time for the level of job you should be applying for / able to handle.

I've run into this issue in different specific ways earlier in my career and no matter what I'd say about skill level / lack of knowing required technologies / being willing to take lower pay in order to justify aiming for another role it didn't matter in the face of the possibility that the recruiter could obtain a larger % for themselves off of that senior / higher-than-in-my-range job's salary.

Maybe this isn't as applicable for senior dev role applicants as it is for some lower level dev roles but at the point an applicant is actually physically there in front of the interviewers I'm sure one of the last things they are going to want to admit is that they wished they were interviewing for a lower paying / easier job even if it's 110% in the favor of both parties.


I'm a senior engineer and have worked with recruiters many times before, and I've pretty much come to the conclusion that they're a big waste of time unless you want to do the 6-month contracting thing to avoid resume gaps or you just like moving around the country and renting rooms.

I've tried working with them before, have gone on job interviews through them, and then generally found that the jobs that hire through recruiters pay peanuts, and I end up finding a permanent job paying much more and taking that instead. I think if a company only works through 3rd-party recruiters, that means the company has no idea how to hire people, and doesn't want to pay much either (because they're paying the recruiter a huge commission).


It's not in their favor to find you an ideal, direct hire / non-contract gig. There is no chance for future money from a client if they find you your perfect fit / long term career type job.

Not to mention most of the one's I worked with early in their career not only had no idea about any of the technologies they were checking to see if I was fluent in but they also acted like they DID know everything about programming and beyond that they tried to flaunt this fact and treat me like I was trying to overstate my skills to sneak my way into a job out of my pay range.

Such a frustrating ecosystem in order to find a career job.

In ten years I can count on 1 hand how many times I've gotten a reply from a non-recruiter job I applied for (applied on my own without a staffing agency submitting / representing me).

It's really crazy how that works.

Another thing I dealt with was people who hired me saying (TO MY FACE):

In this situation I'm making $35.00/hr as a Regular Developer (non-junior / non-senior dev, just a middle of the road contract developer):

CEO of company in front of everyone: "scoggs we are paying way more for you than we are for our senior developers so we are expecting top notch work from you."

Me (used to it by now, unfazed but I hate this situation): "Sir with all respect I'm only getting less than 1/2 of what you pay my staffing agency for my contract/"

CEO: "well they don't really do anything / didn't really do anything but introduce us and get you a phone and in person interview with us."

Me: "But you guys didn't have to pull programmers, HR, and design / copy writing people off of their normal work to create interview materal, submit interview material, review resumes, vet candidates, plan phone interviews and schedule them, set aside time for phone interviews, review phone interview candidates among 'hiring team', review candidates references, plan in person interviews and schedule them, execute in person interviews and have meetings with 'hiring team' to pick who to hire', make offers to people you want to hire, or hire them. All you had to do was tell the staffing agency what skills your were looking for and what kind of company you were. I'm not an employee of your company. I don't have insurance, I don't get paid for sick days, I don't get paid for holidays, and I have no job security."

Of course I didn't say all of this, I said something along the lines of "They take care of the majority of the process" but in my head I feel like I'm always having to live up to the expectations of the people paying the total bill. The same way they think the staffing agency doesn't do any real work and isn't worth the cost / price of doing business -- most of these people feel the same way about developers. They don't understand what we do, they just expect us to solve anything and everything to do with computers / programming / technology and this is ALWAYS the way it is regardless if the final product / project outcome is directly tied to the company suddenly turning a profit based on the success of this project / programming endeavor.

These companies want to underpay contract developers, pile stress on their shoulders, guilt them for the situation they have little to no control over, and then stick them with the majority of the blame if things don't go 200% well (because they are expecting work that's 2x as good as the amount of money I'm being paid).

I've found it impossible to truly and personally (mentally) live up to the majority of expectations laid upon my shoulders by non-technical CEO's / Presidents / Bosses of companies I've worked for.

This is the majority of what I deal with right outside of NYC in Northern New Jersey. There is the rare company that truly respects the programmers they hire (usually companies where programmers have become CEOs / Presidents / people in powerful positions within the company who wield influence).

I love the money I make in this market but I truly hate the way everything makes me feel even when the situation is like this yet I am able to deliver above and beyond expectations. I hate working for people and in situations where it feels like I'm being told I'm robbing the company when meanwhile the recruiter is robbing us both yet they have us pitted against one another -- and beyond that the staffing agency has a 2 year lock on me being able to accept a job from that company without them "buying out" my contract.

The cost of buying out a contract? A university wanted to do that once early in my career (I wish I was still working there frankly, 10 years later), but the staffing agency wanted 2x the amount the university was paying the staffing agency! I was making $60,000/yr at that point so on top of the University paying me somewhere in the realm of $60,000/yr they would have also had to pay the staffing agency ~$240,000 just for the right to hire me. That means they would have had to shell out $300,000 total including my salary and the buyout.

Nothing about that seems right and / or legal. Under no circumstances would I ever be worth that much to the staffing agency and honestly with the amount of work they actually do I feel like it's damn near criminal.

No State University that pays employees with citizen's tax money is ever going to be able to justify spending that type of money on 1 employee. It's just a fantasy of epic proportions.


I lived in northern NJ for a couple of years. You need to get out of that place; it's a terrible place for software engineers. The cost of living is ridiculous but the pay for engineers is mediocre at best. Move to Silicon Valley: the cost of living is only slightly more, but the pay is far higher, and the weather's a lot better too. And $35/hr 1099 is ridiculously low in that area, those are poverty wages.

Anyway, you make it sound like you can't get a job there without a recruiter. There's still plenty of companies hiring directly, I even interviewed at a few when I was there such as Alcatel at Bell Labs and some wifi company in Manhattan. My current gig is with a large company (in the DC area) that hired me directly. Stop wasting time with recruiters and just look for companies that do their own recruiting; there's no shortage out there that I've seen. I've had 8 jobs now that were not contracts and not through recruiters: 3 large corporations, 3 small companies, 1 mid-size, and one a state university research division. Am I apparently special that I'm able to find these jobs?


I think it depends on how you're filtering the people who get to your interviews. If it's purely based on a resume screen, you can get a lot of people who can't code.

Personally, my first-line screen is a short answer (3-5 sentences) to a programming-related question. [1] For me, about 90% of applicants who pass that can also code. I like that better than a resume screen, as plenty of people who haven't officially been programmers are decent coders.

[1] Examples here under the "Join Us" section. https://web.archive.org/web/20151005181908/http://www.codefo...


> 99% of my candidates can code, as in iterate over collections, write case statements, and call functions.

Honestly, that's way too trivial to call "can code". I usually ask something less trivial, yet still extremely easy like "a function to check if a string is a palindrome" (I explain what a palindrome is) or "check if a substring exists in a given string".

You wouldn't believe how many people "with 10 years experience" just lose all motor function in their hands and mouth, even without any time pressure.


Problem is: It's hard to judge the difference between someone who doesn't know what they are doing and someone who does but loses it under the extreme pressure of an interview wherein their entire future with the company is being determined by their performance on a 30 minute exercise.

Throughout my career, I've been in professional situations where I was under a lot of pressure: Having to explain failures to executives, near-impossibly tight deadlines, production outages. At least I haven't had to testify in front of Congress yet. Absolutely NOTHING I've encountered in my professional day-to-day has the extreme level of pressure of a typical silicon valley job interview.


I understand and truly symphatize with stress the candidate is in, but someone who claims to be a "senior developer with 10 years experience" should be able to write that palindrome function in 30 whole minutes in any kind of stress scenario.

Because if I hire that person, they'll be in charge of critical production systems and will face much harder problems in much more stressful situations and if they can't handle the palindrome question, I have very little confidence they can handle the actual job.


People are strange. I can totally picture someone who is an experienced senior, a hero under the pressure of a critical production outage, yet whose brain goes into a crash loop when sitting across from someone who has the power to deny them their dream job.

Not claiming to be that hero, but I have personally had those moments where I came out of the interview, and as the disorienting fog of pressure lifted during my drive home I said to myself, "WTF happened to my brain in there? I know that stuff cold!"

EDIT: I think I read this one from a commenter here, but it's so true: We're interviewing people to be music composers, but judging them by their ability to be performance artists.


Fair enough. I certainly don't claim to know the best interview method ever; simply that, in my experience, starting with trivial coding questions has a higher benefit/risk ratio then other approaches.


I don't agree. They should be able to write that as a solution to resolve an issue. It should be very well understood why they're writing it. Additionally, if you want to keep going under that justification: you should also accept quick alternatives that aren't the naive approach.

(i.e. bring out groovy, using bash, etc)


Even a "systems" language like Go is just as suitable for a "quick alternative" as a "scripty" language like Apache Groovy or Bash.


While I agree in the abstract ...honestly, it's not that hard to write at least pseudocode for problems like that regardless of the "stress" of an interview

Is it reasonable to ask things like that? Maybe, maybe not.

But as stressful as an interview is, it's still a basic problem (I can English describe it to you in 30 seconds...coding (in C/C++) would take me another 5-8 minutes (and, probably, be pretty damn inefficient ... but it's still a simple problem)


It's way more expensive to hire a bad fit than to pass on a good fit. Also, I think I'd prefer the person who doesn't crack under pressure. Flopping an interview is more likely an indicator of lack of preparation than nerves, although nerves are such an easy scapegoat.


Not sure with the amount of sloppy code I have been maintaining for the last couple of jobs.The people appear to be able to do "clever" stuff, but have an inability to keep things simple or follow best practices.


How often do you need to write palindrome functions for your job?

Checking substring in a string is a 1 liner in languages like python.

I was at a final interview with FAANG. One question asked was to load a csv. I used pandas, it's a 1 liner. You could see the wretched face of the interviewer since he was expecting a

with open() as f: do_some_shit

and kept pushing me to write this on the board. I looked at him and said "Why? Why write your own method in a vanilla language? Give me a good reason."


So what is the interviewer supposed to do to find out if you can code? They want to give you a simple, straightforward task to see how you code. Of course, every simple, straightforward task is going to have an existing tool to do the job, but they aren't looking for a solution to parsing CSV, they are looking to see you code.

They can't ask you a complicated problem, because they don't have the time (nor do you) to solve a real problem that requires coding.

You are totally missing the point of what they are trying to ask you if you insist on using pandas for parsing CSV. Understanding the point of what you are being asked to do is critical to being a professional software developer.


I think that is the point. If your business relies on importing csvs that cannot be imported with pandas then explain that, explain the edge cases where pandas has failed, and I will understand. If your business relies on having the best fucking palindrome product then that becomes super relevant.

They can definitely ask a complicated problem. On-sites are 4 hours long these days.


You sound like somebody who would derail all of the technical discussions and pat himself on the back for it because he was "technically correct" while still missing the point altogether.


I hear the opposite. They are stressing a technical opinion here, but for the sake of rationality and efficiency. These are usually not the same people that are overly concerned with "correctness" when communicating with others.


But it is only 'rational and efficient' if you are being obtuse... they aren't asking you to code a CSV parser because they actually need a CSV parser, they are asking because they want to see you code. The 'rational' thing to do is satisfy THAT requirement, which is the real one, not try to bypass the purpose by insisting on using a CSV library.

The ability to understand the underlying need behind a request is important to being a good employee. If a candidate fails completely to understand the purpose of a question during an interview, and in fact continues to argue against you as you explain it, they aren't a good candidate at all.


> The ability to understand the underlying need behind a request is important to being a good employee. If a candidate fails completely to understand the purpose of a question during an interview, and in fact continues to argue against you as you explain it, they aren't a good candidate at all.

I agree with everything you've said, but that still makes it a bad and equally obtuse question. If they don't actually need a CSV parser, they don't need to know that you can parse CSVs, either (if you can - which is the other point - you might be a pretty good programmer if you can quickly parse a CSV, but there are tons of excellent programmers who can't).


I recently couldn't use a built-in CSV parser and had to roll my own because one of our clients couldn't be arsed to send us consistently formatted CSV files so we had to include a bunch of edge cases in there to still correctly format the damn things. (sometimes pipe-delimited, other times comma delimited, sometimes fields surrounded by quotes, other times no quotes except one column (that has a LASTNAME, FIRSTNAME" in it), yet still other times has quotes in the data itself, fields usually have no commas, but sometimes this one field will have it in the middle of it, typos in header names, not including the end of file checksum that others include, etc). The built-in CSV reader you had to specify if the fields had surrounding quotes or not for the entire document, it didn't detect it on its own, for example.

And that was when I found out parsing something that should be as stupid simple as CSV file can actually be pretty complicated.


The ability to figure out actual requirements from poorly phrased requirements is also a very valuable skill to have as a software developer.


I'll argue that a one-liner IS code. That's what I put into production systems.

I understood the "purpose" of the question and I self-selected myself out of a role I would've been bored, micromanaged at, and probably not challenged at.


The question on the interview isn't about the business, it is about showing your ability to design an algorithm. I is an exercise, not a purpose driven task.


So what is the interviewer supposed to do to find out if you can code?

I’ll take someone who gives an idiomatic answer over someone who reinvents the wheel anyday. Who is likely to be more productive on the job?


Depends if the job can be done using one-liners from existing libraries.


That depends on whether or not the job entails inventing new kinds of wheel.


Let's be honest: most commercial programming jobs don't. And the person with a thorough working knowledge of the standard libraries is in a better position to do it anyway.


They want to give you a simple, straightforward task to see how you code.

Overgeneralized at best, LOLworthy at worst. Does this include palindrome questions for Ruby that require the use of linked lists? Because I had that from a CTO of a couple-hundred person company not two months ago.


What would make a good programming challenge for an in person interview?


> How often do you need to write palindrome functions for your job?

That's like answering "how often do you need math?" to the question "what is 2+3?". It is an utterly, completely, stunningly trivial question that even a CS101 student on their first semester should be able to answer.

> used pandas, it's a 1 liner. You could see the wretched face of the interviewer since he was expecting a with open() as f: do_some_shit

I would be perfectly fine with the pandas answer, but my point is "csv parsing" wouldn't be my first question, I'd start with something much easier and if you managed that, I'd gladly accept pandas as a valid and then asked you to elaborate further (what does this pandas function do, how would you implement it yourself etc).


> (what does this pandas function do, how would you implement it yourself etc).

What are the downsides to using pandas? Is that worth brining in that large of a library for a matter like that.


I don't consider palindromes equivalent to math. Unless you mean how to index lists? That would be a valid question since there's different ways in python.


Your refusal exposed you as someone with a hubris that's hard to work with. Who doubles down and refuses to budge.

That you couldn't yield on such a trivial, manufactured matter would make me wonder how you respond in a team environment on real matters in the face of adversity.


CSV is really fucking hard to parse. There's tons of edge cases. "I'd just use a well-known, well-tested library" is a very valid answer.

One of my go-to questions is "sort this array", and if the candidate types `Arrays.sort(input)` they get bonus points, because it shows they have useful knowledge of the language they'll be writing in.


Here's how their scenario would actually play out:

    > I took my sunglasses off, looked him dead in the eye,
    > and said "Why? Why write your own method in
    > a vanilla language? Give me a good reason."
Interviewer: "To see how you might approach it."

It's a synthetic interview question. Why does it suddenly need to be a production-ready CSV parser?

We don't have enough information to say whether these are bad interview questions, and it's really not the point.

Maybe the interviewer just wanted to see if you'd use good fd hygiene (`with open('file.txt') as f:`) and that you can stub out some toy parser code and speak of it intelligently. And that you wouldn't throw a fit when asked to write some code that a library like `npm install fizzbuzz` can already solve.


That's a bit unfair. I agree that Array.sort is the best answer, but array sorting is such a classic interview puzzle that most candidates will expect that that's what you want. So instead of the bonus answer, they'll slog away trying to remember the algorithms that they learnt at uni years ago (or codecademy last month).


When I see someone doing something wrong or poorly I speak up and I speak out. Loading csv using vanilla python is in that boat. Present a good reason why it should be done in vanilla. Otherwise, I won't maintain your code as-is and will refactor it.


Sounds like their interview worked and they dodged a bullet.


Worked both ways, I dodged a bullet and am happier in my current position than can be :).


At least you were given a programming problem. I would have played along a little bit and at least discussed how the layers work. Instead of pandas, use the csv module so he could see a loop iterating over header and data rows as tuples. If he pushed further, outline how you might code the csv reader yourself if it didn't exist. Move the discussion into the difficulties of correctly handling a file format like csv and real-world issues like malformed inputs, validation, and error-handling.

In my case about a decade ago, I became obstinate when given one of those silly math-trivia-puzzle problems like how many ping pong balls will fill an elevator or how long is a string. This was for a relatively senior track FAANG interview, where my background at the time was in HPC, distributed systems, and provisioning systems that were precursors to today's IaaS bread and butter. In my case, the interviewer was adamant that I derive some figures that depended on basic geometry and knowing the diameter of the earth and ratios of landmass to ocean surface. But, he wanted me to first SWAG these figures, after I told him I wasn't a geospatial geek and didn't memorize such facts.

I decided to be difficult. If I were somehow faced with such a task, I would first consult reference materials and even see if I could find the actual answer he wanted, since it sounded only one step removed from what you could find in the CIA World Fact Book or similar. Only if that failed, would I dig up source facts and try to derive an answer. I would focus on getting the answer, not on entertaining myself with a Martin Gardner puzzle at my employer's expense and risk. I would never have to make seat of the pants estimates and act on them rather than doing due diligence. I'd either have done homework, or if there was really no time (like some system availability crisis, if we pretend I was an ops person), I'd choose a pre-planned contingency action to make time.

They didn't go forward with an offer, and I felt a bit of relief at that.


I got this CSV question at some nameless large company in the bay area.

I asked "Do you want me to do the simple thing, and use a library, or write code to do this?"

They preferred the latter. So I did.

As happy as it might make you to be snarky for the often silly questions you are asked, people are, if they are smart about it, looking to see your thought processes.

I recently got a question that I didn't know how to answer, so I started thinking through it aloud. I think they liked it, because they engaged with me during the process. Redirected my thought processes.

Again, good companies will do that. Look for every question, no matter how silly, as a way to show how you deal with situations, even ones you may not like. These are the moments for you to shine.

And after you get home, tell your SO/friends/etc. how wacky they are.


That seems like a fairly arrogant response on your part - it's obvious that an interview is meant to gauge your technical capabilities/knowledge. Even if something is trivialized, the ability to solve certain types of problems can translate to other problem domains where you have to use the same core skills to solve other problems - parsing a file does not seem like an other worldly type of skill (it could even be an unstructured/loosely structured text file for example).

I think to some degree, a suspension of disbelief is reasonable for an interview, and it's not worthwhile to get tunnel vision on a particular problem asked. It's ok to point out that something is trivial with a particular widely used library or whatever, but if it is asked to implement something via first principles or whatever, it is a perfectly reasonable expectation to just do it for interview purposes as a simple mental exercise.


I respect your view but disagree. I use pandas everyday for my job. I use spark. I use keras. I use scikit. I use all these APIs everyday for my job. I can't think of the last time I loaded a file using vanilla python. If it's a requirement for the job (which it wasn't, they showed me pandas code later) then I completely understand and would not be qualified nor would I want the job.

There's a few things going on here. First, there's a frustration from the interviewer who is obviously looking for the cookie-cutter answer to move things along. Second, there's a culture misalignment because they are looking for cookie cutters to do the job while I am not that. Third, they, along with OP, are obviously not a palindrome company nor a load-everything-to-lists company yet this is what they're testing. So again, culture misalignment.

At the end of the day, you are right, I self-selected myself out because I would not be happy at a place like such who are looking for cookie-cutter employees.


Why write your own method? Because it is sometimes better to write 10 lines of your own code rather than pull an entire library for some trivial task.

Creating a dependency to a library is not benign. Even if the library is easily available and mature doesn't mean that everyone has it, or that it will never change. Remember how "leftpad" broke npm?

Having that smug attitude towards the recruiter, assuming he is an experienced developers himself, should rise a red flag. Yes, sometimes projects have constraints (like "no pandas") that may seem stupid, and sometimes they are. But it is first important to understand the context.

So instead of asking "give me a good reason?" to the recruiter, give the reasons yourself. Invent a scenario where you actually need to rewrite that csv parser, bonus points if it is related to the company's business, and finally, write the damn code the recruiter wants you to write.


I don't agree with this reasoning. Perhaps the details are important. The task required loading, filtering, and aggregating a csv. Sure, if all do is "load" a csv perhaps you don't need to import a huge library. But if you want to do anything with that data I'm sure the library becomes very useful :)


Ah. Did I not mention that these CSV files I wanted you to parse... they're generated by an old mainframe system we can't modify the sourcecode to. It does have a few idiosyncracies - for fields like addresses which can contain commas, it uses a special escape sequence where the field just consists of three asterisks, then the value for that field is the content of the next line. Oh, and it uses just CR characters for linebreaks. In EBCDIC.

Can your pandas library handle that?


So presumably the mainframe has code that can parse these non standard files? Just take that code and build "on the mainframe" a tool that converts their non standard csv to something more usable.


Great. Can you show me on the whiteboard what the code for that tool might look like?


Okay, so the interviewer says: “And what if you can’t use pandas? What then?”

Your response is to flip the table and leave? If so, then good riddance.


Testify !


I've had this question and replied with:

const isPalindrome(str) { return str === str.split('').reverse().join(''); }

And the post interview notes said bad performance on the isPalindrome() question since I didn't write out my own functions (interviewer did not mention to).


Personally, if you gave me this answer I would accept it and then asked you how those functions work and how you would implement them. Still trivial, but it would tell me whether you are familiar with the basics or you just memorized some stdlib functions.


To be fair, if you wanted to do a performant implementation, this is probably a more expensive implementation. The expected response is generally for you to iterate over the string and construct a count map of how many times a character appears in a string, and then iterate over the keys of that object and have at most one odd number in the values of the count map.

The interviewer probably should have probed you about performance though if their intention was to judge you on that (they're both O(n) time complexity, but split, reverse, and join are generally more intensive).


Err I always thought that the expected performant response is to have two indexes 0 and length-1 and bring them to the middle checking that the values are equal :)


I'm very confused by this answer. Have I missed a joke? Wouldn't the expected response be to iterate over half the string comparing the characters to the other half (in reverse order)? The criteria you provide seems to suggest aabbc is a palindrome.


Ah sorry, I misunderstood the question - the one I've seen usually is if a string could be rearranged as a palindrome ha.


You're thinking of an anagram, not a palindrome.


Most of the time when they ask for the palindrome questinon they prevent you from using StringBuilder.reverse.


The function to check palindrome will be composed of those building blocks, that's my meaning. Use those basic skills and composing them to solve a problem, such as palindrome, that's coding in the small to me.

Coding in the large is project management and could be tested by a take home, but I'd rather tease it out by asking in person questions about organization, prioritization, and technology choice.


Do you get a IDE with compiler for the palindrom question?

Id say that close to 0 out off 10 engineers at work including me would get a working program without debugger or print statements.

Just tried in c and got me 2 compiles to get it right. Without the output I would just present a nonworking program.


Maybe it's just me, but needing an entire IDE or a debugger to write a function that checks if a string is the same backwards seems entirely too much to me. It's a pen&paper question really.


It's not just you. I use Visual Studio when I'm occasionally looking at C#, IntelliJ on the odd occasion that I'm doing Java, and otherwise it's emacs and I can't remember the last time I used a debugger for Go, Elixir, C, C++, JS, Python, Ruby, etc. Basically, unless the language's standard library is massive and over-abstracted, no IDE is necessary. There's no way I'll ever remember org.apache.some.deep.package.HttpClientBuilderFactory and its 6 constructor arguments, but "import requests; requests.get(...)" is pretty easy to write without an IDE.


Well, this may be a location and tech-dependent problem, but specifically hiring .NET positions in Jacksonville, FL I would say maybe 10% of the hundreds of people I've interviewed can code. Recent graduates, 10 years of experience, everything in-between. It makes no difference.


It's not a myth. I see it when recruiting for my teams. My sample size is too small to give a proportion, but it's enough that not having tests would be a total waste of my and their time since I'd have to fire them on day 1.


I'm amazed you can't tell an imposter just by talking to them.


The real problem is everyone looks like an imposter until you watch them code.


I don't mean this the way it sounds, but if you can't tell if someone is an imposter by talking shop with them in your chosen profession, you probably shouldn't be doing the interviewing.


You're sometimes right, but sometimes really wrong.

I had a candidate for a 3d graphics job, who showed me incredibly impressive things he'd programmed like very realistic water simulations, etc. He talked us through them, was incredibly articulate and clearly knew his stuff.

He completely bombed the coding exercise. It was something like: given 2 hours alone at a computer, with a threejs scene already setup, make it display a few cubes at different heights, colors, etc. Full access to internet, google, etc.

He couldn't get one cube to show up. That's maybe a two line piece of code when given the entire environment already set up, and you can find it by googling Threejs tutorial and copying like the first example.


But yet he "clearly knew his stuff".

Perhaps he really was a very talented developer but your coding exercise was too stressful for him. Shame you missed out on hiring him.

This is not a story to show why we should do coding exercises. It is a story to show why they sometimes don't test the right thing.


Your interpretation does theoretically fit with the facts. But so do a few other (in my mind more plausible) stories, like:

1. He knows how to do extremely specialized things in an existing environment, but can't do things outside of that environment.

2. He was presenting work that he had only a partial hand in creating.

3. He is amazing at e.g. C++, but cannot learn even rudimentary new things in Javascript.

...

Some of these stories make him a terrible programmer. Some of them make him an OK programmer for other positions, but weren't relevant for us. None of the stories make him a particularly great programmer.

"Perhaps he really was a very talented developer but your coding exercise was too stressful for him"

Yes, it's possible. A lot of things are possible. But I'm sorry - if someone can't copy-paste a 3 line program from the internet and get it to work, in their own field, for 2 hours, then... I don't know how I could ever design a test on which they won't fail. Including the actual job.


>You're sometimes right, but sometimes really wrong.

Ya.

As far as graphics, the only time I was ever fooled was by a front end guy in 2005. He brought really nice looking color printouts of his front ends; beautiful stuff. I was really wowed by it; so much I didn't ask him basic programming problems or follow up questions about his prior work, and that was my fault. I allowed myself to be fooled.

His biggest issue was he was lazy. He had grown accustomed to government contracts where you do a lot of prototyping, but nothing ever went live (at least the contracts he had). We had a very aggressive 3 year schedule to build a sophisticated product and he just couldn't or wouldn't keep up. I had to double my workload and he ended up only doing about 5% when he should have done a third and he required a lot of prodding and hand holding. We still managed to ship on time though, but it was really hard on the other 2 people on the team for 3 years.

I'll never make that mistake again.


I think unless you're prideful enough to believe you can identify someone's level of productivity from a short conversation, it's probably better to see the proof. Again, why would I settle for a conversation when I can ask for first-hand proof? It's like a judge dismissing concrete evidence and just basing their verdict on the circumstantial. Sure, it's likely to be right most of the time, but what about when it isn't?


>I think unless you're prideful enough to believe you can identify someone's level of productivity

None of the techniques discussed in this article are even attempting to judge productivity, they are attempts to judge coding ability. It's good that you bring up productivity though; that's what we are really after, isn't it?

>from a short conversation

I'd argue 30 minutes to an hour is not a short conversation. Do you think an imposter could fool you about tech (assuming you are an active programmer) for 30 minutes to an hour?

>it's probably better to see the proof

You're not proving anything except the fact that the candidate has memorized a merge sort algorithm (or whatever trivia you are testing for). What you really want to know is, can they ship?

I think you should at least consider the idea that not only are these tests and homework assignments (past fizz buzz) a poor indicator of ability, they active repel any candidate good enough to be in even reasonable demand. I mean you have a lot of people on not only this thread, but many other threads saying the same thing.

Time for a little reflection, what does your company offer that even a mediocre developer can't get in 20 other places in your city? I mean if I could get "market rates," and a "fine cubicle," and "standard PTO" in 20 places, tell me again why I would bother with this rigamarole?


I absolute agree with you.

I have no idea where the idea that having someone come on site for 6+ hours wasting a dozen peoples time became the norm as opposed to talking to someone for 1 or 2 hours and truly speaking with them to understand what they know.

You're completely right, if you can't talk shop and not know when someone is bullshitting you shouldn't be the one interviewing.

How would this even fly in other industries? Are you telling me that someone can bluff their way discussing the intricacies of surgery to other surgeons without looking like a moron? Or discussing particle physics with a researcher and stand on equal footing? I honestly want to hear a conversation where a liar with no programming skills is able to convince a Senior level Software Engineer that they are on equal skill sets.

The best interview experience I had, where I eventually worked for the company, was having an hour conversation with the Engineering Manager who I would be working under then being ask basic programming Qs based on our conversation we had. Everyone in the company was interviewed the same way. IMO the resulting team was very professional and skilled on various levels from various backgrounds. I actually felt like a human at that job and not a person hired because of x thing.

I'll never forget I had an interview where they asked me a question about finding anagrams. At that moment in my life, I've never heard of an anagram or knew what it was. The Interviewer then explaining what it was to me made me feel very humiliated. I completely bombed every question because I had the audacity to not know about this thing where the interviewers knew about thing therefore I should know thing as well. I immediately understood how cultural biases can effect candidates getting jobs. I can only imagine how worst it is for people not from traditional backgrounds in this industry.


IMO if you're testing for algorithm memorization you're doing it wrong. I prefer to give someone a really simple exercise and then challenge them with added complexity to see how they adjust (since this is what most real work looks like anyway).


You must have a very efficient filtering process. I'd say probably about half of all CS graduates and a greater percentage of people in "coder" positions will struggle to code at an extremely basic level.


Yeah. In all honesty, I would prefer _more_ of our interviews be straightforward coding problems that are reflective of the job the interviewee is going to be doing. Not whiteboarding. Not pseudocoding. Not puzzle problems. Just: write some code, let's see how well you google when you get stumped, let's see how clean your code is, let's see how quickly you implement it relative to other people, let's see how well you document it, let's see how well you collaborate with the interviewer.


This was the one that really stood out to me. MOOCs, bootcamps etc are wildly popular now and vary massively in quality. It's very easy for candidates to look good on paper but not actually know the technical side as well as they should. More importantly, software engineering is one of the few jobs where you can get a (somewhat) quantitative measure on prospective performance so it's natural to take advantage of that.

I'm not saying that All Coding Tests Are Good, but they do provide a modicum of certainty in a very uncertain environment.


The idea that no one can code is often a result of poor interview practices, i.e. coding trivia contests under the gun.

Then misconstruing poor performance as coding ability.


At a former job, I had a set of simple, if contrived examples, that I asked. Programs were short, less than 20 lines; candidates were instructed that the source would compile unless indicated otherwise, in which case they were given the entire compiler output. I would ask fairly simple questions that would expose a lot about the candidates knowledge. Such as:

if (~false == true) std::cout << "true"; else std::cout << "false";

In C++, what does this print? And if they answered correctly, I'd ask them to explain how/why. This question wasn't a deal breaker, but did immediately given an insight into their depth of knowledge of the language.


That is because over last few years it became extremely prevalent industry to make everything about "teams": projects are responsibility of teams, failures are attributed to teams, etc. It used to be that code won arguments. Now it is not ruffling feathers that wins arguments. If I'm not hiring the entire team, how do I know who in that team actually can code based on the result?

What does it all mean? It means exactly what it meant in a high school/college group assignments - one or two people carried the group project but everyone took credit.

I have seen hundreds of resumes where people claimed that they have done XYZ where through some side channels I know they just happened to be on that team but their contribution was limited to tweaking spelling.

I see coding tests as one of the two ways to evaluate that someone can code[1]. I trust it.

[1] The other way is that I know this person and I'm already aware what they can do.


Exactly. The number of people who can't do basic development going into interviews pretending they can is kinda crazy.


Not all technical roles require coding skill.

Unless you meant “lead software engineering role” in which case yes, you are right.


Yes, that is what I meant.

More

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: