I've had some good experiences in hiring by having a conversation with the other person like if they were a human being.
I know it's weird, but I just ask them what they're learning, if they have personal projects and just use that to see if they have experience writing software.
Some people are super-excited about what they're learning, want to talk about the technology and end up making a good impression. Some people are only able to talk about what they're doing at work, the technical challenges and usually make a good impression as well.
Whiteboarding is usually kept to high level concepts like normalization, data flow, logical architecture and things like that.
I like this idea of interviewing by talking about interests and things worked on, though it's complicated, and I can't claim to have figured it all out. A few thoughts...
I used to interview people by looking at their resume, and starting asking curiosity questions about things they worked on, and going from there. I didn't have anything in particular I was looking for, just a vague sense from how they talked about it.
That's more complicated when organizations want to avoid the appearance of IP taint, so, it'd help if they had a side project/interest to talk about. But I'm not assuming everyone has time for that. Also, they might be slammed by work, and in project mental space they can't talk about, and temporarily skewing the impression they give about their curiosity about other things.
What's really unfortunate is all the formulaic, going-through-the-motions interviews these days. We used to make fun of the practice of hiring based on firmness of handshake, how well you dressed, and ridiculous standard interview behavioral questions that might as well be shibboleths for the people who prep for them. Now most young developers are told to prep for conveying culture fit, and answering particular kinds of CS101 coding questions/hazing, which seems to be much the same thing.
That said, though I think I can do better at finding great people than that, I still need to remember to add a big dose of humility, to the idea that I can get a sense of a person as an engineer, just by talking with them. I believe I can do a pretty good job of that, with some conscious uncertainty, and I'm self-aware enough not to simply hire people in my own self image. But of course I surely have more biases than I know, and there's a lot unknowns, and our practice is not science. So I should add in extra humility than I'd naturally have about this, and find the right balance between vision of what I think is needed, and being open to seeing things I didn't already see.
I also need to recognize that I'll have people coming to interview, having read the "Cracking the Interviews of FAANGs and All the Other Companies Who Act Like Stodgy Fortune 500 Hiring Large Numbers of Commodity Brogrammers" prep books, and probably seen that in some/most of their other interviews. :) And if someone shows up expecting the current usual rituals, I shouldn't hold it against them, but rather, it's my responsibility to communicate that I'd like to try to do this a bit differently.
One fear is that I'll explicitly tell people I'm looking to do this "genuine", and some will interpret that as an empty platitude or conceit, in interview prep book behavioral right answer terms. :)
> it's my responsibility to communicate that I'd like to try to do this a bit differently.
Yes, and please, to all interviewers, tell candidates WTF they’ll be expected to do/talk about in the interview, in the greatest detail possible, as early as possible. If FAANG can basically publish study guides you can give more than “here are the people you’ll be talking to, and when.” What about? Will there be exercises. Leetcode? Easy, medium, hard? One of these or is it like half the day? You’ll want to chat about networking? JavaScript trivia most people don’t remember because they use patterns that avoid those parts of the language, and have for years?
Again, if FAANG can give the kinds of guidance they do then Bob’s Mobile Dev Agency can give candidates something about what they should expect without ruining the process.
Like, it doesn’t all need to be scripted but most employers seem to think it’s fun to make candidates come in with no idea whether they’ll be white boarding all day or sitting around singing Kumbaya in rounds
> We used to make fun of the practice of hiring based on firmness of handshake
True anecdote time. A few months after starting my first full time job, I asked my boss why he chose me over the other candidates. "You had the driest handshake" was his answer.
In the man's defence, my first job was as a microbiology lab technician. His explanation was that a dry handshake was a good indication that a person would likely have a better aseptic technique, thus leading to fewer false positive results when performing the tests. Utter nonsense, of course.
>I also need to recognize that I'll have people coming to interview, having read the "Cracking the Interviews of FAANGs and All the Other Companies Who Act Like Stodgy Fortune 500 Hiring Large Numbers of Commodity Brogrammers" prep books, and probably seen that in some/most of their other interviews. :) And if someone shows up expecting the current usual rituals, I shouldn't hold it against them, but rather, it's my responsibility to communicate that I'd like to try to do this a bit differently.
>One fear is that I'll explicitly tell people I'm looking to do this "genuine", and some will interpret that as an empty platitude or conceit, in interview prep book behavioral right answer terms. :)
This seems like a good problem to think about. I don't have a solution, but I think it's a good thread to pull.
It's more that if that is the primary/only thing you are testing, you can get people who ace that but are crap application & product developers/leaders.
Also I personally really hate it when you are treated like some giant imposter when you don't remember some pet CS quiz question when you are 15+ years out of school, and for supposed "leadership" positions. Attitude is like "ho ho, trying to pull one over on us".
It is not expecting Devs to know the foundamnetals, developer should know why it is faster to look for elements in an hash map than in a list. It is expecting to code an hash map from scratch in ten minutes on a whiteboard.
(And hash map are not even that difficult, but I got asked stuff much more complex than that...)
> when organizations want to avoid the appearance of IP taint
So we usually try to talk about what they're learning on their spare time, any personal projects, any cool technology they're looking to learn... then follow it up with what have you done so far to start on this, any small toy projects on this new technology.
It's a weird organization because aside from COVID-19, we've had very little turnover (most people are with us 3+ year with an average of 7 years) so we've kinda changed up hiring a little bit over time.
I currently think that's trickier than I used to. I'd like to mention a few thoughts on my mind on this idea in general (not criticisms, nor making assumptions about the nuances of how you do things).
I live to work, and have had a bunch of open source involvement, and have had serious side interests, so those interviewer questions are to my advantage, when I'm the candidate employee/cofounder.
But a lot of candidates will have families that take up all their non-work time. (IME, some of them are among the best engineers.) And those candidates with families might've heard that many interviewers will discriminate against them for that, wanting people who can regularly put in 60hr+ weeks and surge to 80hr+ of grunt coding. So, I figure they might cringe at talking about how they're spending their "spare" time. I'm absolutely not going to discriminate against them, nor do I want them to fear that they might be discriminated against, or feel alienated because they keep getting questions like that in interviews.
And for the people who're pursuing cool technology through their employer, I'm really hoping it doesn't sound like they prioritized adding this year's unproven hot framework/buzzword to their resume, over the success/risk of an employer's project. :)
The only people who had serious issues were those who said they were not learning anything at their current and had no interest in learning anything new.
Yeah, most developer jobs I've seen require constantly learning new things. And I don't think I'd like working very long somewhere that didn't need people to keep learning -- not so much learning the flavor of the month yet-another-framework, but learning a breadth of things, and picking up whatever is needed. I actually emphasized that in a draft job post recently.
BTW, just to be clear where I'm coming from about families, I think people can focus successfully on both work and family, at the same time. (If they're working in an organization/projects that don't end up wasting so much engineer-time that they think they need people who can be typing 80hr/wk.)
It'd be awesome if all interviews could be done with an informal chat where an expert will assess the interviewee's skills and make a judgement. Unfortunately,
1) You want your experts to be spending their time solving problems, not interviewing people.
2) As the size of the organisation grows and it becomes a more attractive target for lawsuits (fair and otherwise) it becomes crucial to make all of your decisions in a defensible way. 'The tech lead doesn't like this candidate' will get you sued for discrimination if it's the only thing you can say when accused of prejudice.
on #1 and expert that can identify other experts is a force enabler and they offer more value by expanding the field of expertise laterally.
While saying that, just because one is an expert does not mean they are an expert at identifying other experts. It is it a skill set in it's own right.
Yeah, personally I think companies are better off using their best people in the hiring process; the old "A players hire A players" thing may or may not be true, but "B players hire C players" certainly is.
Yes, I remember reading someones arm chair theory that less intelligent people cannot gauge when they are in the presence of more intelligent people, but more intelligent people certainly know when they are surrounded by idiots. It was this persons add on to the Dunning-Kruger Effect and there may be something to it.
The issue with this is that you would be surprised at how some people are good at “talking” but when it comes to doing something simple with code aren’t able to backup their supposed skills
After working for several years in reputable organisation, how bad can somebody be at coding?
Yes there is still a tiny possibly that it is just a talker, but such possibility seems much smaller than the possibility that it pass a coding interview on a whiteboard and then he is not that good.
Depends on what you consider "reputable"; I've worked with people at a large bank doing some great apps and the like, but they were incompetent.
The college project thing applies to the workspace as well; in a team or a project there's always going to be only a few or one that does the most and best work, and a lot of hangers-on.
That is actually a source of impostor syndrome for me, because if I look at my CV there's a lot of really interesting and varied stuff on there, but it was always me as part of a team. How much of what I contributed was original thought vs just copy / pasting someone else's work and changing some values?
That's said impostor syndrome speaking of course, I know I'm a competent developer and ATM I'm working solo on a project. Slowly.
That depends on what you mean by reputable organization. It also depends on what your bar is. Some companies could use "was a Google employee" as a sufficient qualifier, but for others, that wouldn't be enough.
Well, this interview process seem to select for the best talker which I don’t think is the optimal signal for a dev. I know many super competent engineers that aren’t the best English speakers
1. Asking them to talk to you as if they know you is implicitly asking them to reveal quite a lot about their personal life; a life which you, as a prospective employer, has absolutely no business poking your big nose into. Don’t ask about their personal life.
2. As “JMTQp8lwXL” pointed out in another reply, this also opens your selection up to selection bias to prefer your personal preferences, not what would be good for the role. This kind of thing is, unfortunately, rampant, and leads to enormous monocultures where everybody who works at a certain place seems to be the same person; they all dress the same, like the same things, talk the same, and look the same. This is the very opposite of diversity.
Yeah, I don't see having a monoculture as really that terrible, depending on the type of work being done. It can be more efficient and have less interpersonal conflict than a workplace where everyone has a different perspective, gets offended by things the other person doesn't intend to be offensive, requires more communication to get on the same page, etc.
As you sarcastically pointed out, diversity for diversity sake is a false virtue IMO.
I did say that it was implicit, but “having a conversation with the other person like if they were a human being” seems to imply that they are having an informal conversation as in simply shooting the breeze. In such a conversation as an interviewee, it is hard to remain professional and separate your personal and workplace activities, and this seemed to be what the interviewer was aiming for, in order to get to know them better.
GP seems like someone who interviews like I interview. I never ask candidates to talk about their personal life. I ask them to tell stories, and I try to understand whether they can reflect on their experiences and form a narrative. I’ve found that technical ability alone isn’t a great predictor of success. What matters is more that the candidate can communicate their thoughts fluently and that they give some consideration to their goals and how to work with others to achieve them. So I ask behavioral questions. And I ask questions and prompt people because I want to make sure I got a good story out of them. Some people don’t immediately trust you with a good story so I try to establish trust first. I do that by actually telling them what they’d be doing and why and what my employer is about first.
I’d rather hire for those skills plus code reading ability because I’ve honestly never used a single “algorithm” in my entire working life.
People are normally at least discouraged to talk about what they did at a previous employer at any level of detail; there might even be NDAs involved. So, when you don’t explicitly ask candidates about their personal life, but ask them to “tell stories”, what do you expect them to talk about?
Employers seem to be entirely ignorant of the power relationship in job interviews; interviewers all want to talk to the candidates like a peer, even though this is not at all appropriate.
I like this idea. But it doesn't objectively scale in a repeatable manner. Whether or not the candidate's answer was interesting to you is subjective. It could also lean on implicit biases where you end up saying, "hey, I like that guy, let's hire him". But it's reasonable enough for small scale engineering organizations (<100 people).
> you should not put much stock in the interview process. But there is a sense in which it is very impolite to say so.
The interview processes are still very fair compared to things like "performance" reviews and company "levels". People often pretend these types of internal processes are indicators of actual competence when they are usually just based on things like nepotism and corruption.
At least these algorithm interviews can successfully weed out people who can't write basic code. I can't say the same for promotion processes.
Does this mean that only family members and friends of managers get promoted? Or that you have to bribe folks in the company to advance? What do you actually mean?
What I would say is that most companies haven't really put much thought into building career ladders for their people.
Or into defining roles that provide enough breadth to capture a diverse skillset, enough autonomy to get the job done, and enough in the way of pragmatic constraints to scope the role.
Algorithms interviews only tell me that somebody can pass an algorithms interview -- one in which the solution space is very narrow, and there are a few correct answers.
In real-world engineering, this is very rarely the case.
There are always trade-offs to make. Good engineers know how to do things the cheap way now, and pragmatically invest more as we need to -- because they follow practices focused on, second only to delivering value, making the code easy to change.
> Does this mean that only family members and friends of managers get promoted? Or that you have to bribe folks in the company to advance? What do you actually mean?
In many cases I've seen people being promoted due to being close friends with the managers while contributing little to nothing of value. Also dishonestly claiming credit for others work is a great way to advance. Donald trump types get promoted.
> What I would say is that most companies haven't really put much thought into building career ladders for their people.
These career ladder guidelines often make the problem worse. They allow poor engineers to get away with avoiding real work in favor of soft "cross-team impact". Often the lower level engineers working on the difficult technical problems are making the real contributions. Good engineers should be focused on solving the most difficult problems, whether cross-team or not. They should not be focused on gaming silly HR guidelines, people who take these guidelines seriously make the worst engineers.
> Algorithms interviews only tell me that somebody can pass an algorithms interview -- one in which the solution space is very narrow, and there are a few correct answers.
> In real-world engineering, this is very rarely the case.
I don't agree. Psychometrics shows general cognitive ability is very important, and algorithms interviews to an extent seem like a decent test of it. Coming up with difficult solutions isn't happening without proper grasp of the basics and strong general cognitive ability.
From reading this comment, I think it is possible that you undervalue non-coding work. That is, I think it seems like you don't see as much value in it as some companies you've worked for do. There isn't really an objective measure to say whether you're more right or whether those companies are, but it's definitely the case that career trajectories are driven by what companies value rather than what lower level engineers value.
But take my thoughts in this with a grain of salt: my bias is toward believing that most engineers place way too much value on coding work and not nearly enough on the (in my opinion) harder work of figuring out how to successfully execute an ambiguously defined project.
Great read! Another reason companies don't like to do this is due to having less trust as companies grow. Consider a one or two person company's interview process. Because the everyone trusts each other completely (otherwise you wouldn't be trying to build a company), its easier to have an interview process that simulates real work because you trust you colleague doing the interview and don't have time to come up with a text-book process.
As the company gets bigger, the chain of trust weakens. The only way to keep something that resembles trust is to comeup with a process that everyone can trust rather than just trust each other. If such a process didn't exist it would always be one person's opinion vs someone else and there would never be consensus. This is made worse by companies hiring way too aggressively chasing that next funding round or IPO than hiring when they really need to. And would rather hire someone vetted by a text book process than trust their colleagues to look for diverse skills for their teams. This vicious cycle has been going on for way too long and will have to end sometime.
Over the years, I have grown too weary of techniques/methods (that involve things like whiteboarding to prove one's worth as an algoninja) and a 2 hourish interview.
I have found good succes with the following approaches. Being in a small firm helped, but I haven't had too many difficulties following this in a large-ish firm too. (product unit in an Indian firm)
- don't hire in urgency. I believe in building a talent roadmap/planfor the team, capabilities over next 6 months, and thus hire someone who I see fit enough to grow into that role
- have a questionnaire sent before, which has open-ended topics ( views and perspectives on containers, infosec, cloud, imperative vs declarative, microservices vs monoliths, their fitment (and otherwise) to use-cases - etc)
- talk about a recent tech challenge the person was able to tackle and resolve, try getting to extreme levels of detail (to understand whether the solutiion was a true fix or just a patchy workaround), the tradeoffs made while going that route etc
- try to do a systems design session, jointly. the problem statement closer to the actual line of work
- try to use a seemingly open ended objective (cost optimization, reduction in MTTR. increase MTBF, increase scalability etc) relevant to our line of work or current set of problems and see if the candidate can find a good set of questions to find their way towards a workable solution, etc.
- for simpler stuff, hiring interns and then grooming them into productive, trained members of teams is often a useful way
- responsibility doesn't end at offer rollout. setting their teammates for success till they are in the system is the leader's responsibility
However, when am on the other side (am the one who is answering, pursuing a job elsewhere), I have come to hate the find-the-algoninja ritual so much that have stopped taking up any that forces such a deal. Am better off retaining my sanity.
Makes sense that when you're interviewing programmers you don't have to look for candidates proficient in your particular area. You need someone willing to learn
Conversely, if you're interviewing, signaling that you are highly interested in learning turns you into a strong candidate.
I learned this one by accident when I interviewed at a company I didn't care about
Relevant excerpt from a blog post I wrote [1]:
> I was wandering around the booths with my bag full of swag. My eyes caught a pile of rubik’s cubes being given away by some company I’d never heard of. I wanted one! Of course I couldn’t just go up and ask for it directly, so I went and chatted with the guy manning the booth. His name was Vince. A few minutes later I walked away with my prized rubik’s cube in hand. That evening Vince called me up offering to conduct a full interview loop on campus. I already had a job offer from a company I liked, but I thought “Sure, why not? I could use the experience”
> I had no intention of joining the company, they were some boring finance business. There was nothing to lose. So during the interview I felt free to ask any question I wanted. When I thought I got the answer to an interview question wrong, I’d ask “I don’t think I did so great here. What’s the right answer to this problem?” (I wanted to learn the answer for future interviews!). When asked a challenging question I could grin and delight in the problem solving aspect instead of worrying about how badly it would reflect on me. (Remember my smart aleck remark in the intro? That was this interview.)
> Turns out they liked that: the next day I had a second job offer. At a significantly higher salary than my first one. Oh, and they wanted to fly me to New York in two weeks for an introduction to the company. I still didn’t want to join, but a free trip to New York? Sign me up! The company’s name: Bloomberg
> Bloomberg was a practiced hand at recruiting. They had a full two days of events to leave us starry eyed about the company (I came thiiiiis close to accepting their offer). While there, Vince told me I’d made a great impression by fearlessly asking questions, even when I was stumped.
> Since then I’ve stopped hesitating before asking any question in an interview. Let your curiosity run free! Don’t cage it! And as an interviewer, I can attest: sincere interest is always a good sign.
I completely agree that that's the best way to do an interview, without pressure and treating it as an intellectual exercise with some fun. Unfortunately FANGs (well at least FB) have a tendency to mark you down mentally when you show any weakness. To them you're up against 10 other candidates who got to the right answer not only correctly, but fast.
While maybe that's true at FB (it's not true at Google), your brain flips a special bit if you convince yourself that there's no time crunch.
There've been various studies done which show that individuals who are under pressure to perform any creative work actually have worse output in the same amount of time compared to people who're doing that exact same creative work for fun
I think there's a famous study about people being asked to attach a candle to a wall with a box of nails that's the most well known example of this phenomenon.
So even if what you say is true, it's in your best interest to forget that fact and focus on the fun
Getting to the right answer fast is only relevant if they asked the right questions. I think the point of the article is that interviewers are not asking the right questions.
This is almost completely unrelated, but what was the quality of the cube? None of the cubes I ever got from swag booths ever were very usable; I think they were optimized more for visual appeal than ease of turning the side, in addition to likely being cheaply made. It's totally understandable, but on the other hand, a company that gave a cube that wasn't such a pain to use would have gotten some free marketing from me using it.
Getting rid of the litmus test and embracing the idea that the first thing you cut your teeth on might be alien to good developers are both amazing ideas. Working on a toy problem is a terrible way to figure out whether a person will actually be beneficial to an organization. They might take twice as long but produce 1/10 the bugs, they might be someone willing and able to dig deeper than anyone else to solve those long-running problems, or something else amazing. Even giving them a meaningful task manipulating the organization's code might be bogus - with the complexity of current tech stacks there's a damn good chance they've never used yours. If you happen to give them a problem which touches any part of the tech stack they aren't familiar with, they are likely to come up with a clunky solution or no solution at all, no matter how good they are, simply because they are not familiar with the idioms.
Take 4 pages worth of interesting/diverse parts of real production code (Non-IP sensitive) and do a 'code review' where the candidate is the reviewer.
For more junior candidates add blantant defects or bad design into the sample code.
I have experienced this both as a candidate and as an interviewer. As a candidate, I found this a lot less adversarial and less stressful. As an interviewer, IMHO this gives a lot of insight into how a candidate thinks, without being a huge time sink, like 4 hour whiteboard tests or take-home assignments.
I like this idea on principle, but I don't find it easy to jump into new codebases and see what is going on. Its usually after working with it for a couple of weeks that you see what is good and bad. Maybe something like this but given to the candidate to look over in their spare time.
One of the best interviewing experiences that I had was when I was asked to bring in my laptop with some of my code to discuss. I had a side project that I was working on at the time so that was not a problem. I showed the interviewer, he asked questions, asked me to add a simple feature, I did. No algorithimc nonsense (which is basically about as reliable as a coin toss as to whether I will get the answer on the spot). No hours of my spare time wasted on take home challenges. It felt like less pressure than usual as it was code that I knew.
The best programmer interviews I have ever underwent -- as a candidate -- were almost identical to "doing real work".
Either because we actually did real work -- the interview was literally "work with us for a day and see how you like it" -- or because the interview process primarily centered around exploring problems in a wide solution space, collaboratively with the interviewers.
I know this is a very polarized method, but I currently feel that take-home coding problems is the best method for prepping / selecting programmers for interviews.
For it to work, you obviously need to impose some constraints - it's unreasonable (IMO) to demand too much time and energy out of candidates, but it should also be complex enough to highlight a wide variety of skills.
But what's more important, is how you use it. It's an excellent tool in interviews, because you can pretty much spend an entire (technical) interview just discussing their implementation, and have more low-stress discussions around their choices and strategies.
A good and experienced candidate will both showcase good code, and will be able to discuss quite fluidly and in-depth around their various choices. And if you want them to expand a bit one some technical things, you can just ask them then and there.
But, again, I understand this is a wildly polarized method. Some coders would rather spend xxx hours on leetcode / hacker rank / etc. and learn every problem under the sun, and then tackle some random whiteboard question.
I guess if you're fresh out of academia, that's the method which most resembles your previous experiences with testing. Some are good test-takers and love trivia.
But for me, I'll stick with the take-home problems for now. Both when looking for devs, and when looking for jobs myself. Simply put, you get more data out of such problems. If a candidate cheats by hiring someone to do their work, it'll more likely than not still show up under the interview and questioning.
edit: But as mentioned, it all boils down to how well you design such problems, how they are executed, and how you choose to use them in interviews. There are many ways to effectively render them useless, or even hurt your chances at attracting good devs.
There's simply no time for that if you're already working and have a busy life (i.e. kids).
One thing that I don't see much in the discussion is the assumption that we all have time to go through the interview-course material which often is irrelevant to our day to day work -and also irrelevant to the advertised role.
But then again, everyone knows it, interviewing is just another role-playing. You do your part you get the role. That or you get in the company via a good recommendation avoiding all that sh!t.
Like someone in valley put it, "competition is for losers". It applies to jobs too IMO.
If you can't find the time to spend 2 hours on a coding challenge, maybe you simply don't want the job enough?
The hard fact of SE job search is that with most modern companies, you're going to get some sort of technical questions.
For ME, I'll much rather be asked questions about something I've just written, rather than random DS & Algo questions.
In fact, how many experienced know DS&A down to the bone, at the levels of Leetcode medium and hard, and can just wing an interview, with no prep? I'd imagine under 10%.
So yeah, I'd rather take a couple of hours to make some simple application, and review that, that to spend N times that combing through competitive programming questions.
(Most of all, I'd rather not have any of those, and let my previous work do the talking, but that's the world we live in.)
> Ask them "What makes you think you could contribute around here?" and then listen to them.
I think the author didn't lead msny interviews and only imagines themselves as the applicant. I had so many people apply for programming positions who couldn't even type the most basic for loop. The reality is, you need to find out the "level" of the applicant - are they a complete beginner, can they contribute basics, can they design medium sized projects themselves or can they be solely responsible for a larger one. That quoted approach doesn't help with that at all because many people will just tell you what you want to hear. Live coding and whiteboarding can give you good insights.
Everyone talks about how hard it is to hire programmers, but is it actually harder than any other white-collar engineering-type jobs? How do you hire good electrical engineers?
Every method of hiring candidates discussed seems to be accompanied by related horror stories.
I know it's weird, but I just ask them what they're learning, if they have personal projects and just use that to see if they have experience writing software.
Some people are super-excited about what they're learning, want to talk about the technology and end up making a good impression. Some people are only able to talk about what they're doing at work, the technical challenges and usually make a good impression as well.
Whiteboarding is usually kept to high level concepts like normalization, data flow, logical architecture and things like that.