In my experience, it’s fairly easy to judge technical skill. A friendly conversation about technical interests and recent projects can often be enough.
Bullshit. Sounding credible in technical interviews is a skill, not the same skill as actually being a good programmer, and might even (statistically, in the large) be close to orthogonal to it.
We found this out the hard way. At Matasano, we started our work-sample hiring process[1] as a way of filtering out the smooth-talkers. Before work-sample tests, we'd spent loads of time on carefully designed interview questions; interview design was something close to a hobby for some of us.
Of course, it was only after we started doing work-sample challenges that we discovered that not only were a lot of excellent-seeming candidates actually not capable of delivering, but an even greater fraction of the candidates our "friendly conversations" were selecting out were in fact perfectly capable. It was a bad deal all around.
Whatever you do, don't fast-path "senior" developers. Everyone should run the same process for the same job. Not only do you risk hiring people who won't work out, but you're also depriving yourself of the most important data you need to iterate on your hiring process.
I disagree. If you can’t tell the difference between a smooth talker and strong technical competence you are probably interested in the wrong qualities. I have interviewed enough now to see why some companies cannot figure it out. Ask yourself if you really actually want a senior or a strong junior.
It comes down to interest. A good senior got that way because they like solving challenging problems. They are not interested in trendy framework bullshit. In other words talk about the problems and potential solutions. Are things in place to make the job easier? Seniors don’t need easier and this is a huge turnoff.
When I hear companies try to sell me with frameworks and process I know they are blowing smoke. At the very least they are boring and at worst you will be working with incompetent people who are as self-diluted as the company. I agree that filtering candidates is a good idea though.
The reason why some companies cannot figure this out is because they don’t value the problems at hand. They need bodies to put fingers on keyboards and are willing to pay more for people who don’t completely suck. Experience and competence are not the same as excellence but considering the candidate pool I can see why companies compromise on quality.
I hear what you're saying, but I think you're maybe creating a straw man argument here. A smooth talking engineer isn't one who sounds like a used car salesperson or is just hyped about the latest trendy framework.
There are people who can talk through challenging problems at their former companies and how the problems were solved. They can tell you everything you'd want to hear because it's true. Except…they didn't implement it. Maybe they are best friends with the person who did and understand in detail about the tradeoffs and the neat hacks and the insights learned along the way, but couldn't build it themselves.
Those are the "smooth talkers" of the engineering world. Those are the people you can't catch just through a verbal interview.
On a related note:
> If you can’t tell the difference between a smooth talker and strong technical competence you are probably interested in the wrong qualities. I have interviewed enough now to see why some companies cannot figure it out. Ask yourself if you really actually want a senior or a strong junior.
> Those are the "smooth talkers" of the engineering world. Those are the people you can't catch just through a verbal interview.
I agree with this. I was a hiring manager, and there are those that can really talk technical, in detail. You really think they know what they are doing, how to solve complex problems, how to come up with solutions. You put a keyboard in front of them (or pencil and paper), and they go "uhh, errr, ummm." and fail miserably.
I think until you have interviewed a LOT of people, it can be hard to quickly spot this. Some people are masters at telling you how someone else solved the problem as if they solved it, but they can not solve it themselves.
and don’t forget the reverse problem mentioned originally: false negatives.
you can have someone who is a whiz at practical and specific solutions, who thinks critically and analytically and just gets an enormous amount done WELL. And empowers those around them to boot!
they have the reverse problem to speaking about other peolle’s work as their own. instead, they speak of their own work as teamwork.
this effects many great people. also women and poc are particularly likely to do this because they have been socialized to not speak too highly of themselves. “model minority” etc.
if you as an interviewer are already skeptical of what someone says, you will increase false negatives with people who you are asking to verbally “prove” their work and yet have cultural memories of being penalized for “bragging”. they’ll describe a solution and downplay it as challenging or hard because women aren’t likes le when they’re the smartest person in the room, etc.
an interview process should seek to understand many skills: practical, implementation, execution, problem solving, design, high level, communication skills.
a varied process that focuses on a few specific skills, one at a time, is likely to convey the most accurate signal.
The problem with hiring is that a false positive is much more damaging than a false negative. Getting the group of people together to vet a candidate is expensive; recruiters are expensive; for the candidates, taking the time off is typically pulling from a very limited bucket of just a few weeks every year; flying people in to interview is expensive; and ultimately, to go through all that and hire someone bad makes you go through the whole process again. If it takes you a few months to figure out it's not going to work, it's unlikely anyone desirable from your original candidate pool is still available.
False negatives are expected, and honestly probably good overall and in aggregate, because it decreases the odds of a false positive. One of my first bosses that involved me in the hiring process told me one day that the point of interviewing is not to find reasons to say yes, it's to find a reason to say no.
The "find a reason to say no" can be very damaging as well if taken to the extreme.
I've seen people that were entirely qualified for the position be rejected at the company I work for because they made some totally understandable mistake - I'm talking about people that took the time off to take a 5-hour on-site coding assignment, and made a mistake but would have had a passing (and possibly good) grade on an academic evaluation.
And now we have 5 open positions and nobody hired for them.
> I think until you have interviewed a LOT of people, it can be hard to quickly spot this.
What is your threshold for "a lot"? I have given a few dozen interviews. I always ask at least a couple background questions. The number who even have a polished delivery for that part at all are a minority, and the couple that tried to bullshit me we're painfully transparent. Maybe I just haven't done enough yet.
These problems are very simple to solve. Get them to go into detail. Get them to explain their thought process while looking to solve the issue.
If you feel that they're talking about a problem someone else solved, ask them that directly. (Did you work with others? etc) If they're lacking on the technical details either it's been a long time ago or they didn't do it.
> explain their thought process while looking to solve the issue
I would think they and their colleagues discussed alternatives together, collaboratively in for example a Slack chat — so an interviewee can give you good replies about the thought process and alternative solutions that were considered and discarded. I would assume. Or maybe the interviewee him/herself came up with ideas, that his/her colleagues realized weren't going to work, and explained why, for him/her. Then s/he might be really good at describing the thought process.
No, let me give a specific example. Imagine you're interviewing a candidate and they're talking through how to design an analytics service. They begin talking about e.g. database architecture, and how this type of data is most appropriate for a star schema. They start talking about the tradeoffs of row versus column orientation. They mention they'll need to do indexing for performance and talk about the index space versus query speed tradeoff. They say they'll do joins on the x and y tables.
Basically, they volunteer technical challenges they're aware of while simultaneously telling you what the high level solution is. But then you put a terminal in front of them and ask them to set up Postgres in a star schema with some dummy data, and then to write a query joining the two tables they were talking about before. Despite Postgres being on their resume, they'll completely flounder and not even know they need semicolons to terminate commands. Their joins won't just be wildly inefficient, they'll be syntactically incorrect and refuse to run. They won't be able to create, insert, select, truncate, drop, etc. They don't know how to create an index and can't mention any of the options for indexing, let alone the default provided by Postgres.
Keep in mind this example is just meant to be illustrative. Thinking through how to fix the scenario might not generalize to all the ways this can manifest. The kernel of how this arises is a person like so:
1. They read a lot about technical solutions at a high level. They can follow that if you have problem A then you need, roughly, solution B.
2. They have no contextual flexibility or practical foundation for understanding their solutions. They might have read Designing Data Intensive Applications, but they can't actually code and have never administered a database. To the extent they understood the book, they only internalized low hanging fruit.
3. They are charismatic, or ar least comfortable talking about technical topics. They will try to lead the conversation as much as possible, which is where you see them volunteering technical challenges and then offering solutions. But if you force them to answer heavy technical questions which drill deep into a specific area, they'll probably try to zoom back out.
I may be miss-characterizing your point but there have been many times that I’ve sat down at a terminal and not been able to remember how I did something a few months ago. On the job the important thing is knowing the high level goal and the fundamentals of what you’re doing. The syntax can be easily googled.
edit: yeah upon re-reading you’re talking about completely obvious lack of practical experience... fair point
I came multiple times across similar individuals as you have just described. They are very good at talking in details about a tech subject, charismatic, defensive and argumentative all day long over their solution even though it does have lots of holes. But when it comes to actually implementing a task. They usually fall short. They will quickly grap an existing/similar solution from ie github., spend many hours trying to understand it, Then copy-pasted and present it as their work (half-baked solution)
Even though the whole thing could be very simple. Another thing is, they will come up with reasons that the issues with the user story/task for code/solution is due to environment or some other reasons like tools, framework, scalability and "bs".
Wow, I feel like I'm the sort of person this comment is calling out. What advice would you give me so I can be the real McCoy? The only solution I can think of is to keep writing as much code as I can, so I can get real experience instead of just hot air.
Turn off your internet when doing work. Buy a stack of Postgre books. Memorize them. Never, ever, ever use stack overflow. EVER. Don't use google. Memorize everything.
That's what that asshole is looking for. :)
Though to be fair, if you come up with that strategy and can't do -anything- at a sql console, I'm going to ask how you normally interface with the database, because that's like a Linux expert not knowing how to use tar or ls or something.
Most humans use a GUI when they can(with their connections stored and autocomplete and a few other nice-ities. Of course if you only ever use MySql or Postgres through command line, congrats - you know how to do basic connections(I assume using -h makes you an imposter)
As for the Linux/tar piece, I've used Linux on desktop for a few years(both Ubuntu & Fedora) & have used Suse and CentOS for servers for much longer.
I can tell you tar means tape archive. I can tell you I mostly use it with gunzip to compress it. I still google/reverse terminal search what flags to use with it both when archiving it and unarchiving it. I could probably remember some flags if I spent enough time thinking - why would my brain waste that much effort though?
I don't work as a backup administrator. I have better things to worry about knowing/having present at the forefront of my (admittedly human sized) memory.
I don't really intend to call out anyone, so please don't feel that way. Keep in mind what I'm trying to construe is someone who has a lot of broad (not deep) book knowledge, but who can't do even basic practical implementation.
If you can explain how to design a system and you can do it, but don't know the exact commands off the top of your head, my comment isn't describing you. I don't expect people to e.g. know awk like the back of their hand, or to write perfectly compiling code on their first try.
But even if you don't have perfect recall of the commands, it should be pretty clear whether or not you've ever opened an editor and done basic implementation. If the GUI is your thing that's fine. But your knowledge must have some practical foundation which demonstrates you can actually walk the walk.
Writing more code is, indeed, the solution. This may feel challenging or even impossible because it takes away time from doing the research of actually understanding why everything works. Ie, due to time constraints, you may need to implement things more often without understanding how they work. This might make you less good at talking about solutions, but on the other hand better at actually providing them.
> If they understand their friend's work deeply, doesn't that imply they've done something comparable themselves?
Imagine you're interviewing a candidate and they're talking through how to design an analytics service. They begin talking about e.g. database architecture, and how this type of data is most appropriate for a star schema. They start talking about the tradeoffs of row versus column orientation. They mention they'll need to do indexing for performance and talk about the index space versus query speed tradeoff. They say they'll do joins on the x and y tables.
Well, do you think my comment demonstrates deep understanding of database design? I don't feel I have deep understanding of databases, but I can certainly talk to you about very basic things like indices and joins.
Basically it's like someone else said. They read a book and know a lot of answers, but they can't do the most basic implementation of a solution.
Well, do you think my comment demonstrates deep understanding of database design?
No, it seems about on about the same level as being able to paraphrase the abstract of a paper about the system. I would not take it as showing that someone has read past the first page. A high-level overview just isn't enough for that. You have to ask your own probing questions too. Limiting the conversation to the particular problems they bring up is essentially taking them at their word when they claim to be skilled. I've seen lots of occasions where trying to drill down for a bit more detail on some part of what they talked about consistently came up empty (without going anywhere near sitting down at a computer to write a fizzbuzz equivalent).
How do you suggest getting into designing and working with distributed systems without job experience? Maybe virtualizing a data center on a home server?
There is a huge gulf between being able to follow a thought path and being able to find it oneself. I couldn't do the same work in the same timescale as them, I wouldn't make the same decisions or spot the same pitfalls. I am simply not as good. But I spent a bunch of pub/work time discussing it and could project an aura of authority on the projects if I wished.
Look at it this way. I was able to understand all of the maths proofs taught at my degree, but I could not have come up with them myself.
Yeah, that's a great analogy for the core problem.
Imagine putting a math problem in front of someone and asking them to solve it. They correctly identify it as a system of linear equations. They volunteer that they would solve it using x algorithm which has a time complexity of y.
Then you ask them to actually solve it, and they can't even make the first movement towards doing so. They mentioned LU decomposition, but they can't even do Gaussian elimination on paper. They don't know what elementary row operations are. They can't obtain an augmented matrix or put it into (reduced) row echelon form. They don't know anything about linear independence or the rank of a matrix. You put an inconsistent system in front of them and they keep banging away at it, determined to find a solution...etc.
That's what it's like interviewing one of these senior engineers. It's surreal - they confidently pattern match the problem using limited heuristics, and they toss away low hanging fruit to demonstrate knowledge. But when you ask them to do something practical and specific, they either refuse and zoom out into abstract-land again, or they hopelessly fail.
"Principal engineer"? Do you actually have an engineering degree? Have you formal education in engineering?
What is system engineering?
It's not just you; the entire IT industry is suffering from systemic curriculum vitae bloat. That's why the working conditions are so bad in the professional sense.
The subject of what constitutes an engineer or what entitles a professional to the "engineer" title has been discussed ad nauseum here on HN.
Spoiler: "engineer" as a title for someone who does computer programming and software development without a license is perfectly fine and acceptable in the USA. In Canada, however, it's probably not.
Those states you mention do require it, but what I was getting at is that in general terms Americans don't blink when someone refers to an engineer without reference to licensing.
On what basis are you claiming software engineers are not engineers? What constitutes being an "engineer"?
For the record, the National Counsel of Examiners for Engineering and Surveying, which is the US body that regulates engineering licensure and "Professional Engineering", recognizes Software Engineering as a branch of the engineering disciplines:
"Professional (Licensed) Engineer" is not typically used in software development, but if you care about having this credential, you can become a credentialed Professional Engineer in Software Engineering.
Google defines engineering as "the branch of science and technology concerned with the design, building, and use of engines, machines, and structures". An engineer is "a person who designs, builds, or maintains engines, machines, or public works". Software engineers certainly fit that definition from my perspective, because software is a type of virtual or abstract machine.
Software engineering certainly qualitatively feels like other branches of engineering to practice, such as electrical engineering or computer (hardware) engineering. I've never designed a structure, but I imagine the principles are the same: understand the requirements of the customer or sponsor; devise a design that accomplishes those goals within the constraints of the medium you're working with, using principles of scientific reasoning to evaluate what is possible and whether a design will meet your needs.
Last but not least, software can be just as critical to human life and safety as the artifacts designed in other kinds of engineering. (But being critical to health or safety is not a prerequisite for something to be engineering. It's still engineering if you're building a rocket that only robots will fly on, after all)
There may be people whose work on computers does not constitute software engineering. Running some calculations in Excel is probably not engineering. But I think there are plenty of us who build large scale systems that need to operate with high availability under demanding requirements, the properties of which need to be painstakingly planned and analyzed and sometimes mathematically proven -- we who are trained as scientists and engineers, and who apply these principles in our work -- many of us consider our work to be engineering, and consider ourselves engineers. I certainly do.
An engineer gathers requirements through a formally defined process and writes a formal engineering specification which translates those requirements into actionable norms. For example: "server operating temperature shall be between -40 and +70 Celsius at 400 m above sea level and the power supply shall operate at Voltage between 110 to 240 Volts at either 50 or 60 Hz with peak load of 35 Amperes". Or: "the program shall support following options", with a detailed specification on what those options do and how they will do it. Or: "the software shall respond within 25 ms of receiving the request on port 4096. The protocol used shall be TCP". RFC documents are often good examples of system engineering and architecture.
Simply knocking in a program as a code monkey or hacking haphazardly on a program until it works in some way which isn't formally defined isn't an application of scientific theories in computer science into a practical product, which is what defines engineering.
Most programmers I know aren’t simple code mnkeys, they solve actual problems based on the constraints of the problem domain. They model the data and devise algorithms to best process it. Sure, sometimes you haphazardly hack on a program to get it to do what you want, but a lot of the time you have to think critically about it, what you are trying to accomplish and how to do it within the constraints you are given. That sounds like engineering to me and sinse “engineer” isn’t a regulated concept where I am, I have no problem calling myself an engineer. I don’t see how its lying.
Hate to break it to you I worked for a well known civil engineering consultancy and some of our pitch's where economical with the actuality about our engineers experience :-)
I have no idea if there are rules in my country about calling yourself an "engineer", but you can only use the title 'ir' or 'ing', both of which mean ingenieur, which translates to engineer, if you've graduated from a technical university or high-level technical school (technical college? higher trade school? 'ing' is from a level lower than a university degree).
This is a good post. Have been in the UK startup environment it was crazy how many companies got funding but didn't had a strong senior in their software department.
Because of this their hiring process was aimed to hire more "strong juniors" even tho they didn't realize it. A strong senior is a person who completely changes how you are even approaching the problem or someone who shows you problems you hadn't seen before. They often see tech in the context of business as well.
As you said, usually, they can't be bothered about learning the new framework of the month or new backend language of the month, but, they have a set of battle proven tools to solve tech problems for businesses.
P.S: I do find it hard to distinguish between bullshit talkers and people who actually get stuff done in the interview process.
There's times when i think i'm coming across as the bullshitter. "Tell me of an accomplishment you're proud of". I struggle with this one, but I've given this example years ago.
Worked at a company which did nightly data imports. Things worked until 'companyx' became a cEient, and the imports were huge. They would take 18-20 hours. Then longer. eventually they were touching the 24 hour mark - unacceptable. Client's data would be more than a day behind. Granted it was a moderate amount of data, but shouldn't take that long.
I was 'new' there - only started a month before - and the rest of the team who'd put this together had been there a year or more. I reviewed what was in place, took a couple of days, and got it down to an hour. Then worked with the existing team and we got it down to under 30 minutes with some tweaking.
I do see some eyes rolling when I tell that, as I know it can sound terribly self-aggrandizing. However, I had a decade of experience at this point, and the rest of the team was just out of college; they'd never faced this problem before. I basically just took the data and imported in small chunks in to in-memory tables (to avoid hitting the disk), and copied those to disk every X rows, and dropped indexes until everything was done. It wasn't rocket science, but did take someone who had a deeper understanding of DB mechanics.
As I'm telling this, I always realize they have no way of verifying this, and essentially I'm just another bullshitter. The more believable I sound, there's an equally high chance I'm either really good, or just a really good bullshitter, and nearly every time, the person I'm talking to has no idea how to tell the difference. It's worse as you get older, because the younger folks just think you're waxing nostaligic about the 'good old days'.
It doesn't help that in our field the context for many of these past achievements is quickly lost and forgotten.
You can't just say "I did X in Y by using Z": you need to begin by explaining that once upon a time there used to be a thing called Y, and on that thing it used to be very hard to achieve X, but in those ancient days there was also a tool called Z, etc.
This could be an example of the start of a competency based interviewing question. These questions usually start similar to "tell me about a time when". You ask for an example of them displaying some trait you care about, and then dig in to the details of what happened, why they did what they did etc.
CBI is a fairly effective technique for general interviewing, because you uncover how people actually behave rather than how they like to think they behave. Most of the gold is in the follow up digging questions, which should separate the bullshit answer from a real one.
I think it's also important for seniors to look at new ideas at a regular basis. People who don't, tend to overrate the "good old [insert language/framework here]". You should not stop improving yourself, just because you are already able to get stuff done. (Example: Java introduced lambda functions in Java 8 and many Java seniors are still not using them)
I think the difference a senior person can make is to see what things the new language/framework/paradigm still doesn't solve and think of ways to evaluate their risk to a particular project's requirements and how to mitigate that risk. Things like downtime for upgrades or migrations, handling dependencies on ecosystems of community developed libraries without quality and feature change controls, the ability to update the system when it's been in production for 3 years and see few changes have been made and the original developers have all moved on, etc.
> A strong senior is a person who completely changes how you are even approaching the problem or someone who shows you problems you hadn't seen before.
I think this is overstated. Disruption for the sake of it is often not that helpful in the context of the business (although, in fairness, you do go on to state they often see tech in that context).
I much prefer people who are delivery focused to those who are overly idealistic or want to change everything out of the gate: a good senior understands priorities and knows when to make a trade-off to live with a sub-optimal situation or solution in one area in order to deliver greater value elsewhere.
The higher you go the more compromises you have to make.
That said, nobody is talking about disruption, just wisdom.
It just so happens, sometimes that wisdom will tell you that shipping shit out the door in the name of delivery focus is going to cost you more than its worth in the long run. Calling them overly idealistic to justify your laziness just makes you look bad.
I'd say a truly good senior can tell the difference between a startup and a mature company and adapt accordingly. It takes one set of skills to ship shit out the door as quick as possible and an entirely different set of skills to come in and clean up the mess the kids left behind.
This is why you are selling. Not buying. Because nobody wants to clean up your delivery focused mess.
or... nobody wants to pay to clean up the mess. i know people who will do it. I will do it, but I can't clean it up, and delivery loads of shiny new features, and do it all while trying to justify every 30 minute block of time and asking if xyz is 'really' necessary. oh, and you also need to be able to answer some real questions about your own business process, because often your existing practices are just 3000 lines of crap in a function file. when you say "it's broken" and I ask "what's the correct behaviour?" and you don't answer... it'll never get 'fixed'.
I'm not overreading, you said you prefer shipping sub optimal solutions. Sounds like to me.
I didn't call you lazy, I said that shipping in the name of delivery focus was lazy, or at least your argument about idealism was.
Fact is, shipping quality is the far more optimal solution and always will be. Making the trade off and adding technical debt is never a worthy trade long term. The only people who gain from it are you and your team. The rest of the company eventually grinds to a halt and begins taking more and more shortcuts around the code which just reinforces everything in a viscous cycle.
> I'm not overreading, you said you prefer shipping sub optimal solutions. Sounds like to me.
I was talking about prioritisation. I am very wary (or perhaps jaded) with people who want to fix everything all at once with no regard to the wider effects on the business of doing so.
I didn't mention anything about quality with respect to what we do choose to deliver, although I can assure you that user experience - of which quality is a key facet - is our utmost concern.
I thought my intent was clear, but sorry if not, and hence my comment about overreading. I wrote very little from which you (and you're not the only one) appear to have extrapolated quite a long way.
No. The difference between talking (very intelligently) and doing is huge, and a sports analogy might be illustrative. If you ask detailed questions about football to hire a NFL quarterback, your most intelligent responses will probably be from a coach. “How should you change the position of your right shoulder if you see that a fast edge rusher is approaching you from the left side and you have two open receivers?” A coach like Bill Belichick might have a fantastic, detailed answer demonstrating a thorough understanding of every aspect of football past and present, but he could never make the throw himself.
That analogy highlights what’s irritating me about this post and this entire thread discussion - there are 1,696 players in the NFL, each with an average salary of $1.9 million. When people talk about hiring “senior engineers”, they behave as thought they’re auditioning an NFL quarterback - the 1 out of 100,000,000 people who can actually perform at that level. For a salary of, on average, about $100,000. When you start out - offering comparatively little but looking for Tom Brady, you’ll pass over pretty much everybody, because the person you’re looking for won’t just not interview with you, they probably don’t exist. After a few months, you’ll relax your standards, and after a few more, you’ll relax them even more and end up hiring a comparative retard like myself - somebody with only a mere 25 years of hands-on development experience and a bachelor’s and master’s degree in CS along with a couple of publications. But if I presented in the first few rounds of interviews, back when you were looking for the guy who could derive the tortoise and the hare algorithm in 30 seconds in front of five people in a boardroom, you would have passed.
> The difference between talking (very intelligently) and doing is huge
To be fair it is environment dependent more than anything else. Forget competence, charisma, intelligence, and everything else about the candidate that could bias their selection and instead look at processes and code already in place the new candidate is jumping into. Does the environment strongly favor original ideas/solutions or does it dictate the narrow acceptance in the most narrow of boundaries?
I have been on both ends of this as well. It is common in software for shops to define success in extraordinarily shallow terms such as whether you are using spaces versus tabs and indenting properly. Another example I experienced last year is whether you should write a giant monster configuration or a small function that receives a single argument. The reasons why shops coat themselves in blankets of code style and configuration is because they typically don't trust their developers and instead strive for a normalized baseline. They are looking for wonderful solutions, but rather task completion.
The lack of conformance isn't necessarily an indication of lower capability, but it is an indication of incompatibility. Competence and conformance are wildly out of sync when the candidate is misjudged relative to the work available. That is completely an assignment failure opposed to a candidate failure. Having gone through this myself it has taught me to ask very probing questions, as a candidate, during the interview. If, as a senior, I can determine I will not be a good fit I will happily disqualify myself.
> A coach like Bill Belichick might have a fantastic, detailed answer demonstrating a thorough understanding of every aspect of football past and present, but he could never make the throw himself.
Would you really hire a coach to be your quarterback? Is that a thought you would really entertain? Even if that coach could do that job he/she would be more valuable doing other work. I would consider this a solid example of interviewer/assignment failure.
> Having gone through this myself it has taught me to ask very probing questions, as a candidate, during the interview.
Like what questions? I'm assuming the work you're looking for is more along the lines of "wonderful solutions" as opposed to "task completion" - is it as simple as asking, "How anal are you guys about code syntax and whatnot?" and, "How hard are your problems actually?"
Here are some I thought of off the top of my head. These are based upon things I actually encountered.
What if I were to provide a solution that executes much faster, requires less documentation, passes test automation, and is a quarter of the code but ignores the framework or standard code style?
The standard DOM methods perform thousands of times faster than other options for interacting with markup. I can prove this with numbers. Code like that is not popular. Will I be allowed to write unpopular objectively superior code?
If I can reduce the application build from 5 minutes to 5 seconds will you let me rewrite the build from Java to Typescript?
A/B testing is a powerful way to determine preferential user behavior and a measured increase in conversion. Will I be allowed to write inward facing experiments to test developer behavior?
What if I provide a function as a solution the makes use of scope and nested functions but offers no support for inheritance?
Is it better to complete a task in 1 hour with original code plus tests or extend existing components with risk of regression and 4 days of effort?
A couple of these questions would be at least yellow flags for me as an interviewer or hiring manager, as they indicate a strong bias for throwing away existing systems ("ignores the framework", "rewrite the build", "original code").
There's a great quote from Lou Montulli[1]:
> I laughed heartily as I got questions from one of my former employees about FTP code the he was rewriting. It had taken 3 years of tuning to get code that could read the 60 different types of FTP servers, those 5000 lines of code may have looked ugly, but at least they worked.
Those frameworks and existing components most likely have a lot of hard-won experience embedded in them, and I would be uncomfortable hiring someone who did not appear to understand or appreciate that.
That is why I would ask those questions... to excuse myself out of your organization. I have found it painful to be at organizations who repeatedly and intentionally make really bad decisions so that their developers deliberately don't have to solve problems (invented here syndrome). If that is what I were looking for I wouldn't have developed the skills that I have.
Maybe this is unique to sports? Do other fields exist where a BB-esque expert is totally incapable of practicing? The greatest movie critics probably can't direct or write for crap, but they're not commenting on how the movie was produced, just its output
From what I have seen, most of the academic research principals in computer science are not actually practitioners anymore. They are more like coaches. The actual code is written by graduate students, and most of it isn't even very good because the code isn't the point in most cases.
Not unique. One could convincingly argue that this effect exists in many fields. Art (ie art history major vs artist). Music (ie violin player vs film score composer). Restaurant reviewing (ie critic vs chef). I'm sure there are many more.
Why talk to someone so condescendingly? I mean, what percentage of people do you epxpect to HAVE seen this bit of obscure media you seem to hold as seminal?
> The greatest movie critics probably can't direct or write for crap
They may know a lot about how a film is made. It does not follow that they could sit in the director's chair, or write a decent screenplay. (And yet, they could still be great at critiquing films!)
This is a valid point. Just knowing how something is made doesn't necessarily mean you know how to make it. It's the difference between a designer and an implementer. Or architect and builder.
I have been amazed at how many "data science" interview questions can be answered with "look it up in the hash table" or "look it up in the literature".
That last one is a real problem-solving strategy, I can't tell you exactly how to balance a Red-Black tree but I know what a Red-Black tree is and where to look up the algorithm for balancing it and I think that's good enough.
One of the master skills for interviewing is "don't choke." Often good people will make mistakes under the pressure of interviewing and will drop out.
When it comes to a practical session there is the same issue that some good people will choke. Just the sample of who chokes in which environment when is different.
Issues like this turn up in all cases where people try to measure merit. For instance, it is known that some low-SES (socioeconomic status) people will choke on standardized tests like the SAT or IQ tests. It is also known that some high-SES people are as dumb as posts and will have that revealed by standardized tests which are harder to bribe or bullshit.
The hostility towards testing is explained by the combination of those two groups: primarily the Emperor doesn't want you to see they have no clothes, but people have sympathy for poor people.
> The reason why some companies cannot figure this out is because they don’t value the problems at hand. They need bodies to put fingers on keyboards
Exactly this.
Too many companies are availability focused instead of ability focused.
The reason? When your processes and engineering are weak, you need people available 24/7 to put out fires.
When I interview for a position, I'm interviewing them as much as they are me.
One of the biggest asymmetries in the whole hiring process, is that of course every company will tell you they have the best development processes, frameworks and code to work on. They may even believe it because they don't know better.
Then you take the job and are stuck fighting fires on a big ball of mud codebase that takes over your life.
If you want to standout as a candidate, try and probe and really find out how good their 'culture is.' Interview them back.
One of the best questions for getting to the real answer is: "What are your expectations for availability?" If they expect availability from you after hours, then their stack is likely unstable because the need availability to keep it running.
If you are lucky enough when you are senior and good at what you do, you don't have to put up with that.
I often tell them about a the third of the way through the interview: "I'm not an availability guy, I'm an ability guy." They often are surprised by the statement. And the discussion that follows it usually tells me whether I want to work for them or not.
> Are things in place to make the job easier? Seniors don’t need easier and this is a huge turnoff.
What? It requires a special craft to design simpler components. Most of the people don't even see it. This is where your senior skills will shine when you'll make it work like a charm as compared to the previous state.
If you don't see the friction or can't reduce it then please take the first exit out.
I think you are conflating simple and easy. They aren't the same. Simple suggests less code, fewer pieces, and a shorter path between code and solution. Easier suggests least effort and less to look at. Simple isn't easy.
The primary difference between simple and easy are decisions. A good senior will spend more energy on considerations for appropriate decisions than the actual work.
This is surprisingly often not understood, even by people I showed the video. And I am not sure why.
But I do think it's necessary in out field to start understanding this much more deeply, especially for senior engineers.
Good definition of 'Simple'. Simple and easy may not be the same but there are situations where they are similar.
For e.g. - for production deployment, I could either put the entire office upside down. Fires across the departments. Broken applications. Rollbacks. Or, I can automate all of our QA test cases and have a one-click deployment. No huss or fuss. I'm sure after having this experience, someone will definitely say - well, this was easy.
"We will encourage you to develop the three great virtues of a programmer: laziness, impatience, and hubris." -- LarryWall
Laziness: The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful and document what you wrote so you don't have to answer so many questions about it.
Impatience: The anger you feel when the computer is being lazy. This makes you write programs that don't just react to your needs, but actually anticipate them. Or at least pretend to.
Hubris: The quality that makes you write (and maintain) programs that other people won't want to say bad things about.
(That quoite's originally from Programming Perl 1st edition in 1991, the explanations I think didn't show up until edition 2 in '96 or so...)
I agree with your comments, but let me share my 2c on "trendy framework bs".
Most of interviews I've done to candidates ended up in discussing technology trends and googling around for cool open source projects, libraries and so. This, to me, is a good indicator - as long as you bring them up when discussing relevant problems, this means you thought about a problem and researched prior art to avoid re-inventing the wheel.
Also we tend to end up discussing pro and cons of any given technology and so on. This at the end is a key indicator you're talking to a passionate developer. And passion usually makes someone good at programming.
When it comes to soft skills, usually having such a kind of discussion you can figure out also someones behaviour in most work scenarios (to me, having a good "discussion" mode means you're likely to be fit for team work).
Reinventing the wheel is a cliche I have grown to detest. I have found that cliche is abused 9 times out of 10 to avoid the deep dive into a problem. On the other hand, approaching the problem again allows the developer to consider the precision of current solutions or consider a simpler solution. Good seniors aren’t afraid of making decisions.
I don't know how much experience you have as a professional developer but I can barely remember the problems I solved last week let only any interesting ones I've solved in the past 10 years.
I have maybe half a dozen really interesting stories, and maybe a dozen or two interesting but only to the right crowd stories - from a career spanning a bit over 24 years. And a bunch of war stories about why we shouldn't do things that way, because of the problems we discovered trying that in 1998 or 2003... (And occasionally the "we should try that again because it didn't work back when our colocated production servers had 486's and 320Mb disks - but it'll probably work just fine as a mobile app now"...)
The question to ask is "why" they did something a particular way. If they start rattling off something that sounds like the marketing page of shiny new tech, the they are probably doing resume driven development rather than actually solving the problem in the best way that they can. But then some people seem to value that - I don't.
To be honest, I don't think your questions were good enough to filter out the smooth talkers. I was in a senior position at a successful agency and was hired before they changed their interview style to match the eye-rolling "technical interview" that is now gold standard. I repeatedly argued they were getting too many false negatives and only a certain kind of developer with this style, but they believed "if Google and Apple do it, it must be good".
Then I was put in a position to hire backend devs and, even though I couldn't entirely change the format of the interview, I came up with questions I believe could filter out the BS artists/not-yet-good-enough from the talent. And it worked.
Example questions that are truly hard to BS:
1. I want to build a new service that crawls the internet for used bikes and presents them for sale. Roughly sketch out the architecture you would use and how all the parts fit together.
- Why this question is good: it is impossible to successfully answer this question if you don't have experience building systems. Even if you try to BS it, your answer will come off shallow and break under any sort of probing. It also reveals how the dev's mind works when approaching problems.
2. Name your favorite tech stack. What is your favorite thing about it and what is your least favorite thing about it?
- Why this question is good: any dev knows that every tech decision comes with good things and terrible things. I love Python, but GIL, circular imports, shitty deployment/package management, 2.x vs 3.x nonsense all suck. If you haven't been in the trenches, you can't answer this _specifically_... you can only answer it _broadly_. And it's very apparent right away to interviewer.
I had about 4 - 5 of these questions. None of them required a single line of code to be written.
I gave an example of one of our erstwhile technical interview questions below. Batting around interview questions is a hacker sport and I've met dozens of developers who delighted in relaying and evangelizing their preferred questions. I'll admit, candidly, that "what's your favorite tech stack and what don't you like about it" --- a question you can literally just look up blog posts on, and a question I feel I could answer credibly for several languages/frameworks I don't even know --- is a uniquely bad one. But even if I felt it was a good one, I wouldn't trust it. Interview questions don't work.
"a question you can literally just look up blog posts on, and a question I feel I could answer credibly for several languages/frameworks I don't even know"
Really? You mean you could memorize a blog post for random languages, go into an interview and answer a question and not completely fall apart from follow up question 1, 2, n?...
You make it sound like I'm asking this question, getting a monologue response and just moving on. These questions facilitate a dialogue between interviewer and interviewee, which quickly reveals your deeper understanding of a subject.
Now, I think your above approach (lookup, memorize, pretend) works great for the kind of technical interview questions that I've been refuting this whole time.
Finally, I don't think my questions are perfect, nor do I think the interview process in general is that great. I think the best way to find talent is to do a trial run for 1 - 2 weeks and see how good they are at taking fuzzy problems and breaking them down into actionable steps, then executing. This is regrettably hard to do for most people and companies.
> I think the best way to find talent is to do a trial run for 1 - 2 weeks and see how good they are at taking fuzzy problems and breaking them down into actionable steps
How do you get people to do a trial run for a week or two?
Most people start looking for the next job when they are already doing their current one. So how can we ask them to do a trial run for a week or two?
Also what about the remuneration(what kind should be given if at all for this period) and does this work for freshers and absolute newbies too?
> Really? You mean you could memorize a blog post for random languages, go into an interview and answer a question and not completely fall apart from follow up question 1, 2, n?
To me, this is the crux of why I agree with you, rather than the questions themselves. Anyone can rehearse an answer to any question, but a dialogue needs to be created in order to actually find out anything about a prospective hire's competencies.
To be useful, an interview needs to be a conversation, not a monologue.
>I think the best way to find talent is to do a trial run for 1 - 2 weeks and see how good they are at taking fuzzy problems and breaking them down into actionable steps, then executing.
Except even if you could get someone to do that it doesn't filter for the right candidates. Someone who can quickly get up to speed in a foreign domain is not the same skill as being able to build great systems/code/etc in a domain you're familiar with. The day to day of a programmer after the first X months if the latter and not the former.
Aren't you two hiring for completely different position? One is looking for web system developer and the other is looking for the kind of coder that will write exploits and smaller tools. In one position you expect the senior to have general idea about wide range of relevant technologies, because you expect senior to choose between them occasionally (plus hiring manager knows that stuff too). Second position does not require it at all.
Plenty of people can do both or can change from one to another. But it still seems to be that the interview focus should be different as both positions require somewhat different focus.
Writing code is trivial part of what I would expect from senior - making correct decisions about architecture and talking intelligently about code are less trivial parts that make difference between senior and junior. Ultimately, when senior cant write code, that can be found relatively quickly. When he fails at decisions it takes more time and causes more damage. When he cant talk about code, then he is suitable to some positions but not to many others (so it all depends on position at hand).
I don't mind interview having also part where code is written at all. Be it dummy feature or breadth/depth first search or fizzbuzz or any other simple reasonably sized assignment. But in case of senior as in partly decision maker with not much direct supervision in the long term, you need those other capabilities too.
It's even simpler, "tell me about a challenging bug you've vanguished, what it was and how you solved it." People in the trenches love to tell war stories. I do.
For me it tells me a lot about how a person goes about solving the problem, how they're going to avoid it later (if the are), context about the conditions that the person worked in, where their interest lies, and what motivates them.
If their answer involves "well I searched around stackoverflow a lot and asked there" that's a no go for me.
I really wish more interviews asked these questions. They're not only good questions for the reasons you mention, but for the interviewee (at least for me) they tend to be pretty low-stress to answer. Those sort of questions are, after all, something that I might be asked to do or think about during my everyday duties at a new company.
I always sigh in relief when a company I'm interviewing with asks me something like that vs something like "implement x algorithm to solve y niche computer science problem you probably learned in college at some point".
Just to add some thoughts here: I’ve interviewed people who were google L7+ (IC) a couple of times who weren’t very good engineers, at least on the work sample stuff.
I’ve found that the highest correlation to performance in senior engineers is raw algorithmic skill and willingness to say “I don’t know” when you don’t know.
This is not true of hiring devops or sre’s. For those positions, you want the gopher archetype. This is the person who loves hunting down weird behavior and doesn’t give up when debugging. Let’s be real, debugging is you versus the machine and you’re eventually going to win if you can just be honest and patient.
Hiring senior engineers is really hard, I wish everyone luck.
If you're asking questions related to "raw algorithmic skill" you're filtering for people who either:
1) Have had a computer science education and happen to remember the algorithm at hand. This is also a function of recency so senior engineers are less likely to remember any given algorithm.
2) Study algorithms so they can do well at job interviews.
Neither one is something you want to be selecting for. Some of the best engineers I've worked with haven't had a proper CS education. I've known extremely strong engineers with Neuroscience, Mathematics, Physics and Public Policy degrees. I've got a business degree.
Unless you're working in certain extremely hard (and extremely rare) areas you do _not_ need to filter for algorithmic skill. Most ML doesn't count. Neither does Data Science. In 99% of engineering jobs it's more important to be diligent, rigorous, and organized. (Of course, filtering for those is another issue altogether)
Spot on. The reasons are exactly right too; seniority and "ability to recall obscure minutia from college years" are inversely correlated, for obvious reasons.
Companies need to understand that not only are they mis-selecting, but they're broadcasting that they're doing so to all the candidates that go through that process.
Approaching candidates with textbook-style algo or data structure questions merely informs that they're going to be working with an educated but overall somewhat junior lot. That's not necessarily always a deal killer, but it's probably not the image that these interviews are hoping to project.
For well-qualified candidates not applying at an industry headliner like AppAmaGooBookSoft, the interview process quickly inverts itself, and it becomes more about the company selling the candidate on their offer than the candidate selling the company on their skillset. Tread carefully.
It's probably a great analogy because the best musicians all know basic music theory, whether they learned it in school or on the bandstand. As for the advanced theory that they teach in graduate programs, it isn't even applicable to most genres of music.
I bet there's scope to twist it beyond all sensible bounds, and compare the ability of the 99% to the 1%.
I suspect there's top level classical, jazz, and session musicians - who're the industry equivalent of 10x programmers. (And all the other stereotypes probably exist too, I bet there are occasional untrained but gifted musicians who can produce 10x output, but who're amazingly difficult to collaborate with compared to degree level music theory trained musicians... And I bet there are "10 year" musicians with one years experience repeated ten times over.)
The other interesting point there is that probably 99% (or more for, five, perhaps six nines) of "programming" doesn't actually require that much hard-core CS theory. You can get paid well playing covers in bars with a good ear and not being able to read a single note from a chart, just by listening to the originals and copying them over and over in your bedroom. Same as you can make a decent living building basic CRUD websites/apps without having written your own compiler that can compile itself or defended a phd that advances humanities start of the art understanding of something fundamental.
The only problem at that level of musician ship you lose the fun and can end up with some very sterile music that's only of interest to other people who have degrees in music theory.
Btw years ago I did work with a top session guitarist (top 10 hits) who after an accident taught himself to program from his hospital bed.
> If you're asking questions related to "raw algorithmic skill" you're filtering for people who either: 1) Have had a computer science education and happen to remember the algorithm at hand.
Probably true. But perhaps that could be accounted for in the assessment process. After all graduates from Neuroscience, Maths and Physics degrees have got to be some to smartest people around.
Agree. A lot of coding is simply banging your head against the wall, search SO over and over, changing things around, until it does what you want.
Raw algo quizzing skill isn't necessarily the same thing, though you'd think it was somewhat related because when you're learning to code up "find longest continuous run" you also need to change things around for a bit.
Difference is in real life there's never an end. The algo quiz leaves you at some optimum eventually due to being quite a small thing.
Coding is the easiest part. Understanding the actual problem and solving it is the hard part.
> A lot of coding is simply banging your head against the wall, search SO over and over, changing things around, until it does what you want.
It doesn't look like programming to me. Yes, sometimes we miss something, so our code doesn't do exactly what we want it to do, but when we realize it we just fix the code. This view of coding resembles an improved way to write Shakespeare with monkeys.
> A lot of coding is simply banging your head against the wall, search SO over and over, changing things around, until it does what you want.
I don't find myself in these situations nearly as often as I did back when I was a junior engineer. But damn, I'm sure I looked busier (and more stressed out) back then.
I was mostly through that phase of my career before any of those things were available. Toward the end of it, stack overflow had just launched. I mostly relied on printed books for help with languages and frameworks I was using.
It's not correlated at all. I can ask you questions about implementing a b-tree and then give you a job to fix CSS. Which is the case in most jobs and job interviews.
Algorithmic ability has no correlation to the ability to write maintainable code, though. Most time at work is not spent demonstrating ability to regurgitate algorithms.
> Algorithmic ability has no correlation to the ability to write maintainable code, though.
This is not true in my experience. I usually see a strong correlation between algorithmic ability and writing maintainable code. At various organizations I have worked for, I have seen that the ones with strong algorithm skills also happen to be critical thinkers who put a lot of emphasis on simple, elegant, and robust design and code.
So I am very surprised to know that this correlation I observe may not be true in general. How did you come to this conclusion?
I don't have data but my impression is that there is an inverse correlation. My guess as to why is that people with ability to manage a lot of algorithmic complexity don't seem to suffer when code is complex so they see no need to simplify things.
I'm afraid I can also only offer anecdata. Mine is based on hiring and then working with dozens of SW engs since 2008 (my first tech lead role), plus our student Incubator and open source projects (dozens of more junior devs).
The correlation between emphasis on simple, elegant design and code and algorithmic chops is indeed uncanny.
And I'd add "clarity of articulation" to that -- being able to express your thoughts and the problem/solution structure clearly and succinctly is a great indicator as well. Huge overlap with both code maintainability and algo quality.
I haven't seen this ever. Algorithmic skills have never been correlating with productivity or maintainable code in any of those 7 companies I worked in.
No causation, I'd believe any day. But correlation? I'd want to see a statistically rigorous test done before I'd believe that. Correlations are everywhere and pretty much all good traits are correlated with each other. Hell, even vocabulary size and reflex speed is correlated. And this is probably why even terrible methods for selection usually kind of work.
Unfortunately, often (and maybe more so because I live in Europe) it's also the other way round:
> raw algorithmic skill
It's been ages that I've been asked anything remotely algorithmic. My interviews are mostly about frameworks, how you fit in a team and whether you know / can be "agile".
Not even a Fizzbuzz, much less so quicksort or more special algorithms.
> and willingness to say “I don’t know”
That never got me anything in any interview/company. To be fair, I found a few smart and cool friends because of this, but they themselves don't look as if they've found a good job either.
Being hired (valued?) as a senior engineer is really hard.
It really depends on your domain. If you're into low level hacking and distributed systems, there is a lot more algorithmic work. There is demand for software that's cheap to scale and/or low latency. Some fields are bottlenecked by hardware (machine learning, realtime rendering, etc.) and so benefit from better software. Some production systems still need a large amount of optimization to satisfy economic and product constraints.
I don't think the number of jobs requiring fairly deep systems or algorithmic knowledge has gone down, but the ratio has.
I’ve had trouble finding those systems or algorithmic jobs too, like the grandparent, where some kind of engineering quality matters, be it performance or correctness or something. Everyone wants to hire a “full-stack engineer” to write application code that a junior dev could write, but they want someone senior anyway.
I once had a FizzBuzz question in an interview, The interviewer started, I interrupted him, "We are not talking about FizzBuzz, are we?". He apologized afterwards by saying that there was actually on applicant some time ago, writing 100 printfs.
Why doesn't the gopher archetype also apply to development roles? Persistence pays off if you measure your own results. Knowing how to divide a problem is really important, even if you're great at algorithms.
While persistence is good, it has to be applied correctly. It can become the opposite problem as stubbornness, spending too much time in a small thing that is not actually that relevant because the person really wants to get it done.
More important than persistence IMHO is to know when to be persistent and when not, and those two qualities by OP seem to be quite related to it: "raw algorithmic skill" (to know whether something is optimizable or not) and willingness to say “I don’t know” when you don’t know" (seek help, get the right person for the job, etc).
Edit: I know because I was like that in the beginning; it was okay to learn e.g. micro optimizations when learning programming for fun at university, but it'd have been a big issue if I had not been able to correct it.
I can’t overemphatize how much persistence really paid off in past jobs I had. Coworkers that had persistence were such a delight to work with because they didnt give up the first time they got stuck. They did the nitty gritty work of tailing/grepping logs, using a debugger, endless Googling, print statements, and anything to find out the root cause of a bug... or even to understand a legacy codebase.
It's probably such a delight in part because bright kids are frequently not persistent. They are used to things coming easily. If it doesn't come easily, many are quick to throw in the towel.
Someone who is both bright and persistent can move mountains. But it often has a big social downside. People don't like change. Being bright, persistent and also socially savvy enough to sidestep drama is practically a unicorn.
I've done that sort of work. Eventually you realize that nobody notices the guy who tracked down and killed that vexatious heisenbug in the legacy code base. You need to be working on the new, shiny, high-visibility projects or your career is going to stagnate.
Friday demo day (to sales and cust success and everyone else): "hey everyone, know that thing customer X keep complaining about that we've never been able to solve? It's been super tricky. Just wanted to announce that James here figured it out and it's fixed forever. James you're a hero."
Saying “I don’t know” is also very important for devops people, but for slightly different reasons; we’re likely to have a little bit of knowledge about everything, and it’s import for us to be explicit about where our expertise dries up.
I say a lot of things in the form “I’m a little out of my depth in this subject, but my best understanding is that the behavior should be such-and-such; does that sound right to you?”
Honestly speaking, since it's an anonymous forum, what's the difference between L5 and L7? Like if I was asked the same question - the difference between L3 and L5 - I'd say that it's just experience with a few interesting known projects that's paid so much better, while the cognitive ability is the same.
Hey I'm from www.Levels.fyi - this is exactly what our site was built for. The link above is a copycat site and you can see the latest leveling info / compare with other companies on ours.
There’s a reason google doesn’t hire into L7, it’s because their hiring process for algorithms is garbage and provides lots of false positives and negatives when you are looking for real skills.
If saying “I don’t know” and being good at algorithms was enough, you could hire straight to L7 easily or could promote to in a year. Neither of these things happen.
It does always amaze me that literally what you learn in HR 101 in an undergraduate management curriculum is controversial. The r^2 of a ton of different methods for predicting job performance is something large companies are highly incentivized to get studied by academics (and they do).
Spoiler alert: work-sample tests are the only practical and generally tolerated one to use (for software engineers) that is actually correlated with job performance.
@tptacek: how do you feel about people who have large amounts of open source work that is easily reviewed? This obviously doesn’t add new data to your company’s test but is definitely some kind of work sample, albeit not necessarily in a similar environment to the one being hired for.
But it’s not generally tolerated. You end up optimizing for sub-prime candidates because those are the only ones desperate enough to take 4-6 hours out of their free time for a company that hasn’t even bothered interviewing you, yet.
If they want to turn the onsite into one big work sample, by all means, that sounds very effective (and something I’ve seen work well). But in my experience, you’re going to deter qualified candidates by forcing them to do take-home assignments.
Again: this is true of "work sample" tests in their mainstream implementation in our industry (spend 6 hours jumping through a hoop for the privilege of running a standard, nondeterministic interview gauntlet), and those hiring processes are a scourge.
But there's a right way to do it: give work sample challenges and then, at least for the most part, end the technical qualification part of your process there. You spend 4-6 hours at home instead of spending 4-6 hours in front of a whiteboard doing dumb coding challenges.
If I was looking for a job right now, I'd probably refuse to interview anywhere that gave me take-home problems that couldn't promise that a strong result on those problems would take me all the way through technical qualification. But a company that could offer me take-home problems and conclusively make the technical part of the decision on whether to hire me based on those problem would be a strong prospect.
We subscribe to the work sample test as your best option for technical validation, but we also limit ourselves to a small window of time, on site. We pick something we recently worked on, distill it down to something we can knock out in 15 minutes, and give the candidate about 45 minutes to work through the distilled problem and then we explore for about 15 minutes their solution and how they would make it production ready (as a conversation). We then have another 45 minutes of designing a solution to something more complex we recently had to work on, again distilled down. It involves a white board and boxes and arrows. The last 15 minutes is time for them to ask whatever they want of us. We have two interviewers on this technical portion to level out false reads by the other interviewer (I thought this part was a poor answer, but the other interviewer has a different view of it).
I really like it so far. The part I'm battling on it is if the current coding part selects against Java developers. Part of the code we want right now requires a unit test with dependency injection to match an interface. So many of our Java candidates simply can't set up a running unit test. They are used to layers of framework already set up in the IDE and just clicking on stuff. They have full access to Google. Maybe it is good we are filtering out these candidates, but I'm not sure. Still thinking on it.
> The part I'm battling on it is if the current coding part selects against Java developers.
Yes, it does. And let me put it this way: I've used C# and Java at most of my jobs, those theoretically would be my comfort languages, yes?
I do not use those languages on interviews. I often just use C++ (!), or Rust (free unit tests!) if the company tools allow for it, or worst case I'd learn some Python basics. C#/Java are very awkward and boilerplatey in such a small time frame as 45 minutes.
Interesting. To clarify, you are saying that these boiler-platey languages, it is not fair to write a unit test in a 45 minute time block with access to Google? I'm not a Java guy. All the languages I've used, this would not be an issue. Again, I'm wrestling with it because it I feel it should be easy but like 80% of our candidates who chose Java struggle with it.
It sounds like this is a pick-your-own-language type test. I'd suggest scoping down to one, or at most, two, simple allowable options that your team is already pretty comfortable handling.
I've been on the reviewer side of a handful of "choose your own language"-style take-homes recently and found that it's really not good for the candidate if they actually end up using something that wouldn't have been the interview committee's first or second choice. There have been cases where choosing a less-trendy-but-still-totally-viable toolkit has effectively disqualified a candidate, with several committee members not even considering it necessary to look at the code. This is very unfair and lame but an unfortunate reality. I asked that the test be changed to constrain the options at least to a list that wouldn't be immediately disqualifying.
You could advise the candidate ahead of the interview something along the lines of "you'll be asked to write a code sample in either Ruby or Python -- you'll have full access to Google, but you may want to brush up on the basics of these languages if you haven't used them recently".
This does two things: first, it prevents the issue you have, where you're essentially not sure if you're correctly scoring the samples produced. Secondly, it really requires you to constrain the problem to things that someone who has barely used language X or Y can do within 45 minutes.
The theory with pick your own language is they should be able to feel fully comfortable (interviews are already stressful enough). If they picked Rust, Scala, or a lisp dialect, (or anything the interviewers are unfamiliar with), it can even be a better interview because we get more insight on how the candidate communicates and their ability to walk someone else through their solution. A potential other bonus is less biases leak through from an interviewer on "that is a strange way to do that in language X."
> The theory with pick your own language is they should be able to feel fully comfortable (interviews are already stressful enough).
Ah, but there's the rub. Candidates are trying to please the interview panel. If you don't provide guidance, the odds that they'll just use whatever they think the interviewers most prefer are just as good, if not better, than the odds that they'll actually use whatever makes them most comfortable.
You said yourself that since some candidates pick a language that you don't know well, you can't really tell if the failure of a large number of those candidates is reflective of a bad test or just a mismatched candidate pool. IMO, if you're going to stick to the "pick any language" thing, you should at least find out and ensure that any language the candidate picks will have a fair shot.
> it can even be a better interview because we get more insight on how the candidate communicates and their ability to walk someone else through their solution.
You can still get the candidate to communicate and explain his choices if you give an option: "either Ruby or Python" or "either JavaScript or Visual Basic", etc. The problem with having this happen in a language that the interviewers don't know reasonably well is that they are much more vulnerable to the smooth talker who can present incorrect information confidently, and they won't have enough background/anchoring in the subject matter to know the difference.
> A potential other bonus is less biases leak through from an interviewer on "that is a strange way to do that in language X."
I would say that if you're worried that interviewers will load in biases toward their preferred shortcuts etc in a specific language, that you should be equally worried that some good candidates are being excluded for choosing the "wrong" language in an any-language-goes test.
Above, you mentioned that there'd be a positive response if a candidate used "Rust, Scala, or a lisp dialect" -- these are all relatively trendy. What if the candidate used nim, Pony, or some other language that hasn't pulled in to the hype superstation yet? What if the candidate used a language that's not-so-trendy anymore, like Visual Basic, Cobol, or bash? What if the candidate used a programming language of their own design, and brought a copy of the compiler with them on a flash drive?
I'm asking because I've seen this in practice. Candidates for a devops position who chose to use bash to implement the very simple take-home task they were given were laughed off by several other members of the interview committee, despite being potentially high-value senior people -- they were at least senior enough that they're more comfortable performing sysadmin-style tasks in a shell, rather than using a massive CM framework or an awkward amalgamation of Python scripts running os.spawn.
It feels like this type of thing happens a lot, in the same sense that very often, "unlimited PTO" just means "guess whatever amount of PTO is acceptable around here and hope you get it right".
I guess it depends how far back you're starting from. I suspect most Java developers don't spend that much time creating projects from scratch (I do, but the work environments I'm used to suggest I'm an outlier).
I tend to give candidates a simple project already to go, with junit and hamcrest, possibly mockito already available, and ask them to go from there using a provided IDE (which I attempt to get set up as near as they prefer to work as I can). This generally works out fine. I certainly don't feel the boiler-platiness of the language gets in the way, mostly because the IDE is generally capable of doing most of the lifting with that respect anyway, but also because over the timescale of an interview question, we're generally only talking about a couple of classes at most.
It starts from scratch. Familiarity with one's tooling is important. Setting up a project seems like it should be part of the basics. Would it not be unfair to others who choose a different language if Java gets hand holding in terms of initial classes?
For 20% of Java candidates, they do it just fine. Heck, a few echew the IDE and are fine working completely from the terminal (these tend to be particularly very solid at coding). Still wrestling with the idea.
The folks that already work with us (who wrote Java in a former role) see no problem with setting up a project nor do the folks that we hired recently (seeing as they likely passed that technical interview). But that all could be bias.
Maybe you are right. Maybe the next candidate or five in Java will get a base project and we can see how it goes.
Do you give them "their" tooling? I can set up a project of the type you describe in about five seconds, because I have a template for it in my IDE. The best defaults for this that I've seen are provided by IntelliJ (do you provide this in interviews? It seems legally challenging to do) and would probably take me 5-10 minutes to navigate.
I think Java depends much more heavily on powerful tooling to do the heavy lifting, and my experience of using that tooling when I haven't had a chance to configure it in advance has been pretty miserable.
If they're coming in, I'll hand them a laptop with the project set up and ready to go, with IntelliJ running. If they're normally an Eclipse user I'll swap to Eclipse shortcuts and help them manage the IDE as we go.
If it's remote, I'll ask them to share their screen with me with a project set up and ready to go, having emailed them a copy of the interface we're going to implement about 30 minutes before the interview.
It depends on context as well. Where I currently work, they wouldn't have a chance. The corporate firewall will prevent them talking to Nexus, for example. It just wouldn't be fair to expect them to navigate that sort of thing in an interview.
In general, if I'm asking them to code on my (or the company's) hardware, I'd start from an existing project, if only because I wouldn't expect them to be familiar with the installed tools (oh, you use maven? I'm a gradle user ... etc.) On their own laptop, I'd expect them to be more comfortable.
I'm doing a remote interview on Wednesday. That'll be on the candidates machine (because screen-sharing is easy, getting them inside the corporate network, not happening), and they've been told they'll need an IDE ready to go. I'll expect a project set up and ready to go before the call even starts.
On your own hardware, I'd expect you to be able to have a blank project up in minutes, so yes, I expect somebody who's not travelling to our site to be able to take 5 minutes out of their busy day to create a blank project.
> Familiarity with one's tooling is important. Setting up a project seems like it should be part of the basics. Would it not be unfair to others who choose a different language if Java gets hand holding in terms of initial classes?
Also, how much of their day-to-day work is going to be setting up new projects? I sometimes feel a better test is to be thrown into an existing code base and asked to make a change. It's far more indicative of the sort of work somebody is likely to actually be doing.
We ask that the code they write have a unit test. The nature of that unit test is that it has a dependency. They can mock however they like, but passing in an object that represents the dependency (dependency injection) is the easiest and most straight forward way to do that. That object should have a method on it (a know method, with a known signature, also known in some circles as matching an interface).
If I was doing this at Google, I would spend a lot of time thinking about test fraud. At a 40-50 person company? Not so much. We do simple follow-up things that raise the amount of effort you'd have to put in to fraudulently submitting work sample tests, and we know pretty exactly how we'd randomize our work sample tests to make it hard to cheat (at least, hard to cheat without doing something we'd be interested in anyways) --- but it's just not worth it right now.
Work sample test fraud is one of those things that sounds like a huge deal on message boards, but when you game through what would be involved in doing it in real life, it makes very little sense.
To my mind, the simplest and most straightforward way to combat any fraud is also beneficial because it gives you even more information about the candidate:
Talk to them about the code they wrote.
Have a conversation, as if they were already your co-worker, with the exercise as the subject. Go through it, ask them -- non-adversarially -- why they did X, what they thought about requirement 2, how they could get better test coverage for Z. If you see something that seems to be a mistake, talk about it. If you see something awesome, discuss it. I can't imagine someone incompetent being able to bullshit their way through detailed discussion of code they were supposed to have written.
And this provides valuable info about how this person thinks and communicates about the work you want them to do.
That seems like a good idea, but does somewhat detract from the notion that once a candidate does the work-sample he is done with the technical part of the process.
A nice approach I liked (back when I was interviewed by Thoughtworks years ago) was that one of the interview stages was to take the assignment you'd done and apply some new requirements to the story. It rapidly makes it clear if the candidate actually did the assignment or not.
If majority doesn’t, why create a process that punishes everyone? People are lying on the resumes is a very common argument made by creators of insane interview processes. It feels weird to start a relationship with a company with assumption that I’m a cheater.
Who said anything about punishing anyone? 1) tptacek advocated for a specific kind of interviewing, 2) I asked him if he had ever seen a problem that I thought might be an issue with that method, 3) he explained that he wasn't worried about it because of X, Y, Z but acknowledged that for other companies it could be a problem, 4) I thought his answer was very reasonable. Also wool_gather and swish_bob added some useful ideas.
I'm not sure why you felt the need to come out guns blazing.
I agree with that. Sounds like a good process. Although, I am too cynical to believe any company that tells me this will be the only technical part. Too often do recruiters lie/misrepresent the recruitment process. Some seem to operate on the sunk cost fallacy, where you just see it through because what's one more round after already doing several?
I understand your cynicism but: who's got two thumbs and can offer an existence proof that there are companies that really do hire this way? :)
My best guess is that there aren't going to be many companies that will give you definitive statements about what their process will be after challenges who are lying about how they digest work-sample responses. But I don't know and am prepared to be wrong about that.
I've only had one take home assignment task that really worked like that. I did pretty good at it and the interview.
I was co-owner of a startup at the time, the guy I interviewed called me up and said you did really good but I think you really don't want to work here - my wife said 'Wow, that guy was really smart' considering how some things went later it might have been better if I'd said no I really really do
The extremely obvious problem with this is that there's no way of preventing someone from completely blowing a hole in your interview process by simply paying for or hiring another developer to do the take-home problem for them. At that point they've gotten past the technical requirements and now only need the soft skills to execute on it once the rest of the interview process continues.
This is why take-home problems are almost completely irrelevant except for filtering out good candidates. Eventually those problems optimize for perfection which help out those who 'cheat' at the process and people that otherwise put in earnest efforts are rewarded with denials. This is something I've experienced before in my job search where I would put in an honest effort and get 90% of the problem solved, but get denied because my solution wasn't flawless.
So allow me to call bullshit on your own claims that this is the right way of determining technical qualifications.
Not all companies handle it like this. I had a take-home exercise as part of an interview last year. I hit a real snag on a fairly small part, I couldn't figure it out, and I ended up leaving a bug in my submission because I simply ran out of time. Very frustrating.
It was raised at the interview; I admitted that I knew it was there, and that I hadn't been able to figure it out. We discussed possible causes: it actually turned into a pretty interesting, though minor, technical conversation. The interviewer eventually told me that he had figured it out after a little investigation (and I expressed my gratitude for the explanation!)
I ended up getting an enthusiastic offer from them.
I'd couple the take-home with a substantive discussion following submission. Harder for a fraudster to talk about how they came up with or tested their solution and how they'd improve it in a real production version.
> I would put in an honest effort and get 90% of the problem solved, but get denied because my solution wasn't flawless
It's possible that the company stated the take-home work in terms of non-negotiable deliverables, and their baseline for allowing a candidate to move forward is 100% of those, and they prioritize candidates who take initiative and do more work beyond the base requirements.
... not that I would agree with such a process (it biases towards people with more free time, like people with fewer dependents and people who are currently out of a job / underemployed), but it's very possible that this is what you faced.
As a new immigrant, my wife had trouble finding work as a designer until she was given a take home work sample test. After that, the company that gave her the test hired her quickly and she has been doing well with them ever since.
I’m currently doing the gauntlet thing myself, but none of the interviews I’ve done have asked any technical questions pertaining to the role that they actually want to hire me for. It’s all generic stuff that frankly I would have done better on 20 years ago.
The best interview I ever had was a 2-hour onsite work sample, followed by 1/2 an hour discussing what I'd come up with. I was offered the job the next day.
Surely most people would prefer this to whiteboard tasks?
The best interview I had was a few hours of friendly conversation about the details of my resume.
Then I was hired under probation, as everyone there was, and the understanding was that I could be easily dismissed if it was clear that I wasn't working out.
There is no way in hell I'd ever sign up for that. To be expected to quit your current job on the hope that the probationary period works out is insane.
Yes, I've learned that the hard way. Provided they give you payment in lieu of notice, they don't need much of a reason to let you go.
Myself and a co-worker were once let go for "performance reasons" at 10 months - just after project completion (successful). It was beyond my probationary period, and no issues where raised in the 2 performance reviews.
Their notice period was just 1 month. We were effectively cheap contractors.
My advice now is to treat offers with a low notice period (of them telling you) as a red flag. The norm is 3 months, after probation.
This is only true in the most extreme case. Many companies have formal procedures around firing people for performance issues. Short of hr violations or literally refusing to do anything, I can't imagine someone being fired before 6 months.
Yes, above a certain size, companies typically have some formal procedures. But typically those are a fig leaf.
In many labour markets, there's a legal 90-day probation or equivalent. You bet your boots some people get dismissed at 80 days. Or the job was contract-to-hire, and the contract doesn't "get renewed".
But on top of that, literally every company I've worked at or any of my friends have worked at (including lotsa startups, two of FAANG, and some in-betweens) will terminate when they want to terminate. In most non-European labour markets that I'm aware of, there's a penalty for doing so, and the company just pays that penalty and gets on with it.
Sometimes there's more security than that, I've heard (but not experienced). And sometimes the company puts in large effort to cultivate the underperforming employee first (had that happen to me once; they tried and I tried but it didn't work out). But the overwhelming majority of cases of my first-hand and second-hand experience, dleslie's summary is about the whole story:
Most places I've worked at in the US have a 90-day probationary period. There's still red tape, but not as much. More common now, however, is to hire people on as contractors for 3 to 6 months. If you work out well, they'll fast-track you to becoming full-time without a second thought. Otherwise, your contract is up and they choose not to renew it. Which makes things less dramatic if there are issues.
Fair enough. It's to some extent a cultural thing (there's no need for explicit probation in the US since most employment is at will), but I too wouldn't necessarily like a probationary period, even though I don't foresee it actually being an issue.
In the US there often is a formal probationary period at larger companies which mainly accomplishes one thing: reduce the HR red tape if a new hire isn't working out. During the probationary period it's generally easier to make a case (i.e. little or no documentation needed) that 'they're not working out' and HR will be OK with it vs. after the probationary period, you typically have to 'document' them out of the company.
I'm in the USA, and this (probationary period) has been the case with every job I've had in the past 30 years. I've never heard of a company not doing this in fact.
Same here. Though many mid-size / smaller companies might not advertise this fact (their HR policies are often a bit more ad hoc than larger companies if they haven't been involved in as many labor lawsuits)... but pretty much if there's an HR department, the probationary period exists.
I'm not sure how much this is considered but in my state, which is an at-will employment state, being unable to preform job tasks due to lacking the knowledge or technical skill is explicitly defined as NOT a demonstration of cause for termination that would absolve the employer of their financial responsibility toward unemployment compensation.
I suppose you do sort of feel a little stressed during the trial period but I've never seen anyone fail it and it applies at every company, so there's no escaping it anyway. When it was introduced some people got quite upset but I can't really say I think it's had a bad effect.
I guess from the companies perspective if they realize they made a grave mistake they can back out of the hire, but they are still very careful and rigorous in the hiring process just like always. It also allows the candidate to bail if they realize the company wasn't what it said it was. It goes both ways. Again, in practice it seems mostly harmless.
Perhaps the US wouldn't do so well with a similar policy maybe even just due to the crazy healthcare situation going on over there. I couldnt say.
> I suppose you do sort of feel a little stressed during the trial period but I've never seen anyone fail it and it applies at every company, so there's no escaping it anyway.
I'm not sure what you mean by "it applies at every company". Getting hired and then fired a week later is virtually unheard of. This is not a fear I have, at all.
But if you told me it's probationary, that is totally a fear I'd have, I'd get paranoid, so I'd rather work somewhere else. You're basically telling me it's not a real offer in my eyes, and I should not expect stability.
> Again, in practice it seems mostly harmless.
It's extremely harmful in a place with poor labor protections that is the US, for reasons that I don't feel like expanding on and that you can educate yourself on if you wish.
By "it applies at every company" I mean it's part of the contract no matter where you work because by law the employer is allowed a 90 day trial period, so they all put it in the contract, so one offer is just as real as the next and it's just something you have to go through and yeah I guess it's probationary in nature.
Not everyone likes it or agrees with it, and I can only comment on the software industry here and not other industries but it's not the end of the world and the sky doesn't at all fall. When they introduced it a lot of people tried to make arguments like it would be abused etc and as far as I can tell there hasn't really been any drama. YMMV depending on country.
The best I had was a guy asked me to bring in my laptop and show him some code. We talked about it, he asked me to add a simple feature. I think it worked well for both side. Not much of my time wasted. No gotchas because of some configuration issue of setting up a project for the first time.
I had perhaps the most amazing interview experience recently where it was an open discussion. It wasn't a technical drill down but more a Q&A where you discuss topics relevant to the area you claim to have knowledge of and how it relates to the job you're interviewing for. It was the complete opposite of code this technical problem and a missing semicolon will get you a flat out rejection.
What clicked was I was finishing their sentences and knew precisely what they were asking. It was an incredibly rewarding experience which led to a same day offer.
As a junior I’d much prefer a whiteboard where I just need strong CS fundamentals and reasoning to a work sample where there are 10+ different dimensions I could be judged on like style, maintainability, whether I used the latest language features, how I solved the problem ( use libraries or from stratch?) there are just way too may variables and potential bias in the judges VS did you correctly turn the problem in a DP algo.
Hah, I had a sample test that I had to do after first being met on-site, which I thought was great.
Then they said my work sample was amazing, and they’d like to do an on-site Q&A about it, but when I arrived the engineer hadn’t even seen my work, and proceeded to just quiz me on obscure JS trivia.
Which I promptly failed.
They then rejected me even though they were happy about my work :/
In addition to covering all interview expenses such as travel food and lodging, candidates need to be paid for their time interviewing by the companies interviewing them.
Six hour take home tests is fine, but I want $1350 for that in advance as a consultation fee, and if I hit 6 hrs and it's not done yet they can keep paying until I am done or we can just end it, no refunds.
The idea that I should spend six hours doing free programming for random companies that say they are desperate to find anyone qualified is absolutely absurd and insulting. No one should put up with that, particularly anyone with an established and verifiable career.
It's even worse when you're a consultant. At the moment I'm faced with scheduling my fourth round with a company, and of course it's 5 hours onsite. I have the option of skipping the free lunch, they said. It's nice of them, but my monetary loss for that time (including commute) is enough to pay lunch for a number of people. All that, and they can still can you a few weeks or months into the job if you're a dud.
It's employment, not marriage, guys, lighten up with your strenuous and time-draining processes. We senior employees aren't as, is the word excitable, as the entry-levels about joining your workforce.
That does sound excessive. I've never had more than 2 rounds of interview, and even at that the first round was typically over the phone and the second on-site.
This is pretty silly. Candidates routinely spend integer multiples of six hours running interview gauntlets for tech companies that are notorious for negging candidates. None of them expect to get paid for interviewing. An at-home work sample challenge is strictly less onerous than an interview gauntlet, but because it has the appearance of something people have heard other people get paid for, it's commonly suggested that they should be paid, too.
why are you even interviewing for these "random companies desperate to find anyone qualified" if you are so disgusted by them?
The elitism of some engineers is mind boggling, instead of being grateful working in an industry that has so much demand that you can easily find a job at anytime, you complain about the process being insulting to your oh-so-important persona.
If i really want to work for a certain company because what they are doing excites me, then yes, i'd do a 6 hour take home test where I can probably also learn a thing or two and I am willing to bet a lot of other developers would too. As an interviewer, someone charging >$1000 for a take home test would be an immediate red flag and our mindsets probably don't match up.
It's good to see a full gamut of opinions on here. I'll say that mentalities like yours are why I'm a contractor in the first place -- you should be grateful for the opportunity, etc and so forth. No. I exchange my time for your money. If that's entitlement to you, we come from opposite worlds.
I'm not the guy who says he should be compensated for take-home tests above. I am however a senior engineer. I'm spurning any long interview processes because they cost money. I don't think it's entitlement, but if that's what you want to call it, so be it. It's called hours worked, hours paid, and in the West it's been a concept since at least the 18th century.
I agree with the statement "they cost money for both parties" but I disagree with "in that sense it's fair."
Individual consultant: loses 5 hours of interview time (and commute time) or take-home exam. Let's call it $800 for the sake of argument.
Company: loses 5 hours of interview time, plus the time it takes to "quiz" the exam.
Individual loses money that he / she uses to pay their mortgage.
Company loses profit because the time spent interviewing the candidate could have been spent working on feature Y of the application. So shareholders / VC's lose.
So you're saying it's fair that the individual contributor who loses half a day's pay in your interview process is equivalent (you DID use the word "fair" so that's an equivalence argument you made) to a company's loss of a few hours out of the many thousands of man-hours they rely upon? It's a .0001% of their profit, assuming the employers don't work a few hours more to make up for the lost time, because they will (they're salaried!)
Consultants also do not get paid vacation, or paid sick leave. They also don't get any of the benefits that a lot of regular employees get. But that's part of the game, they have to account for all of that which is why they earn much more per hour. A lot of those "customer acquisition" tasks cost time and may not necessarily yield a return, but that's part of the extra risk consultants have to assume and why not everyone is willing to do it.
Maybe all true with regards to cost considerations, but it doesn't support the notion above that a candidate's wasted time and money (vacation time and sick time costs money for an FTE) and a company's wasted time are somehow .. 'equal'.
That sounds like disorganization and poor internal communication, two traits of the organization probably not just evident in their evaluation of you. You may have dodged a bullet.
Gotcha, yes that makes more sense. I find a lot of unicorns are already doing this sort of interview, anyway. While I have never observed studies, it seems to provide better signals than white-boarding.
>It does always amaze me that literally what you learn in HR 101 in an undergraduate management curriculum is controversial. The r^2 of a ton of different methods for predicting job performance is something large companies are highly incentivized to get studied by academics (and they do).
Large companies are not "highly incentivized" to optimize they're hiring process. Once a certain throughput is achieved, there's very little reason to revisit it, because if they revisit it, "they're lowering the bar", or not "optimizing for recall" or whatever.
Mindlessly copying large companies isn't all that useful. Microsoft famously loved brain teasers. Can't get the fox, chicken, and grain to the other side of of the river? You must not be able to code either. Google, loved asking your SAT scores because, "obviously" someone that got some arbitrary score on a standardized test, a minimum of 6 years ago, certainly means something today.
Large companies aren't immune from bullshit. In fact, they have the habit of metastasizing bullshit, because of "Well, X does it, so it must be good." X only does it, because they had a stupid idea, became successful because of completely unrelated means, and then fooled themselves into thinking their process was good, "Well, I've been hitting candidates with ball-pein hammer for years, and I'm successful, so screw you."
Interestingly enough, eventually both Microsoft and Google abandoned these interview questions, because eventually, they realized that one had nothing to do with the other, but only after years doing it, and others copying them.
Microsoft and Google do not collectively hire that many people and are not who I am talking about. “Brain-teaser” type questions are explicitly not work-sample tests and, yes, have no correlation with job performance.
IQ does correlate with work performance, and SAT tests are IQ tests. Brain teasers are an attempt at seeing how well/quickly your brain works to solve problems, is a rudimentary IQ test.
The problem is that requiring IQ tests for employment introduces liability that employers do not want.
Neither IQ nor SAT scores correlate with job performance. SAT scores were requested at Google for years. Explicit aptitude tests have been used in the past, and continue to be used. They are quite legal, as long as they are used for their intend use.
SATs are IQ tests? A large portion is math, and the other written language, both taught disciplines. You obviously haven't taken a real IQ test, which is more abstract and has a large portion of spatial geometry type questions.
Hr isn't exactly known for rigorous scientific research and I don't recall it being hr that introduced this trend of hazing candidates.
Also have you got any examples of this research that actually proves this correlation I haven't heard of any and I would have as I have spent decades semi involved in IR (industrial relations) in the UK
Hazing candidates is mostly negative reinforcement introduced to keep people from wanting to go to interviews and change jobs. It also selects for those willing to do a lot and put up with a lot for a corporation.
> highly incentivized to get studied by academics (and they do)
Why do you think they are incentivized to get studied? If anything, power dynamics in large hierarchical organizations keep away any studying done that can threaten the power. Sometimes studies do happen though, but large organization can never apply the results to anything on their scale. They are more worried where to even get such massive stream of professionals to hire.
You’re getting too philosophical here for something that isn’t that complicated. This isn’t related to power dynamics. Asking someone if they can flip burgers is obviously not as effective as asking them to flip a burger and seeing if they can do it correctly. Getting employees who are unquestioning robots with no ambitions for power is a different interviewing issue.
Totally agree. I suck at interviewing and freeze up on the spot. Even on-site coding tests I'm not great at. Give me the time to sit back in my own comfortable environment and I'm confident I can solve 95% coding problems (and reasonably quickly).
I've been involved in a lot of hiring and I always advocate a choice of a couple of fairly in-depth (maybe 4-6 hours) take-home coding problems. I think people should be paid for their time on these as well.
I’ve got a friend who is a lot like you. I’ve worked with him, so genuinely know he’s good, but he sucks at interviews. He freezes up and can’t answer simple technical things, things he knows and has successfully done often. Some people simply suck at interviews.
I think practice help with this "Some people simply suck at interviews".
If you practice at home, after work, try with other companies before applying the ones you really wants and you will improve your interview skills by a lot.
Equal problem: Trying to find the best chess player trough friendly conversation about chess.
We could talk all day, but I could not pick Magnus Carlsen over any other professional chess player in conversation. Worse. I can't even differentiate someone below my level from grandmaster level if they are huge chess enthusiasts.
If you want to find programmers who are better than you or rank people close to your level, only the actual work shows the differences. I can recognize someone who is better programmer than me only from the code they write.
However, I've walked away from interview processes that require a work sample that'll take more than a few hours or require lots of domain knowledge to get started. It just isn't worth my time, especially as a barrier to entry before talking to anyone.
Things I am happy to do to see if I'm a good fit:
- Pair with a developer on the team. I learn just as much about the team I'd be joining as the hiring company does about me.
- Talking through code that I've written, talking through how I came to the implementation and how I might refactor/enhance further. You can't really bullshit this without the interviewer being a nit.
Things I'll do as a 1099:
- Fix a real bug
- Refactor real code
- Do a project that isn't trivial, start to finish
I recall an interview I was brought into early in my career to talk with someone being brought into be a senior architect... in my mind this meant this dude must be crazy smart right. Everyone in room had all basically made up thier minds and loved the guy. I gave a variable like
a = [1,2,4,8,9];
And asked if he could write a loop and print out the values of each element. He embarrassingly could not.
I always ask this question and then talk and then have everyone solve a problem a simplistic one but it always involves an Http request and some loops...
I like to see how people type too... if you can’t type it’s usually easy to tell if you spend most of your time with a computer or not...
> I like to see how people type too... if you can’t type it’s usually easy to tell if you spend most of your time with a computer or not...
I disagree. When nobody's looking I type pretty fast, but when the whole interviewee team is looking at my screen I suddenly forget all the shortcuts and mistype a lot due to anxiety.
But are architects supposed to write code? The example you give is embarrassingly simple and anyone should be (expected to be) able to write code for it. But aren't architects designing architecture. I don't think the architect or anyone in his team at my previous organization's IT dept could write code, but he could design system architecture.
I know a few architects (as in for buildings and civil structures) who wouldn't be able to pour a foundation, frame a wall, or run plumbing properly. They're working at a different level of abstraction and are concerned with different problems.
Or to put it another way, if you can code does that mean you should be able to design a CPU, even a very basic one? After all, how can you write a program if you don't understand how it works at the instruction, transistor, etc level?
Civil Architect is to Builder as Software Engineer is to Datacenter Technician. Software architect is to software engineer as partner at an architecture firm is to architect, or something.
You don't expect a plumber[0] or a building constructor[1] or a civil engineer[2] to become an architect[3], they're different degrees or certifications. A software architect is normally considered a software engineer who works at a grander scope of design. That doesn't map correctly.
I wouldn't expect a building architect to do a great job pouring a foundation, framing a wall, or running plumbing. But I would expect them to be able to do it, at least to a basic level.
Architects that don't understand how to do the fundamentals of implementing the designs they make tend to make unrealistic designs, that are expensive to implement and may look pretty, but function poorly for the owners and occupants.
This applies to architects of buildings, architects of software, and architects of hardware. Probably to most high level design and supervision type positions.
Edit to add: many of the best architects of buildings are able to do amazing things specifically because they've studied the fundamentals and are able to do innovative things with materials. Of course, time will tell it the innovation worked out well or not.
> I wouldn't expect a building architect to do a great job pouring a foundation, framing a wall, or running plumbing. But I would expect them to be able to do it, at least to a basic level.
waaaaat? That is completely utterly 100% unreasonable. Unless you mean that you expect every adult member of society to do those things?
I agree with GP. If the civil architect doesn't know that you need to put up plywood to pour the concrete in, they will make impossible to build designs.
And I'm not saying that everyone needs to do everyone's jobs. But you shoupd be able yo do at least a bumbling job of the layers of abstraction you work on top of if you want to make a decision in how those things should work.
If an architect that hasn't coded in 5 years or more dictates technological decisions to you, when they are out of touch of current best practices, how do you bridge that gap?
Building codes change. Architects are required to design buildings according to the new codes. There is no such thing in software.
When I first started in industry, storing plaintext passwords in the database was what everyone did. We have moved on from that.
I've worked on j2ee web apps, where the EJB tier absolutely had to be on a separate server or servers. Times have moved on from there.
I've met very few "software architects" that have kept current without writing code at least 5-10% of their job.
Times do change, but it doesn't mean you need to know every single detailed implementation. If the standard practice was plain text, and then became hash...the architect dictates that passwords needs to be hashed. They don't necessarily need to know how to code it.
In my company my 'enterprise architect' oversees at least 40 applications in my space in various languages and platforms (and he works in other spaces too so I can't give an exact number). I need him to give a good direction on what apps are needed, how apps talk to each other and certain detailed design. He also pitches to management and the business to defend how/why we do certain things.
Last thing he would be required to do is remember to write basic code (though I would hope he can do a swag).
Most software architects aren't working at a level of abstraction that's as much higher as that. They're making decisions like "we should use this framework" or "we ought to have 3 9s of reliability", which can't be meaningfully analyzed without knowing the details of how the coding will be done.
> Or to put it another way, if you can code does that mean you should be able to design a CPU, even a very basic one? After all, how can you write a program if you don't understand how it works at the instruction, transistor, etc level?
You'd be surprised, but this is what computer engineering actually is. From p-n junctions and transistors and all the way to distributed systems.
Difference of terminology I guess, but I worked at a Pharma company and I am talking about the Enterprise Architect, so maybe not the same kind of architect you are talking about. He and his team made decisions related to tech stack. They never made software design decisions. Over at the Pharma company IT higher ups weren't technical anyway.
I don't understand how one could possibly design a system that they couldn't even begin to implement. Not framework/tool specific stuff, but can't right code? At all?
I'm not sure if this is a common sentiment, but wouldn't an architect that does not know how to code have a very hard time convincing the other (seniors, but in particular juniors I guess) engineers to follow their designs? I guess it is possible to fake a while , but only a while...
I don't expect my architect to know the ins and outs of Go, I expect them to have a strategy that is forward thinking a few years out. Plans on cloud integration, ci/CD requirements, updating legacy systems to be containerized, managing kube clusters, etc. They should know the pros and cons related. They don't need to write a for loop.
Hi, author here, thanks for your feedback. I think you might be right, thinking about rewriting that paragraph a bit.
For me it's been really obvious in interviews whether someone knows what they're doing or not. But probably I've just been lucky in this so far.
Looking forward to reading your post about it. Haven't read it yet, but if it's not in the post, could you give a few examples of what people said, that turned out to be bullshit? And the other way around, too? Not questioning you, just genuinely curious.
I think your key point is less that testing is easy than you should let the candidate prove their worth in their own way. This sounds key to me. Your original quote is… yeah, “easy” happens, but it’s not always elementary.
Testing a candidate is actually very candidate-dependent, I found out; however, it is a lot less hard when you let the candidate pick their own approach (and you supplement it subtly). I found it easier than most interviewers who tend to do scripted, one-size fits all tests.
I always ask candidates to talk about what they’ve done (I’m in data science, so university projects typically can be relevant for day-work; standard CS might not be so lucky). If there are omissions, ellipses, I might prod a bit.
The best candidates can easily juggle between business objective, method, architectural and technical choice, how they scaled a release into marginal improvements from prototypes to a full-fledged platform. It typically takes me one question, two follow-up and 10 minutes to know they are strong. From there, I can shelter them from more technical tests, or at least apologise for the process; ask if they are comfortable with white-boarding, etc.
The worst candidates are not that different: they can sweet-talk their way, but might not be able to scale up or down abstraction. You can prod from them if they are comfortable receiving feedback (based on how they react to you re-framing the question), their learning style (based on how excited when I ask details about specific implementation).
If you are uncertain, which remains most candidates, I like to put them in front of what I have been working on recently. It gets Legal a little nervous (it really shouldn’t) but it puts them in a rare but effective situation: I might not know as much as they do. I probably have had more time to explore, but no idea that they suggest, I can dismiss: either I had the same or it’s new to me and worth trying. I’m not the most structured worker, so I often have a bug, something that could be improved but it’s a priority, some high-level conceptual concern, etc. I make sure that they go for something that excites them or the problem that I’ve identified earlier.
I rarely need more than an hour to have a clear idea, including data preparation.
I'm really not comfortable talking about specific patterns of questions and answers that turned out not to be predictive, because I feel like they'd be traceable (or at least appear, to the people involved, traceable) to particular candidates.
Happy to answer any other questions you might have about our erstwhile process at Matasano (which is more or less our current process at Latacora).
I have recently gone through interviews in few companies, and in almost all of them I've been skipped the first small technical test. I have a strong online profile [0][1][2] so I was very glad not to have to spend several hours doing some simple fizzbuzz. Don't get me wrong, you could say that I even enjoy doing these small puzzles, but with a fulltime job and trying to select several companies I just didn't want to spend a lot of time on that now and I'm glad they all skipped it.
However, later in the process they all had a more technical test with people. If there was no technical test at all I'd feel the companies didn't know what they were doing and hiring people who didn't know how to code, but if there was too much BS initially I'd feel that they also didn't know that they were doing in the way that they were missing senior devs with many options because they were making it difficult. This just seemed the perfect balance.
When I see people mentioning fizzbuzz, it's always to filter out people who really can't program. If you just give me ten minutes to solve a fizzbuzz-like problem, that should be plenty of time for me to solve it given that I do know how to program, and just that single test would be enough to filter out those who don't. I don't see why it would have to take several hours.
Fizzbuzz is an example, I've never seen it on its own. A company doing a pre-screen test would normally put 2-3 small tests, the easiest one fizzbuzz-like, for a total of 1-1.5h.
Multiply that by the number of companies you are interviewing and that can easily be 8-15h in total. I'm glad I didn't have to do that.
I think this might be the best thing I've read on hiring.
As an engineer, I love work-sample tests. And I also like the insight you had, that a test should test code-reading as well as code-writing.
Some people complain about the amount of time that they take, but if you're good at what you do then you should only need to complete one work-sample. Whereas you could easily need to attend many interviews that accurately assess your skill. Some companies even have many-stage interview processes, and these definitely take longer than creating a work sample.
The most annoying thing is when you show yourself perfectly capable of doing the technical challenge and still get passed up on the interview because they completely ignore the results and/or then check your ability to script on a whiteboard. I’d say the work sample is a waste of time if I didn’t view it as a learning exercise.
I couldn't possibly agree more. The current cargo-culted trend of take-home problems is a scourge. If you can't hire almost directly off the results of your challenges, you're wasting time and shouldn't do them at all.
> The current cargo-culted trend of take-home problems is a scourge
I hire based on take-home problems. But then again I don't have candidates whiteboard.
I give candidates time to produce something in the comfort of their own environment that goes beyond solving puzzles on the fly and is actually a reflection of the work people will do day-to-day.
I think something people tend to miss is when these problems are appropriate. If you are hiring from a pool of recent graduates for a junior position, or any such situation where you have a demand for the position, then this might be a good idea as a first step.
For more senior people this kind of test might even be more appropriate, because they are not always working on exactly the same type of thing in recent memory but in their own time will come up with a good solution quickly. However, for more senior people this type of test should certainly not be one of the first steps of the interview process.
You should ideally have much shorter and simpler technical tests during the first interviews to try to weed them out while also giving them exposure to what kind of environment they can expect to be working in, but also tell them this kind of task will be coming and why you are doing it. Once they're more committed to actually wanting the position they will be more likely not mind spending their personal time doing this kind of work exercise, or you could even offer to pay for their time on it to show you are not the kind of company that expects them to do work for free.
If you don't do this, you'll only come across more senior people who are desperate enough to jump through these hoops, which is rare and you better know for sure why they are desperate.
>I give candidates time to produce something in the comfort of their own environment that goes beyond solving puzzles on the fly and is actually a reflection of the work people will do day-to-day
100% this.I find whiteboarding to be the actual scourge, as it involves unrealistic pressure in an unrealistic scenario. It reminds me of tests that inadvertently just test whether people are good test-takers versus assessing their command of the actual subject matter.
I believe hiring devs who are also good at whiteboarding have a bias--even blindspot--where candidate whiteboarding is concerned. They oddly believe it to be a prerequisite for a job that infrequently or never requires it.
As such, they may miss out on good devs who simply thrive in an environment more akin to reality.
I see whiteboarding as a skill that is very useful in presenting or defending design ideas. I've had several cases where I used a whiteboard to layout a design interactively to an audience in a design review. I find I am at my best when I have marker in hand as it gives me control of the conversation.
Yes, but that's a different application of white-boarding.
In the case you describe, you've already done the more "valuable" dev work of thoughtful analysis/design, and you are merely presenting that work.
That's far different from being forced to devise that design at the whiteboard, singlehandedly, with onlookers.
That's not to say that devs don't hash out designs collaboratively via whiteboard (yet a different application). That certainly happens and is useful. But, you're likely to find that--even in those situations--team members have considered the problem offline prior to the collaboration.
> Bullshit. Sounding credible in technical interviews is a skill, not the same skill as actually being a good programmer, and might even (statistically, in the large) be close to orthogonal to it.
I don't agree with this. I think the issue here is that your interviewers aren't going deep enough in the conversation. If you ask someone to tell you about some interesting technical project they worked on and they give a good high level answer, push a level deeper. Still get a good answer? Push another level deeper. Repeat. At some point one of two things will happen: The candidate is capable of talking about the project intelligently at a lower level than the interviewer or the reverse. If the first happens, re-think whom is doing the technical interview, if the later happens, you've found their level of competence.
Strongest possible disagree. There are people who can talk all the way down to the point where they're basically dictating code to you, who nevertheless cannot deliver on a real project. There's a difference between knowing, intellectually, how to solve a problem with code, and actually being able to deliver working code.
In practice, though, what really seems to happen is that technical interviews are split between whiteboard-grade programming puzzles and "friendly deep discussions" about technical problems that both (a) can be prepped for with rote memorization and (b) are extremely game-able by people with strong interviewing skills (for instance: the skill to respond to questions with questions, or to redirect the course of a line of questioning back to safe ground for the candidate).
It's not like we were asking candidates "hey, how did you enjoy your last software security job?". We were doing something closer to "here's a piece of code with a heap overflow in it; talk us through how you'd write an exploit". It didn't work: there were people who could answer those questions indistinguishably from an exploit developer who nevertheless couldn't write a 1995-era stack overflow exploit if their life depended on it.
> It's not like we were asking candidates "hey, how did you enjoy your last software security job?". We were doing something closer to "here's a piece of code with a heap overflow in it; talk us through how you'd write an exploit". It didn't work: there were people who could answer those questions indistinguishably from an exploit developer who nevertheless couldn't write a 1995-era stack overflow exploit if their life depended on it.
Something about this example is not registering for me. I can't imagine how this situation is possible. Going from a detailed, step-by-step description to a code implementation is a trivial rote task in my mind. I must be missing something.
I mean, I've seen very technically proficient devs fail to deliver because there's too many cooks in the kitchen, or because they drown themselves in tooling or bikeshed their project to death. Maybe something like that happened?
There's definitely tons of senior devs out there that care way more about linting and unit testing than they do about writing the code that implements the spec.
Nothing hit quite as hard as the punch to the gut from the bit about linting and unit testing... spot on. I've already decided to move on and such, but my current company is so this that it hurts.
Happened to me. I hired someone onto my team who did great during the interview. The questions weren't puzzles, but live coding sessions. He got all the way to the point at which he had written a semi-functional web server.
This guy couldn't deliver a single project and would spend days debugging issues before he asked for help and I or someone else on the team would solve it in a couple of hours or less. I have no idea why this happens, but I've seen it twice.
> Strongest possible disagree. There are people who can talk all the way down to the point where they're basically dictating code to you, who nevertheless cannot deliver on a real project. There's a difference between knowing, intellectually, how to solve a problem with code, and actually being able to deliver working code.
I'm a little confused by this statement. Is your issue that they aren't _capable_ of writing the code or that their job performance isn't there once hired? i.e. Are the technical chops not there or is the work ethic not there? It seems like you are describing the later to me, or we are talking about different levels of conversation.
I haven't experienced this, not if you are digging into detail on how they solved previous issues. I'm not saying it doesn't exist, but I've never dug into detail and had coding be the problem. If you leave the conversation at a conceptual level, then I can see it happening much more often.
I'm not OP, but when I've experienced this, it's very similar to something like "I've seen a million heist movies, but absolutely could not steal shampoo from a CVS". There's a very big difference between talking about doing something and actually doing it; there's a level of executive function that's required.
More specifically, you make so many small executive decisions when coding. Naming, how functions and classes are designed, which file/module this goes in, is this already tested, oh I found an unrelated bug, do I make a ticket, a new PR, or include it in this one, etc etc etc. The only thing I've seen that can answer those questions is homework--maybe previous work, but homework is better.
But I think you're remiss if you're not asking your senior engineering candidates how they make those decisions, not because the answers to relatively trivial questions are important, but because you should know--and they should be able to articulate--how they make those decisions, and they should know the pros and cons of their approach and other approaches.
> It didn't work: there were people who could answer those questions indistinguishably from an exploit developer who nevertheless couldn't write a 1995-era stack overflow exploit if their life depended on it.
Do you think the problem was that they couldn't write that particular kind of code, or was it the more general problem of not being able to code well in general?
If you've never been around real bullshit artists it's hard to imagine, but there are people who can talk the talk so well and say all the right things, and yet their actual programming is terrible.
There are also the opposite. Folks who are not great at verbalizing programming principles, but do deliver solid code in reality.
Bullshit artists are honestly easy to spot. They are easy to trip up because they think they are good at fooling people. If they said all the right things, the questions were probably low quality. I would throw a zinger in to see if they can react to it. I honestly expect a candidate to say, "I am not familiar" or "I'm not sure I'd have to look that up" once during an interview.
The people I talk about aren't even deliberately fake. They really think they're awesome. They can say all the right things. They're charismatic and sound real smart. They have great anecdotes, dripping with wisdom.
What happens to every candidate who gets flustered by the questioning and seems to bow out earlier than their technical depth?
How do you hire people who are more technical in a subject area than your current staff, if your test bottoms out at the depth of the current interviewer in the room?
> What happens to every candidate who gets flustered by the questioning and seems to bow out earlier than their technical depth?
I'm not sure I get this question - I'm not talking about a white-boarding exercise. I'm just talking to the candidate about a technical project they have worked on in the past, the pain points, how they resolved them, etc. If that results in the candidate being flustered it's not a good sign on numerous axes.
> How do you hire people who are more technical in a subject area than your current staff, if your test bottoms out at the depth of the current interviewer in the room?
That is a _very_ different hiring situation. If you have no one in the company that is equipped to properly technically interview the candidate they obviously you can not properly technically interview the candidate. You'd likely have to leverage the connections of the management team to find the right person to lead up unknown territory.
One of the very best people we ever hired was so flustered in his final interview with us (a formality, after work-sample challenges) that he was visibly shaking during it. If we had taken the "numerous axes" on which the was "not a good sign" seriously, we'd have missed that hire --- and, I think, an entire business unit in our firm wouldn't have been started.
I generally think software developers (I count myself among them) know far, far less about psychological assessment than they think they do.
Of course that happens, but you're taking a real risk in hiring someone who can't get through the interview. I'd rather miss out on a good hire than make a bad one. You only have so many ways to determine whether or not someone is competent.
I don't know what to tell you. We switched to pure work-sample hiring and hired several dozen people that way. We retained all of them. I can't say that about our interview process prior to that: not only were we not picking up the "moneyball" candidates I'm talking about here, but we were also hiring people we (painfully) ended up having to let go.
I have an anxiety disorder, so in depth conversation with a stranger is going to cause me to be flustered. If you asked me to give the answer in writing I could. If you asked me to code on a whiteboard I could. If you asked me to do a takehome I could. But a verbal conversation with a stranger judging me in an interview setting is the worst out of all possible options for me.
100x this. Whiteboarding is so outside the typical engineering process that it just feels cruel, especially to people with any kind of anxiety. It's like if you were hiring construction workers and you asked them to perform an interpretive dance about concrete in the interview. It's just not at all the job and very difficult for many of the people who go into programming in the first place. It's hard for me to think of it as anything other than hazing.
> If that results in the candidate being flustered it's not a good sign on numerous axes.
Are you hiring sales reps or engineers? Why are you asking for an engineer to sell you on their past project(s)?
Interviews by their nature are about judgment, and knowing you're being judged by a complete stranger who holds the power over this potential path of your future is enough to get anybody flustered, especially a brainiac that just wants to work in relative peace behind their desk.
> Are you hiring sales reps or engineers? Why are you asking for an engineer to sell you on their past project(s)?
I don't ask them to sell me, I ask them to dive into the technical issues they faced and how they solved them. It tells me a lot more about how they will handle similar issues in the future than scribbling DFS on a white-board would.
Just to clarify, when you say "how they would handle similar issues" do you mean talking to a hiring manager about a past project during an interview setting?
I'm a bit unclear on your question. I'm assuming a technical person with relevant knowledge performs the technical interview, if that is what you mean.
The interview setting, with someone who might or might not be technically capable, with your job / career / earnings at stake is a very different situation than explaining past projects to a colleague at work.
Programmers in a company work with other programmers. If they cannot explain some work they've done then they are useless. Or would you hire a programmer who does not speak English?
I think what the OP is hitting is the difference between being able to explain something and being good at it.
With a deep enough interview process you’ll find people who know what they are doing. But it won’t tell you anything about their practical coding skills; it won’t matter if you are hiring an architect, but if it’s for a main developer you’ll want a decent level of productivity, proficiency with the tools, application of best practices and reviewing skills.
Those aspects are a lot easier to understand with actual code produced.
There have been instances where I have forgotten vocabulary that makes me seem knowledgeable about a project, but I have nevertheless done the work. I sound incompetent but I actually am competent.
IHMO you're comparing two different selection processes.
The process you tried was to spend loads of time on carefully designed interview questions which then get used to filter out candidates.
The OP was not suggesting this approach, but instead suggested engaging in a friendly conversation about technical interests and recent projects.
I can see why the first approach might fail, only because it isn't that difficult to memorize several answers for well know questions and topics.
However, to hold a conversation on a particular topic would require a greater depth of knowledge, so I can see how this would be much harder to fake.
I suspect the second approach is a lot like how police catch people out who are lying. As number of lies grow it just gets harder and harder for the individual to keep all the lies consistent and coherent.
You’re assuming the friendly conversation doesn’t have a structure that shows viable experience. Get somebody to geek out in detail about something they worked on and you’ll be able to tell very quickly if this person legitimately knows their stuff or not. You also have to be able to translate and validate this type of conversation though.
How do you get senior candidates to spend any time and effort on submitting "work samples"?
I'm a senior engineer. Last time I interviewed, I received multiple offers from brand-name employers. None of them required any work samples, just hard technical interviews.
A few times I saw a small unknown company that piqued my interest. I talked to them. Occasionally one would require me to do some sort of a task.
Why would I do it?
I don't need to do any of that for the top employers in the market. Why would I spend hours of my time, and precious coding time, especially on some small company I never heard of before?
I haven't done that in years, and doubt I'll ever do it again. The only way you could get me to do it is if I was desperate for a job.
By requiring this sort of time and effort from candidates, you are screening out the best ones, those who can get multiple offers from great employers without jumping hoops for you.
This seems extremely off to me - from my POV who the hell has time to study algorithms and implementing data structures every time they need to find a job? Trying to make sure I can whiteboard a random CS problem from a massive set of potentials on the fly in front of observers is drastically more involved and requires much more time that serves no other purpose than facilitating interviewing, versus just spending an hour of my time programming a solution to a problem at home.
I think it may depend on how much your day-to-day work involves data structures and algorithms.
Almost any work on compilers will involve good working familiarity with graphs
Work with concurrency will tend to involve linked data structures, like treiber stacks, or modeling computations as directed acyclic graphs of data dependencies
Working in a memory constrained environment tends to involve data structures and algorithms tailored to that domain, etc.
Now it's totally true that many algorithms questions just reduce to brain teasers, or are so esoteric and interview-specialized (Find out if this linked list has a cycle in O(1) space!) that they only serve to stoke the egos of the interviewer. But I wouldn't say that using algorithms or implementing a data structure is a skill that only exists to facilitate interviewing - and being able to explain your thought process as you work is really important when making contributions to other teams' code. I try to focus my interview questions on miniature versions of problems I've actually had to solve in the course of my job. For example, the futex state machine I wrote here:
If you subtract the interrupt and timeout handling, you have a real problem with a simple, approachable (for someone with some concurrency knowledge) solution, and you only need to know about two futex calls which have simple behavior and are easy to explain in limited time. I wouldn't ask this question of someone looking for a web development position; but then again, I don't interview those candidates because I know nothing about web development. But it's a nice question for someone who listed concurrency on their resume.
Most of the top employers in the market have similar types of interviews so you have to study anyway unless you want to rule out all of them and any time spent preparing for such interviews amortizes across all of them. Meanwhile, a take-home test is almost always a one-off and takes much longer on a marginal basis and rarely is even used to make the final decision - every time I've done a take-home, I've also had to go through the full interview loop.
Also, "an hour of my time programming a solution to a problem" is likely not an actual work-sample/take-home - it's likely a replacement for a phone-screen that doesn't require an engineer to conduct, thus can be given to many more people to keep the top of the funnel large. The only time I've done a real take-home in less than 10 hours, it was rejected for not being polished enough.
I'm fine going through tough technical interviews.
I'm not fine spending 3-5+ hours on some trivial coding task that can be gamed and was issued to me by folks who may not have even thoroughly checked my resume.
Given the current supply/demand curve, and the fact that top employers like FAANGs do not require this sort of time and effort (and for a good reason), I don't have to jump through that hoop.
I think what they're arguing is that since most engineers don't use the algorithmic knowledge on a day-to-day basis (and are thus likely to be rusty), the time they'd need to spend reviewing/prepping to pass a whiteboard coding interview is greater than the time it would take to do the work sample.
I kind of agree and kind of dont. I have a super simple inital question I always ask in interviews: Write me a function, in any language, that takes an array (or list) of integers and returns the sum. 90% of people that can do that are decent, and sadly i'd have to say i'm shocked by how many people struggle with this question, or the follow up of "ok interesting implementation, now what can go wrong at runtime?"
It's a great 15 minute question that I find often correlates highly with the people that get offers.
Honestly, just try it. Have a significant preface about how it’s simple by design, and you are not trying to trick them, if you are hiring, say, a VP or a CTO; call it a warm-up if you want. But absolutely try.
I’ve had a majority of candidates with five-year experience as a data scientist not be able to load a csv file using np.read_csv() The problem wasn’t that it didn’t work (the csv intentionally had a minor quirk that demanded that one uses a non-default option) but the reaction to it not working, or, in a majority of cases, the inability to search (on Google) for “what function allow me to import a csv file into a table” was incredibly telling. From there, you can have a conversation about what they did and it often leads to a more honest portrait of their previous functions. Sometimes, that led to offering them a more suitable position.
If I were hiring a VP of engineering or CTO, my only goal in administering such a basic test would be to see whether they are offended and prepare to politely leave the interview. Not kidding here.
Sure, CTO duties can run a broad spectrum, depending on company size, etc. It could essentially mean "dev lead with a small team", all the way up to public company CTO with a good command of financials, managing budgets, large multi-level teams, etc. But, at either end and anywhere along that continuum, I would want to skip to probing higher-order thinking around architecture, strategy, people-management, etc.
So, if I've brought in people who I believe are adept at skills like these, I'd expect them to be confused and perhaps even insulted when I pull out a fizzbuzy integer summation problem.
100% of those that couldn't do it are not qualified candidates.
Some of those that could do it are good candidates.
When you bring in someone that has a nice looking CV and experience and recommendations and you ask them a very basic question and they can't respond at all, it is a safe disqualifier. It also doesn't take six hours.
That's a different argument that gets us back to the original premise I disputed, so you're now just begging the question with numbers.
That is, your numbers are only valid if I accept the premise that I'm rejecting. I don't believe that 90% of people who can answer such a basic question are qualified.
The reality is that some jobs are social and some are heads down technical. More and more companies have no problem keeping an engineer in 20 hours of meetings per week which make people feel good but offers little in technical accomplishment. The rare exception is the shop where a team lead participates in many meetings leading the junior developers. This is an effective multiplier, most meetings are not.
I don’t know what your work-sample looks like, but unless it’s time actually working with your team on a real project, it gave you no indication of whether or not someone was “capable of delivering”
You may think you filtered out people who failed your test, but you have no indication that that’s the case – you could just as easily have filtered out people who don’t like tests or people who don’t test well
I trust you’ve found some decent hires, but I’d suspect you passed on many as well
And, I don’t know about your work environment, but I’d prefer someone who can have a friendly conversation and do good work... I’m surprised you bothered giving your work-sample test to people who didn’t pass the earlier test
> but unless it’s time actually working with your team on a real project..
I've ranted about this before on HN, but that's very much illegal in my neck of the woods. As soon as someone does useful work for you, they're an employee. YMMV but you can't expect new candidates to do actual work until you've hired them.
There are plenty of programming assignments, exercises or questions you can have them do instead.
So have a basic filter, hire based on who meshes well with the team and have a trial period during which they actually get evaluated for competence. They’re technically an employee during the trial period, but with the understanding (and contractual agreement) that they will be evaluated at the end of the period and that the employment will be terminated if they do not meet the criteria.
Trial periods, too, are no longer allowed over here. They used to be permissible up to 6 months (which was an insanely long time to essentially live with the knowledge that you could basically be sacked any day). But now they're gone. Firing people isn't terribly easy either, might need to give two months notice!
Of course, the net result is that self-employed contracting is on the rise. Especially the kind where people are self-employed only in name, but really work for a single employer. The only difference is that they send an invoice at the end of the month, and the employer can terminate the contract at any point (depending on the terms in the contract, of course).
Hiring is hard. Especially so when hiring the wrong person can cost you two months' wages.
I will ask you about you projects, then I drill down till I find something technically interesting and have discussion about it. I will concentrate on failures, not on successes.
Gives me better signal than solve a problem on whiteboard question.
What do you do when the person you selected ends up being a wrong fit? Do you try to rehabilitate (makem them fit, if that’s even possible in a small team), of do you cut your loses? What’s worked out for you?
Do you think it's always worth the effort to get that lower false positive rate? Other professions seem pretty happy to do a trial/probation period after less elaborate interviews.
Was there a specific attack vector you commonly saw from candidates in interviews? It would seem pretty difficult to fake having built a real product or made significant open source contributions. In fact would probably be as difficult or more difficult than faking having written a book, and as far as I know book publishers aren't constantly trying to weed out would-be authors who have faked their previous publications.
I'm not saying you're wrong, especially since we hear about this kind of thing all the time, but I just don't see anything about coding that would seem to make it uniquely conducive to this sort of deception.
I have interviewed candidates who:
* Have impressive products that they "designed and implemented.
* Co-authored a book.
* Founded and ran a {programming-language} user group.
In all 3 cases, the candidates came across very well when talking at the surface level. Digging deeper, they (in the first example) overqualified their contribution, and none of the 3 could write the simplest of code.
When a guy tells you his experience, and you call bullshit, it's hard to tell whether you doubt his sincerity about having the experience at all or whether the experience itself was not representative of your reality. Either way, you're telling us not how it is, but how you are, and that's simply not useful.
In "The Last Starfighter", the recruiter/hiring manager Centauri is constantly accused of using "the Excalibur test" with derision, and which he denies. While I've never seen it explicitly defined, I'd imagine an on-the-spot work sample test to be what it means. "Pull this sword out of this stone and you get the job."
What proponents of the Excalibur test fail to realize is that their methods often spoil the data. Treat me with enough derision and pressure off the bat and I guarantee I'll fail your test even though I do fine at other companies.
I won't apply at, or respond to recruiters representing, those companies that have a reputation for a long and/or torturous hiring process.
While I have no interest in being interrogated with high-stress puzzle solving, it is not the interview process itself that causes my disinterest; it is the understanding, from experience, that how a company treats its potential candidates is often reflective of its general culture.
I was referred to a company that eventually handed me “6 hour” (likely much more to complete sufficiently enough to advance) take home project a few weeks ago and I still can’t bring myself to start it. I read their Glassdoor reviews and the number of folks are reporting either being ghosted or getting a negative response minutes after submitting, after putting days of effort into this exercise, is disturbing. It seems like you lose by not playing the stupid game of asking questions while working on this inane project, so correctness and effort isn’t even enough — you must also pretend to be a moron who can’t figure things out on their own.
I also submitted one of those exercises for a different employer — that one took me about 6 hours but of course they estimate it at “about two”. I submitted it a week ago. They acknowledged receiving it immediately. And that was the last time I heard from this employer.
I’ll follow up tomorrow, but I’m kind of ticked as I was pretty happy with my code, and it’s not even something I can open source if they don’t like it. I also just got an offer elsewhere and I bet mentioning that fact will expedite their process more than me putting in effort into dancing around like a monkey.
Anyway, I’m done ranting, and I’m done doing take homes. I used to be a fan, but only because I preferred them to whiteboard algorithm puzzle questions.
> it’s not even something I can open source if they don’t like it
What? Of course you should... make sure you create a repo with the company name and people can create issues to collaborate on a better ToDo sample code.
It is mostly correct. Unless you sign very carefully worded IP assignment agreement to that effect, even work you get paid for remains under your own copyright. And frankly anyone who would sign that kind of thing on a employment test exercise is... Naive.
To be a work for hire, the work must be produced by an employee within the scope of their employment. Consequently, you cannot do work for hire if you have not been hired.
This is not true. If the problem statement itself is a unique creation, and the solution is publicly viewable and gives details on the question itself, the question writer may have the copyright even if you signed no NDA when solving it. Even if not, if a company wants to sue you to try to keep their interviewing IP private (as they see it), they can tie you up in court at a trivial cost to themselves but a life destroying cost to a lone developer.
Source: I once had a dragged out legal dispute with a company that used a GitHub DMCA takedown to tell me to remove a public repo in which I solved their takehome challenge (I had signed no NDA or otherwise any type of agreement of any kind prior to solving it).
If it's not an original question find the original source (a source that is older than the company would be especially nice), and post a solution to the question posed by the original source. The world is a big place, a lot of question are not original.
Downvote facts. I don't care about the fake worthless internet points. It is just pathetic that people downvote actual facts without having an actual rebuttal or intellectual counterargument. Nope just be a coward and downvote because I don't like it.
Not sure why you got downvoted, but I’m also not sure what point you are trying to convey. Facts are facts like 1 + 1 = 2, but it’s unclear what that fact brings to the conversation.
The top companies with these leetcode tests probably don't care that good people being rejected or avoiding them because of the amount of preparation required. Middle sized companies and startups doing leetcode tests are missing good people and probably can't afford the same number of false negatives as someone like Google with an endless supply of candidates. It seems like smaller companies often lazily copy these processes despite it likely costing them good engineers.
Having gone through the process recently I avoided companies with puzzle solving hiring procedures because I was confident that I could find well paid, interesting, work without jumping though those hoops. As you get more senior there are more people you have met and worked with with employment opportunities. It gets easier to find work by asking around and finding someone who will hire you based on recommendations or having worked together before.
This also applies to when you're actually interviewing. I once went through three rounds and they wanted a fourth. No is such a powerful and unexpected word. That they're still talking to you means they're interested but that they haven't offered you the job means they're not 100% sure. This one scenario I said No they came back with an offer. But it was rejected.
Would you do a 4-hr take-home assignment? What if I gave you a $100 amazon gift card?
There's a big difference between making someone jump through countless, one-sided hoops and asking for a little effort. The challenge IME is finding the balance.
Personally, if I can get to an onsite with another company after a recruiter chat and phone screen, I’m not even going to do a “2 hour” project. In my experience, actually spending the recommended amount of time on one of these take home projects is a losing proposition when there are people who will spend more time on it.
He’s implying the author fell victim to a cognitive bias.* he’s not doubting their honesty , he’s just being harsh on the lack of discipline when it comes to battling your own brain’s tendency to lie to you.
Avoiding biases is hard and painful work, which goes well with being resolute and non compromising. When this accidentally bleeds through to others, because you’re human and you forgot to throttle back, it can seem blunt and disrespectful. But it’s often not meant that way, at all.
I think it’s fair to give him benefit of the doubt, here. From what little I know, Thomas is a man with great respect for data. :)
* Selection bias? Whatever, it’s a bias. Apply the “pick one to sound smart” rule. :)
> "Treat me with enough derision and pressure off the bat and I guarantee I'll fail your test even though I do fine at other companies."
If only more companies realized who they are able to hire is a function of how they hire. And it starts with the job / opportunity / role description.
As a side note, it seems to me, all this friction in hiring (senior level talent) makes a good case for developing and promoting from within. I realize it's not that simple. None the less, taking a known and grooming that person isn't rocket science either.
If management is rude and naive enough to be bad at hiring, they can be expected to be equally bad at promoting and retaining internal talent.
For example, there's the simple pattern of ignoring employee careers and expectations until, after the least patient ones have left, suddenly there's a lack of personnel, and the best remaining people are needed in their current dead-end role.
The mistake I see __consistently__ is that when I'm discussing an opportunity / relationship with someone who is "hiring" is that they don't realize I'm interviewing them as well. Fairly often, those who don't realize this are the same ones who will hint / complain about retention and such. They want better people but that have no idea how better people think / feel and how their approach will never get them the type of people they need.
Once I interviewed at a large fashion company for a Python programmer position. I asked to see IT offices (no way, even if I had required a second interview expressly to see them; the building layout suggested some sort of overcrowded basement), I asked about salary (matching my current one would have been a stretch), about benefits like a car (oh no, only for top management!), about training (studying their software after hours, on my own time), about lunch (internal mess hall only, and nobody can leave), about their software engineering approach (overtime, and proud of it).
They unconsciously went out of their way to demonstrate a corporate culture of paranoid secrecy and to show that IT had very little consideration within the company. I tried hard to find a reason to like them, and when I found none and refused their offer I got an angry call from the manager: I wasn't "motivated". He thought I had wasted their time.
Ironically, Patrick, Erin, and I did a company after Matasano that was based (sort of) on work-sample hiring (really, more based on CTFs) that was called Starfighter.
>So, err on the generous side with your scoring, and down-select with interviews. Iterate until your interviews almost feel pointless. We got there!
I'm having hard time parsing this. First sentence suggests that I should be permissive in the scoring part, and rely on the interview to weed out candidates if we had too many. So interview becomes important. The second sentence says that interviews will stop being important. Can you clarify?
Funny. I agreed with the author. It's relatively easy to gauge technical ability vs those who can pretend. I only use a simple take-home problem when I truly can't tell which type they are. The really great part is that I usually tell a faker from the resume. Maybe I should blog about this.
I want to highlight a third possibility that isn't discussed which is people who can talk about code, but can't deliver (or can't deliver well), but really want to learn how to deliver.
It's fine if you don't want to hire that person, but I'm really annoyed that they often immediately get labeled a bullshit artist in a lot of these threads.
Considering how many shitty positions there are, it's honestly surprising we don't reflexively go "oh, you never learned to do X well because you haven't been with a good company that can teach you to do X well".
I feel the same way. Obviously I can't measure false negatives, because they don't get hired, but I can't remember having been duped before. My process is to ask them to describe their role on some projects on their resume and then keep asking them deeper and deeper questions about it until one of us hits the limit of our knowledge. I only ever ask people questions about things they claim to have done. This probably creates some false negatives with really shy people, but it's easy to suss out the bull shitters.
There's a company that turns up here on HN sometimes that claims to find the best engines. They were looking for part time remote screeners/ evaluators. I figured I'd apply. Filled out some stuff online, and then their system wanted me to take a test on java script or something - not my language. It was then that I knew they don't really know how to interview people, so I stopped the application process.
Good to hear you say take-home problems, as I believe it's a much more realistic assessment than is whiteboarding per what the job actually entails.
Being a good dev is about being thoughtful in assessing a problem and designing/implementing a solution. This vs standing up in front a group of people and hastily trying to devise a solution to a random challenge.
I've turned down roles in companies that took this approach to hiring. It raises a massive red flag in my mind on the organizations technical pedigree and the sorts of people I would be working with.
Sounds like the problem lies within your organisation. My guess is that you lack someone who can actually evaluate a candidate properly based on conversion and need to resort to other means.
It's a process similar to creating a minimum failing case for a regression: take a tightly scoped problem which you'd actually reasonably assign an e.g. intermediate engineer to in your company. File off the serial numbers if you need to, then start reducing all of the unnecessary scope from the production environment until you get to just the core bit that can reasonably be explored in $TIME_BUDGET.
Package that up in a self-contained Vagrant image / Docker image / whatever with all of the skeleton required to run a full answer to completion. Then, take out the full answer. Consider leaving a reduced test suite so that candidates can see if they're making progress in a positive direction.
Now write the rubric by which you'll assess candidate answers. Generally, you'll want to pick ~5 areas which are important for you and write prose describing what 0 points through 4 or 5 points are worth in each of those areas. Then, determine what a passing score is, perhaps by calibrating through running it with existing engineers at the company. And then (this is the brutally difficult part): convince your organization that the rubric now makes hiring decisions.
Then, create a way for people to submit these into your hiring process. This might be as simple as "Create a private gist of this file and email me the link" through something with material tooling developed.
(This answer may or may not be exactly the same as Thomas' answer, but we were co-founders at a company which was quite related to this problem.)
This is the only correct answer. White boarding is to coding what Foosball is to soccer.
In my last in-person interview they asked me a bunch of trivia that was a simple google search away - so of course, why would I bother memorizing those things? I got a dear john letter back saying I wasn't smart enough or some such, and it's like "oh great, being called an idiot by an idiot". And that's most of my interviews if I can get past head hunters that don't even bother to do their homework. Do you know "go long"? How would you solve this "cash-ay" problem? Yuck.
Sometimes I just wish people had some common sense and were reasonable and could think. But by and large people just do not give a crap to be good at their job because people by and large are just lazy and suck. Call me a pessimist but man are there some stinkers out there.
Agreed. In my experience, senior engineers are even better at glibly skirting what specific contributions they made at a company were, or what their true technical skillset is. They need to be tested.
The fact that a lot of time senior engineers will get pulled in to interview from other companies because someone knows someone and has worked with them for a long time only exacerbates this problem.
> because someone knows someone and has worked with them for a long time only
And this is the #1 thing that gets a near auto-hire from me. I'm not saying don't interview them, but most interviews don't really tell you what it's going to be like working with someone. On the other hand, WORKING with someone (for a long time) 100% tells you what it's like working with someone.
If someone has worked a long time with someone else and is willing and wanting to do so again, then hire them.
My experience is, most interviews suck trying to figure out how someone is going to do on the job. Working with someone a long time and wanting to work with them should weight heavier than anything else
I can't tell if you're saying "look at their resume and see if they've successfully worked on teams for extended periods of time" (don't do this; it's a debacle of a signal) or "hire them if you yourself have worked with them before" (this works fine but network hiring can lead to brittle teams that fall apart as soon as 1-2 people move on).
Someone I've worked with before and whose work quality I respect may get an "auto-hire" from me, but that doesn't necessarily translate to a "auto-hire" from the team that is going to be working with them. I'm fine introducing good candidates I know if into a recruiting process, but not forcing them through just because I know them. In that sense, I want them to be able to demonstrate their abilities in an interview to the rest of the team, as well.
And yeah, I have strong opinions on this - I've been on both sides of the table for a long time, and don't think the average interview is a good indicator of how they'll do in the job.
Isn't it just... mostly money? I mean, if you want to hire someone good, who presumably has a good job, you have to pay better than what they are getting now.
I think this is probably especially true in Europe; I mean, as far as I know, people get like a 2x-4x raise moving to silicon valley from europe. If you pay more than your local competitors, you will largely have your pick of the candidates.
I mean, yes, things other than money matter a lot, too. But as an employee? I think that's really, really hard to judge from the interview. The money is pretty easy to judge, and as far as I can tell, total comp is a good 'honest signal' that I'm going to be treated in other ways. How much money they offer is a better indicator that I'll be treated well, in my experience, than anything anyone can say at an interview.
I think as workers, we're not doing ourselves any favors by pretending that an employer can get the best people without paying the best rates.
Like I said elsewhere in this discussion, I disagree. Of course, salary is very important, but I (and many people I know) would easily choose (and have chosen) a lower salary for any combination of these perks:
- Work fewer hours
- Work from anywhere
- Work in a specific location
- Work in a certain environment
- Work in a specific kind of organization
- Work with specific technologies
- etc.
People are very different and have very different wishes and needs. Salary isn't always priority #1.
I think #1 is mostly a pay issue. If two companies pay the same and one expects more hours, they don't pay the same.
(note, I'd also take a small hourly cut to work fewer hours a week, but as far as I can tell, the paycut I would need to take to do that is so large that I'm better off working a few years on and then taking six months or a year off and eating the tax and career disadvantages of that)
For most people in my area, #2 and #3 are also mostly pay issues. Like #1, it's not entirely pay, but the most common reason I've heard for people wanting to work places other than here (I'm in silicon valley, as you'd expect from someone trying to maximize salary) is that it's less expensive to live almost anywhere else in the world.
Yes, you are right, is mostly money. But there is other stuff that at the end adds up, specially if you are a Senior Engineer in a FAANG company:
* Very few recruiters outreach mentions any kind of compensation expectation. Why would I bother with hours of chat, screening and interview loops if I know only a handful companies offer the same kind of compensation I'm used to receive?
* The bar of a Senior position is quite high, so high that most Senior engineers would not pass it, it requires months and months of preparation. I would do that only if I was absolute sure I wanted to leave my current job.
* I would be willing to receive a little less if a company could offer something that I don't have like a better commute, work on something I'm more passionated about, etc... However, most companies think that working from home is good enough to justify a 50% lower compensation.
The only exception that I can think that compensation is not the main point is if you are trying to hire someone so successful and rich that money doesn't really matter for them. But my guess is that this kind of people are even more rare than senior people.
I've paid off my mortgage and am now more open to working on something more altruistic than selling ads for slightly less pay. The problem is I can't find any interesting without a large paycut.
Really? Are you in the Bay Area? If so, I find it hard to believe there’s not at least one startup within 10 miles of your current employer doing something at least vaguely interesting (or, at least more interesting than selling ads).
that's the thing; I mean, yeah, we'd all take small pay cuts for a lot of things; but dropping out of a FAANG company to one of the companies that is having a hard time hiring is a massive pay cut.
Like, for me; I'd be so happy to take a 20% (hourly) pay cut and work half-time for 40% of my current total pay and go to school the rest of the time. But as far as I can tell, the hourly paycut would be more like 50+% just on the hourly, so I'd be looking at like a quarter the total comp, and the prestige cut (and thus hit to future earnings) would be huge, as I'd have to go work for a startup/small company.
I guess this will make more sense if marginal tax rates go back up.
You have to be a senior at Google or Facebook, maybe Bing, but nowhere else would be cutting your pay for a fixed amount of experience unless you can't get an interview at Netflix
One problem that I do have is that, although I'm prepared to take a relatively large pay cut if I'm really passionate about something, one thing I do care about is working with capable colleagues and being able to actually achieve the thing I'm passionate about. That's less likely to be the case when a job doesn't pay well, unfortunately.
The term "FAANG" initially comes from the stock markets, where those 5 where hot new tech stocks, and Microsoft was a boring, established player. Nowadays it's more used as a shorthand for "the big tech/internet companies", not the specific members indicated by the letters (e.g. Netflix is an outlier, and Microsoft probably closer to the others than it, in many ways)
Not really, you have to pay competitively of course. But you can safely assume that everybody else does that too. I noticed a pattern in Berlin where I'm based where the less attractive jobs indeed come with higher salaries. When the work is shit and they bleed engineers faster than they can hire them there's nothing else they can do than up their rates.
So, when I see a top rate advertised, I immediately ask myself: what is so wrong with this company that people don't want to work there? Usually it's fairly obvious when you meet people that used to work there (they'll have a grudge). Usually it's a combination of mismanagement, bad culture, and corporate stupidity. It happens. Big companies can get away with that for surprisingly long.
When hiring, what I look for is very simple. I try to sell the gig and look for positive response to the sales pitch. Does the person have the skills I need? A few simple questions can get you a good hint there. Red flags would be bullshitting when they don't know something or plain ignorance for stuff they ought to know. Beyond that, it's about fast decision making based on imperfect knowledge. I'm well aware that most interview processes are completely ineffective so I don't waste too much time on that. Either they obviously know their shit or they obviously don't.
So, I simply put myself in the frame of mind that I've already decided to hire the person and start talking about what they'll do, how they'll do it, etc. It's classic sales strategy and it works; I've hired some great people this way.
> Not really, you have to pay competitively of course. But you can safely assume that everybody else does that too.
In silicon valley, the difference between what a FAANG company pays and what almost everyone else pays is... pretty big. Like maybe not double? but certainly half again as much. I mean, you could say they are competing for different people, I suppose, but when a recruiter tells me that they "pay competitively" - that means drastically different things for different companies, even if I am limiting myself to companies within 10 miles of santa clara.
> I noticed a pattern in Berlin where I'm based where the less attractive jobs indeed come with higher salaries. When the work is shit and they bleed engineers faster than they can hire them there's nothing else they can do than up their rates.
Huh. That's really interesting, 'cause my experience (In the US, mostly in silicon valley) is the opposite. There were times in my life where I actively sought out harder and even more dangerous work for more money; but even casting aside any concern over who I worked for, even if I was willing to go work in a war zone, for my skills? the pay was better sitting on an expensive chair in a safe office in mountain view.
Even within silicon valley, the smaller companies and the "scrappy" startups or small companies seemed to me to be a whole lot higher stress... which for parts of my life? sure, I'll take extra stress for extra money, but no, those companies pay a lot less. Which, again, is fine in certain parts of my life; I'm not educated, so I needed to get a lot of experience before I was given a shot at a job in one of the FAANG companies. And it was... really a very different sort of experience; I was 20 last time I had a job where I felt that junior compared to my colleagues, (and I had a hell of a job when I was 20).
I've seen something similar around layoffs; I've been through several layoffs but have yet to get a pink slip myself. and I've always found myself dissapointed that I wasn't laid off; those in the first round get nice severance, a retraining bonus, and a chance to get another job before too many other people are laid off. Those of us who "dodge the axe" end up having to do twice the work with almost no hope of a raise and with co-workers who are afraid of getting laid off in the next round.
> I mean, as far as I know, people get like a 2x-4x raise moving to silicon valley from europe.
One factor to keep in mind here, though, is that the cost of living there is so insane that even a multiplier on pay could go with a total quality of life decrease.
If I hire a senior Programmer in Bulgaria for $35,000, they can afford the house of their dreams and lead a very good life with holidays to nearby scenic destinations. Any excess profits that the company makes translates into a huge bonus for them. They have the chance to work in a growing company and be revered as the top of their society.
If that same person moves to the US, they are just a cog in a huge machine, and will be unable to afford to buy a house. Not to mention that getting the visa to go there would be very difficult.
Not to mention that when you work for a small corp, you spend all of your time on dubious products changes for the sake of a product manager looking good for an interview panel.
In developing markets there is no such 'bullshit', to be frank. The company delivers a good product to the international market or it dies. If you can find a talented Westerner to lead the team and product design, even better. This kind of leanness is exhilarating to work for as long as the company approach, payments and bonuses are transparent.
>The money is pretty easy to judge, and as far as I can tell, total comp is a good 'honest signal' that I'm going to be treated in other ways. How much money they offer is a better indicator that I'll be treated well, in my experience, than anything anyone can say at an interview.
Eh, not in my experience. I've often seen it as an excuse to treat people poorly. As in "You're being paid a lot to put up with BS" and "Highly compensated shouldn't have excuses or whine." It also seems to correlate well with their trying to solve all legitimate work headaches with money.
Hands down, the number one problem in hiring and retaining senior engineers is being able to match the compensation level. FAANG-type companies have a huge difference to practically everyone else. Startups, especially, can’t match the salary, and few make up for it in equity comparable to the risk level engineers take on by accepting the job. Senior engineers typically know better, and so you end up with mid or even junior applicants for senior roles, and out of desperation, they end up being the pick.
I disagree a bit here. Of course, salary is very important, but I (and many people I know) would easily choose (and have chosen) a lower salary for any combination of these perks:
- Work fewer hours
- Work from anywhere
- Work in a specific location
- Work in a certain environment
- Work in a specific kind of organization
- Work with specific technologies
- etc.
People are very different and have very different wishes and needs. Salary isn't always priority #1.
The problem isn't lower salary, it's the paltry options most startups offer these days.
The YC startup I worked for got acquired recently. I was employee #38. I've still made more in RSUs working at a big tech company than I would have made had I stayed the full four years and vested all my options. In fact I got a bigger RSU award this year than what I got from my stock in the acquisition.
Startups would be well-advised to hire fewer people, get rid of under-performers quickly, and reserve an extra slice in the cap table for really high performers. If someone comes along that creates entire new features on their own initiative (features still featured prominently in your marketing) then may I suggest you continue to award that person either their initial grant or half (depending on how much it was) on a yearly basis? Even if that means you skip an extra hire that year you'll still come out ahead, both by keeping your high performer happy and by saving the extra salary.
(I want to be clear: The founders were well within their rights to deny my requests for more options and I hold no ill will toward them. It was their business, not mine, and they were free to run it however they wanted.)
You can get those with a startup. I sort of did, for some definition of "certain environment" and "specific technologies" and so on. I still do actually, post-startup.
Notably, I got what most software developers would describe as "fewer hours". We tracked time and got paid for actual hours worked. (still do) Since the company would be paying more, we were/are seldom asked to work beyond 40 hours per week. Much of this comes naturally with government contracting. You must track time, and you must pay employees for their time.
Being onsite is a positive. I'm never expected to work at home. Work is work, and go home I can focus on my family or hobbies or sleep.
And even if they did make those promises and honored them, you're taking a risk that they won't. And it's not like you get the FAANG salary back if they can't follow through on working hours, the tech stack, or whatever other consideration is important.
Work from anywhere is a fairly powerful lever. You can reduce taxes and cost of living extensively and still be able to save the same amount of money in the bank and have the same amount discretionary income. I estimate something from a %30-%40 'discount'.
From my experience, it depends entirely on the startup. There are enough success stories now from startups that treat their employees very well and came out on top.
I find this "salary first" a little bullshit too. For the most part it's true, but it's not strictly true. If Amazon offers me $180k but I need to work like a slave, but a startup offers $150k but is very chill, I'm gonna choose the startup.
What if Google or Facebook offer you $300k no risk, with a relatively standard work life balance. Then you have to really carefully weigh the startup equity package and how much you enjoy the respective work environments.
> What if Google or Facebook offer you $300k no risk, with a relatively standard work life balance.
If. People here sometimes make it seem like a senior engineer can just walk into a FAANG, chat with the receptionist, and walk out with a $300k offer. While it is certainly not the hardest thing in the world, these companies are still pretty selective and only employ about 10% of the total seniors in the market. More likely than not you are not getting that kind of offer. This isn't even taking into account downleveling, which Google in particular is notorious for.
Who cares if you're downleveled? The compensation bands are wide enough that they overlap. This usually means that even though you expected level N, you can get your target comp at level N-1 if you negotiate.
That was not my experience. Google was pretty inflexible and offered me (with 16 years of experience) well under $300k for an L4 position.
A month later Twitter offered me 35% more than Google's best and final offer for a more senior position. I didn't even have to negotiate or mention the rejected Google offer.
Oh, I should add that, in my case, the downlevelling was up front. The recruiter wouldn't even put me in for an L5 interview.
I don't know the details of your experience (were all 16yrs of experience as a SWE demonstrating a deep technical expertise?), but Google puts engineers with even 2-3years of experience on an L4-interview loop.
If you knew that you were being slotted for L4 upfront why did you even bother interviewing? I've seen L4s rack up multiple job offers from different companies and use that as leverage to negotiate; they've all been offered $300k+ or more. Not all experience/education is the same but you gotta know how to play the game.
> I don't know the details of your experience (were all 16yrs of experience as a SWE demonstrating a deep technical expertise?), but Google puts engineers with even 2-3years of experience on an L4-interview loop.
It's all engineering, and the majority (10+) was pure software engineering.
> If you knew that you were being slotted for L4 upfront why did you even bother interviewing?
I didn't find out until late in the process, I thought I was going to get a better offer than I did, and I was trying to correct the mistake I had just made by accepting the job I was then in. I only realized my mistake after the offer came in and I considered what dropping back to L4 would mean.
> I've seen L4s rack up multiple job offers from different companies and use that as leverage to negotiate; they've all been offered $300k+ or more.
That would have been nice, but I barely had time to keep up with one company process at a time without arousing suspicion. I was close with Facebook, but I decided before the actual interview that I wasn't going to take L4 at any price. Facebook rejected me anyway, so it wouldn't have mattered in the end.
It means: You interview at Level X, but they give you later on offer for Level X-1. As a reason, they will claim you haven't scored good enough in the interview loop - which might be true, but also might have been the result of bad questions or an incompetent interviewer.
Sources: I know people in person who got these kinds of offers.
"Standard" work life balance is relative though. I know lots of people who don't want to work full time. This is especially true for people with family/kids (who incidentally are usually older/more experienced).
I have, and instead took the job for almost half that at a startup. It's not that tough of a decision if you properly weigh the quality of your time.
Keep in mind that you spend roughly half of your waking hours at work. It's important to do what you love during that time. I'm personally not a fan of spending my life to make tiny increases in some ad placement algorithm.
There are other places in the US that are hiring senior developers outside of Silicon Valley and the FAANG companies. There are major metropolitan areas in the US where you can start a company, get good senior engineers and not have to pay the wages that FAANG companies pay.
You may not be realizing that half or more of the compensation at the big SV tech companies is not salary.
Total compensation of $300k or more is pretty standard for the public companies. Check levels.fyi to see examples.
The benefits are also much better than I’ve seen elsewhere.
I moved to SF from Raleigh because even after cost of living I earn significantly more (ie. put a lot more in savings every year)
Housing here is definitely not as nice as other parts of the country, but if you work hard for a few years here you can leave and pay cash for a nice house in most of the rest of the country.
That’s why engineers keep putting up with the drawbacks and moving here.
And that’s only true at four or five companiess at the most....
Especially if you’re married, I know a lot of developers whose spouses income goes straight into savings, investments, etc. but it’s another example of HN Silicon valley thinking that software engineering starts and stops at four companies. Married dual income couples especially can make out like bandits in any of those cities.
I live in one of those cities and depending on how you classify me,I’m either somewhere above the median or well over the 75% percentile. Our nice big, new (comparatively) cheap house in the burbs is less than 20% of our take home pay. How much do you think we put aside in savings?
Because assortive mating is a thing - it’s statistically likely that two college educated people will end up getting married and I’m trying to keep this as generic as possible. If one was a software developer making $135K a year in one of those cities and the other was almost any college educated professional making $65K a year. The net of the lower paying spouse could pay off your nice house in the burbs in ten years or less.
I don’t know of any couple who is in that situation (and I know a few) who would think about moving to the west coast just for one spouse to make $300K when their combined income is already above $200K in a much less expensive area.
I get your point, but I think you’re overgeneralizing.
(1) the salary / equity combination I’m talking about is not just FAANG, it’s basically every large SF tech company. I don’t work for a FAANG but the deal is almost as good.
(2) I specifically moved with my wife and kids to SF to make it easier for my wife to stop working — she wanted to focus on the kids for a few years. One income here was worth more than double what I could earn in Raleigh and this is the first job I had in my career that offered comprehensive medical for the whole family, not just me.
We live in about the same amount of space as we had in Raleigh, it just costs more.
Lastly, while I would prefer to have a little more space for a less absurd mortgage, I’m not interested in living in any suburb. So when you compare housing in the central city of Raleigh (or Austin) versus SF, the comparison is less unfavorable.
So, sure, if I was willing to live in the suburbs I could have more house. But I’d rather deal with less space than is ideal than have to live in the burbs. No judgment intended, it’s just not the lifestyle I want, in the same way it sounds like city life isn’t the lifestyle you want.
I am always telling people that you reach a point where you have enough money to live the life you want. That point is different for everyone and it also depends on where you are in your life.
If I made twice as much money where I live now. It wouldn’t have an immediate effect on our life. Even now when I get a raise it just goes to “increasing my net worth”. It doesn’t really change too much of anything.
We could retire earlier. But my hobbies are computers and working out and traveling. I dont see myself fully retiring, maybe just doing consulting/contracting part of the year. As far as health insurance, my wife specifically took a job with the school board so we could get benefits whether I am consulting, contracting, or working salaried and we are guaranteed coverage for the rest of our life after she has been working ten years. I work with someone whose wife gave up her demanding job that she didn’t like to work in the school cafeteria for the same reasons.
Recruiters and consulting companies are baffled why I don’t accept an opportunity to be a high price consultant and I am sticking with this small company I work for when I could easily be making 25%-50% more. But living in a low cost of living area gives you that optionality.
Heck, there is a 50/50 chance I could get a job with Amazon working as an in house AWS consultant without moving (but with traveling). I’m just not interested in large companies. I worked at one at the time Fortune 10 (non tech) company for two years and I said never again.
I think the thing here is that cities are a better deal if you can command a top salary, and worse in most other situations. For example, talking about FANG, a senior position at Google (the lowest terminal level) comes with an average salary of $360k. Whereas someone who can get a senior position at Google is looking at your $135k (I get the feeling you're quite senior) or maybe a bit more in lower CoL areas. Even adding in a significant other that doesn't see a bump moving to the Bay, you're likely looking at 2x in the high CoL case.
On the other hand, if you can't get one of those top positions in the Bay you're probably looking at closer to $150k-$200k in a senior position, which makes a $100k or so low CoL position look awfully nice.
This gap widens as you go up the salary scale. I don't think I've ever seen even $300k in a low CoL area, whereas in the expensive areas 7 figures is possible (although quite rare).
How much would it cost to get a five bedroom, 3-1/2 bath newly built house in a top rated school system in Silicon Valley?
That’s easily affordable with one income, with a family making the median software developer wage. Any competent developer can pick up the phone, call a few local recruiters and have three or four offers in three weeks. Of course the pickings get slimmer the further to the right of the bell curve you go. But I do know developers who have been a lot more aggressive about their career a lot longer than I have and usually takes them two or three months to get jobs over the 80th percentile.
I picked $135K because that’s a pretty easy mark to hit if you have any negotiating skill, keep your eye on the market, and keep a warm network.
$160K+ is doable as an architect or an overpriced consultant working for a consulting company.
I've never lived in SV, so I'm probably the wrong person to answer that. But your real question seems to be "how can I live exactly the same life as now, but in an expensive city", which - sure, that's probably not possible. But neither can someone in a cheaper area live within half an hour of, say, a major international airport and several world-class museums, and within a 5 minute walk of a dozen high-quality restaurants. It's not really a function of cost or CoL or whatever; in both cases you're trying to find something that doesn't exist at any price in the given area.
Which is to say - the finances are only one part of it! Living in a large city is qualitatively different from living in a small city which is different from living in a rural area. If you have a strong preference, giving up the best financial situation to live the life you want is super reasonable. I know plenty of people who could live more comfortably in a cheaper area, but some people just want to be in a big city. I'm sure the reverse is true as well.
Well to give away where I live, it’s about an hour away by car from the world’s busiest airport, or less if I drive to the train station and then take the train directly to it - not exactly Small Town America.
None of the cities I listed are in what you would consider rural America. Yes we live in the burbs, but if living in the city were the lifestyle we wanted, we could buy a smaller condo for about the same price and I could still have access to the same job market.
The difference being that the FAANG-type companies don't pay well enough to make up for the cost of living. Senior engineers seldom want to live in tiny apartments.
See if you can prove me wrong: find a 10-acre lot within a half hour commute, with a large single-family home, that is affordable to a senior engineer. Alternately, make it 1 acre, but within a 5-minute commute.
I understand what you’re saying but I think you’re underestimating how many people like city life.
I work in SF and most of the people here are here specifically because they DONT want to live on ten acres in some rural place, they love that the city is walkable and dense and interesting.
For most folks the issue of living in condos or apartments it’s an issue at all until they reach a certain family size. At that point, most of the people in the city would be thrilled to live in a 3-4 bedroom townhome in the city — but THAT is where SF gets really difficult, because those cost $4 million or more these days.
Actually for many people they feel trapped because they’d like a LITTLE more space, ie. the Victorian townhome, but they specifically DO NOT want a car or a yard and do not want to leave the historic, walkable city. At that point you’re really screwed because outside of SF there is basically nothing else walkable west of the Mississippi. So there’s nowhere to go.
If somehow magically you could build a 1900s Victorian townhome city about 30’ away by train I think you’d find a couple million urban Californians wanted to move there.
But of course the sprawl all around the job centers, and the regulatory environment, both make that impossible.
> Actually for many people they feel trapped because they’d like a LITTLE more space, ie. the Victorian townhome, but they specifically DO NOT want a car or a yard and do not want to leave the historic, walkable city. At that point you’re really screwed because outside of SF there is basically nothing else walkable west of the Mississippi. So there’s nowhere to go.
There are a few other big cities on the west coast, and many of them have towns or small cities nearby that have walkable centers.
LA is world-famous for being non-walkable (even as it simultaneously is getting much better for walkability, hilariously). Seattle just isn't walkable compared to SF.
Surrounding towns of LA, like Pasadena, have nice walkable inner cores, though maybe you'll have to bike commute to work. Also, I found the LA transit system remarkably pleasant and useful, despite its reputation.
Seattle is walkable enough for me. It's not a binary variable.
I would love to buy something... anything better than a fixer upper, within a few minutes of a BART station that would allow me no more than a 60 minute commute into SF, but that doesn’t look realistic on less than a $400K household income. Owning pretty much anything would beat apartment living, even if it is in a rent-controlled $2200, 800 sq ft 2BR (my current situation).
First of all, SF doesn't really qualify. How many live in a neighborhood where their workplace, medical, grocery, and social needs are all walkable to/from?
Car-ownership-hostile != "walkable". Using Lyft/Uber isn't "walking".
Anyway, as to your question: parts of Sacramento, Auburn, Carson City, Eugene, Bend, and Boise all qualify, and that'slimiting to cities I know firsthand. Other cities that I hear this from others include Salem, SLC, ...
Much depends on what you consider to be "normal adult lifestyle". If one's definition of that includes walking through waste and needles to the Folsom St. Fair, SF probably does have a monopoly there.
I think your bar for walkable is lower than mine. Yes there are other places where you can take a nice walk, but I’m talking about living without caring or noticing that you haven’t driven a car in weeks.
Things that I do within two blocks of the flat I live in:
- get groceries
- go to a bakery
- go to one of three coffee shops
- go to one of ten or so restaurants
- go to a comic book store
- go to a game store
- go to a pub
- get my hair cut
- go to the dentist
My two kids, both under school age, at this point can safely walk with me to any of these activities, and, except for the dentist, they often do and quite enjoy it.
I could walk a bit further, but usually bike, and occasionally take transit, to my office, and to literally everything else in my life except the following:
- Trips out of town
- An occasional visit to a Lowes on the other side of town
That’s the bar I’m aiming for.
Note 1, I’m not talking about Manhattan. There’s almost nothing over three stories for several dozen blocks around me, this is all just really ordinary small/medium scale stuff from circa 1900.
And note 2, you asked how many people “really live like that,” and as best I can tell in San Francisco the answer is somewhere between 50 and 90% of the ~million residents. (I don’t know how good every neighborhood is, and I think some are less convenient than mine, but mine is pretty ordinary for SF.)
SF has decent public transit. Disliking public transit is fine, but ignoring it is something else entirely. Muni and Bart do make it possible to get around most of SF without walking or using uber/lyft.
And, given where I live, my medical, grocery, and a significant chunk of my social needs are truly within walking distance. The only one that isn't is work, and that would be accessible via public transit if I was willing to compromise on commute times a bit (or if I changed employers/offices).
Carson City has 4 bus routes serving 150 square miles, San Francisco has close to 100 serving less than 50. Those aren't remotely comparable.
The parent comment asserted walkability. If your environment is walkable, that means you don't need transit/carshare for everyday needs.
Carson City has 4 bus routes
Again, walkable means walkable without transit. My father lived there with daily needs within walking distance. SF is not only not the "only" walkable city, SF isn't generally walkable overall for all everyday destinations (especially hilly streets).
Walkability in the city-planning sense is defined by "lack of need for a car". Improved public transit improves walkability (see the walkability scores on pretty much any site ever), as does mixed use development.
Carson City has a walk score of 34 (although apparently there's a 4 block area in downtown that gets up to the mid 70s). SF has an average of 86. My apartment scores 99. Like I said, there isn't a genuine comparison to be had.
Exactly this. New York City is the very definition of a "walkable" city because you walk to the subway, stand on the subway, walk to work.
Not because you live within walking distance of work, which almost nobody does. That definition of "walkable" would be so tiny to render it almost meaningless.
Muni and Bart do make it possible to get around most of SF
Possible? Yes. They serve most attractions in SF. Not feasible to most of SF, not by a long shot. Look at your transit options between southern Sunset and Dogpatch, for example.
BART cuts one diagonal swath through the SF core only, and it doesn't even connect directly to Caltrain in SF county. It's hardly the Underground.
> Look at your transit options between southern Sunset and Dogpatch, for example.
L transferring to the KT? It doesn't even require a bus.
>BART cuts one diagonal swath through the SF core only, and it doesn't even connect directly to Caltrain in SF county. It's hardly the Underground.
Indeed, I'm not suggesting transit in SF (or the bay area as a whole) is perfect. Far from it. But compared to anywhere else west of the Mississippi (and most of the places east of it), there's nowhere better.
Plenty if you can cycle and get paid over $300k. If you had to, you could buy several walkable-to-work lots and join them into one big one in some cities. Or Uber literally everywhere.
Huh? I was saying a FAANG salary in Columbus Ohio (for exame) probably buys you a car free lifestyle with the style of home you want, especially if you have a working spouse.
Sure, but I bet even FAANG don’t pay $300k in Columbus. Google has an office in Ann Arbor, IIRC (which is a reasonable proxy for Columbus in this scenario). What does a senior engineer make there?
Google does indeed have an office in Ann Arbor. As far as I remember, it's quite light on engineers, though; I'd be surprised if any seniors are there.
Most Silicon Valley companies, including FAANGs, pay less to people outside the Bay Area. Some of them have an official HR policy of doing so and have standardized the salary adjustments for the other cities where they have offices.
Even if you are already at a FAANG in Silicon Valley, you will likely get your salary adjusted down by the standard amount if you move somewhere else. You'd actually have more luck avoiding a pay cut at a startup because they almost always have less rigorous HR policies than FAANGs.
There’s a big difference between places where you can walk (technically that’s almost anywhere) and a city that was built for walking (ie was large before cars).
On the east coast these are much more common, for obvious reasons. Places like Charleston and Savannah have a very similar feel to San Francisco. Some of New Orleans. Most of the center cities in the northeast. A lot of Chicago.
But the common thread here is age of the city and how well it has held on to its historic core. Out west there’s just not much old enough to qualify. Sure, there are some nice small towns here and there with historic main street areas. Portland, Seattle, and Boise have a neighborhood or two. But just ask the question “would you live here without a car?,” and then look at what percentage of the population does. Outside of those aforementioned bits of the country, not owning a car would make life extraordinarily difficult.
> See if you can prove me wrong: find a 10-acre lot within a half hour commute, with a large single-family home, that is affordable to a senior engineer. Alternately, make it 1 acre, but within a 5-minute commute.
That seems excessive. You certainly don't need to live on a multi-acre lot to have a good life.
I work at a FAANG and most of my colleagues own single-family homes and have no problem paying their mortgages. Of course they aren't on 10-acre lots -- we aren't in the middle of nowhere, we are in a city.
A 10-acre lot is a ludicrous metric, and even 1 acre (for a residence) metric is unrealistic. I doubt most technical types even want the burden of upkeep for an acre+. At that point, the main thing you're buying is isolation.
Reading this thread, I find both sides at extremes - one side from the Valley who's quite out of touch with the rest of the country. And the other side that is putting a fairly high metric.
1 acre is not at all unrealistic. I work at a well known tech company and live in an area where you can afford acre (and multiacre) lots. While owning a 1 acre lot with house is certainly not the norm in the company, it is also not at all noteworthy. No one gets surprised when they hear someone owns one. It's seen very much as a preference thing. Some people like apartments, some townhouses, some houses, and some larger lots.
And yes, they're mostly "tech types".
More realistically, in most parts of the country, a tech job will get you a 2000+ sq ft house built no earlier than the 90's. And in many of those places, a short commute to boot. Now I know SV folks often talk about how they don't need a house that big or one that "new"[0], the reality is once you're outside and can afford one, you won't find a good reason to buy a 1960's house, or a 1200 sq ft one. The preferences of SV folks are really post hoc rationalizations.
[0] Many of my coworkers reject houses built in the 90's - it's too old for them.
It makes perfect sense as is. It is almost my experience.
A coworker of mine bought 11 acres. (That's one more than 10. These lots go to 11...) He has sheep and goats and chickens, and he can shoot his AR-15 in his yard. Interestingly, he grew up in San Francisco. His current lifestyle is ridiculously incompatible with his city of origin.
I settled for a little 0.4 acre lot with a modest 3500 square foot house. (1619 square meter lot, 325 square meter house) By settling for that, I got to live within a mile of the beach and within a mile of work, and I paid off the mortgage in 8 years without trying terribly hard.
So that's close. I suspect you'd normally have to commute about 15 or 20 minutes to get a whole acre, and they go for $220,000 to $500,000. There is a house on 1.3 acre (like a football field) selling for $320,000 that is 11 minutes away, but the house was built in 1957. If you don't mind paying a couple million, you can be 5 minutes away with a modern waterfront home on an acre or two.
So this is non-urban life. It's mostly not even rural; we have an airport with about 7 passenger jet flights arriving per day. Senior developers live in nice houses, paid-off unless they really go nuts. They can have waterfront access, or room for a horse, or room to shoot and raise sheep. They can get McMansions. They can live within walking distance of work.
I just can't see giving that up. Sure, cable cars are cool and all, but that doesn't compensate for what I'd have to give up.
Aren't things cheap if you go well to the west, almost to the coast and away from a rail station? Japan has rural areas. If people would live there, they could afford to have large families.
Why? I'm on 10 acres. I like it for the isolation. Just got it, so maintaining and improvements are going to be interesting and a big learning opportunity.
Why would you _want_ 10 acres? What are you going to do with all of that space?
I like walking to the grocery store (or bar, or restaurants, etc). I like biking to work. I live within walking distance of literally 6 parks, so it's not like I'm starved for green or open space.
I don't understand your point of view to the exact same degree that you don't understand mine.
Here’s my answer: I have no interest in living so far away from a major urban center that I could have 10 acres. I have no interest in doing anything with land, farming, maintenance, whatever.
My life is very simple and I’ve never lived in an apartment bigger than 600 square feet. The idea of 10 acres is insane to me.
I’m in my late 30s and I’ve never even driven a car. What would I do with a carport lol. It’s just a different mindset, I guess. I don’t have a ton of shit and just enjoy the minimal upkeep, simplicity, and other benefits of living in a small urban apartment.
Forget about a tech hub, what city with jobs at all. Ten acres is more like a small farm than a house, by definition at that lot size almost nothing can be within five minutes, lol.
My area almost qualifies. Right now there is a home on 1.16 acres going for $275,000 that is 7 minutes away from my employer and 10 minutes away from a rather large employer.
You can get more than ten acres if you are willing to commute a bit. I think 2.5 acres is the minimum lot size if you go south about 14 minutes from one of my workplace's buildings, or 22 minutes from the one I'm in.
It is a small farm. Senior engineers can afford to live on hobby farms. Some of them expect to do so. It's not just here either; I've known people who did that in relatively rural parts of Massachusetts and Virginia too.
I recall somebody on HackerNews proposing an interesting explanation for how people react. The available jobs get dismissed because you'd have to seriously consider moving (which is annoying) if you admitted that employment in these locations is viable.
Uh, no. A quick perusal of real estate listings in towns all over flyover country will disabuse you of that notion. It's a little harder to find a house on an acre of land with 1Gib fiber, but if you're willing to settle for mere cable speeds, it's very, very easy.
Your somehow assuming cost of living is only high around FAANG. Bay area is full of non-FAANG companies that pay significantly less. Faang doesn't struggle to hire senior because they're the only places that affords to pay enough.
10 acre is a lot of land. 435600 square feet intact. When I lived in NZ our home was on 9000 square feet. And that was a lot of space... I would think a fairer example is living in a landed home instead of an apartment. I’ve spent the last 11 years in apartments and just had a baby, now wanting to move back to NZ to have a landed home and back yard...
Stop looking at only the bay area. Google has offices in Pittsburgh, Kirkland, Austin, and Boulder. If you are willing to work at one of those locations, you can get pretty close to getting everything you want.
If you want a 10 acre lot then your lifestyle is simply incompatible with city living. It has nothing to do with money or cost of living; even the billionaires in San Francisco don't have 10 acre lots.
Context: 15 years of experience, multiple public talks (including AWS reInvent), own chunk of infra serving hundreds millions users.
Got a message from yet another self driving startup. Agreed to do a phone screen out of curiosity. Interviewer is 6 months out of school and his first question - why do you want to work for us.
My answer - I actually don’t, I didn’t even know you exist and it is your job to convince me. Thoughts - i’m wasting my time.
This a big problem I've seen as a senior dev. Interviewers almost always default to "why do you want to work here, prove yourself, etc", and question accordingly. This is hugely off-putting to passive candidates who are busy with their current roles, etc.
I was very close to do it and I did a poor job hiding my annoyance.
It didn’t stop there. The coding challenge was standard bs done in a glitchy online tool. Yet another clone of coderpad which implemented poorly. I asked the interviewer why did he pick this specific question and how does it correlate to what they do - answer was not encouraging.
As you guessed, I didn’t pass he interview :).
I think the article is reasonable but it suffers from the imprecision of "senior" as many things do. What is a 'senior' engineer in a startup, or at a mid-size private company, or a large public company? All of those companies have "senior engineers" but they aren't easily swapped out for each other.
These are my definitions, at a startup a senior engineer is one that is broad enough in their experience to have gone deeply into one area and at least lightly into several others. They understand what technical debt it and when to use it, and when to avoid it. What is more, they can communicate what needs to be done to a more junior engineer.
In a mid-size private company a senior engineer knows all of the forces that are being applied to their area of expertise, they know what the cost constraints are on the solution, they know what the time constraints are. They understand the area fully and can review other's work and participate productively in design reviews and product planning sessions. They can be an axis of excellence by leading by example and surfacing things that need to be fixed early.
At a large publicly traded company a senior engineer is one who can relate the business to the engineering effort to insure that they are doing the engineering that is needed and more importantly they are not doing engineering that is not needed. To paraphrase, "With great resources comes great re-invention." Large well funded engineering teams tend to re-invent the wheel because they can, not because they need to. And as a result can become mired in self inflicted complexity. It is the role of the senior engineers to see to it that it doesn't happen.
Why should not all those be together and be the bar for senior starting at the start-up layer? I think the only big difference is working/navigating in a small or large org, no?
If you look at the both extreme, large public company and a start up, in the former case, you already have customers, a viable business model, and a legacy product. As an engineer, your first requirements would be to "not break things" going forward. Sure you want to gain some market share from competitor but you already have some momentum, otherwise your job would not exist. So you need to upgrade or add features to your product in a controlled way. Your 20 years customer might not want to have to learn a new interface just because you can deliver it.
At a start up, even if you have your first customers, you want to deliver a product or more features as quickly as possible starting from scratch. Possibly, you also want to be able to pivot totally if the first idea does not stick.
It is great if you can do well in both environment, but usually, this is different skill set, and I would even say personal inclination. You'll likely have a hard time to convince a start up engineer to work on COBOL code base on a mainframe.
The author is from Austria. As a dev who moved there 4 years ago I noticed it's probably some archaic Germanic cultural trait for any job postings to be very employer focused, acting like they're doing you a favor by hiring you and not caring about what the employee might want, even for senior roles.
Maybe a local is reading this and can explain why this is still a thing.
Author here, true, software jobs are a bit extra weird around here. I think it's because companies are run more "traditionally" here. Meaning that the people running the company are usually managers. They have a business background and don't understand what most engineers want and how they think. Even many self-proclaimed "startups" (which are often just consulting shops) are run this way.
The cause for this might be that it's culturally pretty unusual to start a company around here. I'd say people are extremely risk-averse and starting a company is almost frowned upon.
I am from Austria, and just want to say that this assessment matches my own experience. Sadly I don't have an explanation for it either. I moved away 4 years ago but that's one of the many cultural reasons why I don't miss Austria much.
I agree with a lot of the points here. One specific pet peeve of mine is job listings that refuse to nail down which specific technologies they're using.
Some will say things like "Front-end web development using a framework such as ExtJS, React, Vue, or Angular". Neat - which one? I'm happy to work with 1 or maybe 2 of those. Do you mean that you actually use React but you're fine with people who come from some other framework? Or do you mean that you are converting a legacy app from Ext.JS to React so you're hoping the candidate has used both?
Sometimes they'll bury the actual framework in the "Nice to Have" section but it's pretty common for it to be left vague. I think what they're trying to do is widen the pool of applicants, but that strategy seems suboptimal for attracting folks who want to work with a specific tech stack.
While this is only my experience, when I write job requirements that cover multiple languages or frameworks it really is 'any of these are fine'. The theory is that you don't want to filter out people that would otherwise be a good fit - because for the most part smart people can learn new frameworks and languages pretty quickly. You are correct that it is suboptimal for people wanting to work on a specific tech stack, but if you're not getting any applicants because your requirements are unnecessarily strict it's not in anyone's interest.
It makes sense that you wouldn't want to prematurely filter people out, but in that case I'd think it might be better to explicitly state that. Something like "We use React, but experience with another framework (Angular, Vue, etc.) is fine. We assume you can learn React."
I think the unintended consequence of leaving it broad is that you might lose people who are specialized in the very framework you use.
In my own case, I'm pretty biased toward React, having written a book & been blogging about it for a few years. I definitely pass over job listings that leave the framework open to interpretation. The cost of finding out what they actually use is too high, unless the job looks exceptional in some way.
I've thought about that a lot, and I wonder if the number of frameworks mentioned in a job posting indicates poor job quality or just an inept hiring manager.
Either way, I purposely avoid those type of job postings. I don't want to waste so much as 10 minutes on a phone screening to find out that even though you mention Ember or React that what you really want is someone with Angular 1 experience.
That's hilarious and so true. Last year I interviewed for a senior back-end engineering position at a well known mid-sized company in the Bay Area. They put me through 2 technical phone screens and a 6 hour onsite interview, grilling me in detail about system design, domain knowledge, personal experience, and of course doing endless exercises of writing solutions to algorithmic problems on a whiteboard. I really felt like I nailed the interview but they still rejected me because they thought my technical skills weren't good enough. 7 months later I start applying to jobs again via LinkedIn, and guess what, that exact same position on the same team is still open. It seems insane that during the past 7 months they couldn't find a single person to fill that role. Makes me wonder if they know what they're really looking for.
There's a whole metagame of HR metrics within companies. KPI's are set to interview X amount of candidates or extend Y offers. Arbitrary hiring is a common goal after a fundraise. Not every HR team sets thoughtful efficient goals or gets reviewed for waste. This can lead to perpetual interview pipelines and fake work to keep the Hiring team employed. This is equivalent to software engineers building knowledge moats for job security.
It's also possible someone quit. LinkedIn has/had a feature where you can find people who used to work at company X. I interviewed somewhere and got an offer. The work was up my alley, and the interview was a difficult but fair interview. But it felt it was a bit too "fast-paced" for my taste. I used that LinkedIn feature and saw most engineers stayed about a year there.
"senior engineer for team y" is a position that's constantly been hired for at most growth companies. The fact that they're still hiring for it probably doesn't mean much.
Respectfully, I don’t think you can really determine much from the outside, there are just too many things that happen. Maybe they hired someone and then they ran into visa issues, maybe someone switched to a different team, the possibilities are endless.
One thing I will say is that for many employers, a "no" now doesn't mean a "no" forever.
Since the Author is from Austria, it's not common here to specify the actual salary but the minimum allowed by law according to a collective agreement which is always a laughable sum for a dev.
This is really annoying as companies try to keep the employees in the dark regarding the market rates hoping to catch someone gullible who doesn't know them, and often they do.
This is probably true, but it's not the only way to attract seniors. As in the other discussion above, I'm convinced there are many very good people that will gladly trade a significant portion of their salary for other perks.
Money is not all but FAANG have benefits and work that are comparable or better than most startups/companies. The overall package is just better... It's obvious to the market at large. Only niche audiences are willingly going to choose startups/companies with certain perks. (fully remote work, for instance - it is a niche thing regardless of what people here say)
I have just skimmed through the article. But my personal experience is lot is ads (at least in Australia) don't mention the salary range in offer. what I would be interested in is the maximum they can afford for a right candidate. since then I stopped applying if there is no pay in the ad and I don't know the company. Lot of companies assume that their pay is higher than Market.
One of my experiences was a recruiter for a startup (which plans to be leader in XYZ) reaching out for me on LinkedIn. Since I found the general domain quite interesting, I responded. From there on we have a short chat, but instead of getting really insightful information on the job I get forwarded to a bunch of URLs which aim to describe things. I get that it's convenient for the company to save time - but I don't want to spend hours reading through things.
Then there is an interview loop. 2 of the interviewers are apparently fairly junior, and spend the whole interview asking the typical graduate coding problems. One of them even wasn't sophisticated with having things 80% solved - it needed to be 100% - even though that meant the whole timeslot was used up. In the end of those in-person interviews I learned nothing about the job because there was no time left.
The main thing I learned was that there are people there who can be pedantic on interviews and can solve basic coding problems. If anything that gives me a negative impression. I want to work together with people who know interesting technologies, know how to write well-structured code, etc. The interview might give the impression that they don't know these things, because all they ask for is how to solve a leetcode problem.
After that the company asks for another interview loop at a different place - which obviously would mean spending even more time on this. At that point I'm giving up.
Having spent a lot of time being interviewed recently, I've noticed a certain type of question asked in technical interviews that I think can sabotage perfectly qualified engineers. I'm not sure if my view is sound, but maybe someone can pipe in and tell me why I am wrong.
Imagine being asked this question in a technical interview:
> When do you decide to use frontend validation?
(yes, this is based on questions I've been asked IRL)
The problem that I have with this type of question, even though it can be answered and seems reasonable, is that it's not even how engineers tend to think most of the time. It's about a solution looking for a problem.
When I'm faced with this kind of question, I often stumble to come up with a hypothetical need for the solution being proposed. It's not that I can't come up with an answer, but I end up lacking eloquence and saying "ummm, well, okay, so..." a lot while using my imagination. It would be like handing a hammer to someone and asking them to find things to nail with it.
In essence, it's backwards from how I would tackle most problems.
Here's the sort of question I would rather be asked:
> Let's say that we've built out a web form that the user can submit, and there are a lot of inputs they have to fill in. They can only know if they made a mistake once they've hit "submit", due to backend validation. How could you improve this experience for the user?
I find this preferable because it's a lot more realistic and natural. Instead of looking for problems to fit the solution, I have a problem and need to come up with a solution. To extend my hammer analogy, this is like providing the pieces of a wood table and allowing the candidate to show how they would assemble that table. Anyone can memorize terms like "frontend validation" and what sort of problems they are associated with, but it demonstrates more skill to be given a problem and know where to go from there.
My guess is that there's a lot of good engineers out there, even senior ones, who get weeded out in the hiring process because they're being asked not to approach problems like engineers but like they're contestants on Jeopardy.
I think that could have been a poor choice of words on my part. From memory, when asked nearly the same question, I don't think it was suggesting frontend validation exclusive to backend validation. Then again, maybe that was how it was phrased. Ugh. Maybe I'll edit my original post.
The sheer amount of recruiters and interview requests that act they’re doing you a huge favor for the opportunity is rediculous. My conclusion is they truly do only want to hire people who don’t know what their skills are worth.
I think it's really just a question of mindset. A lot of recruiters don't have a background in tech. They don't understand that the market is completely upside-down. It's a huge change in mentality and it takes a bit of humility to see.
Also experience. Many recruiters I see are either part time contractors or recent grads that are just trying to get some (any) work experience before they move onto something else (like sales).
While the definition of senior engineer is a bit squishy, I agree completely that as some one gets more senior they increasingly choose the company rather than the other way around.
I'm soft-launching a product to help companies recruit and I've done about a dozen customer development interviews with hiring managers (5 person companies through FAANGs) in the last two weeks. One thing I heard constantly was that referrals are their best source of candidates and that for roles with 7+ years of relevant experience they actually don't believe good candidates will come in by just randomly applying on their website/LinkedIn/etc. I had an engineering manager tell me "a senior engineer actively searching for a job is a negative signal of candidate quality" as opposed to going through somebody in their network or connecting with somebody at the company first. I had another manager tell me for the senior roles he's hiring for he only considers candidates explicitly sourced by his internal recruiter who searches LinkedIn for the specific skillset or people who are referrals for the specific job. This isn't some CTO level position; this is just a role for a ~10 years of experience individual contributor. Applications from the website go straight to the trash can. The implication was people with those skills aren't looking for jobs.
Whew, that is a hard thing for me to read, it actually really bummed me out (apologies if this comment winked in and out of existence...wanted this to not be one giant blob).
(Bear with me, gonna anxiety dump a bit here ;) feel free to skip to the end where I have a pertinent question...=) )
Background: I've worked as a high-level contributor both at the management and engineering levels for the last decade of my career. Decently respected software dev/leader/whatever, lots of big and small projects and teams, all successfully delivered. People like working with me! I've had friends ask me to come work with them. And I like my friends. They are great people.
However, I'm also known to be really picky about the things that I want to be doing with my time and my career. I'm really protective of it overall, so when I have the opportunities to "do things" with friends or whatnot I have generally said no, for a variety of reasons. They are great people and wonderful developers (legitimately wonderful, brilliant human beings) but in many cases they work for environments or domain spaces that wouldn't really be "my thing". Sometimes I've just pursued my own stuff for a while, since I have had that ability...but what you're saying above sounds like that has been hurting me.
At the end of December, I was in a work situation where the entirety of the technical team was laid off, including myself. I figured at the time, given my general "level" and experience I'd be all right when applying for work.
While I haven't applied to lots of jobs since then, I did apply to a few that seemed like a great personal fit on paper, and that I knew based on my experience I should probably at least get an initial interview.
No dice. 0% response rate. I was shocked! I mean, it's been a while since I've been doing the interview dance, but 0? So I had my CV looked over by people ranging from C-level guys to other software devs of varying levels and no one could figure out why I wasn't at least getting into initial discussions. "It's not you, it has to be them", I got told. But really after a few of these things happening, you stop thinking that "it's them". That can't be, right?
I made a comment to one person that I felt far more desirable to companies when I had much less experience/professional success than after I had more significant accomplishments under my belt.
I hate mysteries - and I hate the not knowing. It's one thing for me to go through an interview process and not be a fit, if such a thing were to happen; it's another for me to never even get to the interview process and not understand why!
So while reading the above is kind of dejecting, thank you for writing it...I guess it provides some illumination for me on what might be going on.
Now rather than hop into my shower and softly weep - do any of your engineering manager friends spoken about above have any advice for people who may have that experience but just happen to be picky about what they want to do, don't want to move themselves to the USA, and don't always jump at referral from friend X?
As one becomes more senior the hiring manager expects a candidate to do their due diligence and ask some questions beforehand rather than hopping right into the application process. For a referral, that has already happened somewhat as the candidate already presumably talked to the the referrer about working at that company. You don't have to be a referral but a hiring manager expects you to do this due diligence. You don't necessarily need to go through a friend and you don't have to work with your friends. However, having an acquaintance at the company even in a different team is a massive help.
Think of it from the company's perspective for a second. They don't want somebody who just "wants a job". Money is always a factor but if somebody takes a job solely for money then they'll jump ship the moment somebody offers more. Companies want somebody interested in their space who wants to work there. You mentioned that you purposefully pursued some roles that looked like they'd be good fit for you. That's smart! Make sure the hiring manager knows it. By just clicking apply on the website, that may not be communicated clearly.
Here's two concrete pieces of advice:
- Before going in the front door and applying for roles, reach out to the hiring manager and ask them some more details about the role and what working for the company/team is like. If you know somebody at the company who can put you in touch (even if it isn't the same team), that's better, but even if you don't that's ok. They will talk to you and if they don't you probably don't want to work there anyway. LinkedIn is useful for this.
- If you're applying for a local job, try to meet the hiring manager or somebody else at the company in person if possible. Don't stalk them but to give you an example, I know that a CEO for a local security startup runs the local OWASP meetup and I told a person interested in working in security to go to the meetup and talk to him if she was serious about working there. I've been a hiring manager before and I looked way more closely at any candidate I talked to in person. Don't sound desperate but show them that you're interested and serious enough to go out of your way to talk to them. I had a couple candidates who I didn't know reach out to me via Meetup.com for roles posted to the company website and I had no problem meeting them for coffee.
On a related tangent, I help organize a local meetup and I get asked the question "how do I land my first dev job" a lot. I've helped half a dozen people land their 1st or 2nd tech job. Assuming they already have tech talent, I essentially offer them the above advice. It works at that level but it works for senior roles too! As a hiring manager, you want to talk to qualified people about your open role.
- The #2 source of candidates was almost universally meetups in my customer development interviews. Meeting people in person works! You don't have to be a referral although it does help.
- It's pretty easy at a small company to figure out who the hiring manager is and contact them via LinkedIn/Meetup/etc. At a big company, this can be hard or impossible but you are much more likely to know somebody at a big company who can put you in touch. Absolute worst case, reach out to the big company recruiter on LinkedIn asking for some details on a specific role and they might put you in touch directly or they'll get your questions answered some way. The recruiter 100% will talk to you. They are paid good money to find qualified people like you.
- While I said earlier it isn't a numbers game and you should only apply to roles/companies where you're a fit, to some degree it is a numbers game. Sometimes jobs are posted to a company website and the role isn't really available. Maybe they already have a candidate in mind. If you apply to ~3-4 jobs and don't hear anything, that isn't super uncommon. However, most people feel obligated to actually give you a direct answer if you contacted them directly.
"When hiring senior engineers, the company doesn’t choose the candidate, the candidate chooses the company."
That's totally true. I also choose the environment where I will be working and what conditions will be.
I prefer working remotely and have flexible working hours. It keeps me motivated.
"Your job posting probably sucks". I agree with it too. Because some recruiters write me that their company offers free alcohol in their office for stressful situations. I told them it looks unprofessional and seems that they just drink at their office. She said to me that people hate boring descriptions and I'm wrong.
But you know that more people like a healthy lifestyle. Especially in Silicon Valley. Do you agree?
So, if you sell your company, find more information about a candidate (their interests, lifestyle, values). Instagram, Facebook, Twitter show you who a person is. It helps you to write the best offer to a candidate.
Currently, I'm using three ways to find a job: FB groups, Networking, and Freelance Marketplaces. I used to get projects on Upwork but right now I check job boards.
BTW, if you want to save your time searching for projects, I found a service on Reddit comments that aggregates freelance projects from job boards (https://periodix.net/). I like it because it doesn't sell me bad projects.
Yup, and with remote, you can buy from the entire country, maybe even the whole world! talk about a huge candidate pool. compare that to hiring locally: where you're limited to 30 miles (4 times more than 15 squared), or maybe even 15 miles, if the infrastructure is really terrible.
In the article the author says there is more demand than supply for senior developers. I’m saying if you hire remote and pay a decent wage there is more supply than demand.
All of these recurring interview threads makes me wonder: why are there only two interview preparation-focused boot camps right now? (Outco and Interview Kickstart)
It’s one thing to take an algorithms course, and it’s another to excel at competitive programming, and still another to use those skills on the spot in an interview. Surely like any other admissions process, one can train for technical interviews in a group setting.
The point that is being made by the original post is that the process is bad in many companies especially when hiring senior engineers who have a proven track record and have other options. These engineers don't want to spend time and money learning how to crack the coding interview in some bootcamp. They can find work and are put off by the process some companies use and such companies are missing out on good people. Suggesting bootcamps for leetcode and whiteboarding as a solution misses the point as these are not great interview practices.
They definitely aren’t, but I wonder if their existence will force big tech employers to realize their interviewing process is being gamed and is even more of a sham.
I think it's ok for big tech companies to do this (from their point of view) because they have an endless list of candidates and don't care too much if many good people are put off or fail. They will get good people through the sheer volume of people applying and these processes do get good people if you can afford false negatives.
I think the middle and smaller companies need to change their processes more. If they just copy what the large companies do without understanding they don't have the same ability to attract candidates and pay high salaries they will just lose good people that they actually really need.
There is a lot of complaining about hiring processes these days.
I think both algorithmic questions and work samples have an important role.
Algorithmic questions are presented with a time lot, you get an idea of how the candidate thinks through problems. These questions I think reflect general intelligence.
Work samples reflect effort and experience, which is also important.
People like to complain about one or the other like they're entitled to a job. It's hard to hire senior engineers because there aren't very many good ones, a mediocre engineer who hangs on for 5-10 years is now a "senior engineer."
You're not special. If you want a high paying job of your choosing you have to earn it. Only a vocal whiny minority is heavily complaining online. If you think you're highly competent and the problem is with all of these companies, in whose best interest it is to hire good people, you should think again.
>I think both algorithmic questions and work samples have an important role.
I'm a senior engineer at one of the top tech companies. I hold the opinion the interview processed is heavily biased towards college grads or kids who are fresh on algorithms.
I think this helps us hire mostly kids who are book smart but aren't necessarily great (or even "good") employees... I'll do some hand waving here and define good as some combination of being driven and motivated, being able to work independently, and actually caring about what they're doing.
The question is then - well, how do we do this better? Honestly, I'm not sure. Hiring quality workers is a difficult problem. Technical interviews allow you to somewhat measure one of the key aspects of your hire. That being said, some people aren't made to stand in front of a white board and get judged. They don't handle that situation really well, and just because of that, they'll make mistakes that they might not otherwise make.
> I think this helps us hire mostly kids who are book smart but aren't necessarily great (or even "good") employees... I'll do some hand waving here and define good as some combination of being driven and motivated, being able to work independently, and actually caring about what they're doing.
Absolutely. I'm at FAANG, been here almost a decade. I've met many fellow engineers who lack basic problem solving skills, curiosity to learn more about the environment they're developing in (e.g. Linux), and diligence to do a good job. School doesn't teach you those things, nor does conventional whiteboard interviewing select for these qualities.
I spent way too much time/energy on my last team covering for other people who were more senior than me.
I think it varies by company, but at mine the questions are very much domain based for senior people. The focus is greater on algorithms for new grads.
The questions being asked aren't extremely challenging, they can be trained for. Willingness to learn new things (even if useless) and put in the effort to pass interviews is a positive signal imo.
On that note, it seems like HN has assumed the only work that exists is writing CRUD apps and not worrying about performance. There are many roles where you do need to have a strong understanding of performance and algorithms, and those are the difference between a smooth rollout and crashes in the middle of the night.
What aspect of performance though? Even that means a variety of different things to different people and it isn't just algorithms. I've met people who are bad at typical CS algorithms yet know their language of choice very well and write low-allocation high performance tight code that "does less". Or people that are used to handling concurrent code which sadly is still somewhat of a rare skill. Only the algorithms are touched on in typical CS courses; most aspects of performance tuning come from experience and persistence spending hours trying to get more performance and less resource usage out of an application.
If you are truly a senior software engineer with all of the common skills living in most metropolitan areas in the US in 2019 and can’t find a job, it most definitely is the fault of the job seeker.
The author is right that the firms are not selling enough and often they are too picky, have unreasonable slow and effortful hiring process with many stages of which some don't test the engineer but rather humiliate the person.
I am running a tech recruitment agency and this month I had TWO engineers who refused to take offers because the companies (two different ones) basically grilled them too much in the processes and showed off an insufficient company culture. I know both firms rather well and their environment is actually good. Seems like firms who aren't yet having a brand name like Google but who are sufficiently good tend to be arrogant enough to scare of engineers but not good enough to make them accept an offer.
The arrogance of some firms might seem irrational but it can be explained by "catastrophe avoidance". Most of the time, hiring a bad engineer is really much much worse than rejecting a good candidate - that is especially true in Europe, where it is hard to fire people. Also, every job description is a set of tasks that is supposed to be done by a person who'd better be REPLACEABLE. If you wait sufficiently long, YOU WILL FIND SOMEONE ELSE WHO FITS ALSO OR BETTER. Due to the nature of the employer-employee relationship, firms will still be "the ones on top".
I definitely agree with some of the points (job descriptions and onboarding are generally dumpster fires, to highlight just two), but it has a little too much of a stereotypical young developer feel for my tastes. There are fine engineers out there who aren't overly concerned about the dress code, for example, but care deeply about diversity, social good, and/or benefits other than basic salary.
There is so much more to technical hiring that a leader has to know about and consider. If anyone would like to read a different perspective, I recently finished a long blog series (~60 pages if it were a book) that tries to consider a more complete human approach:
Hi, author here, thanks for your feedback. I did mention that not everyone will care about these things, but I like your point about diversity and social good. Probably things I should've included.
I had an interview for an engineering manager role with a London fintech startup, from 11AM to 4PM, with each hour designed as a 'gauntlet' of some kind.
There were no breaks. When I went to urinate, it took time out from their quiz. Someone stood outside the door.
Interviewing is mentally exhausting: a couple of hours in I started to flag from lack of food. By the end of the 5 hours, after eating and regaining my abilities, I thought that an environment that didn't give me a moment to eat during the courting phase, wouldn't care that much for my welfare once I'd actually signed on.
After a couple of days they got back, assuming I was still interested, and told me feedback which consisted of "really knows his stuff, but is often a poor communicator". Yes, so is everyone when they're presenting for 5 hours.
I might send this to the 1000’s of clueless recruiters that spam me every day, and can’t understand why I don’t want a .net job in Wigan for £12,000 p.a
On the subject of recruiter emails, I just recently got one for a job at my former employer, a clear indication that the recruiter didn't read my resume at all. Also I've gotten an offer for a 3 week contract senior developer position on the other side of the continent. I swear that recruiter must have been mental or desperate.
From my perspective as a job seeker, it's pretty obvious that recruiters really just don't care. And the weird mental games being played on applicants with increasingly stupid and inflated requirements only means you're going to have fewer applicants.
I have talked to several interviewers with a buying mindset, and the thing that annoys me the most is that they always ask why I want to work for $COMPANY_NAME, especially if it's a well known SV company.
They don't understand: I don't need your job, your recruiter called me, so what is it $COMPANY_NAME has to offer me that's better than my current senior level gig?
One thing that employers should keep in mind is that the candidates they would like connect with probably already have jobs. Many people I know simply do not reply to recruiters if they are adequately paid and reasonably happy in their current job. The reason why is simple. If they did, they would simply be volunteering to spend a bunch of time doing take-home projects and coding exercises for free without even knowing if the company administering said technical assessments would pay them more or offer them a more fulfilling job!
So it's not just that you're selling (as opposed to buying) because the candidate is "senior", you're selling because you're trying to lure someone away from a job they already have. Asking someone how much they're making now so that you can offer them the same salary or a measly ten percent more and then giving a homework assignment is not exactly enticing someone to switch jobs!
As a happily employed person who occasionally interviews for another job, I find coding samples a turn-off.
Why? Two reasons:
- If the coding sample is in a language (or large framework) that I will learn on the job, I don't think I will have any success. Something that's a 20-minute job for one person could be a 5 hour "learning exercise" for me.
- If the sample is something like a bugfix or enhancement where I have to go through the whole process, the time commitment is just too large. I have to come up to speed with new frameworks, conventions, idioms, layouts, designs, which is time consuming. Then, the review process itself may be time consuming, or slow to get feedback if company engineers don't see my work as a priority.
I prefer coding samples that require very little ramp-up and process. Let me use a language that I'm familiar with. (If I will learn a language on the job, let's agree on a language that I and the evaluating engineer are comfortable with.) The coding exercise should be fun, without assuming knowledge of extensive frameworks or processes.
In general, good coding assignments look like a high school coding competition, or a topcoder question.
[Edit]: What I've really liked, in the past, was a company that did a very basic coding sample instead of a phone screening. It took me about 90 minutes to write, and was rather fun.
What I had to do was write a small program that navigated a maze. Navigation occurred by loading parts of the maze through a web service. Other than that, there were no requirements for language, framework, process, ect.
In short, I didn't have to onboard myself, or have extensive domain knowledge, in order to do the exercise.
I find coding exercises that require extensive domain knowledge are only useful when screening subject experts. IE, we're hiring someone who has extensive knowledge in language X with framework Y and design patterns Z. Sometimes teams need domain experts, other times they need experienced engineers who learn the tools on the job.
[Edit 2] The statement "When hiring senior engineers, the company doesn’t choose the candidate, the candidate chooses the company." is very important when choosing a skillset. Part of what sells a job to me is the opportunity to work on a new language, framework, ect. This is why coding assignments that require extensive domain knowledge are, ultimately, a turn-off.
My pet peeve is getting a coding assignment with a bug or syntax error or misplaced curly bracket that turns the coding exercise into a trick question. The compiler or editor would catch all that stuff so don't expect me to do it by hand unless you're paying me to work in notepad.
While I agree with a lot of the suggestions in this post, I disagree with the "you’re not buying, you’re selling" part. It's just not that one-sided.
I think the proces of hiring or applying works best when it's a conversation between equals. Both the employer and applicant have something of value to offer, and both expect something of equal value in return.
Many employers and applicants act like the other side should be happy to talk to them (they're granting them a favor). If you really think the other side has more to gain from talking to you than yourself, why did you decide to have the conversation in the first place?
Sure, markets are not perfect, so the most popular employer in a city and the best engineer with nowhere-to-find expertise might both be able to act as a buyer. For the rest of us: We're buying, and we're selling.
This is good and all, and many of the comments here explore the nuances. Something all of it seems to miss is that one persons "senior" engineer is not another persons "senior" engineer. This designation is meaningless and it usually is a failure of a company to properly assess their needs when creating a role. Some approach it by using a wishlist of framework and language knowledge (that quickly turns into a checklist). Others approach is by years of experience doing x,y,z. Fact is, a "senior" engineer is code for "person I want to hire who is right for the job and who needs little guidance". Many people fit that bill.
Hot take: many (most?) startups don't need talented senior engineers and won't even pay for them. They have products that do not have performance constraints or that would benefit from mastery of any one specific technology.
What startups need is engineers who can show up to work, understand requirements, point out potential shortcomings, write code on an approximate timeline and be willing to fix their mistakes. As it turns out, you don't always need a senior engineer to do that. You certainly don't need a 10x rockstar ninja engineer.
Where I've seen startups struggle in the past is fostering a culture of mentorship. When you're hiring fresh grads or people early in their careers, your employees will want someone to give them honest and personalized advice. It's pretty hard for people that are early in their careers to fill that role, so you may need some senior engineers to do that.
Overall, I think this article is a bit fanciful in the expectations it makes around recruiting. Sharing stuff like internal processes, team size etc are definitely helpful to the candidate but are costly to the employer. This information may be confidential or provide a competitive advance. They may also vary by team will definitely vary over time. Overall I think it's unrealistic to expect the company to be very forthcoming about this type of thing until they make the offer.
The interviewing stuff also seems to come from a good place but represents wishful thinking. Companies (even startups) hold significant leverage in 99% of interview situations and therefore are strongly inclined to err on the side of false negative. Their thought process goes like this: "sure, you may be amazing but because you're not willing to jump through my arbitrary hoops (6 hour coding exercise, 3 rounds of onsites, etc) I'm willing to hold out for someone who is equally as good."
I'm not saying any of this is good for the candidate, but it's unrealistic to expect companies to start to treat senior candidates any better given the way the labor market is set up. In fact, I'd expect this kind of treatment to get worse as the bumper crop of younger engineers trained in the late 2000s starts to level up in experience.
In my experience, one of the key ways to get an engineer interested in a role is to tell how much it pays up front, and make sure that number is appropriate for the experience level you're targeting.
Author here, you were right though, that was the actual intention. I even cheated a bit on the highlights. I just wanted to get my point across and not get lost in details.
Not revealing the employer (without going through the work of unblurring as the HN detectives have already done) was just an added bonus.
If you only start thinking about selling a candidate when you _meet_ one then you missed the bigger opportunity to build a corporate reputation of supporting developers in their careers (e.g. with open source, developer blogs, etc.).
With that in mind, focusing on the bits highlighted here doesn't make sense. Attract smart/experienced/motivated candidates and your boring hiring process or your flawed onboarding process doesn't matter as much.
This is clickbait to sell a book. "Having trouble doing the thing? The problem doing the thing is you. Buy my book for $45, then you can do the thing."
Hi, author here, thanks for your feedback. What did you think about it was clickbait? I genuinely believe that most companies can improve a lot of things when it comes to hiring. And they don't know how to do it, because they have the wrong mindset.
If something about this was too flashy, please let me know.
Sorry, I see this at the end and I can't really take the article seriously any more -
> I have more than 7 years of experience in Software Development, Product Management and Project Management
7 years is enough now, to be making industry recommendations about difficulty hiring senior staff?
I'm not saying you can't get a good idea of part of the industry in that time, but it seems a bit of a stretch to claim sweeping knowledge of the whole space.
[Sarcasm on]I think the most shocking of this is that an engineer with 7 years is already a senior these days. I even heard more shocking things like people with 3-5 years are already seniors. So what about people with 15 years? How shall we call them? Super senior? How about 30? Call them mega-giga-super-senior? And 40? Principal-giga-senior?[Sarcasm off]
Having said that, I actually agree with everything the article says.
I think 7 years is definitely enough to be a senior but it depends on your experience. If you have worked for Google last 7 years it’s very likely you have been promoted 3-4 times and have a lot of experience with large scale production systems. So I’d definitely say such person is senior.
I’ve Been fortunate enough since my first job to always find a place that wants me and sells themselves. Once in a while i ran into companies that tried the testing and other bs and i either just left or told them it would’nt happen and they accepted. Stuff like being allowed to use Linux in a Windows only organisation or not be locked in for x months was something i could talk through.
You need to ask open ended questions and expect detailed answers that only an experienced engineer would be able to answer. And keep asking more and more details. Only another experienced engineer would know if that are bsing. Most companies put way too much on whiteboarding. Failing whiteboard guarantees you lose the job seem to be very inefficient.
I think that this should be true for any position. I find you can generally get better people working with you if you put effort into making them feel like they will be valued, compensated fairly and that the role will be meaningful. And then delivering on it.
Similar to adding the words "bad economy" in discussions regarding performance evaluations and raises. Cop outs based on mostly arbitrary "market mechanics".
Thought I would add my own thoughts to the difficulty here and just talk about the process I brought into the last place I worked at. I came in as a Director at a time when the company had a...let's say not too great set of developers overall from a skill level perspective (the company thought they were amazing! That's maybe a story for another time). I'm proud of the process I introduced there because it led to my meeting and befriending some top tier gentlemen. While it may not work for everyone (and I understand it perhaps weeds out some candidates) it seemed to do all right. Here it is, for anyone to steal if you so feel it useful:
* Candidate applies
* Within 24 hours, the candidate has a personal reply from our in-house recruiter. That E-mail explains the entire interview process in detail and sets up a brief 15-20 min phone screen with the candidate if they're okay with said process.
1) The phone screen is mostly nothing more than a brief "psychopath test" (is this person nice to talk to or are they rude or condescending?). The recruiter talks to them about their own goals, aspirations - if we approached them, we skip the "why do you want to work here" part otherwise we ask them what appealed to them.
2) The candidate is given a take-home assignment (which has no time limit but has been vetted against internal staff to take no more than really about an hour), with a note attached that says "Hey, you're likely a great and talented candidate and I'm well aware that you may be thinking that this is a waste of your valuable time. What I can assure you is that a) I'm only doing this because our industry is a sewer ;), with no real standardized screenings b) As discussed in the first E-mail, there are no further technical "screenings" coming forward - if we go to the next phase, we'll just discuss your approach to the assignment for a bit and then it's "getting to know you" time.
Internally, this technical assignment is not one where two reviewers can have different "reads"; there are things we're explicitly looking for to determine overall "level", mainly with readability, approach, things like that (as just one example, a senior level candidate will know about unit testing and show something along those lines in their solution).
3) Someone reviews the submission. We reply within 48 hours.
3a) If the candidate is a no fly we don't go forward, and tell them so. If the candidate asks why, I tell them explicitly what the assignment was looking for, provide links to get acquainted with those and encourage them to reapply at a later date (we hired someone who did exactly this - those are the kind of passionate ones you want to be working with).
3b) Candidate is a go-forward;
the next step of the interview is simply a discussion about "The Alliance" (Reid Hoffman, great book, at least the first half) of what they'd like to do with their career and whether the company can harmonize that into something both parties are happy with, and a getting to know the candidate a bit deeply. I told candidates explicitly at this point not to stress, as this wasn't an interrogation but simply us understanding that - to tie this back to the original posting - they were interviewing us as much as we were interviewing them.
Our hire quality sharply increased after that. I left the company a year and a half ago but that process is still in place, relatively unaltered. In fact, one person left the company, went to another company and basically "stole" that interviewing approach because he thought it was so good for finding people but also treating them with respect (which is critically important and all too rare).
TL-DR; our industry, particularly around hiring, really sucks and it's an absolute crapshoot on both sides of the equation. It's sad and really depressing; I try not to spend too much time thinking about it and more time trying to mitigate it in my small corners of the world. =) I wish it were different.
There are a lot of comments in the thread claiming that there’s some kind of “smooth talker” senior engineer candidate, but frankly that is preposterous and I simply don’t believe all the impossibly perfect anecdata thrown around.
It is incredibly hard to talk clearly about complex projects you’ve solved before in the specific tradeoff circumstances of some past project or job. Someone who can do that with great clarity is extremely hard to find and valuable.
I saw one comment that said something along the lines of hearing a candidate talk fluently about tech details (at a very specific implementation level) of past work, but then put them in front of a keyboard and they go “hmm urr uhh” or whatever.
This is frustratingly dumb. First of all, senior engineers should spend way, way more of their time thinking and directing architecture choices or prioritization than writing code, _especially_ in early stage projects or companies.
Second, I’m absolutely certain that the coding question you’re asking to generate this himming and hawing is some Cracking the Coding Interview bullshit trivia question that this senior engineer hasn’t thought about in 20 years since their algorithms class in college because they were too busy actually solving real business problems for people (including a few rare problems that involved detailed deep dives into novel or complex algorithmic solutions).
We harass and haze people with bullshit trivia, then turn around and don’t even ask them what problems they have really, actually solved before, or invent reasons to rationalize that if they solved a lot of problems before, they’re just some sort of smooth talker and somehow the only proof is if they memorized Floyd’s cycle detection algorithm or some other ludicrous bullshit.
i can't believe all of the replies to this thread. us nerds get a bit too myopic about
the details and fail to see the forest for the trees. let's zoom back and look at the Earth from the Moon.
what other industry has hiring rituals premised on assuming The Candidate is a shape shifting imposter and then testing The Candidate using various Gom Jabbars to verify The Candidate is a human who can code? do attorneys and doctors and professors device elaborate exams and gotcha questions to spring upon The Candidate because The Candidate might be a total fraud? to be honest, this implicit assumption that The Candidate is an outright liar and con-man and that we have to "catch" them in their lies before we let them into The Club has been an enormous turn off for me in this entire industry. and i'm saying that both as The Senior Engineer put in charge of hiring developers as well as being The Candidate dozens of times and having to go through multiple Voight-Kampf tests to prove "yes, i can code."
why don't attorneys and doctors have the same problems we do? they pay more, so the incentives are there for The Candidate to rote memorize a bunch of text books and talk impressively while being unable to perform The Work themselves. is the problem that anyone can call themselves a Software Engineer and there is no way to prove it? is the problem inherent to the nature of software itself, in that the problems we solve and the solutions we apply are constantly changing ephemeral logical abstractions which cannot be repeated because every context has a totally different set of constraints, so it always reduces to Apples vs Oranges? is the problem inherent to the nature of Nerds and Autists and the personality types who tend to become software engineers? is being a Coder the Nerd's version of The Jock's hazing rituals? are we Nerds just as big of bullies and assholes as The Jocks, bullying our own and shaming each other against meaningless goal posts? we often hear now about how our Industry lacks "diversity" and how we need to be more "inclusive" to women and blacks. well how do you think women and black people feel about our bizarre and harsh hiring rituals of hazing and shaming and accusing of being The Imposter and your Interview test is to prove to us that you are not? i think we as an industry need to tone down our bullying hiring rituals if we hope to ever stop being an elite club of oppressors.
just last week i got rejected as The Candidate because i didn't have 10 years of React experience. "sorry, you're not senior enough", even though i had twice the number of years of raw dev experience than the punk kid over ten years younger than me who was in charge of hiring. i have seen this happen to me a dozen times, where i am rejected because i know too much about a Domain, and can go into gory technical detail about internals and implementations, which intimidates The Juniors who know one Javascript framework and think they are Seniors and who don't want a Real Senior around who might take their promotion slot away from them.
i don't even think in terms of "Junior" and "Senior" anymore. those are not useful buckets to divide the vast world of software development. instead, i think in terms of "what rad shit have you coded and what gets you excited about code and what kinds of things do you wish you could code or what do you code for fun on nights and weekends?" sadly, the vast majority of "professional coders" draw a blank at that question and they have zero answer, because for them coding is just a job. they never even knew it was For The Love of the Game. general interest and curiosity seems to be a better litmus test, because "interest is aptitude", and if you're the kind of brain who takes things apart for fun to figure out how they work and then try to put the system back together, then that proves you're a left-brain thinking INT(J/P) type of mind who is naturally suited for software engineering. the rest can be taught or learned on The Job as needed to fill in the gaps.
Thought I would add my own thoughts to the difficulty here and just talk about the process I brought into the last place I worked at. I came in as a Director at a time when the company had a...let's say not too great set of developers overall from a skill level perspective (the company thought they were amazing! That's maybe a story for another time). I'm proud of the process I introduced there because it led to my meeting and befriending some top tier gentlemen. While it may not work for everyone (and I understand it perhaps weeds out some candidates) it seemed to do all right. Here it is, for anyone to steal if you so feel it useful:
* Candidate applies
* Within 24 hours, the candidate has a personal reply from our in-house recruiter. That E-mail explains the entire interview process and sets up a brief 15-20 min phone screen with the candidate.
1)
The phone screen is mostly nothing more than a brief "psychopath test" (is this person nice to talk to or are they rude or condescending?). The recruiter talks to them about their own goals, aspirations - if we approached them, we skip the "why do you want to work here" part otherwise we ask them what appealed to them.
2) The candidate is given a take-home assignment (which has no time limit but has been vetted against internal staff to take no more than really about an hour), with a note attached that says "Hey, you're likely a great and talented candidate and I'm well aware that you may be thinking that this is a waste of your valuable time. What I can assure you is that
a) I'm only doing this because our industry is a sewer ;), with no real standardized screenings
b) As discussed in the first E-mail, there are no further technical "screenings" coming forward - if we go to the next phase, we'll just discuss your approach to the assignment for a bit and then it's "getting to know you" time.
Internally, this technical assignment is not one where two reviewers can have different "reads"; there are things we're explicitly looking for to determine overall "level", mainly with readability, approach, things like that (as just one example, a senior level candidate will know about unit testing and show something along those lines in their solution).
3) Someone reviews the submission. We reply within 48 hours.
3a) If the candidate is a no fly we don't go forward, and tell them so. If the candidate asks why, I tell them explicitly what the assignment was looking for, provide links to get acquainted with those and encourage them to reapply at a later date (we hired someone who did exactly this - those are the kind of passionate ones you want to be working with).
3b) Candidate is a go-forward;
the next step of the interview is simply a discussion about "The Alliance" (Reid Hoffman, great book, at least the first half) of what they'd like to do with their career and whether the company can harmonize that into something both parties are happy with, and a getting to know the candidate a bit deeply. I told candidates explicitly at this point not to stress, as this wasn't an interrogation but simply us understanding that - to tie this back to the original posting - they were interviewing us as much as we were interviewing them.
Our hire quality sharply increased after that. I left the company a year and a half ago but that process is still in place, relatively unaltered. In fact, one person left the company, went to another company and basically "stole" that interviewing approach because he thought it was so good for finding people but also treating them with respect (which is critically important).
TL-DR; our industry, particularly around hiring, really sucks and it's an absolute crapshoot on both sides of the equation. It's sad and really depressing; I try not to spend too much time thinking about it and more time trying to mitigate it in my small corners of the world. =) I wish it were different.
In my experience, it’s fairly easy to judge technical skill. A friendly conversation about technical interests and recent projects can often be enough.
Bullshit. Sounding credible in technical interviews is a skill, not the same skill as actually being a good programmer, and might even (statistically, in the large) be close to orthogonal to it.
We found this out the hard way. At Matasano, we started our work-sample hiring process[1] as a way of filtering out the smooth-talkers. Before work-sample tests, we'd spent loads of time on carefully designed interview questions; interview design was something close to a hobby for some of us.
Of course, it was only after we started doing work-sample challenges that we discovered that not only were a lot of excellent-seeming candidates actually not capable of delivering, but an even greater fraction of the candidates our "friendly conversations" were selecting out were in fact perfectly capable. It was a bad deal all around.
Whatever you do, don't fast-path "senior" developers. Everyone should run the same process for the same job. Not only do you risk hiring people who won't work out, but you're also depriving yourself of the most important data you need to iterate on your hiring process.
[1]: https://sockpuppet.org/blog/2015/03/06/the-hiring-post/