The best interview process I've ever been a part of involved pair programming with the person for a couple hours, after doing the tech screening having a phone call with a member of the team. You never failed to know within a few minutes whether the person could do the job, and be a good coworker. This process worked so well, it created the best team, most productive team I've worked on in 20+ years in the industry, despite that company's other dysfunctions.
The problem with it is the same curse that has rotted so much of software culture—the need for a scalable process with high throughput. "We need to run through hundreds of candidates per position, not a half dozen, are you crazy? It doesn't matter if the net result is better, it's the metrics along the way that matter!"
I dislike pair programming interviews - as they currently exist - because they usually feel like a time-crunched exam. You don't realistically have the freedom to actually think as you would in actual pair programming. i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview, but it's pretty realistic of real life work and entirely a non-problem. It's probably even good to test for at interview: how does a person work when they aren't working with an oracle that already knows the answer (ie: the interviewer)?
Pair programming with the person for a couple hours, maybe even on an actual feature, would probably work, assuming the candidate is compensated for their time. I can imagine it'd especially work for teams working on open source projects (Sentry, Zed, etc). Might not be as workable for companies whose work is entirely closed source.
Indeed, the other problem is what you mention: it doesn't scale to high throughput.
> i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview
In all pair programming interviews I have run (which I will admit have been only a few) I would fail myself as an interviewer if I was not able to guide the interviewee away from a dead end within 15 minutes.
If the candidate wasn't able to understand the hints I was giving them, or just kept driving forward, then they would fail.
Exactly! Who calls "researching how to build X, but then letting my pair-programming partner fall down a rabbit hole so I can feel superior" "pair programming".
That's definitely up to the interviewer, in which a lot of discretion and trust has been placed. I think a lot of it also comes down to the culture of the company—whether they're cutthroat or supportive. As you get better people into the company, hopefully this improves over time. I know that when we did it, it was never about nailing it on the first try, it was literally about proving you knew how to program and were not an asshole. So, not the equivalent of reversing a binary tree on a whiteboard. The kinds of problems we worked on in the interviews weren't leetcode type problems, they were real tickets from our current project. Sometimes it was just doing stuff like making a new component or closing a bug, but those were the things we really did, so it felt like a better test.
Exactly. People go like “the ideal way to interview is <whatever they themselves are the best at>”. Pair programming interviews suck and don’t scale, just like every other alternate way of hiring.
IMO interviewing is the biggest bottleneck and if interviewing was decoupled from hiring then it wouldn't be a problem. But this requires a Guild-like organization to manage interviewing/vetting and for companies to use said Guild for hiring. The companies could then do a single team culture meeting (if they wanted) before hiring.
I wish I could remember who, but there is a company out there who conducts tech interviews to create a pool of candidates for their customers. Pretty sure there was a post here about it, but it's lost in my ocean of unread favorites.
Create a scalable, practical interview process where the result is reliably the best candidate, and you'll take over the world.
At this point in my career I no longer even believe that "best candidate" is a meaningful description except perhaps for highly specialized roles. We're all stumbling blindly in the dark.
Exactly. Its almost like optimising for finding your best possible match for marriage. You don't go over a billion prospects, you choose from the ones locally available to you, as they come.
it isn't about scale. It is about core principle of the tech hiring - all the companies hire only the best. Not only it is impossible to scale, it is plain impossible. Even if all the companies hired only "above the average" it would still be a pretty tall order :)
There is no one-size-fits-all for software developers. Different companies are looking to hire people at different experience/pay levels, with different skill sets, etc. Just as with dating, most companies are also going to have some understanding of their own "attractiveness", and not try to date out of their league.
FAANG companies offering industry leading compensation packages and prestige are in a position to be able to hold out for the best (even if their interviewing practices may fail to achieve that), but most companies are just looking for someone that checks the boxes and seems like a good fit.
Employers trying to hire for fit and culture has resulted in the most inhumane and counterproductive processes I've witnessed. If that's what you really want to do, let the person work for at least a month in the team.
Provisional hires (which often exist in theory but they're pretty much a formality in general) don't work for the most part. Lot of overhead for the company. And in many cases you're asking for the candidate to quit a job and possibly relocate on the possibility they'll get a new position assuming that they click in a short time interval.
Relocation used to be pretty common for professional jobs. Don't know about today when there's more remote work. And maybe companies aren't as willing to pay for in general.
There's even more overhead on the people being provisionally hired.
Yes, sometimes things just don't work out. But, if someone quits a job and maybe relocates, that's a big personal cost. It's just the way things work in some limited contexts (e.g. professional sports) but it's not and shouldn't be the norm.
I suppose you can give a huge sign-on bonus with no claw-back provision, but that's never going to happen in most cases.
I'm fully convinced the way to make better hires is to invest more, which will be more expensive. Which wouldn't be a problem unless we expected something else. It starts with quitting pretending the current process is working, or even close to optimal.
Has hiring ever really worked, anywhere? Especially as roles and need evolve? I guess you could argue that it sort of did, apropos of a play I saw last night on the astronaut program--and maybe the military in many cases more broadly.
But, in many cases, I'm not sure how I, as a candidate for a tech job, would feel about a company offering me $200K--no strings attached--with the proviso that I statistically only had a 25% chance of making it through the next 6 months. (And is that really long enough anyway?)
There are tournament-style professions. But I'm not convinced most professional jobs are or should be among them in general.
yes. Best place i worked at - we hired only by internal references and only people from our University. Up until the company grew around 200 people. We didn't do technical interviews, just a short talk. And we were among top employers, including salary-wise.
When you have high trust a lot of other processes become unnecessary. When that trust is broken, and surely a lot of grifters BSed their ways into jobs, that’s when all kinds of barriers were added.
When I was doing interviews at my old job we did an hour of pair programming and we very intentionally designed the exercise such that
1) if you're qualified to do what we're hiring you for this stuff should be your bread and butter. We were doing restful web apps so the prompt might be "A restful API that takes in a string and returns that same string but backward." If you chase your tail on that for 15 minutes, you're not right for this job.
2) We weren't looking for a specific implementation. You want to make it a GET call with a query param that returns just the string? Neat. A POST with request and response DTOs? Also neat. We're not gonna ding you for doing it one way rather than the other but we're definitely gonna talk to you about why you made the choice that you did and maybe try to tease out another choice you could have made so that we can ask you to compare them. Again, the only wrong answer here is one you can't defend.
3) I'm not here to test your ability to memorize a particular language's syntax, your IDE's autocomplete or your ability to google. If you can articulate that you're trying to create a POST endpoint I'll give you that the correct annotation with Spring is @PostMapping or little things like that.
I agree. But what I mean is: that's not how it's perceived in the current interview structure, which lasts maybe 45 minutes or so. Ultimately, going down a dead end means you'd now have 30 minutes to find the right solution and code it up. So the oracle (the interviewer) would probably help you realise sooner that it's a bad idea, so you don't waste your time. That's assuming they know the problem and solution well; if they don't, you'll just lose them and burn through your time.
In a 2 hour pair programming session on an 'unsolved' problem (like an open issue / minor bug / minor feature in a public repo), yes, it will likely not matter if you tried a bad idea, and would both be more realistic and a positive signal.
this is an interviewer problem, unless the candidate is totally silent. A candidate that can't ask questions isn't going to succeed, anyway.
If the candidate will communicate with me, I will offer them a LOT of guidance. It is still very, very easy to tell who knows what they are doing and who does not. You give them a basic but domain-specific task, you give them whatever extra context they need to do it, and you watch them hammer out the code.
It should be a task that is sufficiently familiar to the person applying to the role that they do not need to do -much- looking at docs, and as the interviewer you should be prepared to help them quickly find the docs you already know they will need -- you designed the task after all -- so that they don't waste time with that.
What's important is that they ask for docs when they need them, and that they can understand them, quickly, and use them. It will be obvious if they are using AI because of how long things will take. It will be obvious if they don't know when to reach for documentation, and it will be obvious if they cannot understand the documentation.
Then, they should write a test for their solution. This weeds out 95% of candidates. Talk to the other 5% and you'll find someone who can both actually write code and also discuss design.
A lot of good people will close-up and not voluntarily ask you any question if you put them under pressure. (What they automatically are on that exercise.)
What is not to say that you are making anything wrong. But watch for bias there.
> A lot of good people will close-up and not voluntarily ask you any question if you put them under pressure.
This sounds like the kind engineer who won't push back or ask for clarification on unclear requirements - and happily spend a month solving a problem the business doesn't have.
So maybe seeing that at interview time is a good thing?
Might not mean the candidate doesn't fit - but can clarify what kind of roles would work?
> So maybe seeing that at interview time is a good thing?
Perhaps, but an interview is a fundamentally different environment than day to day work.
There's no way to solve the interview/interviewee problem though, the whole thing is impossibly fucked and is going to have false positives/negatives no matter what.
I do 1hr pair programming interviews for my company and you have to strike a balance between letting candidates think through the problem even when you think it won't work (to see their thought process and maybe be surprised at their approach working/see how quickly they can self-correct) and keeping them on track so that the interview still provides a good signal for candidates who are less familiar with that specific task/stack.
I'm also not actually testing for pair programming ability directly, moreso ability to complete practical tasks / work in a specific area, collaborate, and communicate. If you choose a problem that is general/expandable enough that good candidates for the position are unlikely to go down bad rabbit holes (eg for a senior fullstack role, create a minimal frontend and api server that talk to each other) it works just fine. Actually with these kinds of problems it's kind of good if your candidates end up "studying" them like with leetcode, because it means they are just learning how to do the things that they'll do on the job.
> maybe even on an actual feature
I don't think this would work unless the feature were entirely self-contained. If your workaround is to give the candidate an OSS project they need to study beforehand, I think that would bias candidates' performance in ways that aren't aligned with who you want to hire (eg how desperate are they for the role and how much time outside of work are they willing to put into your interview).
> if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview
Eh, if it's a reasonable bad end and you communicate through it, I wouldn't see it as a fail. Particularly if it's something I would have tried, too. (If it's something I wouldn't have thought of and it's a good idea, you're hired.)
I did a couple of rounds of this with my manager as the interviewer. Personally I really liked the process, and the feedback I got from the candidates was positive (but then again it always would be).
What worked well for me was that I made it very clear to my manager, a man who I trust, that I would not be able to provide him with a boolean pass/fail result. I couldn't provide him any objective measure of their ability or performance. What I could do was hang out with the canditates for an hour while we discussed some concepts I thought were important in my position. From that conversation I would be able to provide him a ranking along with a personal evaluation on whether I would personally like to work with the candidate.
I prepared some example problems that I worked for myself a bit. Then I went into the interviews with those problems and let them direct those same explorations of the problem. Some of them took me on detours I hadn't taken on my own. Some of them needed a little nudge at times. I never took specific notes, but just allowed my brain to get a natural impression of the person. I was there to get to know them, not administer an exam.
I feel like the whole experience worked super well. It felt casual, but also very telling. It was almost like a focused date. Afterwards I discussed my impression of the candidates with my manager to ensure the things I was weighing was somewhat commutable to what he desired.
All in all it was a very human process. It must have taken an enormous amount of trust from my manager to allow me the discretion to make a subjective judgment. I was extremely surprised at how clearly I was able to delineate the people, but also how that delineation shifted depending on which axis we evaluated. A simple pass/fail technical interview would have missed that image of a full person.
This is exactly how I interviewed people and what my replies were like. But I was one of many interviewers, so I would only take 15 minutes, which always annoyed my manager, but I never gave a bad "fine by me", so I still did interviews...
I've (unfortunately) been interviewing the last two months and the main pattern that I've noticed is that a) big companies have terrible interview processes while b) small companies and startups are great at interviewing.
Big companies need to hire tons of people and interview even more so they need some sort of scalable process for it. An early stage startup can just ask you about your past projects and pair program with you for an hour.
I hear this all the time, but I have yet to experience it. It may be because the small companies that I interview with are all startups, but I have yet to be able to get a call back from any other kind of small company. And the startups I do interview with have a full FAANG interview loops.
There seems to be a weird selection bias that if you're FAANG or FAANG adjacent these small companies aren't interested.
At a former gig we had a newly hired ex-facebook employee give notice within a month because she didn't like that dev setup had bugs that devs themselves had to fix. At fb they obviously can spend millions of dollars for a whole team that ensures that working dev env is always a button click away, a startup (even a scaleup) usually can't afford to. This is just one example out of many I can tell...
And I’ve heard just as many horror stories about companies hiring from small companies that the engineers haven’t kept up with engineering, culture and practices and are coding like it’s 2004.
Also, those types of stories tend to pop up with any engineer who’s only worked at a single place.
My point isn’t that there’s not bad engineers at Facebook it’s that there’s bad engineers everywhere and filtering based on random signals like this is not useful.
The sad truth of hiring is that you can't afford to interview everyone, and have to keep in mind that prospective employees are showing you a different version of themselves than they'll actually bring to work.
Especially in a small company where your hiring manager may also be busy with development and sales, and not have an HR department to run the process for them, you're much better off interviewing candidates you are 50% sure of being a good fit vs 5%. Personally I prefer interviewing candidates coming from FAANG-ish companies and often make exceptions for candidates that demonstrate exceptional skill/interest, but when you can only interview 1-10% of your applicants you have to prioritize those who are likely to succeed at your company (keeping in mind implicit bias and such).
> filtering based on random signals like this is not useful.
In aggregate it most likely is useful for those companies.
> And I’ve heard just as many horror stories about companies hiring from small companies that the engineers haven’t kept up with engineering, culture and practices and are coding like it’s 2004.
Big cos can afford to onboard their engineers for months, sometimes years. Startups usually can not
>”There seems to be a weird selection bias that if you're FAANG or FAANG adjacent these small companies aren't interested.”
Many smaller companies have noticed that former and wannabe FAANGers are looking for FAANG-type jobs, and are not good fits in their niche. Small companies often have more uncertainty, fewer clear objectives, less structure, and often lower pay. They’re not a good substitute for megacorps.
The problem is that many smaller companies have hired people from FAANG, and had them quite after a short tenure, so they're unwilling to try their luck again, as it's just not worth it. You may be different, but they've heard that story before, and had it not work out.
As someone that never had a desire nor ever made an attempt to work at any of those companies, do you mind elaborating on the mentality of such places?
I'm just your boring below-average to average dev, so I know I'm not cut for those types of places, but it never truly bothered me anyway. Any reason that I can personally think as to why I would work for such a company would either be due to my own egotistical desires or for monetary reasons, but those were never strong enough to actually compel me.
I am just mainly curious about two things:
1. Is working at those places all it's cracked up to be?
2. Assuming one had to work hard to get into such companies, was the juice worth the squeeze?
I've often wondered if one's experiences for these companies is often something akin to the old advice of, "Don't meet your heroes." In other words, was the conflicting dyad of expectations vs. reality present?
It has turned into something similar to what people in trading companies on Wall Street deal with. Constant grind, unrealistic expectations, and projects done in order to get a promotion instead of because it provides value to the customers or the business.
That said the amount that you make is insane some of the smartest engineers I’ve ever worked with have been at these companies and a lot of them have really strong engineering cultures, and standards.
The current work environment seems designed to use up bright young engineers, and burn them out within a few years. This is a significant shift from 15 years ago, where it was a much more sustainable place to be.
I could see that happening, especially given that they're young and feel powerful, nearly invincible. I experienced a similar attitude form younger folks outside of FANG, some kind of derision of experience and advice from people who have been around and choosing instead to obsolete it. After the first burnout may of them become a lot more thoughtful and humble. I may have been like that too when I was young, I don't remember being exactly like that but maybe I was just not aware. I still did respect experience and I still do to this day.
Sure, but in this context, your FAANG experience is a negative signal for people who don't know you well yet. It's unfortunate for you, but a genuine factor you now need to account for.
Your path through will probably look like having the luck of breaking in at one of these kinds of companies, and then staying for several years to demonstrate earnest commitment/fit while building a new network of connections, and then leveraging those connections to get more opportunities if it becomes necessary to do so. If you have connection from your previous non-FAANG work, that's probably your best route.
It won't happen overnight and you'll always be at a disadvantage when you find yourself applying through resume portals. Good luck!
In my time at tier one companies I have worked with the best engineers I have come across in my entire career (even the worst engineers were more than competent) who were working on deep issues that could affect the revenue of the entire company because they’re laser focused on providing value to the business, instead of doing engineering for engineering’s sake. I have grown by far more in these kinds of roles than I have anywhere else because the kind of problems you encounter at such a high scale just don’t exist elsewhere. And most of them have been there for at least five years if not longer you don’t make those kind of contributions to accompany without a long tenure.
> In my time at tier one companies I have worked with the best engineers I have come across in my entire career
You’re throwing a giant red flag right here. First of all, FAANG isn’t “tier one” except to people who idolize these companies. More agile startups are trying to disrupt these dinosaurs and do not thing very highly of them. Many of us who have worked with FAANG and ex-FAANG engineers were not impressed.
I mean, I'm just sharing the practical ground truth of how a resume like yours effects recruiting in certain contexts.
Just like there are innumerable brilliant, effective engineers who would contribute tremendously to a FAANG but don't suit the modern interview funnel (leetcode, etc), smaller companies surely do miss out on strong, suitable FAANG engineers in anticipation of negative experiences they've had with others.
There are a lot of people who accumulate FAANG entries on their resume and many of them really don't suit smaller companies for a number of reasons.
Honestly, though while I'm only seeing a very narrow picture of you here, it sure sounds like you see these "tier one" companies as a desirable place to work, with prestigious colleagues and profound learning opportunities on high scale problems that just don't exist elsewhere, and surely for much more money. Are you sure you're really going to be happy somewhere else? Or might you get restless? That's precisely the kind of concern these smaller companies carry when seeing FAANG stuff on a resume, and it doesn't seem like it should be baffling that they would do.
I definitely used to. The work culture and attitudes, particularly of management, passed the breaking point for me a few years ago. I realized work was not my whole life nor did I aspire to that.
>"And most refused to look at anybody deviating from their ideal background in my experience."
This is often because the culture of job-hopping for better pay every 18 months has eroded the willingness to pay for training or adaptation. Why pay for someone to learn if they're just gonna leave soon; the pre-trained person is a better deal if you'll have to pay to retain anyway.
Which was caused by cost cutting measures, MBA disease, in companies to begin with.
We’re just seeing the end of the cat and mouse struggle that’s been going on since the 60s. And massively accelerated in the 80s.
It’s unfortunate for companies though because they’re the ones that will lose out in the end when all the experienced people start retiring and they have no one to hire.
It’s an untenable position to not train people, period. There is no schooling you could go through that would educated junior dev to the level of a senior dev. And it’s the same for any other role. Experience is not optional.
I think the primary stimulus which creating the “job hopping culture” was actually the hot labor market for software developers. Other fields experienced real ‘cost-cutting’, without resulting in a lot of ‘job-hopping’.
I agree that this situation is undesirable, but it seems to be stable, somewhat like the result of repeated play of the prisoner’s dilemma.
That definitely massively accelerated it but you’re looking way too short term that’s only been in the last 10 to 15 years.
I agree that other industries are not YET at the point where software is , but you’re not looking hard enough if you don’t see the short tenures compared to the 25-30 years they used to have.
And yeah, it might be in an equilibrium now, but how long can it stay in an equilibrium? I’d guess at max 10 to 15 years.
I'm guessing the majority of people now in their 50s and 60s in computer-related careers had very eclectic jobs before settling down in computer-related stuff. After all, many never used computers at all until college or beyond.
My understanding is even in the early 2000s it was pretty much just firmware versus desktop software with a small niche for Mac developers.
Edit: my point was not that specialized software applications didn’t exist. It was that people were expected to be able to jump from stack to stack when they change roles in a way that has disappeared from modern job applications.
Well, and mainframes. And trading and financial systems. And numerical/scientific computing. And network services. And web sites and e-commerce. And flash, java applets, and browser plugins. And control systems. And operating systems and tooling. And cell phone applications. And games. And video/image/audio/music processing. etc etc
It was probably about as hard to move between those domains now as it is today. Which is to say that it's pretty hard and needs some concerted, non-trivial effort in shaping your experience and how you present it before trying to make a transition, and often either some kind of inside reference to vouch for you or an employer that was especially hard up for candidates. Or else an employer that straddle multiple domains and actively supported internal transitions.
Depending on what you could bring attention to in your prior experience and the size/needs of the new orgs you seeking to move to, certain transitions were more feasible than others, but you could easily spend decades working in mind-numbing enterprise applications while wishing for opportunities in game development or trading or whatever and never get your resume so much as looked at. (And vice versa, even, for those who dreamed to "retire" into the supposed quiet of enterprise apps or government IT or whatever)
I basically agree with your edit. There was a lot more fluidity among roles and even just moving into computer roles from other engineering (and even non-engineering) fields. But that's not really what you wrote initially.
I've had a few good experiences with interviews at small companies and startups, so they do exist.
But I have also had really terrible experiences, similar to what you've mentioned. Sounds like you've just gotten unlucky and gotten the terrible ones.
I'm not talking about being watched closely, I'm talking about regular work and then making a decision. You have to leave people space to do their thing if you want to see what they're capable of.
Sure, that would be even better. But how would that even look?
In the best case applicants needs to apply multiple companies. Companies need to interview multiple applicants and have a way to compare those applicants.
Those are the most basic constraints I can think of. How do you make that cost tens of hours for each round?
For me as a job applicant even in the best case I would need to do 3 to 5 interview interviews. The same is true for companies in the best case it will take at least 3 to 5 interviews to find somebody. Are they supposed to have 3 to 5 temporary staff for weeks at a time?
How much time should that take per interview? How would somebody that currently has a job manage that kind of time commitment?
You seem stuck on the idea that you need to verify an employee to hell and back before investing anything, that's not what I mean by committing. If they look like they might be a reasonable fit, give them a month to prove it, then you'll know for sure. One interview per person should be enough to make that decision.
In a small company, you can tell your buddy “just have a chat with the candidate and if you like them and you think they can do the job, hire them”.
If the person interviewing your candidates messes up, you’ll know soon enough. In a large company, the bad people will take over and your company is dying a slow death.
That approach doesn’t work on a large scale. Some interviewers are too nitpicky, elitist, others approve anyone who uses the same language as them for side projects. Some are racists, sexist, or have other kinds of biases. Some might have a crush on the candidate. Sometimes the interviewer thinks about their own task while they squeeze in an interview. In some countries, “undoing” a bad hire is hard, so they need to make sure that the candidate can work on any team (or at least on multiple teams reasonably well).
IMO for large companies it makes sense to standardize the interview process.
Also, in my opinion grinding leetcode is also a good personality check for FAANG hires: it shows the candidate can suck it up, study hundreds of hours, and do whatever they need to do to pass an arbitrary test, even if they themselves think it’s a broken process. The larger the company, the more this quality matters in candidates as they will need to deal with a lot of things they will probably not like.
I'm quite conflicted on this. While I do not think one needs to remember/memorize a bunch of brainteasers or past computer scientists'/mathematician's PhD discoveries in order to build CRUD applications.
However, I do feel like there is perhaps some amount of truth to the thought behind the interview questions, no? As in, I would imagine someone that could invert a binary tree in 15 minutes on a whiteboard could probably learn React. However, I am not sure everyone that can learn React can invert a binary tree in 15 minutes on a whiteboard.
However, maybe I am projecting my own insecurities because I wish I could invert a binary tree in 15 minutes on a whiteboard as well as being able to solve all those other problems.
No, it's not actually about mastery of binary trees or other abstract computer science concepts.
leetcode is really a culture fit test and success points to the combination of "smarts", diligence, and conformity to some acceptable degree. It shows you have some baseline of familiarity with computing, can focus on an arbitrary task to pursue a goal, and will conform to an arbitrary process when it's asked of you.
Those are genuinely essential skills in an organization with 1000's of white collar professional workers.
More precise insight is gained when the test covers a binary tree inversion than that were it just some contrived logic puzzle like the LSAT or a critical reading exercise. The computer science bit does provide some signal and isn't completely arbitrary, but it's only a small part of what's being evaluated.
Among otherwise strong engineers, it tends to filter out the especially willful, prideful, independent, meandering, creative, and pragmatic ones. These personality types can be extremely valuable in some work environments and can still sometimes in through leetcode challenges, but spoil big bureuacratic systems like FAANG's when they're overrepresented.
The culture thing is huge. If you can’t find a few good leet code problems to enjoy, I don’t know if we are interested in the same field.
Having done a lot of interviews at least 70% of the time it’s filtering candidates who just aren’t very technical and haven’t studied computer science. The kind of people who hate reversing a linked list are the kind of people who haven’t touched a polynomial since high school.
It’s not unreasonable for a junior tech interview to expect you studied something like CS or EE. It’s a blessing that our field is open to all to give it a shot, but if that doesn’t describe you - you need to recognize that there is effort and study to fill that background.
> In a large company, the bad people will take over and your company is dying a slow death.
Why is this the assumption. I would rather say any big org. is converging towards the average talentwise by necessity. It is like Hawaii can't have 100 olympic level swimmers no matter the recruiting proces.
But I'm talking about the idea of pair programming for an hour or two. Why can't the standardized test still involve that? What doesn't "scale" about it?
I think it’s easier to compare candidates who might have been interviewed by different people possibly weeks apart if you have a coding challenge… could they solve the problem we gave them, if yes how quickly and how well. Interviewers can write their notes, the session can be recorded and the committee can compare candidates relatively fairly.
Comparing candidates based on how they “vibe” with the interviewer during a pair programming session is a recipe for lawsuits and bad hires.
I’m just speculating here, I don’t have any significant hiring and interviewing experience
If you're worried about interviews weeks apart and cross-team testing, then the most that can hurt efficiency is making you do the test with the team they'll be imminently hired into, which means you're on par with the small company, nothing lost nothing gained.
Avoiding pair programming for the reason you listed sounds like lawyers getting in the way, not scaling. But yeah we'd need to hear someone say if that's actually happening.
> If a startup can spend 20 man-hours filling a single position, why can't a big company spend 1000 man-hours filling 50 positions?
Because big companies are run by bean counters and they also don't require the same kind of talent that is useful to startups. There's less competition for hyper-specialized seniors and middle of the pack generalists.
> small companies and startups are great at interviewing
Small companies have the benefit of the pressure to fill a role to get work done, the lack of bureaucratic baggage to "protect" the company from bad hires, and generally don't have enough staff to suffer empire-building.
Somewhere along the line the question changes from "can this candidate do the job that other people in this office are already doing?" to "can this candidate do the job of this imaginary archetype I've invented with seemingly impossible qualities".
Do we have different definitions of "marginally qualified"? Idk, I feel I'm a decent engineer - I can certainly do whatever leetcode medium they throw at me, as much as this counts of anything - and can actually code, but I still get maybe 1 callback per 50 applications.
Does "marginally qualified" mean "Ivy League Competitive Programmer PhD" or something?
> Do we have different definitions of "marginally qualified"?
Not every candidate is an interview. I recently hired. I got 90 applications and from these, 80% were an instant "No". They didn't match the job description or had no permit to work in my country. I invited the rest. Simple interview, pair-program a dead simple App with a prebuilt skeleton with me with any framework of choice. Make one GET request, render it and realize one needed optimization which needs to be implemented. 90% (I'm not joking) of candidates failed the first task. In half an hour, they couldn't send a GET request and translate that into JSON. All were allowed to google, open any documentation they liked. Of all 18 who failed, 16 asked me if they could use an LLM for this task, which I denied.
The GP is still passing through hundreds of people, dozens of them capable, until he reaches somebody that convinces him of their competence. You were passed down because you weren't convincing enough.
Or maybe he is getting resumes from a channel that has been victim of machine-gun filling, and there indeed thousands of incompetent people posting resumes into every channel and just half a dozen real applicants.
TBF, I have no idea how to fix either one of those problems. Hiring is just completely broken.
I have only a resume to convince them. I have job experience at major companies, with examples of what I've built when there, personal projects I've made, a github link, a great GPA from a good school.
And I know similar Junior-mid people in the same boat. We can all do Fizzbuzz, we've all built things, and somehow we're not getting interviews, but people that can't do Fizzbuzz are.
Do thousands of incompetents also machine-gun apply to, say, mechanical engineering, accounting, marketing, HR, or finance gigs? Is it just tech?
> Do thousands of incompetents also machine-gun apply
Enough that it's HR or some automated software that first screens your application.
> Something isn't adding up.
Yes. The pipeline "posting > application > screening" is now completely broken. In your case it's quite possible it's HR and screening software that your resume is not convincing. I have been hoping for anecdotes and studies in this direction (from people who have access to the HR and/or software screening and are inclined to report) - but it's at least not common. What we do hear from, is tons of people who can program and who are not getting even first interviews.
Big companies and small need to agree on some standards and create qualifying exams that they will actually accept as proof of competence. Degrees somehow don't prove anything, experience doesn't, blah blah blah. It's exhausting to have to prove to interviewers that I'm not mentally disabled at every turn and it's a waste of time for everyone.
Create certifications that actually count for something and aren't just a blip on a resume that may tick a box, but will actually move you past technical trivia questions. I know some people have a deep repulsion to this and I think it would be fine to have a technical interview gauntlet for those that choose not to engage with any type of certification and a simplified interview format for those that have passed the prerequisite tests.
I don't care how long, rigorous, or ridiculous the tests are. Just agree on some effing standard.
Watch out for what you ask for. Plenty of big vendors have certification programs. ... And for some combinations of field and vendor, they are red flags - rather than pre-validation. That is, far too many applicants have the certification but do not have the grounding knowledge without which the certification is sort of useless - potentially more dangerous than validating.
>The best interview process I've ever been a part of involved pair programming with the person for a couple hours... You never failed to know within a few minutes whether the person could do the job
There is something funny about the "best interview process" taking "a couple hours" despite giving you the answer "within a few minutes". Seems like even the best process is a little broken.
Lightly ironic indeed! Though I’m not sure “broken” is exactly the word I’d choose.
I can only speak for myself, but I imagine myself as a candidate approaching a “couple of hours” project or relationship differently than I would a “few minutes” speed round. For that matter I can think of people I know professionally who I only know through their behavior “on stage” in structured and stylized meetings of a half hour or an hour—and I don’t feel like I have any sense at all of how they would be as day-to-day coworkers.
If we sat down to work together, you’d probably have a sense in the first few minutes of whether or not we were going to work out—but that would be contingent on us going into it with the attitude that we were working together as colleagues toward a goal.
That's mainly because there were multiple pairing sessions, and even if you knew the person was going to pass, there are still a couple more people who need to meet them, and a schedule to make sure they're available to do that. Plus due diligence, etc.
Nor am I saying it was a perfect system, just the best I've seen in terms of results.
The biggest victims of these non-scalable process is people without a good network. As an intl PhD student, I am that person.
So now I have this weird dynamic: I get interview calls only from FAANG companies, the ones with the manpower to do your so called "cursed" scalable interviews. But the smaller companies or startups, ones who are a far better fit for my specialized kills, never call me. You need to either "know someone" or be from a big school, or there is zero chance.
I've been interviewing recently and got through to the last round (of five...) with an interesting company. I knew the interview involved pairing, but I didn't expect: two people sitting behind me while I sat on a wobbly stool at a standing desk, trying to use a sticky wired mouse and a non-UK keyboard, and a very bright monitor (I normally use dark mode). They had a screen recorder running, and a radio was on in the next room.
I totally bombed it. I suspect just the two people behind me would have been enough though.
Pairing on something close to whatever real work they'd be doing, but familiar to the applicant is my favorite way to evaluate someone (e.g. choose a side project, pre agree adding a feature).
I don't care if someone uses modern tools (like AI assists), google, etc - e.g. "open book" - as that's the how they want to work. Evaluating their style of thinking / solving problems, comms, and output is the win.
Very few people doing this sort of interview (they tend to be our best, most empathetic developers) are likely to cut a multi-hour planned process short after a few minutes. It will eat at least an hour of their (very expensive & valuable) time.
Also how am I supposed to filter the 100's of AI-completed assessments? Who gets this opportunity?
We didn't do assessments (if by that you mean take home assignments). This was partly a solution to that, since nobody thought they were a good idea. If you mean the phone screen, I think that would be a problem, yep, but it wasn't an issue back in 2016. Having them pair would weed out cheaters, but we would have to figure out a way to weed them out during the screening, I agree.
We also did not require the employers doing the interview to be our most senior team members. They probably did it more often than most people, but often because they volunteered to do it. Anyone on the team would be part of the loop, which helped with scheduling. And, remember, we were working on actual tickets, so in a lot of cases it actually helped having the candidate there as a pairing partner.
For a little extra detail, the way we actually did it was to have 2-3 pairing sessions of up to 2 hours apiece. At the end of the day, all the team members who paired with the candidate had to give them the thumbs up.
This. Interviewing for a sr dev position with a web app, backend stack is the bog standard java, spring, SQL abstracted away via JPA. We did a first screen, then the tech interview was two of their senior devs shoulder surfing me as I built a simple API. We chatted, I built, they asked questions, I defended my decisions (sometimes successfully, sometimes gracefully conceding defeat), they left knowing that I was who my resume said I was and the reminder that popped up in the middle of the interview to feed my sourdough starter showed them that I'm a culture fit.
I think you're onto something with that last paragraph but I want to try being a bit more generous with why things are the way they are. The question seems to be "When there are hundreds of applicants how do we give everyone a fair shake without hiring an entire team of devs who do nothing but interview?" From that perspective the intentionality is different and even sensible but the end product is likely to be the same. Even when someone is chasing a metric it's because someone else wants what's best and has decided that metric is a sensible way to make that happen. At the end of the day they really do want to hire the best candidate out of a pool whose size is extremely variable and that's challenging.
I've been a proponent of pair programming since the early days of Agile, when it was still seen as part of extreme programming. Unfortunately, it’s not often employed in workplace settings.
With that said, would your perception of the interview remain positive if the outcome had been negative?
A common challenge across all interviews is a mismatch in personal dynamics, which can significantly impact the experience for both participants.
Consider a scenario where a senior developer, who prefers simplicity, is paired with a mid-level developer who is still captivated by complexity.
Or a "just start typing" person is with a "mull it over first" person. By the time I am typing code, I want to have 90% of it already completely worked out (at least till I type a "c.Lock()" and suddenly realize I hadn't considered thus and so synchronization issue.
I work at a company which has 11 engineers and competes with companies with 100s. The hiring process was a screening call with the CTO to not waste the prospective team's time, then a call with 2 of my prospective colleagues to gauge competence and cultural fit. Since then I have been involved in hiring most of the team I work with now. The CTO is one of the most competent engineers I have ever met and he designed this process. He also has very high EQ. One of the points I sell to prospective hires is him as a person to work with, as well as our team. He has also flatly denied people I suggested before and that's fine.
I have been here 5 years now and I'm working with the most competent team I have ever worked with. My take away from this is that hiring doesn't need to be commoditised and scale, it just needs to find good people and give them an opportunity to show you that you do or don't want to work with them.
Similarly, in general the best interviews I've ever been part of (either giving or being) turn into discussions where people's experience, opinions, and stories get aired (going both ways). You eventually get a good sense of each other and things get more relaxed when you both realize that you know what you're talking about (this is harder for Jr roles, though).
Being peppered with questions very rarely gives any insight.
For junior roles, you want to interview for intelligence and shall we say an interest in learning rather than specific skills.
Even for senior roles, that's what I want to interview for, although it is true at times a business case can be made for someone that is good at some specific complex skill and doesn't need to listen to other people to do ok work.
I used to love getting to know the interviewer and doing things like that but IMO the market has shifted fundamentally on both ends for this to be effective anymore for most SaaS roles. This is anecdotal for US/Canada tech market over the past 10 years so YMMV.
Developers Side:
Since developers don't have job security anymore (at least for those who work on common languages like Go, Python, Java and Typescript) they are better off learning and keeping in touch with leetcode and system design questions, looking for new opportunities and interviewing in "batch mode" when looking for a job. The idea is to clear as many interviews as possible using the same concepts, get in and make money asap before you get laid off. No incentive for collaboration or for fulfilling but esoteric stuff like Haskell and Scala. Career security > Job security.
Companies Side:
On the other end software companies have less trust in developers staying long term so they want to make the interview process as quick and risk free as possible. In essence they are betting that by perusing 100s of resumes and hiring someone who seemingly knows CS concepts they can get some value out of them before they leave. Standardized tests/vetting > team fit.
TLDR; The art is gone from this job, its become akin to management consulting or investment banking. Quality and UX seems to be regressing across the board as a result.
I meant the “grind”, short term profit mentality of SWE market has become similar to professionals in those fields, not that any of these fields are similar.
I think what makes it work is that our code pair is pretty low stakes. I was told that I didn’t have to finish the problem and I was free to use whatever tools or language I needed. They just wanted to see how I work and collaborate.
Thats what we did, pair program on some real production code and tickets. This way the person could get a feel about what they potentially were walking into and you get a good idea of how they think and approach problems.
The problem with it is the same curse that has rotted so much of software culture—the need for a scalable process with high throughput. "We need to run through hundreds of candidates per position, not a half dozen, are you crazy? It doesn't matter if the net result is better, it's the metrics along the way that matter!"