Hacker News new | past | comments | ask | show | jobs | submit login
Interviews in the Age of AI: Ditch Leetcode – Try Code Reviews Instead (chrlschn.dev)
189 points by CharlieDigital 7 months ago | hide | past | favorite | 272 comments



We use this process, and it has helped us hire some great developers without having to use third party staffing and headhunters:

1. Post well written job post on major job boards.

2. Immediately reply via text to every applicant (yes, we have opt in on the application)

3. Initial screening including several knock-out questions via text. Immediately let candidate know if they are not selected and why.

4. 30 minute can you write a working function from requirements code test. Allow one re-try. Immediately let candidate know if they are not selected and why.

5. Interview and code review session. Bring a side project, show us how it works.

6. Offers out.

7. Background checks start in parallel with on-boarding. First week is spent with business unit teams to learn what the company actually, really does.

8. End of first week, we occasionally have to let someone go on background issues. Otherwise, week 2 is first week coding, expect a working pull request by end of week. All candidates informed position is filled, ask if they would like to be alerted if similar job opens.

Works like a champ. Helps we write software that does this.

Average developer tenure is two years, company is three years old. We write telecom software that does texting, voice and video interviews.


> 7. Background checks start in parallel with on-boarding. First week is spent with business unit teams to learn what the company actually, really does.

Yikes!!! I’ve never heard of a potential employer giving offers before background check. Do you tell candidates this before they quit their current jobs?


Yeah this sounds like it should be illegal, although it's probably allowed in the US. There's no way it is clearly communicated to candidates, I don't think most people would agree to a contract that says "FYI we might fire you after a week if HireRight doesn't like you, no we can't check beforehand you'll just have to hope you meet our standards."


Yes, I agree that the company should be clear about this upfront.

I would think that people affected by this are probably used to this coming up and are up-front with what happened before the accepting the offer. "Hey, so great to hear about the verbal offer. So about white collar crimes..."

Unless the company is somehow getting expunged records or something like that, it's likely that the person is lying about something they shouldn't have and got caught.


Exactly. A lot can unexpectedly go wrong during a background check, some companies ask employees for (very old) W2's they may not have, etc. There was a time when I worked at <redacted> and a new co-worker suddenly disappeared without a trace, and I was told in confidence by a VP they "were not who they said they were" which I presume means they failed a background check. This was during growing pains, they were probably doing checks after hiring.


We're not doing a salary verification so we don't make the new hire provide a bunch of data, aside answering questions about where you lived, when. We are doing a court record and criminal background check. When we have a problem 100% of the time the new hire has misrepresented themselves. We tell people exactly what the background check will discover, and we've even hired people who have disclosed issues prior to the check (nothing serious).


If an employer ever asked for an old W-2 I'd refuse, and if it was enough of a problem I'd recuse myself. That's private, sorry. Not that it would matter for me personally, I've been self employed for 15 years. But still there has to be some self respect.


Unfortunately these types of requests come after you have presumably accepted the offer and declined alternative offers...


All “high-horse” claims work better when yo have cash in the bank and don’t need to get that job right here right now.

I also decline signing NDA, providing data they should not request. Asking to cross off anything that encroaches intellectual property I might create after hours.

But if I would need that paycheck ASAP all of it goes to trash and I would comply to most of shenanigans, maybe not all but still.


> Asking to cross off anything that encroaches intellectual property I might create after hours.

We're pretty big on making sure that our contract allows this as we encourage developers to do side projects - open source, etc. We do have an anti-double dipping clause.


> "were not who they said they were"

I worked with a guy who it later came out that he was convicted of conspiracy to commit murder when he convinced a buddy of his to kill his ex-wife and her new boyfriend. Probably should have had better background checks at that company.


Your example seems like something that the employee likely knew of though.

If it’s fully communicated that the background check happens in parallel and the new hire doesn’t expect anything to pop up, it seems like a good option (provided the employer is reasonable about trying to solve unexpected inaccuracies that may arise).

In my last two positions, I’m pretty sure my employment contract included a section indicting that at any time they can do a new check on me. And I believe my most recent one explicitly asked if I expected anything to come up.


> Your example seems like something that the employee likely knew of though.

That's true, but consider this: by running these checks in parallel you are exposing your employees to whatever risks the new element brings in. Perhaps they were convicted of SA etc. and decide to stalk someone at your co.

It just doesn't seem like the best idea.


…why would you need an old W2?


People lie about job history


So you reference check with the company’s HR department, no? There is way too much info on a W2 that no future employer should have.


There's a school of thought around comp where what someone's made in the past should inform what you offer in the future. Some use comp history as an indicator of quality. Some use it as guidance for making sure they don't offer too much.


As someone not familiarized with background checks, why is it weird to do bg checks after handing the offer?

If the employer does the bg check before handing the offer, that means the candidate hasn't resigned yet from their current job. So wouldn't the bg check expose the candidate? (E.g., my boss would know I'm thinking about leaving)


Doing the background check after the offer is fine, it's insane that they do it after the candidate already starts the new job. So what, I leave my current job to come work for you, and in the first week you tell me "sorry you failed our background check" and let me go? Who the F would ever agree to this kind of risk?

>>8. End of first week, we occasionally have to let someone go on background issues

Yeah, that's bonkers.


We're not talking like government security clearance here. If you have some major legal or financial issue in your life then maybe it depends what exactly the BGC turns up or how it's adjudicated. If not, there's basically zero chance it's going to affect anything. Just a formality from the candidate's perspective.


If it's just a formality, why not do it before you start the job then?


If it’s just a formality, why would you delay the new hire’s start by several weeks while it’s pending?


As an employer you wouldn't - but I don't understand how any employee agrees to starting before it's complete.


You already know whether you have a criminal record or financial catastrophe. If you do, maybe you need to wait and see what they think of it. If you don’t, there is no chance of a problem.


This is normal for non-tech employers. It is novel in tech, but not something that is bonkers in other sectors. We're able to have developers on the payroll three-four weeks before offers are out by competitors.


I wonder where you find developers desperate enough to accept such conditions - I'm not sure I'd want to work with them.


This has never been a problem, except when someone is currently employed and is leaving a job to take one at our company. Right now, there's a lot of really good talent that does not have this issue.


>>except when someone is currently employed and is leaving a job to take one at our company.

And in that case you do the check before they start with you, yes?


Of course.


In that case it seems like all of the criticism in this thread is moot.


> So wouldn't the bg check expose the candidate? (E.g., my boss would know I'm thinking about leaving)

HR at the orgs I've worked for call that a reference check and it is usually concurrent with the other background check processes before the offer. The background check I'm thinking of is more like criminal history, work eligibility status etc., potentially drug screening etc. I know I have no criminal record but I'd really like to know that whatever service HR is using to screen agrees with this before I quit my job.


Background checks don't call your old boss. How would they even know who that is?


Have you ever done a background check with HireRight? They ask you for all of your previous companies and contact them to verify. I think there is an option not to call your current employer.


How far can you legally go with those in the US?

Here in the EU at most you can ask if they worked there, nothing else.


There's a huge background check fetish here in the US, and it's facilitated by a massive lack of privacy and an equally big fetish for everlasting punishment.

Want a job or rent a place? Background check, baby!

Not just prior employment but even civil court cases and criminal history are all a part of it, and it doesn't help that even something as small as a traffic ticket shows up on your records.

Have you ever sued a landlord to get your deposit back? There's a good chance your new landlord doesn't want you.

Were you convicted of petty theft years ago? That might cost you your prospective job.

In most of the EU, much of this information isn't public, and in many countries, criminal records are inaccessible.

Often, in the latter, you can get a declaration of “good behavior” from the government if your prospective job has some sensitive elements. The government will then issue one or decline based on the specific job and its risks with your record in mind.

Were you caught committing a DUI? You won't get one for a job as a cab driver, but you can safely get one for working at a bank.

Have you got caught embezzling money? Then you can't get one working at a bank, but you're welcome to become a cab driver.

In the US? Well, you're SOOL. Enjoy being marked for life as your options to get an income legally have significantly shrunk.


Oftentimes, in practice it's the same.

Aside from things that may lead to accusations of violating equal opportunity employment laws, you can ask just about anything. Many previous employers won't do more than confirm employment though, as a negative review may open them up to liability. As a result, the norm in many industries is to only ask (and receive) confirmation of employment.


Nearly the same. “Did BigJ1211 work at your company? Would you hire them again?”

But that signals to their current employer that they’re applying for other jobs. Unlike the EU, employers in the US can be retaliatory about that.


On one hand it sounds off putting, on other hand it can be very awkward that you have hired a mole from competition who is there to understand your secret sauce and get out or that you have hired somebody who was in prison for computer crimes.


A week -1 background check would find anything a week 1 background check would find.


I work for a large, well known tech company and can confirm I had a formal offer before doing any background check.


Yes we disclose. We have to. Incentive is on the candidate to be honest.


I can bring my kids in but I don't think I could tell you how they work. I've been in this industry for almost 20 years, but I don't have side projects, because, well, see above.


My go to side project right now is training a neural net to operate a bio-mechanical process to automate waste management for consumers.

By which I mean I'm potty training my 3 year old.


Which is why this is a dog whistle for ageism.


You’ve never written a utility at work to help that isn’t part of your core work? You’ve never had a side project in 20 years ?

Maybe your view would change if you were actively looking for work? Spend a week on some fun project while taking a break from resume writing.


I've never heard side projects should be recent or ongoing. How do you not have any side projects? Have you only ever programmed professionally?


I doubt they would want to be judged on code they wrote 10 years ago, though.


I doubt they would want to be judged on code they wrote 10 years ago, though


Freelancers can quickly build out some side projects for you to use on your resume if you wish. It shouldn’t be a big deal.


That seems super dishonest and the kind of thing that would get you in big trouble if discovered. Hell if you're going to lie like that might as well lie on your whole resume.


There is zero chance of being discovered. You pay your freelancer, own the project, throw it up on your GitHub, no one will ever know.

Meanwhile, you get to spend more time with your kids.


Agreed not everyone has side projects. But being asked to build something of your own interest in about half a day seems like a good alternative.


I don't know what I could do in half a day that would impress anyone. Especially if other candidates are bringing in their serious side projects.


>5. Interview and code review session. Bring a side project, show us how it works.

So you are filtering out candidates that have families or lives outside of work that do not spend their free time coding?


Yes that is precisely what happens here. It's intentional.

It also doesn't scale up high enough, because the pool of candidates is smaller. It's a big part of why you won't see large companies hiring this way, typically. The other big part is that big companies have lawyers who see potential for lawsuits due to discrimination.


Same thing leetcode does tbh


It's great that you quickly notify candidates if they aren't a fit! Wish more people did that.

But....

>Bring a side project, show us how it works.

What do you do if someone doesn't have a side project (or want their side project to be reviewed by someone they don't know)?

Some people code to live, not live to code.


I'm not defending the practice of requiring that a side project be presented. I think that's over the top and wouldn't do it -- however, if a candidate volunteers one, that does give a huge advantage to that candidate.

> Some people code to live, not live to code.

Totally fair. But there are lots of companies who prefer people who code because they love coding over those that do it just for the paycheck, and that's also totally fair.


I love to code, hell, I do it almost 40 hours per week. And I get paid to do it, isn't that cool?

Sometimes a side project catches my eye. But I have more than a singular interest. And kids. And to be honest, when a side project catches my eye, work suffers a bit because those tiny moments I have to fill with thoughts are spent on my side projects rather than work projects.


I don’t think this was intentional, but you missed the third category: people who love to write code but only have time to do it for the paycheck.


Oh, I think it is very intentional, and the whole point is to not say it out loud.

If you don't have time to code for "love", you won't have time to jump at unreasonable requests and pull tons of crunch time overtime either.

Given the choice, companies would love to hire someone with no responsibilities outside of work, this is just a legal way of filtering out parents, people with dependents etc


> If you don't have time to code for "love", you won't have time to jump at unreasonable requests and pull tons of crunch time overtime either.

As opposed to doing leetcode? when you also have to spend time outside of work to practice and memorize them?

If you are looking for a new job, you have to invest time and effort. One way or another. Unless you are a known quality.

Can you suggest a fair way to evaluate a candidate?


>Some people code to live, not live to code.

That doesn't mean not loving coding.

It means that you might have other hobbies that you want to spend time on, or other obligations you have to spend time on, etc.

I like what I do at my job. I would even go as far to say as I love it. I also don't do it for an extra 20 hours a week (beyond the 40 hours I already do for work) because I have other things I love to do too (sometimes more!); quality time with friends, family, my other hobbies, etc.


I can sympathize, but keep in mind most professions expect you to commit to a certain number of hours of continuing education. Often you can convince your employer to let you do this during work hours (or even pay for it!), but not always.


Continuing education credits is pretty far from "bring in a hobby project for code review for this interview".

Is there even a CE credit available for coding up a hobby project (and who from)? Completion of a course, attending a lecture, attending a lunch & learn, etc. are the CE credits I am familiar with (spanning a few different professional bodies).


Sometimes you can declare "self-learning", depending on the order.

But anyways, that's not really my point: My point is just that in professions it's not uncommon to be expected to perform some kind of extracurricular activities related to your job. Often software "engineers" aren't members of a professional order, but I'd argue that the idea still applies. Tbh learning by working on a hobby project is way more appealing to me than watching some PowerPoint presentation...


>My point is just that in professions it's not uncommon to be expected to perform some kind of extracurricular activities related to your job.

If my employer _expects_ me to do something related to my job, I get paid for it. I really wish we'd all stop normalizing working for free.


I don’t understand your view. How do you improve your skill in the craft ? Or learn new technologies ? You wait for your employer to tell you to learn something ? That’s a sure fire way to always be behind the curve.


I don't understand how you reached that conclusion.

If it is a personal interest or hobby, I do it on my own time. If it is something required for work, I do it on company time. If there is a lot of overlap, I do it whenever.

Other than that, I learn and improve like any other person does.

Continuing education credits, which is what started this subthread, is something required by the professional body that my company wants me to be a member of. So they happen on company time and dime.


Employers don't require their employees to be members of a professional order because they think professional orders are nifty- It's because certain jobs are only legally allowed to be performed by a member of said order. If you were a dentist and ran your own clinic, you'd still need to be a member of a professional order (at least in Canada and the US afaik) to practice dentistry legally, which would come with obligations outside of your usual working hours.

Software engineering exists in a sort of gray area where you can often be a professional software engineer without having to be a member of any order, which is great in many ways. But I feel like one could argue that the informal expectation of software engineers to care about software outside of their work is similar to what is expected in other professions with more rigid governing bodies.


>Employers don't require their employees to be members of a professional order because they think professional orders are nifty

I didn't say they do it for nifty-ness. At risk of repeating myself again: if it is a requirement of my position, I get my employer to pay for it. Why it is a requirement doesn't matter to me.

If you want to pay for and do continuing education things on your own time and dollar, I'm not going to stop you.

>But I feel like one could argue that the informal expectation of software engineers to care about software

I didn't say I don't care. I just have plenty of other things that I care about that take priority when I am not working (spending time with my family, friends, doing other hobbies, etc.).


Then those people get ranked lower (for good reason).


Here is a dark side of side projects (pun unintended). I went to an interview which was a nodejs shop. I was excited as they wanted to see a side project. Most of mine are just things I do for fun. Anyway it was in golang (with lots of control plane like features for a "fun" saas offering). Cto wasnt happy that I had chosen go and wanted to know why I hadn't chosen Node instead. (Never mind my Fe was in typescript). He wasnt happy with any of my explanations and came back with retorts like "well you have async/await so you don't really need goroutines).

First of all nobody should be forced to be judged on side projects alone. There are several talented "family folks" out there. And even if not there are enough "recharge on my time " folks which is totally fair.

Secondly just like in leetcoding interview all it takes is an inexperienced interviewer with a script to degrade the experience!


Yeesh, sounds like you dodged a huge bullet. What an insanely narrow minded individual.

I want to see good engineering practices, I might be curious about your language decision, but I'd never tear you apart for your choice.


right, and i even explained that node was for single-language simpletons, but he didn't like that either


Yeah, I provided a side project once during an interview process and the only feedback I got related to my use of the 'static' keyword and how that could introduce an undesirable side effect when deploying into a web server (or something like that.) Not sure where they got the idea that the side project would ever be used in that way, unless they were trying to do that themselves.


> Bring a side project, show us how it works.

"As you can see, it's based on a I-IV-V chord progression..."

"Usually I try to take advantage of the negative space, but it's not cheating to add a little white gouache in spots like this..."

"...and here you can see that I did more than 12 reps, so next time I'll up the weight."

"When she makes that face she might be pooping. If you ask her and she says 'HHHNNNNNO!' she's trying to hold it in, so bring her knees up like so..."


So you have someone start before you've decided to hire them. This sounds incredibly sleezy.


Sounds incredibly like a non-starter for people who aren't completely desperate. Then again, maybe that's who they're looking for.


It's bizarre to do with background checking specifically. Like WTF, you're gonna have someone come in and then fire them because of their past.

I thought it was bad enough these people trying to get you to start as soon as you clear HR.

I'd never even thought about the idea of starting before getting the greenlight... this is insanity.


In the U.S. you'd get title IX'd eventually. All feedback will be used against the company. This is why leetcode was spawned, to weed out the 99% in a way where nobody can claim discrimination. The last 1% can be picked with discrimination by failing the undesired candidates, the "not a culture fit".


> we occasionally have to let someone go on background issues

What kind of issue would that be?


Not having a side project going on in the background.


Undisclosed felonies is a popular one.


Had they disclosed the felony would they have made it that far?


Maybe. But they definitely should have brought it up before accepting the offer.

Embezzlement or something? Probably a no-go, especially if you are anywhere near money.

There's a whole host of crimes out there that you could have committed and be hired after. But the opportunity to explain yourself looks much better if you do it before accepting the offer.


I have heard of employers allowing you to explain them, yes. Example: Felony DUI: explain your remediation steps, AA, etc.


I fail at #5.

Most all the coding I do is for the firm I work for, occasionally some of it escapes to the open source world if I can convince some manager to let it.

At the end of the day, if I've wrung out all I got, that's all there is. It is best to recharge and get ready for the next day.

I've written some pretty cool stuff. But often the code isn't 1/4 of what is interesting about what got written :)


Man, I forgot how much people on hacker news absolutely hate the idea of side projects, a.k.a. passion for engineering and an interest in creating new things.

Somehow the implication is that if you build even one little side project for fun that means that you spend every single hour of your time coding. It's such a reductio ad absurdum argument.

If I was hiring an architect, I'd expect to see samples of their work.

If I was hiring an artist, I'd expect them to have a portfolio on Behance or Artstation.

etc. etc.


It sounds like you have the mentality that everybody must be a front end developer.

I work in a domain that's very much not front end development, I've participated in interview rounds for at least 100 different candidates, I don't think I've seen "side projects" ever be a factor in someone getting hired, not even once.

The one thing that did put an applicant way ahead of the pack was contributing to open source projects relevant to our domain. It didn't have to be anything super duper impressive, we just liked to see a history of submitting meaningful PRs and getting them through code review.


AI could do code review better than coding.

90% of code review is BS - and AI excels at BS.


Are you hiring now? :)


"Bring a side project, show us how it works." Can people in the industry stop this please? You have to be mentally sick to ask people to code review their side projects. It doesn't make any sense.

"Average developer tenure is two years, company is three years old." Got it. I don't think this statement says what you think it does.


Agree. Interviews taken into account side projects are biased towards:

- people who have free time to work on side projects (usually young people have more free time than people with families)

- people who don't mind sharing their code with others

- people who work on interesting side projects. If your side project is boring, that will probably bore your interviewers -> no offer

- people who work on side projects on regular basis. If I get to work on one side project every year, well, chances are I may have forgotten the hell I did on that project (depending when the interview takes place). If I work on side projects constantly, I have no trouble picking up a fresh one to talk about


> people who have free time to work on side projects (usually young people have more free time than people with families)

That's the whole point. Companies would always prefer someone with no responsibility who can be guilted and pressured into doing overtime over someone who can point to actual needs outside of work that they need to attend to. It's a great way of filtering out people who will have boundaries.


> people who work on interesting side projects. If your side project is boring, that will probably bore your interviewers -> no offer

This hasn't been my experience at all. The interviewer doesn't care how "interesting" the side project is. What they want to see is what your actual working code is like, that you demonstrate a good grasp of programming concepts, etc.


I wouldn't be worried about boring my interviewer and getting rejected because they consciously recognized that they were bored, I'd be worried about there not being a level playing field between projects when the hiring committee is discussing it later. The guy who built the fun video game will probably be remembered more fondly than the guy who built the CLI tool for tracking the weather.


    > You have to be mentally sick to ask people to code review their side projects.
I would love to do this; it would give me an opportunity to showcase my side projects, my coding style, and all with code that I'm already familiar with.

It would be great, IMO.


> You have to be mentally sick to ask people to code review their side projects.

Could you expand why? We allow people to substitute a tech interview round with side project showcase. Don't see why it doesn't make sense in your view.

> "Average developer tenure is two years, company is three years old." Got it. I don't think this statement says what you think it does.

So what does it say?


It creates an extremely subjective comparison between candidates.

Part of it is like a sibling comment said, if an interviewer's interests lie more in line with devops, then a devops project that solves a problem he's acutely aware of will be given much higher praise in his mind, and allows him to engage with the candidate more. If the interviewer rarely does any devops and isn't totally sure of the problem space, he may be biased towards thinking it's overengineering a problem that doesn't exist, or may not fully grasp the problem in the time allotted. The candidate could articulate the side project exactly the same but have completely different interview outcomes in both outcomes.

And an argument could be made that "it's a test to see if the latter candidate can articulate a problem and peak the interviewers interest" but in this case the former didn't even have to do that, and thus you are judging candidates by different subjective metrics. And it's especially an issue if the side project isn't directly related to the role (e.g. video game vs cli tool vs database vs website, etc...).

It also means that some candidates will be judged on system design (if a side project is wide scoped) while some candidates will be judged based on algorithmic design (if the side project is very narrow scoped with the details in the implementation).

It just adds so many variables to the interview project that you are no longer comparing "which candidate is the best fit for the job I am actively hiring for" and instead "which gave me the best vibes", specifically because you did not actually measure all candidates by the same measure.


very clearly said, this is such a subjective process.


> So what does it say?

It depends on the definition of tenure... if tenure is defined as a closed interval (i.e. defined only for developers who've joined and then left) then it means that, of the developers who have departed, none had wanted to grow with the company, or the company had let them go. For a startup company this might not be a good sign.

If tenure is defined as the length of time the developer had worked, OR, has worked so far, then it means they have a core group of developers and aren't growing the development team particularly quickly. Again, this might not be a good sign for a startup.


> then it means that, of the developers who have departed, none had wanted to grow with the company, or the company had let them go. For a startup company this might not be a good sign.

What. Of course that "of the developers who have departed" all have left/been fired, that's a tautology. How is it "not a good sign"? Or is it supposed to be a bad sign that there even is a person who is not working there anymore?


I think parent meant:

Assuming tenure is defined ONLY for those who have already left, 2 years is a bad sign. For a 3 year old company, if that definition is used, it is indeed pretty bad. It means that _of the people who are leaving_, people stay a couple of years, then bounce. This means people stick around long enough to get past the warmup of new employment, get used to your stack and tech, then bounce for greener pastures.

The very important metric this leaves out though is what percentage of the company actually left. If there's 7 people in this 3 year old company and only one person left last year . . . that says almost nothing. Any single person can leave for a great variety of reasons. You'd need a decent sample for this to matter.

The flip side definition of "tenure" is worth considering though: if the average tenure of 2 years includes people still working at the company, there are a lot more variables to content with before you can know anything. A 3 year old company could have an average employee of it have been working there for 2 years and not a single person who joined the company having ever left (e.g. if at year 1 there was a decent amount of hiring). I think this is probably why parent wanted to restrict the definition (and why it's worth thinking about this angle) - because otherwise the company ramp up and trajectory and hiring patterns become hidden variables and you can't glean anything out of the tenure numbers on their own.


I still don't get what's bad about that. So what would be a good tenure for a 3 year old company if 2 years is bad? 2.5 years? 1 year? 6 months?


> Could you expand why? We allow people to substitute a tech interview round with side project showcase. Don't see why it doesn't make sense in your view.

As an option to replace a different interview segment I think that's quite fair. But both choices have to be clearly presented to the interviewee, and they should understand that either option is treated as equally good. OP's interview format seems to assume that everyone does the side project and the tech interview round isn't an option.


People are forgetting why companies these days do Leetcode in the first place.

The initial reason why we have Leetcode today is that Google originally determined through their interview research that the smartest candidates were the ones who were best at algorithms. Google wanted to hire the smartest people, not necessarily the best coders, so that’s why their interviews were mostly algorithmic.

Of course everyone started copying Google without doing the same research and the message got lost. They just blindly asked coding algo questions and that’s why we have an entire industry built around coding algos and the signal is entirely lost now.

It’s funny that people have lost that intent and are now creating derivatives around that, ie thinking code reviews are a good replacement.

It might be, but there needs to be data done to research it to see if you get what you want. If you interview using code reviews, you get good code reviewers. If that’s what you want, great.

But remember that Google’s initial goal was to hire smart people, and there’s no research to suggest that good code reviewers are smarter than average.


When you are a company that builds core software services algorithms are actually important and not proxies, and can be studied and learnt by heart. For most software companies algorithms they are irrelevant since they are glorified CRUD shops. What google ended up with that workforce, is a percentage of failed projects (in terms of real world business value and longevity)) that would have brought other software companies to their knees. Google produces nice languages and frameworks tho.

They really need to hire less geeky people (disclaimer, also geek but with self awareness). We hire a percentage of people whose first degree is not programming and have formed relevant career paths.


> What google ended up with that workforce, is a percentage of failed projects (in terms of real world business value and longevity)) that would have brought other software companies to their knees.

It's not clear to me why this would be a result of their engineering hiring practices as opposed to their business strategy and product management culture.


The problem is that it might have been true at some point but now there are resources to “learn” for interviews, so currently you will get people willing to grind.

Nobody happy with their pay and job is going to grind leetcode.

They could just ignore leetcode and ask puzzle problems instead to test ability.


> They could just ignore leetcode and ask puzzle problems instead to test ability.

One of the hiring fads before leetcode was to ask puzzle questions, with a heavy emphasis on Fermi questions and trick questions. These had the same failure modes.

It turns out that hiring is hard, involves some risk, and always tends to illustrate Goodhart's Law: whatever process you put in place to avoid that risk will ultimately become worthless when people start optimizing for it.


> Nobody happy with their pay and job is going to grind leetcode.

I'm not going to grind leetcode regardless of how happy I am with my pay and job. I have very strong objections it. If a company requires it, I take that as a pretty strong signal that I won't get along well at that company.


The people who are good at algorithms are grinders. There is a tiny percentage of geniuses who don't need to grind algorithms. The rest of the people who were good at algos before leetcode became widespread were also grinders

In fact this is how you become "smart" - some talent and a lot of hard work (grind)


> In fact this is how you become "smart" - some talent and a lot of hard work (grind)

Hmm. I think this is a different definition of "smart" than I'm used to. I would say that some talent and a lot of hard work gets you "skilled", not necessarily "smart".


There are many different types of intelligences or smarts

Skill is a form of "functional intelligence", where skill = talent + grind. The more talent, the less you need to grind, but combined the two are killer. To me, "smart" where the grind is 0 is not "smart", but just "lucky" (born with talent)


Honestly, being "willing to grind" might be a good thing to target if it's a good proxy for being willing to do the work.

However, I agree with the general sentiment that almost nobody is doing the research to ensure their interview process selects for the things they actually want.


I’m willing to grind if you present me with a real life problem you need solving. I’m not willing to run on a hamster wheel, it seems illogical to me.


And therefore you are not an employee they want.

These companies don't want an employee who starts to question "why" things should be done or why things are important. Demonstrating that you are willing to apply yourself at something objectively useless simply because you are required to is a huge plus.


This is the situation at most big companies.


1. Learning algorithms and practicing leetcode is not actually useless. A deep knowledge of algorithms falls under the category of high value knowledge that is not frequently needed. Its still high value when its needed and you cant predict when it is needed.

2. Its not a hamster wheel because you get a high paying job for your efforts


My sentiment too. These Leetcode style websites will take an algorithm and then come up with a contorted problem that can be solved with that problem. Often the problem is not even worded precisely but people who have been wasting time on solving random problems can see the patterns apparently.


Weeding out candidates who aren't willing to ride the hamster wheel is a huge point of leetcode.


> Honestly, being "willing to grind" might be a good thing to target if it's a good proxy for being willing to do the work.

Grinding is a poor proxy for work ethic. It is a good proxy for submissiveness.


You might be right, but I think it could be both.


> The initial reason why we have Leetcode today is that Google originally determined through their interview research that the smartest candidates were the ones who were best at algorithms. ... It’s funny that people have lost that intent and are now creating derivatives around that, ie thinking code reviews are a good replacement.

Have you considered that what's actually happened is people are finally realizing that they don't want to optimize for the characteristics that Google is optimizing for?

Setting aside the question of whether "smart"s are something that can be sought out without a more specific definition, the average business doesn't need "smart" programmers. They need programmers who are predictable, who reliably produce good-enough software, who communicate well in speech and in writing and who work well on a team. Algorithmic ability provides essentially zero signal for these capabilities, so it's not so much that people have lost the intent that they're realizing that the process they cargo culted isn't appropriate for their hiring needs.


> The initial reason why we have Leetcode today is that Google originally determined through their interview research that the smartest candidates were the ones who were best at algorithms.

How did they measure "smartness"? The Leetcode algorithms are mostly solved problems and I know people who can "solve" any leetcode you throw at them, but they are very poor at actual real world problem solving.

It's quite a bit like saying people who memorised Wikipedia are smart, because they know things.

To me being smart is to know where to look.


"When a measure becomes a target..."

Also, I wouldn't take Google's (nor anyone's) claimed research about hiring at face value.


I'd wager that the best mathematicians would fail on these "algorithmic thinking" interviews. Modeling a problem, even a simple one, as a computational problem requires time, trial and error beyond what you get in an interview.


It seems like chasing golden unicorn super smart bois has been more of a problem than a solution in SV and companies like google. It starts to look like a schoolyard game or mensa.

I'll just get the smartest of the smarts and rule the world! If I only get the alphas of the alphas, I'll surely win.

Reminds me of all the NBA players getting schooled in FIBA, because the international teams have better chemistry. At one point they had to do a total re-boot, because it was so embarrassing.

I know of at least one second tier, non-sv shop that goes hard to hire the best developers in terms of intelligence. It actually leads to huge quality problems during recruiting and poor diversity. There's just something about mega-nerds with extremely poor social skills that keeps the women and minorities away.

What I'm talking about isn't just awkwardness and lacking charisma. I'm talking about randomly telling women that they can tell she is on her period and things like that.


> It seems like chasing golden unicorn super smart bois has been more of a problem than a solution in SV and companies like google. It starts to look like a schoolyard game or mensa.

> What I'm talking about isn't just awkwardness and lacking charisma. I'm talking about randomly telling women that they can tell she is on her period and things like that.

These don't seem limited to SV and Big Companies™ like Google, all it takes is a cursory scroll either through this website (even this thread!) or a trip to any of the CompSci-focused boards on Reddit. Additionally not limited to women and minorities, but anyone from a non-traditional background as well.

If anyone feels like working on these issues, the problem is extremely local, you don't even have to leave HN!


Maybe true, but in other industries, such as ones that are more "business and engineery, there's less of this obsession with getting an ultra-genius, although we know how valuable engineering talent is.

As a result you get pretty average people. Even the weirdos tend to be just quiet and reclusive. My brother on the other hand, went to work for a smaller B tier to google, and he had multiple co-workers that were just beyond bizarre.


It’s just a proxy measure for IQ really. They could have handed out Ravens matrices and got better outcomes prediction.


I assure you, they would if they could, but its illegal.


A quick search indicates that it is not illegal in the US as a whole, but maybe California?


What's illegal is IQ/ability tests having disparate impact on particular racial groups. You can only use them if you can justify how they're related to the job (which is much easier for algorithms stuff in SE roles).



You can also train for IQ tests.


People misunderstand the reason for this. Test takers can improve, but only up to a point, and only because they getting comfortable with the test format, not because they're getting any smarter. Solution: give the candidates a few practice tests first.

This extends to every test though. My ACT score jumped massively on my second try. The reason: I didn't realize how much little time they gave test-takers, so I ran out of time on some portions. The second time around, I rushed through as fast as possible.


You're certainly not getting smarter. It's the test format that is not very reliable.


Is memorization how we should measure IQ? If so then LLMs have already far surpassed humans.


A LLM would do pretty/very well in a leetcode interview


In a "have you memorized the code to solve this problem" interview? Probably.

In a "can you solve this new leetcode problem that just came out and is not in the training data"? Awful.


There are probably about 30 or so leetcode problems and then endless variations made for those 30 problems. The best people on leetcode practice and memorize.

If you want to see an example of a new problem check out https://news.ycombinator.com/item?id=37910297. You'd think if leetcode makes you good at solving problems then it should be doable any higher ranking leetcode programmer.


> There are probably about 30 or so leetcode problems and then endless variations made for those 30 problems. The best people on leetcode practice and memorize.

The comment I replied to claimed that "LLMs would do pretty well at leetcode interviews". OpenAIs own report [0] shows how GPT-4 fails miserably at medium/hard leetcode problems that are not in the training data.

[0] https://cdn.openai.com/papers/gpt-4.pdf


I wouldn't consider doing 31/41 easy problems, 21/80 of medium problems, and 3/45 of hard problems a failure. GPT-4 wasn't built to solve these problems but can still do 3 of 45 hard problems. Hell I don't know if you sat down 45 random programmers if 3 of them could solve those 3 problems GPT-4 was able to do, and nobody could solve them in the time it took GPT-4.


Hell I wish they would tbh. I have a decent IQ (140), but can't (or just unwilling to) grind leet code


An IQ score of 140 is considered near-genius or genius level and places you in the top 0.25% (99.75 percentile) of the population.


Actually 137 if test was accurate, I still don't do well with leet code whiteboarding though, shrugs.


Maybe you just need to find something you enjoy doing, programming might not be "it"?

Or do you do well solving the problems on your own and just have trouble when being interviewed?


The latter, take home projects I do well on (projects, not HackerRank quizzes)


That's a believable score around here. There are over 8 million Americans with that score (assuming OP is American) and this place probably has a disproportionate number of high-IQ people posting.


My IQ is 125 and I feel the same......I love solving problems but only if I feel like they are real world problems.

I did extremely well in algo classes in school.....but since then I have built actual applications that run businesses and haven't used algorithms at all.

Grinding leetcode feels like running on a hamster wheel for me.


I agree. You have to do the test that really relates to what you are looking for.

For backend web devs I'm hiring I established a modified version of a process that has worked for me well before:

1. Interact with a RESTA api and solve the problem by doing cURL requests and some lightweight string mangling and computation to generate key. Then submit the result with your BASE64 code in the request.

Your code is analyzed with ts-node (I look for TypeScript devs) and we check the validity of your generated key.

On success I get an email with the person info and the code. To follow up.

I give the url of the challenge to anyone whom I get their CV and seem interesting for the position. A lot of people disqualify themselves because they dont know a REST endpoint needs the correct content headers, or dont know how to send Auth Bearer token, etc.

Then I see the code, and if I like it, i send them a link to self book a 90hr interview. 30/50 mins are to talk about their experience, resumé in hand, and the rest is to make a version of the server to process the requests from the client they submitted. The problem is common knowledge for both of us, its basically what tgey will be doing, and it allows me to vouch for how they code. Mind, I dont care if we dont finish it, I just want to see if we can solve a problem together with the candidate driving.

And that's it. At the end of the interview I can tell them if I'm going to make them an offer .

I try to do interviews the way I'd like to be interviewed. I hate tech interviews, its stressful and seems pointless sometimes. Why would I do that to people who will be my peers? Bad taste.


sounds like a great process. are you hiring? email is in my profile.


Sorry, I'm pay Mexican pesos, as I hire in Mexico. Theoretically I could hire from outside Mexico, but people in US and Canada are expensive


no problem. best of luck with your company!


How do they measure which candidates are the smartest? And if such a method exists why they don't use this method in interviews directly?


by asking them leetcodes... Leetcode also has the advantage of being tangentially related to software engineering and you are able to improve by practicing. Employers want people who are willing to sacrifice their life for the job so leetcode is better than a straight iq test.


Be the change you want to see in the world.

I refuse to interview at places that are Leetcode-heavy. Never prevented me from getting 7 figures offers at FAANG-type companies.

Similarly when I interview candidates, I don't ask them Leetcode questions either. I like to get a sense of whether or not a candidate can reason in semi-unknown territories and has good intuition, not if they can parrot a textbook.


7 figures? I appreciate your stance, but if you're getting million dollar offers I'm not sure your advice is relevant to most of us. It's like a movie star saying "why bother auditioning, just wait for a director to request you specifically".


You can't just leave that hanging there. What "FAANG-type" companies are making 7-figure offers to anyone on HN, without a Leetcode fratbro hazing?


I sense some dissonance here:

"I interview at FAANG companies and refuse to interview at FAANG companies"


What are some FAANG-type companies that don't ask Leetcode-heavy questions?


Same boat. I've interviewed hundreds of engineers, using thirty minute code review kind of setup, who have proved damn good in the depth of their knowledge.


I'm a CTO for a small software studio. We've used mini projects and code reviews in our interview process for years, and it works great. The code review is a great way to explore a candidate's ability to code, and also to communicate their decisions about a piece of code.

In the "age of AI", I'm not sure we'll ditch the mini projects. I think testing a candidate's ability to _use_ AI to generate code is an important skill. However, it's so new, I'm not sure what the best practices might be.

I'd be curious if anyone has any advice on this front...


I think mini projects + code review is the way to go for smaller places. Make the projects large enough that an AI couldn't generate correct code right off the bat and then in a follow up conversation dig into the code and ask for their reasoning to make sure they understand the code they wrote so you know that they either wrote it or fully understand it to the point where it doesn't matter if an AI write it and they tweaked it.

To be honest, as a CTO (not currently hiring though) I want people that can use AI to speed up their coding but who know how to use it well. If anything I suspect those devs are better than devs who don't use AI as they are practicing using their code reading abilities more often and filtering away any bad code written for them.


I use AI to generate code sometimes but it often doesn’t work as expected. Other times I get just awfully inefficient answer. Someone who has programmed without AI for a few years will spot those things. Someone who doesn’t have any “gut feeling” formed about code and just relies strictly on AI will do awful.

I think your approach is good, I would introduce things that work but are terrible design decisions and have the person interviewing figure out what could be improved.


Would you allow employees to subcontract and outsource their work?


> Would you allow employees to subcontract and outsource their work?

No, but I do encourage them to use the best tools possible. :-)


"Sir, there is this approach where I feed in requirements and get back results, can I use it?"


You can't fix the stupid ones no matter what hiring strategy you use.


Just to add to my previous comment. We use AI heavily today, and we're seeing a 25%+ productivity improvement. And our Devs are happier, as they get to focus on high order problems.


Leetcode being asked in an interview is a marker of an incompetent organization.


Especially when they don’t pay FAANG money.

If I were capable of looking good during the only experience in my adult life that feels like having a solo in a middle school strings recital, I’d be applying to those places, not yours.


I don't know if this is what you meant, but to me this reads as: "I don't know anything about data structures or algorithms".


Not him but

1.) DS and Algo knowledge are way overrated in day-to-day job functions. I have yet to have to implement a single DS or Algo from those classes (they are almost always available to the framework, standard library, or language), runtime complexity has literally never once come up in my life, and the most I've ever needed to know about data structures is "which one should I use here?" and 99% of the time its either a vector (arraylist) or a map (dictionary).

2.) Just because you don't have a proclivity for code-golf does not mean you don't know DS and Algos, it just means that you don't waste your free time writing unreadable code for useless puzzles which serve only as interview questions.


> runtime complexity has literally never once come up in my life

There are definitely different types of programming jobs, but for some contrast: I have never had a job where runtime complexity did not come up as a significant factor somewhat frequently.

It really comes down to what you’re doing, but at a certain scale or size of problem you really do need to know about algorithms, data structures, and runtime complexity to avoid having your runtime explode, server costs become untenable, opening the door to extremely long running requests and other problems.


In my experience the biggest bottlenecks are in the network. Most optimizations are also very very low hanging fruit because the original code was written by a moron


Also, using scripting languages and misusing browser engines unironically yet expecting performance...


Using electron isn't a performance decision, it's a labor one. It's easier to write a website once and then just export to electron than it is to rewrite your application in QT or XAML or another native desktop framework and have to hire C++ or C# developers


If Browsers can do Figma and VS code, they're already a far capable and approachable platform at least for most desktop applications than Qt especially for anything that has some data entry such as ERPs etc. And it's free. Qt is not.


> There are definitely different types of programming jobs

That's what most people on HN forget. The programming world is huge. Even more so, if you include all the stuff we wouldn't traditionally include, such as people using code to optimise business or production processes.

It's always best to assume that you don't know what "most software jobs" are like, because you're very unlikely to have seen more than a tiny slice of it.


I'd estimate that 98% of companies are just receive-deserilze-validate-transform-store a.k a CRUD shops though some might have borken it down into distributed chunks (Micro Services) without any of the distributed tooling or mindset and awareness in place.

So... That's less likely where you'll need your dynamic programming or palindromes in constant time or space.


As a different perspective: I end up writing a fair bit of code that involves data structures and algorithms. Runtime complexity does come up in my line of work, and I've definitely had to solve a couple of issues related to code being accidentally quadratic. So I do value basic Big-O analysis, and it's definitely nice to know some core data structures, especially arrays and hash tables.

But what leetcode and leetcode interviews optimize for isn't all that valuable. In the context of the job there are usually different space/time constraints than what you'll see in leetcode, and there are also integration constraints that can change/invalidate textbook approaches.

Most importantly, though, you always have more than 30-60 minutes to figure things out, and can consult whatever references you want.


I think if you don’t know it, you are blind to it. We had some AWS lambda function timing out and the person working on it kept adding more and more resources to get it to complete on time but it would inevitably slow down again. I took a look at it and realized that they were building a huge array and performing a lookup in it each time. Converting the array to a set immediately dropped the runtime and fixed the issue for good.


I once had a lambda that was running at like 10mins and timing out. It downloaded a bunch of data from an API, ran some statistical functions on it, and then uploaded it to a database.

I found that the biggest performance increase came from async downloads. By sending multiple http requests at once it went from 10 to 2min. After that, optimizing the queries sent to the API reduced the amount of data that had to be downloaded. Finally, I found the best way to bulk upload data to mysql was to use file based uploads which simply copy themselves into the database. Sending 500 inserts at a time was wayyyyy slower. Managed to get it down to 30s. I also switched from python to Java but I never measured the difference between the two


Their point (to me) was that leetcode was hard and if they knew how to solve leetcode, they would apply for FAANG companies, but leetcode is just data structs and algorithms


This reads to me as '90% of interviewers are incompetent at interviewing'. Most developers have no training in how to interview properly and little interest in learning, to most of them its a distraction and a bother.

Most developers tend to ask questions about stuff they found challening or interesting, without consideration for how this related to the day to day job.

Like asking a web developer to perform binary shift operations, idenity magic bytes, work with complex numbers, non-trivial question about geometry from people who have no reason to use geometry in this or previous jobs.

I had two interview stages at AWS, second interviwer asked the exact same questions as the first one, and became angry when I pointed out that they probably had a mix up.

Also you sometimes get poor communication from interviewer, unclear instructions, and interviewers literally making mistakes in recursion but telling the candidate they got it wrong.


Yes. Communication is the biggest hurdle in any collaboration. I have not been part of multi stage interview process, so I don't know all problems there.

But my point was more that if you find leetcode hard-to-impossible on difficulty scale, it probably means you are self taught and didn't bother with data structures and algorithms lecture/book/video/course.

Again probably bad communication on my part.

Our interviews is slightly modified fizz buzz. What we are looking for is basic programming skills and ability to test your code. For take home also basic git usage.


The material’s not the hard part. You can just study that.

It’s the social environment and performance, often sustained for hours, that’s entirely unlike anything else I do. It’s not like emergencies (great at those), it’s not like being in a meeting with a hostile client (good at that), it’s not like any socially-difficult thing I do in work or life outside of it, and it’s certainly not like normal collaboration. Programming while being watched and judged is horrible and energy-sapping in its own special way.


It feels like performing in a theater while someone is distracting you with math questions.

If it was like a normal exam with own and paper, that woupd be easier


I agree, but it's near-universal. Not sure what that says about our industry.


That most are incompetent organizations, I guess.


There is research into how an interview should be done to find who will be a good or bad employee. Most companies (not just our industry!) don't appear to have even looked at it. (I'm not sure how useful the research is for real world hiring)


Links?


I don't know how to search the academic literature to find it. I'm not even sure it is online to link. I just know it is done.


Google, Meta, Apple, OpenAI, Citadel are incompetent organisations. Wow lads


What if I told you any large organization can be incompetent? Not saying they are, but there becomes a point in a corporations life when it becomes full of people who flock to it due to prestige and pay. The original founders are often out of the picture and their original product is beginning to turn into shit.


I like this article a lot and have had great success with this tactic (as a hiring manager) over the years. I often give a page or two of code as a take home assignment to discuss at the interview. They get a chance to look at it and research it on their own. And they also get a bit of the crucible of face to face thinking and reacting on their feet while you discuss it with them in the interview.


In the year 2023 tech companies are not "buying" (supply > demand) but "selling" (supply < demand); that means the hiring process should at least be equal in terms of time spent on it. Candidates spend way more time in home assignments (hours to days), while companies spend in the order of minutes to review the assigments. We think this is "normal" because "the company is OFFERING you a job"... that couldn't be less true nowadays, it is "me the one CONSIDERING working for your company". A big difference.


That's not my perception of the industry right now. I think what you said was true two years ago, but not any more.


Well in my case, this code review was only two pages and it was only given to people who accepted an interview. So you had about a fifty percent chance of landing the job. Contrast this to the leet code dedication of a couple of years on the side effort...


Considering all the layoffs, the closed positions, and job listings suddenly skewing heavily towards "staff" engineers, we are in a buyers' market (ie. the employers have all the leverage right now).


why opting for an "home assignment" instead of doing that as part of the interview?


In my experience, most candidates prefer it. They get to use their own computer and own environment and nobody is pressuring them while they code.

It’s also most similar to the job, where I won’t be standing over their shoulder doing live coding.


Possibly because many quality candidates work / think better when not being watched. It is also likely far more natural for most people.


Better simulates the conditions of the job. I never have to write code on-demand in front of my boss with my job on the line. I get assignments and I do them asynchronously. Less pressure that way and I can do it right.


I think a company with a mature code review culture wouldn't be able to benefit much from this kind of interview.

The candidate wouldn't have to figure out what the code does, because there would be a PR description and/or walkthrough video explaining the code.

There wouldn't be any bugs to find, because code reviews aren't a great way to find bugs. The bugs are found by automated and manual tests.

There would be no stylistic feedback to give. The linter already enforced everything enforceable, leaving only things that are a waste of time to fight about.

There's no point in giving architectural feedback, as the architecture was already socialized and agreed upon before the code was written.

Now on to what code reviews are good for. The changeset is probably a small piece of a bigger picture, because the code is being shipped incrementally. The candidate has no comprehension of the bigger picture to weigh in on downstream issues.

The candidate has no broader understanding of the codebase in general, to suggest reusing an existing pattern, or avoiding a common pitfall with a private API.

The candidate can be well-versed in the OSS framework, and suggest better ways to write some particular boilerplate, or avoid a framework pitfall.

I fear an interview structured like this says more about the company than the candidate. If your company's development lifecycle is mature, there's not much a candidate can weigh in on.

The most useful case that jumps out to me is gaining an understanding of the candidate's familiarity with a framework. A contrived diff falling into lots of common pitfalls could cover more ground than having them write something in the framework. But having them write something in the framework shows you a lot more than just their awareness of pitfalls, so I'm not convinced this is better still.


Not sure if it would be any better, and I say this as someone who hates leetcode-related interviews.

Code reviews are way more subjective than writing a piece code (at the end of the interview session, if the code works, then that's a huge thing already).

If I highlight something like "Perhaps we could inject that dependency, instead of harcoding it": sure, that seems probably the most correct assesment, but again, what if we are dealing with a really small project and hardcoding the dependency is the best way? Same goes for variable names ("i" is too short? Depends on the scope, right? Well, probably it depends on the interviewer's mind), what about composition vs inheritance? Again, subjective as hell. And don't let me get started with anything related to microservices vs monolith...


I've come to the conclusion that any sort of job interview has a high level of subjectivity. Basically as an applicant, you roll the dice and hope that you have a good day and "click" with the interviewer.


I think job interviews are subjective and not a bad thing. Some objectively good candidates are not good for certain roles and vice versa. To me, your answer would be the best and would allow you to fit into more roles. You give options for a situation, rather than dogmatic statements like, you must inject that dependency, or the variable names should be done this way. But that is my subjective assessment.


I like this idea, but still think that time and people watching you will add pressure.

Most coding challenges that non-FAANG companies give aren't terribly hard; they mostly want you to work through your process out loud, ask questions, interact.

These soft skills are as important as technical aptitude in determining whether or not you'd be a good fit with the organization.


    > but still think that time and people watching you will add pressure
When I conduct interviews, I always try to set a conversational tone from the very beginning. Code reviews in general lend themselves more to a conversational interaction and discussion rather than trying to read, process, construct, and respond in a coding challenge.


It's admirable, but the interviewee is always under some kind of pressure to perform. if you are taking a conversational tone, they're going out of their way to be similarly nonchalant


But you will have people watching you work and you will be under time pressure in normal work as well, so if you just turn to spagetti at interview are you actually going to perform any better at day to day?


When was the last time you had to nail out a fully working nlog(n) longest valid bracket sequence in to prod within 45 mins, without spending 3 weeks in meetings, design doc reviews, capacity plannings and allocations and alignments with the said bracket owners?


meetings, design doc reviews, capacity plannings and allocations and alignments are not supposed to be used as ways to hide mediocrity


No one - literally no one - expects perfect implementation of anything in an interview, where are you even getting this notion? What actually *is* expected of you is to describe how you would approach the issue.


I literally never have anyone watching me code, and the time pressure of an interview is way shorter than a 2 week development cycle


So you have never worked in an office with other people?

And yes couple hours of interview is different than two weeks sprint, but you are very naive if you think two week of work days means 80 hours of work. Plus I expect a lot more from you during a sprint than during an interview. During an interview I am fine with pseudo code as long as you can explain what you want to do. During actual sprint I actually expect tested deliverables.


> So you have never worked in an office with other people?

I used to, but even then nobody actively watched me code. Everyone else had their own jobs to do

> During an interview I am fine with pseudo code as long as you can explain what you want to do. During actual sprint I actually expect tested deliverables.

So you're optimizing for the ability to write pseudocode under intense time pressure and social awkwardness. Is that the most important quality you seek in a candidate?


I've done something similar for product designer interviews in past roles and at my current company: rather than any variation of take-home or design challenge (which are unethical and unhelpful), we ask candidates to do a design critique with us centred around a third-party product of their choice.

It gives us great signals into the things we care about in designers: communication skills, design and product thinking, their mindsets and approaches to thinking about and evaluating products and design, and even their craft.


> For hard problems, I may go for a walk, talk to a co-worker, research algorithms, build small toy codebases first

Everything about the JIRA/timetracking/project management regime is expressly designed to prohibit that - you're a programmer, program! Faster! Know the answer RIGHT NOW!


He's on the right track, except instead of ditching LeetCode use it as a source of code to review. I think many people overlook that each LeetCode problem has an associated forum where people discuss the problem and post solutions.

Here's what I'm going to try if I ever again have to conduct interviews.

1. I'll present the candidate with a LeetCode problem (or let them pick from a list), after emphasizing that I'm not going to be asking them to actually solve it.

2. After they read it we'll discuss it, starting with the problem statement and examples, until we are both sure we think we know what the problem is actually asking. We'll also talk about any edge cases we anticipate might give us problems.

3. We'll talk about possible approaches to the problem.

4. After we have come up with some approach that with sufficient hand waving could convince someone that we could actually right a solution, we'd go the forum for the problem and start looking at the solutions people have posted.

5. For some of those solutions we'll try to figure out if they would work, if they handle the edge cases, spot bugs, see if they are well organized, talk about how well they appear to be written, and stuff like that.


> For some of those solutions we'll try to figure out if they would work, if they handle the edge cases, spot bugs

Most of the solutions will have been posted after submitting to the platform and passing hundreds of test cases, so you'd have to pick a very specific problem and solution to find any bugs/not handling edge cases.


Let’s break down the problem:

* hiring is a very high-stakes decision

* the normal interviewing process has no hope of directly measuring a candidate’s fitness for a particular position. There simply is not enough time. You’d want to evaluate the candidate over maybe a couple weeks of full time work, which, of course, a large number of candidates aren’t willing to do.

* So you have to use the limited time you have to get as much information as possible and accept that direct knowledge really just cannot be obtained.

The idea of leetcode exercises is that someone capable of the analytical problem solving skills and determination necessary to do well at those puzzles will also be capable enough to handle the harder parts of software development jobs.

It’s an imperfect approximation, but for a problem where there is no closed-form solution, it could be one of the better of a number of poor options. (That’s the idea anyway, I don’t know if it’s true.)

All that said, I think a code review is a good idea. Maybe do a couple leetcode puzzles and a code review (and a design discussion, and a “explain a recent project of yours to me” discussion, etc)


> Bug hunter

> Intentionally introduce some logical flaw or defect and see if a candidate can spot it. A good idea is to go back and find recent bugs that were solved and pull the source before the fix was applied. Can the candidate identify the root cause? How would the candidate suggest resolving the defect?

I've had this exact idea on the back-burner for years! https://news.ycombinator.com/item?id=18794465

The example bug that gave me the idea initially is this one I reported on the "Satsuma Graph" dotnet package: https://sourceforge.net/p/satsumagraph/tickets/2/

Very keen to hear from anyone that has experience with this, from either side of the interview fence.


I had a bug hunt as part of an interview once (or, I should say, once that I remember). It was different from that proposal in that the code was not part of the company's actual codebase - it was entirely fabricated for the test. It was fun, though; I enjoyed that more than livecoding and live-whiteboarding.


I had one that wasn't fabricated when I worked at Strip. There was an old bug in the requests library for Python where it was sending wrong data when posting a stream and the response had to be retried. The bug had already been fixed upstream.

We cloned a repo, ran they tests, it failed, and the task was to identify why it failed.

It was a tough process for me, since I've never really used Python debugging tools, so I actually had to learn during the interview, but I was able to reason well and picked things up quickly enough that I found the cause, and we had time to discuss ways it could have been fixed.

It was different. I do well during Leetcode questions, so stepping out of that framework was uncomfortable and stressful. I got an offer, but I felt like it was closer than it should have been.


Thanks for relating your experience. I'm sorry to hear that it was uncomfortable and stressful, but it sounds like you must have reacted pretty well in those circumstances to get an offer. Do you feel like the skills assessed by the interview were reflective of what you ended up doing on the job (assuming you accepted) ?


There are always going to be parts of life that are uncomfortable and stressful. This one wasn't terrible, and I appreciated a new outlook on how to assess skills. The interviewer was very understanding and we meshed well, so I don't think it came off as negative.

Were they indicative? shruggie. I mis-typed, I interviewed there, didn't work there. And I haven't used the python debugger since, but I also haven't worked on much python code.


I like this. In the last dev organization I worked with, we made an artificial practical coding problem, like adding a small feature to an existing purpose-made codebase. It was nice to directly compare the results of the same task among several candidates, and we were fortunate enough to be authorized to pay them for their time. All great but it was a ton of work... Making the challenge, doing code reviews, debating valid differences in approach or unexpected but understandable interpretations of the challenge...

I think the code review method could tell you a lot of the same things about a candidate and show whether or not people can issue criticism without being jerks.


For take home reviews, I wonder what the impact of generative AI would be. I can see a lot of people feeding the code segment into an AI and using it's comments instead of generating their own. Of course, people will do that with programming questions as well, so I suppose it's up to the hiring manager to decide on the best method to handle this situation.

What strategies do people leverage to avoid purely AI answers?


A good take-home interview process usually involves a review phase where the candidate talks about their solution and how it works.

This is remarkably effective at filtering out people who had a little too much help in solving it, either from friends, Googling it, or now using an LLM.

It doesn’t happen often, but from time to time someone will come in with a solution they supposedly wrote in the past week but they are unable to discuss it. “I don’t remember exactly what I did here…” and other excuses. It doesn’t take much discussion to reveal someone who never actually understood their own submission.

Of course, I always continue the technical interview to cover the odd possibility that maybe they were too nervous or something. So far I haven’t had anyone who is unable to discuss their own take-home yet shows well in the rest of the interview process.


Time will tell, but there optimists are already predicting that in just a few years great programmers will feed requirements into AI and then review the code it produces, not write code themselves. I'm not sure I believe it, but given that someone who can make AI produce good results is worth a lot more than a great programmer without an AI.


With a take home, and honestly any kind of interview, the key is not in the deliverable it's in the discussion about it. To be totally honest as a hiring manager I don't care if a take home was fully written by AI so long as when we discuss the solution the candidate can clearly explain how the code works and why it was written that way. Of course in the case of an AI the why might be flimsy but if it isn't then who cares. The goal is to hire someone who's capable of solving problems and in turn explaining their solutions I don't care who wrote the code.


I can't tell you how sick I am of doing technicals anymore for a job. If you can't tell from us talking shop and my resume and your due diligence i probably don't want to work with or for you.

The arrogance most software "engineers" have is astounding. 99% of companies aren't splitting the atom. Companies shoot themselves in the foot and waste so much talent this way.


I don’t get why people always hate on leetcodes.

Anyone can train and get good at it, it makes interviews short and not time consuming, it’s a standard accros industry so you can focus on studying it and not have to read one book for each company you apply to.

What would be a better alternative? Hiring based on connections? Take home assignments that takes 15 hours to complete? Domain specific knowledge grilling?


Because adults with hobbies and families have better things to do. How is this so hard to grasp? Bonus is that 90% of devs literally will not use most of that knowledge.

My biggest problems are getting requirements, understanding the problem, working around a legacy system or deciphering old code, not actually writing code when requirements and documentation is on point.


> Because adults with hobbies and families have better things to do

If you really valued your time you would spend a week studying for the interview that gives you a 50pct or more increase in salary so you can work that many fewer years


But it still feels wrong if progression is measured by an arbitrary measure orthogonal to what one actually does in one's career, no?


Leetcode isn't progression, it is changing to a better track. Progress on that track doesn't require leetcode, just entering it.


In the context of this conversation we are discussing Leetcode as the status quo primary means of passing interviews for new, higher-paying jobs at FAANG-equivalent companies, as career advancement.


To be fair grinding Leetcode to the point of proficiency takes longer than a week.

With that said I agree on your other point, putting in the work and sacrificing time, even months, will pay dividends long-term with a salary that allows someone to retire early.


If you are willing to spend your younger, healthier years grinding just to (potentially, there are so many things that are outside of your control that can go wrong no matter your work ethic) get a few more years of retirement at the end of your life - then sure. Grind away.


These interviews are not designed to require "spending your younger, healthier years grinding". They require grokking a fairly small set of concepts and being comfortable using them to problem solve under time pressure. That's it.

Hearing some people, you'd think they ask you to memorize chess openings or something.


That's because it is memorizing chess openings, or basically the same thing.

Most Leetcode Mediums fall into the category where the type of solution is obvious (sliding window/graph search/etc), and the code isn't hard to write, but there's a "trick" embedded in the problem and if you don't already know the trick, you're never going to figure it out in 30 minutes.

So the best strategy is to find a "Top N Leetcodes" list like Blind 75 and just memorize them. Of course you can't expect those exact questions to come up in every interview, but having a few dozen solutions already memorized should let you do some pattern recognition and lower your cognitive load while you're trying to figure out what that problem's trick is.


> Most Leetcode Mediums (...) there's a "trick" embedded in the problem and if you don't already know the trick, you're never going to figure it out in 30 minutes

I am sorry to say, this is just not true.

> but having a few dozen solutions already memorized should let you do some pattern recognition

Chess opening memorization is pure "remember the moves, play the moves" that's it. No thinking or pattern recognition involved.

If this is about being able to solve problems by generalizing from having seen a few dozen solutions, then I don't think "memorizing" is the appropriate term here, at all.


Do you have any evidence that isn't true? How often do truly new problems come up? Nearly every problem I see is after someone gets an an algorithm named after them.


While it is true there are many factors outside of our control I'm still going to prepare the best I can for the factors that are in my control. FAANG salaries allow someone to retire decades earlier which is the significant amount of time


You can work 80% and still earn more, so you have more time your whole life.


I never said I didn’t grind or I won’t grind to move up. It doesn’t change my opinion.


One problem is that they keep making the leetcode interviews harder because many people spend a lot of practicing. Simply solving a problem isn't good enough anymore. Now you have to solve two problems in 40-45 minutes with the optimal solution. The only way I've been able to solve two problems in 40-45 minutes is if I've seen the problem before.


Surely you'd find something to discuss about one's previous twenty years of building software. Especially if the last ten years have been in the same domain. Somehow it all worked just fine before 2010 or so. Also, one's tolerance for silly things and short-term memory for useless puzzles don't increase with age.

You know you live in a clown world when memorizing nonsense makes one more employable than learning some front-end stuff to be a little more full-stacky.


> Domain specific knowledge grilling?

Why would you not do this? If I’m hiring for a specific role, I want to make sure they are good at that role. Doubly so if their resume states they’re an expert at something - let’s find out.

This doesn’t need to be hostile or one-upping, but to prove someone’s actual depth of knowledge in an area they claim to have expertise in.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: