Hacker News new | past | comments | ask | show | jobs | submit login
Tell HN: Interview take home assessments without feedback are frustrating
338 points by shakes_mcjunkie on June 23, 2023 | hide | past | favorite | 479 comments
I'm in the process of interviewing for senior and staff front end positions. I've done several multi-hour take home projects now and a couple were rejected with generic rejection messages. It's pretty insulting to be frank. Spending 2-4 hours on likely valueless work is a substantial amount of time relative to the week day. If I've spent the time making the project and they've spent the time recruiting me and reviewing the project, the recruiter and reviewer could spend 5 minutes sending constructive feedback.

Anyone else share in this frustrating experience? Have you successfully asked for feedback before?




I took a take-home assignment from fly.io a while back; they promised that every completion would receive real human feedback.

As far as I remember, I was told it would take a few hours, maybe 4, but the assignment looked rather fun, so I thought, whatever, let’s do it.

Maybe I didn't understand the assignment fully or missed some cues; I don't know. But it took me roughly 10 hours spread across two days. After 5-6 hours, you don't feel like throwing the work away; you just want to finish. It was pretty frustrating.

After returning the assignment, I waited the two weeks they asked for and heard nothing. I sent one email, waited a week, got no reply, sent another, waited some more, and still received no reply. I ended up sending a handful of emails to various addresses I could find. I even sent a DM on Twitter to one of the founders to let them know. No reply anywhere.

Overall, it was a pretty bad experience.


Mirrors my experience.

The job listing promised mountains and that it will take few hours and they’ll take time in their feedback,etc.

except i just got a generic reply that “we’re not satisfied with the solution”. For something that took ~10 hours, i was at the least expecting a vague pointers on whys. I even sent an one pager explaining style decisions, caveats, etc. It was pretty insulting.

And when asked for a follow up, got a more generic bs about how the evaluation criteria is honed from their years of experience and that is not something they share outside.

One good thing that is it made me realise interviewing still sucks and i just stopped looking for jobs.


I went and looked at your submission, and I am comfortable with how we handled it and the level of detail we gave you. Would you mind sending me an email so I can hear a bit more? It sounds like we might've implied something we didn't mean to.

These should not take 10 hours, unless you're learning Rails from scratch or something (some people do this). We've tried different ways of saying "don't spend more than 2 hours on this unless you really want to" and it doesn't always come through.


The "detail"I received is the same generic reply you've shared in the other thread.

I don't know what fly's internal processes are, but I suspect the reviewer had to have submitted some bit of internal feedback on each submissions on why they're okaying it or not. A filtered version of it or even a separate candidate specific feedback would be just enough.

I've been at the hiring side of the table for many years and every time we share a handcrafted feedback to interviewees that we have passed on, we always heard back good things. They know we respected their time (and them), even if they don't agree with the particular feedback.

This feedback is not meant for them to get "better" or something. My point is, it could even be 2 hours the candidate spends, the least they deserve is knowing that it was actually looked at.


No that's not how ours works. We have, basically, 20 checkboxes for "things that are good about this work". These range from "strong user focus" to "logic to prevent <some problem we've previously had with billing>".

The submissions that we continue with check a bunch of boxes. The submissions that we pass on don't.

The problem is, we're not hiring people to build to a spec. We want to assess your decisions and ability to go from a basic problem to a first implementation. If we shared the rubric with folks, they'd focus entirely on trying to check the boxes and I don't think we'd get an accurate assessment.

Your point about valuing peoples' time is important, though. We have not yet found the right balance for everyone.


"The problem is, we're not hiring people to build to a spec. We want to assess your decisions and ability to go from a basic problem to a first implementation"

You could have told the candidate that, then. That would have been some useful information to have for the hours the candidate put into their submission. Or like "hey, we're going to just skim over this with a subjective list of check boxes".

I have enough experience that I probably would have smelled this before I got that deep into the process and noped out, but for the hopeful mid-level/new-senior devs this sounds pretty demoralizing.


I’ve worked with, and been on hiring committees with people like mrkurt before. What always happens is they reject a bunch of candidates toward the beginning of the process, then eventually the time comes where they MUST hire someone. Because someone or something they are accountable to (investors, their boss or commitments they’ve made) asks why they can’t hire. Then there is a mad dash to interview and hire someone where standards are greatly reduced. They then end up hiring someone with similar skills and risk profile of people they have previously rejected anyways. The net result is just a bunch of time wasted for everyone.

If you ever get asked to do one of these exercises, it’s useful to try to determine where they are in this process. Ask about their hiring timeline, how long they have been interviewing for the position, and try to get a feel for how fatigued they are in general.

If it feels like they are just starting, ask how long you have to complete the exercise. If there isn’t a time limit, it’s better to wait as long as possible, so they can reject other candidates first and burn themselves out. If there is a time limit, ask to pause the interview process (insert some excuse here) before they send out the exercise.

If they are near the end and sound exhausted, that’s a good sign and the effort might not be wasted.


Sound and practical advice from a fellow grizzled veteran that I will put to work my next go-round. Thank you, sir.


You know, "take-home" projects are so cargo-culted, and so poorly executed at so many places, and in such bad faith, that this I think turns out to be perfectly reasonable, valuable survival advice, even though it applies literally not at all, in any way whatsoever, to the hiring process you're commenting on.


Submissions are supposed to predict and handle issues that you guys have had in production? That doesn't seem very reasonable for a 2 hour task.

I wonder if your hiring process is skewed by the perfectionists who spend tens of hours of their submissions. Arguably that's the type of thing you guys would want to select for but it's not at all fair for those who timebox themselves to 2 hours or whatever.


We're actually selecting for experience. One way submissions demonstrate experience is if they account for the big issues that come up in real life.

I don't think you can spend more hours and get these criteria.


That's exactly how it works. Spend more time, cover more edge cases.

This sounds like a classic "my developer reckons he can do this in 2 hours (but never actually has)". The reality is your developer is crap at estimating.

A good developer will be lucky to produce 100 lines a day. if you doubt that check your git history (excluding autogenerated crap).

So unless the project you're setting people needs, on average, 25 lines, whatever you're setting people is clearly not a 2 hour project. And that's not even taking into account whatever hurdles you've inadvertently put in the task, strange build config, obscure libraries, out-of-date libraries, non-standard formatting, etc.

Could you tell us what the average length of a submisson is?

One task I got given a year or two back had an old version of Vue, linting rules that conflicted with the defaults of the 'usual' IDE you'd use, wanted you to create 3 new backend API endpoints and a new frontend page with non-trivial functionality. Plus they wanted units tests.

That's a 2 day job. They also claimed it would take an hour or two.


Sorry, this smells bad - It sounds a lot like the old "We've just solved this really tricky bug that nobody knew how to deal with - it took us a month, but we're experts now - now you do the same in 2 hours"


Even a very experienced engineer might not have encountered this specific bug/situation that you’re testing for in your take home assignment. How do you make sure that you don’t filter out good engineers by testing them on a situation they haven’t encountered in their career?


Presumably the answer is "we don't, because we're ok with filtering out a few very experienced developers if that reduces the risk of a bad hire".

It's pretty rare IME to encounter interviews which are closely related to the actual work you'll be doing, mostly it seems like the main purpose is to filter out the heaps of bad/mediocre developers.


> The submissions that we continue with check a bunch of boxes. The submissions that we pass on don't.

What detail is shared about what "checkboxes" the candidate met and didn't meet? From the parent post and your reply, it doesn't sound like much at all?


Woah, I've been a hiring manager long enough that it's been a while since I've done a take-home code exam myself but I don't even think I would grade a LLM that way (because I moved to AI now). Unless your code exam is super trivial or the boxes themselves are table stakes ("code runs without errors", "code includes more than one function"), coding is creative enough that it's hard to come up with 20 checkboxes that cover whether a sample is any "good" let alone "shows better decision making".

I've had a few bad experiences when sharing feedback with candidates myself and I would understand doing the checkbox approach for feedback and/or just never sending detailed feedback, but actually grading submissions pass/fail based on a subset of criteria you jealously guard from candidates essentially selects for lucky people. If I wanted to do that, I'd just shuffle the submissions by number of bytes and discard everything that's a multiple of 5 or something.


> Your point about valuing peoples' time is important, though. We have not yet found the right balance for everyone.

Did you try paying candidates for the home assessment? People's time costs money. Paying people not only helps attract candidates but also helps the company reduce the list of candidates to take the assessment.

I'm inclined to say the interview process is imbalanced with more power in the hands of the employer. If you're looking for the right balance, try turning over some power from the employer to the candidate.

The candidate's cost was 2-hours worth of time. Was the employer's cost 2-hours worth of time?


shouldn’t take more than x hours - if i had a dime for every time i heard that…


If it takes more than 2 hours, there's a pretty good chance it's the wrong job for the person doing the challenge.


Presumably you can't tell if the applicant spent 2h or 6h, and in practice the 6h solution is probably going to look much better. So sure, in theory you may be right, but in practice I bet you select for people who spent (much) more than 2 hours.


And for every company that means it there are five others want to find people that will sink massive amounts of time into these so they can do the same once they start.


The poor communication roughly mirrors my experience there as well. I had expressed interest in one of their infrastructure operations positions and was given access to the repo with the project after a wait of about a week to hear back. Due to some life events I wasn’t able to start looking at the project in earnest for about another week, whereupon I had some questions. So I sent questions to the individual that originally contacted me. Never heard back.

Was it my issue waiting a short time to start it? Maybe, but you’d think they’d at least try to answer a question or two. Oh well, I just gave up because if they weren’t going to put in any time, why should I?


It's hilarious watching the CEO here trying to do damage control in a highly visible setting after so many burnt programmers emerge to tell their tales.


I hope my responses don't come across as damage control. Talking through this stuff is super valuable, we tweak our hiring process almost continuously based on feedback. Sometimes when people have a bad experience, we've just fucked up. Other times, we set the wrong expectations and there are simple improvements we can make.

I doubt anyone will read frustrated comments and my responses and think "oh boy, I didn't want to apply there but that one guy responded and made me change my mind".


> Talking through this stuff is super valuable, we tweak our hiring process almost continuously based on feedback.

Isn't that ironic, indeed hypocritical? You take feedback from job applicants, consider it super valuable, but refuse to give anything useful to those very job applicants.


I'm not completely following. Can you be more specific?


Sounded pretty clear and specific. You value feedback on your interview process but don't extend the same courtesy to applicants who would value feedback on their rejected application.


"We don't really give individual feedback on the take home challenges." https://news.ycombinator.com/item?id=36458881


I think people read it as damage control because you're trying to address the feedback while arguing that you guys aren't wrong. That's pretty typical damage control behavior these days.


Yeah it probably does sound that way.


Between your responses here and the treatment of candidates that appears to be pretty common, Im a big no on ever applying to your company. Your hiring process is broken.


Everyone thinks they’re going to give feedback because “we’re different, we’re going to do this right”. Then life happens and that shit gets dropped by the wayside.


We don't really give individual feedback on the take home challenges. Where we struggle is explaining this to people, it's not obvious to every dev _why_ we don't give specific feedback. Here's what we send:

---

Unfortunately, we’re not going to be moving forward with your application for the backend role at this time.

The first thing we want you to know is that we got a lot of applications for these roles. Like, a lot a lot.

The way we evaluate candidate submissions is that we’ve built, over a year or so of running this challenge, a written rubric for what the strongest submissions look like. We started with something sane looking, and then iterated over time as we got a sense of what candidate submissions actually looked like.

We score submissions according to that rubric. Different members of our platform team score different applications; they’re just jumping in and looking at the code and making grading decisions. In the pool of candidates we’re evaluating right now, your code submission missed our cutoff.

A natural thing to want from us next is a copy of the rubric, or specific information about what your submission was missing. We’d want that too! But we can’t give that to you, for (at least) two reasons:

(1) The outcome on your code was a weighted combination of a bunch of different factors, so there isn’t a simple answer that doesn’t just dump our whole rubric to you.

(2) We want to preserve the opportunity for people to apply to these roles in the future, and spoiling the challenge would break that.

What we can tell you is that the most successful submissions had a combination of these attributes:

- Heavy user focus. Successful submissions explained how the system could do good things for users. - Straightforward database models that make the queries we care about fast to run. - Transactionally safe sync with Stripe. For example, the best submissions mentioned that Stripe API requests had to be idempotent to ensure the user is not charged multiple times.

We sincerely hope you stay in touch and re-apply in the future. In the meantime, we'd like to give you some Fly.io credits to play around with! If your email address here matches your Fly.io email, click here to get them. If not, please let us know which email you use for Fly.io and we'll set that right up.

Please feel free to reach back out to us about other roles, or about this role in the future. Thank you, again, for doing this.


Seems like you probably are not explicitly stating this policy beforehand? It's not very surprising that explaining why you don't do as an applicant might expect afterwards is not going well. From the Github examples you've linked earlier, right at the end:

> When you’re ready, let us know and we’ll schedule it for review. We review submissions once a week. You’ll hear back from us no matter what by the end of the /following/ week, possibly sooner if you submit early in the week.

+ 1 sentence expressing "we will tell you if you passed or failed, we have a policy of not providing actionable feedback"

> Remember, we are not timing you and there is no deadline. Work at your own pace. Reach out if you're stuck or have any questions. We want you to succeed and are here to help!

+ 1 sentence "but make sure to not spend significantly more time than X hours"


Are candidates made aware of this policy before spending hours on your toy project?


I think that your heart is in the right place with a message like this, but my feedback would be to keep it shorter, perhaps to a paragraph or two. There is a phrase called “the message is the medium” and I think it applies to hiring.

The best way to reject a candidate IMO is to call the candidate, deliver short feedback over the phone “Some of the feedback we got from the interviewers was we like to see the problem solved in a shorter amount of time/ less space complexity/ whatever). If I was on the fence, I would them know that if they want to try again they can give it another shot in 6 months, a year, whatever. To me this seems like the most humane way to do it (This was actually my experience with Google, maybe 8-10 years ago).


You know what, that’s actually fair enough.


> Then life happens and that shit gets dropped by the wayside.

Correction: It's not profitable.


Companies also start worrying about getting humiliated by their interviewers.

The less feedback they give the less embarrassing it is likely to be.


Replace “humiliated” with “sued”.

At the core of everything people-related a company does or does not do is the fear of a lawsuit, and in the case of interviewees, the only thing that leads to a lawsuit is telling the interviewee anything at all - therefore, no feedback.


No, replace "sued" with "humiliated". Every time. It just sounds more prudent to say "legal risk" than "we're worried a member of staff will make us look like absolute morons" or "we're lazy". It's obvious when you think about it.

"We didnt hire her because she seemed like she might get pregnant" isnt the kind of feedback most people write anyway and if they did a 2 minute pass via HR is enough to eliminate the risk.

The companies that are actually worried take pains to tell you what not to say during the interview. This is rarer than you'd think. Most companies aren't actually that worried about getting sued over interviews. That doesnt stop them from tossing this lame excuse at you though.


This is the only correct answer.


I agree with this, a LOT of programming assignments have numerous effective approaches to a solution, and it's quite often, I believe, that the interviewers are simply unfamiliar with the submitted solution and don't know how to evaluate it or what to look for, not that it doesn't work.

And of course, this is more likely when the interviewee submits a more advanced or higher-level solution, meaning the best strategy is to "dumb-down" the submission rather than trying to use the latest language features.


I did a fly hiring project and was simply told I wasn’t good enough to move on. Zero explanation as to why. I found it very frustrating too.


Similar experience with DuckDuckGo - except I didn’t get past their bullshit first part which was “write 7 pages describing a project you designed in the last year”.

They claimed it would take only a couple of hours but for the 15 or so bullet point requirements that was simply not realistic. Spent a couple of days on it and the only feedback I got was no thanks.

They had previously stated they pay for every stage regardless of outcome. After sending my details they never paid me.

Feels vastly hypocritical given their stated people first work culture.


Had a similar experience and made it to the 2nd stage, and also got paid for both.

I even took the time to learn enough Perl to write the solution. They said it wasn't required but would be a nice touch. It was an interesting project to work on and it was fun coming up with a solution. I thought it fulfilled their requirements pretty well, but also just got a short "no thanks" email.

I was left wondering for a long time what they didn't like about it..

Someone suggested they may use these interview submissions as cheap labor to get ideas to solve internal problems. I thought that was ridiculous until I reread the disclosure I had to sign at the beginning:

> [DDG] will be the exclusive owner of all right, title, and interest in and to the work product resulting from the project, including all intellectual property and proprietary rights.

Could have just been boilerplate disclosure stuff, but seems a little weird.


It’s sad seeing so many comments saying they got paid while I didn’t! I was interviewing for their new privacy based browser, it was a C#/.NET desktop app role.


If you have documentation of that, you might enjoy suing th om small claims court.


Is there any point though really? For 100USD while I’m in the UK probably adds even more paperwork


> Is there any point though really?

I really meant the "you might enjoy" part. I wasn't suggesting it's an efficient way to make money.

Some people are really bothered by misdeads going unpunished.


We do pay people at every stage regardless of outcome and so not sure what happened. If you email me I’ll figure it out though.


I had the same experience with DDG, but I will say I did get paid the 300 dollars or so (by PayPal) they promised.


Lucky!


You should just share it or create a "leetcode" site for takehome assignments!! Shameful that these companies think so little of your time!

Don't get me wrong if you had fun (and I enjoy them too) nothing wrong with that.


I just add them to my open source portfolio on github and make it look like I love side projects

I also use them as starting points for the next take home assignment and even live interviews (I’ve been able to keep the project open in the IDE on a separate monitor and copy and paste code even when doing a camera-on + screenshare interview)


Nice. Keep up the good fight mate!


This is the one: https://github.com/fly-hiring/infraops-challenge

Here's another: https://github.com/fly-hiring/platform-challenge


Damn, I thought fly.io was better than that.


We're not. We've had issues losing track of candidates over the last year. This is one of those all or nothing problems, a 0.5% failure rate is pretty bad for many human beings.


Come to think of it, the only material that said that was their own.


That's very frustrating, I'm sorry you had that experience. The most time I've spent is ~4 hours and I've definitely gotten to that point where I'm like I don't know if it's worth it but I should just finish it. And yea, to not get anything back from the reviewer/recruiter after that just feels bad.


Hi Martin: I'm really sorry we ghosted you. That's not good, and it's happened more often than I'd like. We struggled with this in March, especially, because we were dealing with reliability issues.

I'll make sure we get you a response within the next week.


Had a similar experience in Klarna, but I they returned with feedback.

Feedback was something like "Assignment is private and you can't store code on GitHub, you should send zip archive"


I had pretty good experience interviewing with Fly, on the other hand.

The most fun task I had so far, the feedback was pretty good as well. Kudos to tptacek for that.

Maybe they had bad experience with sharing feedback directly?


I'm not sur how it would excuse anything.


Nobody here is asking to be excused.


leetcode costs you at least 20 hours per month and you don’t create anything you own


If you want some feedback email me and I’ll check your work. I don’t work at Fly.io but I have over a decade of backend experience including Google, Apple, etc.

If they said it would take 4 hours and you spent 10 hours on it, it’s likely your approach was ineffective compared to a more skilled engineer.


In my experience, the number of hours the company suggests is always BS. It takes 2–3x as much time to experiment and then polish your solution. It might take the states amount of time if you’re the person who came up with the problem and knows how to solve it right away because you’ve put much thought into it and have seen dozens or hundreds of solutions already.


The context is also radically different.

Let's say I come up with an assignment and I pass it to a junior to see how long it takes. The junior does the work, likely reports the time rounded down, but also with no stakes. Jim job is secure.

As an interviewee you're gonna spend time polishing the code. Every variable well named, lots of comments, good functions, well considered parameter order etc. It's gonna take longer because , for you, the stakes are higher. You'll likely build some code to test it, and so on.

This is before we discuss domain knowledge that the junior had, and also that the company implemented the task in the first place and didn't just spitball the time expected.


Yeah I’d be very surprised if even half of take home assignments were “tested” on juniors vs just having time estimates spitballed. You know, like every dev does when asked “so how long do you think this ticket will take to implement?”


With the unfortunate difference that when estimating a ticket you will have to complete, most people have learned to give safe over-estimates, but they don't want to look bad when a prospective hire complete the job in 1/4 the time estimate...


When I've created these sorts of take homes I always do them myself and have at least 1 other person on the team do them. My thinking was that we should be able to do them in 1.5 hours if we tell the candidate it will take then up to 3 hours. I would give my own time to complete it as the expected time for a candidate.


You can't create the take home yourself because you're drawing on your own personal domain experience.

For example I'm an expert on real time video streaming in embedded systems. It's trivial to me to program a real time system that will stream video at a lower resolution in real time from scratch. I pass that problem to your average backend web developer they're going to take 100x longer then me.

The issue is, while it may seem obvious to you that the domain is too specialized, when you yourself spend to much time in a specific domain you become biased and you start to think such knowledge is basic because you get too good at it. It's invisible to you.

Time shouldn't be the factor measured here because people have such varied backgrounds.


The take homes I created in the past were not so extremely domain specific. They were relatively common problems like "read these JSON files and generate a tree of dependencies between them based on references, where file1.json might contain an item like 'file2.json'".


If I'm hiring for a real time video streaming platform, I probably don't want to hire a backend dev who has 0 experience in the field and takes 100x thr time, so this seems fine.

The point of the assignment isn't to make it "fair" (in some loose sense), but to filter out candidates.


True to some extent, but you usually don’t want to filter out all candidates who don’t have extensive experience in the exact domain you’re working in, otherwise you’d never hire anyone.


It is also going to be very hard to stand out on them without taking extra time because like it or not for a top company the competition is going to be spending that 2-3x time. At least in a technical interview the problem is time boxed so that the other solutions you are being compared against also only got 45 minutes.


We design these so people with relevant experience can bang them out in about two hours. They're not trick challenges, though, they're very straightforward. Getting people to believe they should only spend two hours on them is the challenge.


It's not BS in the sense they are lying. It's BS because they're wrong.

The reason they end up getting it wrong is usually because they pick something domain related to what the company does. So these guys literally work in that domain every single day of course they know it in and out.

When they throw it at someone completely foreign to the domain it's going to take a longer time to figure it out. Most interviewers just lack the ability to see this.


Often the difference in time is simply whether you’ve seen a similar problem before or not. I wouldn’t jump straight to suggesting someone is less skilled. You can also end up spending more time on something simply because you’re trying to stand out or have some reason to believe more effort translates to a better results. If you’re a test driven fan, simply making your code more testable and adding meaningful tests can add time.


> If they said it would take 4 hours and you spent 10 hours on it, it’s likely your approach was ineffective compared to a more skilled engineer.

They promised this person if they completed it, they would get an actual human response back.

Not only did they not get a response back, they didn't get a response back from anybody, even when directly trying to follow-up.

And this was what was promised for spending 4 hours of your time on something.

At the very least, that seems unfair.


I'm doubtful. In my experience the hour requirement hours number is under estimated.


it is a nonzero probability that they are full of shit


One company I interviewed with told me to spend no longer than 2 hours on a project. I complied and followed that time boundary, as I wasn’t paid for the interview and I didn’t feel it was fair to me to spend any more time on it than that.

Long story short, the company passed on me, saying that my code was easy to understand and high quality, but I didn’t handle all of the corner cases. I listed most of the corner cases they mentioned in the readme as limitations from the time limit. It was clear that while they said to limit my time on the project, they didn’t really mean it. They wanted an exhaustively finished product in a ridiculous amount of time.

At this point in my career, I’ll just decline interviews that don’t respect my time.


I once had a recruiter send me the project template in a .zip for the take home with the instructions to take less than 3 hours (the email wasn't timed to start right away either). No server with countdown, no forking from github, just download this zip and then email back in less than 3 hours. I thought to myself: will they check the download time from the share drive (if they even can)? will they just pass me by default for starting one day after the email delivery? Open up the zip. You need to add 6 endpoints, 2 of those are some fairly complex aggregate queries and I had to take a quick refresher on the legacy ORM they were using. For the frontend, bootstrap a react app from scratch and implement 6 flows (some even required you to go beyond the api tasks). They even encourage going above and beyond and adding unit testing and some integration tests for the UI! So then it was obvious the metagame was cheating and you'd be compared against people cheating. Are people sending edited git history? doing it in groups? what was the catch?

I obviously noped out, but am still wondering if they are aware that all the people working there 100% cheated on their take homes.


I always do exactly one commit with a commit time of exactly when they sent me the take home. If I actually had to fork something I would create the Github account specifically for just that take home and fork it right before pushing. Creation and modification date of files are the same.

Either they don't care or they see it and figure I must be some sort of whiz. I say that because the amount of people that don't know how git works or that file attributes are arbitrarily changeable is amazing.

And no not specifically to "cheat". Just on principle. If they require BS I give them BS. Else it's just fun. Sometimes the people actually hiring/doing the interview are also not the ones that care for this but HR/senior leadership only.


  always do exactly one commit with a commit time of exactly when they sent me the take home

  If they require BS I give them BS
So, that's editing git history to fit a narrative. I'm not judging though! I guess it really depends how bad you want the job after all. But personally I wouldn't even agree to a process like this that ensures cheating is the only way to be competitive. No company that does this is paying nearly enough to make up for the possible reputational damage, and not counting the hazzle.


What do you mean editing? I only commit at the end. I happen to set the date and time explicitly :) For all anyone knows my timezone was still off from traveling when I committed.

And yes, lots of companies have an interview process that state these things. The process makers are not the same people as are interviewing/going to be your team. I know from the other side where I have in the past internally had to advocate for proper take home practices. They really wanted to extend the take home and to say "should take 4-8 hours" (meaning really it's 2 days of work) . I told them that's BS and I wouldn't be with them right now if that was the practice back when I interviewed.

The worst thing that can happen is that they think you are cheating and they never even call you back. Bullet dodged. The best thing that can happen is that they do call you in for an interview and the interviewer tells you that you're so far the best sport and apparently know your git very well and you bond over it.


> I always do exactly one commit with a commit time of exactly when they sent me the take home.

Amazing I'm mixed on companies looking at start and end times. I think it would be ridiculous for them to actually check, but at the same time I'm curious how many people are going way over the time limit. I wonder if I should go over the time limit or not.

> And no not specifically to "cheat". Just on principle. If they require BS I give them BS. Else it's just fun.

Maybe this is controversial to say, but in reality, we're all "cheating" because we have to.


they can definitely enforce a strict time limit by only showing the exercise on a website that starts a countdown and telling you so. I'm okay with that. What I'm not okay with is being ambiguous about this and expecting me to game-theorize the risk-reward ratio of competing with people cheating.


I would rather feel the metagame is that you are being free labor for an actual problem/task they are having in the company.


I had a company ask me to put together a performance improvement plan involving proposed changes, instrumentation and engineering changes. They expected me to look at all their existing code, talk with engineering and other employees and put together this plan in 6-7 hours, an already ridiculously large time frame. I told them that allotted time was way too small for what they wanted and withdrew my application. It felt like they wanted me to put together a plan they could implement without paying or hiring me. Worse, I'd be working around their schedules to have these conversations so it wasn't really a take home assignment.

I wish I had saved a copy of the assignment. It was ridiculous.


So was the company the best in their field? I sometimes see small / badly managed companies pull such interview processes and it is ridiculous. You would expect they would hire 100x engineers with these tasks but I wonder what they are exactly looking for.


My theory is that they’re looking for compliant, docile employees who won’t have the courage or the liberty to refuse unpaid overtime demands or complain about shitty stressful work conditions.

By having such an abuse and convoluted requirement for applying, you’re selecting for people in this situation. Then all you have to do is hire the best on the technical solution and you’ve got one more employee you can abuse in your company :)


No. They were just starting out trying to sell their open source product. In fact, I had interviewed with a different company solving the same problem whose tech I thought was way more impressive.


Had the exact same experience. I actually got the role, but the feedback provided asked why I didn't build out something fully featured, despite them stressing to me the importance of doing it within 4 hours.

The worst part was that after working there I found them to be terribly unproductive and it would have likely taken a team there a week to build what they were suggesting...

But yeah as you said, either ignore the time constraints if you really want the role or just politely decline due to the red flag / inconsiderate nature.


It’s self reinforcing. Other people spent more than 4 hours on it and lied, so their expectation of what can be achieved in 4 hours is warped. It’s really just a way of selecting for liars, although that goes for most interview tactics.


well, obviously you failed their test but you were the best option they had.


I'm in FE, and FE interviews can be like this so much. "Here's a takehome for something that in reality would take a week to complete but do it in an hour and don't spend more time on that!"

And I'm like.. an hour is not long enough to implement any reasonable webapp... I have no idea if other people went over or not, and it almost feels like a test of commitment (if I was serious I would take the actual 4 hours it would take and pretend I did it in 1 or something like that).


Yeah even a basic to do list or calculator web app can take several hours if it’s your first time with the specified framework, build tool or CSS library.

Given the diversity of FE it’s quite likely for a take home project to hit one of those criteria.


> ... if it’s your first time with the specified framework, build tool or CSS library.

Depending on the position they may been testing specifically to filter people out who are using a tool for the first time. If I'm hiring a Senior Flutter Developer then it better not be their first time using Flutter.


Unless we're talking about something truly enterprise (Magento at 4 million lines of PHP dependency injected MVVC hell comes to mind) or very obtuse (hiring ocaml devs to work on their compiler) senior means senior engineer, I don't care what you worked on to be honest. Senior perl engineers can flip to ruby in month and be totally proficient. A good Java programmer with 10 years experience will have plenty to bring to the table in C++ .

If you are a senior developer and haven't seen half a dozen frameworks, tool sets, or languages under your belt. What value are you in 2 years when we shift gears


I don't know about Flutter, but my issue with these takehomes in a React world is that React is extremely basic. It is only a view renderer, and leaves up to you all difficult questions of: routing, CSS, components, data fetching, state management, even things like how files are organized and named are up to you. As a React candidate, your takehome probably touches on all of these topics, and you (rightfully) probably want to express some kind of proficiency in most/all of these areas. The more senior you are, the more likely you have opinionated ways of doing these things. But in an hour long span, you are left with two uncomfortable options:

1. Write it all yourself. Express those opinions as clearly and bug free as you can. Try to get it all in as much as you can and give up on finishing all the features in the spec. Definitely give up on the UI looking nice -- despite what anyone says, default HTML is not pretty.

2. Use OSS libraries. Are you actively up to date on OSS libraries handling these things? If you're a working React professional, you probably are not, because you're using whatever system your company uses. Do those OSS libraries properly express what you think is good code? And what does it say about you to use those OSS libraries? Do you look like someone who actually understands React and understands those tradeoffs, or do you look like the kind of person that creates the type of React app that HN is always complaining about, one with a billion dependencies?

Sorry if I sound ranty. It's not directed at you. I've just been dealing with a lot of these this week.


the worst part about it is that when they give a time limit they make it sound like they're doing you a favor.

"we have a takehome, but don't worry! it's just a quick little thing that should just take an hour to throw together. we're not like those other companies that give you a huge task"


Exactly. Then I've been wondering, how many people are actually going over the time? Are the reviewers actually checking any time stamps?


What can they realistically check?

I see two data points they could have: time when the template was downloaded (assuming it’s a unique link only you can access, and you didn’t download it > once/all downloads are logged) or emailed, and the time it was submitted.

All other time stamps can be spoofed.


No, I agree. I just mean like when they say, hey this will take 2 hours, how do they know? Do they even care? Or just everyone cheats and does more than the allotted time?

I did one take home that was timed in coderpad. I thought that was a little more fair.


They have no idea how long they take. They come up with a task first and then slap a time limit on it afterwards so they don’t feel bad about asking too much.


This is explicitly what bootcamp graduates are asked to do. It’s possible, they were just filtering out candidates that can’t work that fast.


Same happened with me. Funny enough the other candidates solutions were shared on GitHub. Two funny aspects:

1. Other candidates were clearly spending 8-9 hours on the problem. I saw their commit logs. A commit every 1-2 hours for 9 hours. Several candidates public repos were like this. The assignment was like 2 pages of requirements supposed to be done in I believe 1 hour.

2. They told me (thru the recruiting firm; I never spoke with them) that it would have been resourceful to use the other candidates solutions.


"resourceful"... Wow I'm impressed by the level of bullshittery. I wouldn't want to work as an engineer in a place where honesty and following the rules is a detriment


"works well with others ... code"


Paid interview FOR THE WIN.

> ... as I wasn’t paid for the interview and I didn’t feel it was fair to me to spend any more time on it than that.

I have been paid for an interview, a coding interview where I fixed a bug and added a feature while my pair (future colleague!) chilled .. I think he surfed and I'd ask him clarifying questions occasionally? Anyhow, I got paid like $400. Best interview ever.


That's amazing. I got a $30 gift card for one interview. It wasn't much, but TBH in the interview gauntlet I've put myself through it was nice to get even the most basic recognition that my time is money.


Hey, that’s still a win. It sounds like they respected your time and knew that you’re a rare bird.


> At this point in my career, I’ll just decline interviews that don’t respect my time.

This is my philosophy now. In my younger days when I had no experience I had a lot of my time wasted by recruiters and companies - never again. Now I ask up front and politely what the interview process will be like. If there are any home projects or mentions of whiteboarding I end up passing. No big deal - I know there are probably loads of developers willing to waste their time competing for the position, but not me.

It's worked very well for me and eliminated a lot of stress when I've been looking for other positions. The only negative I can see is the strategy wouldn't work great when in between positions and really needing something, which luckily I've never had to experience. I suppose in that case I would force myself to go through with the hazing process.


Was this us (Fly.io)? I'm not sure how many companies give the 2 hour guideline, but we definitely do.

We don't fail take homes because of edge cases though. There are things we expect people to handle in the challenges that are important, but I don't think more/less time really influences those.


Please don’t tell me that the compensation ranges I’m seeing for fly.io are accurate - between $130k - $170K.

Those are standard enterprise CRUD salaries. I’m not trying to be demeaning. Those are the kinds of jobs I had most of my career until 2020. But they are a dime a dozen and I definitely wouldn’t do a take home test for one.


I've been job searching for the past few months. Advertised salary ranges seem to skew lower than 6 months ago.


Don’t get me wrong. I’m not so deep in the tech bubble that I don’t realize that $120K-$170K is the the standard range that most of the 2.7 million developers in the US make. I was one of them for decades until 2020.

But at least from around 2002- 2008 and 2012-2022, if you have experience in a mainstream stack, and you live in a major metropolitan area in the US you could easily get a job that pays in that range without even coding interview, let alone a take home test.


Most of our jobs are ~$120-215k base salary. The equity compensation is typically 3-4x that.


I value “equity” in a private company at $0.


Yeah you should not work here. That's a completely reasonable stance.


From the data that I could find, it's perhaps at best 10% of all tech companies that even go public (and very possibly as few as 1%). It's far more reasonable to assume a company is the 90% or 99% of companies that do not go public rather than anything else.

My experiences are the same that equity in private companies is not significant. Worse yet, in all of my experiences the private equity I could have bought or did buy were all 100% losses.

[1] https://www.quora.com/What-percentage-of-Tech-startups-eithe...

On a different topic, I think your response mrkurt is a violation of the commentary guidelines:

- "Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. Comments should get more thoughtful and substantive, not less, as a topic gets more divisive."

- "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."


He was responding to me and that’s a very reasonable response. When I was in the market for your standard enterprise CRUD Enterprise dev jobs, I would have taken a cash compensation of $150K - $160K and zeroed out the equity.

What I wouldn’t do is jump through any hoops like a take home test when jobs in that pay range are (were? I don’t know what the current market is like) a dime a dozen and I could just do one phone screen and a soft skills interview for half a day and get a job making that much.

If a company expects me to jump through hoops that take more time, the resulting reward need to be commensurate with that.

But not all jobs are for all people.


Oh, I guess that could sound snarky. I wasn't feeling snarky, though. We are the wrong place for a lot of people to work.


Yes, I had a very similar experience.

Honestly 2h from scratch is a rush to do anything? However simple it is, you're asking for an 'entire' greenfield project, of course I have skipped things and missed corner cases.


Especially when bootstrapping takes the most time on projects. 2 hours to code up a game of War, ok. 2 hours to spin up a useful laravel site? Ehhh


In the past when I’ve given take homes (I don’t anymore) I genuinely would have wanted you to stop after the time limit. It would definitely mean that you weren’t the right candidate if you can’t do it in the time limit, which is OK! It’s meant to signal that it’s a bad fit. I wouldn’t want you feeling like you have to work after hours in the actual job, either.


Takehomes have been ruined as a concept by companies that spam them out before applying a first pass.

I’ve passed interviews at demanding companies like hedge funds, Google, Facebook, Snap. Not once have I “passed” a take home interview even when I know I got the solution correct. Sorry, but I refuse to do them now. Companies need to be able to show a token investment in time with an actual engineer interviewing me for me to be convinced they are actually considering me as a candidate and didn’t just send me a takehome link without any due diligence because it costs them nothing to do so.


I agree. I'll give you like a 4-to-1 ratio on a takehome assessment, give me a 30 minute phone call with my potential manager or fellow programmer and let me ask some questions and get a feel for the company and culture, then I'll spend a couple hours on your assessment. If not, well, same rate applies, I'll spend 4x0 hours on your assessment.

A nice way to say this is, "I'd be happy to complete this takehome assessment, but it will take a significant amount of time and first I would like a short phone call to ask some questions and get a feel for the company, nothing too long, I just want to make sure we're all on the same page before I spend my time on this. When can we schedule a call?"


I agree with what you're saying here:

> I would like a short phone call to ask some questions and get a feel for the company

I've had some places do behavioral first and others do coding first. I think I'm more onboard with behavioral first because communication a lot of times is more important than raw technical skill. Everyone can save time if you can suss out early on that the person is a bad communicator or bad fit. And generally you can spend time in this interview asking questions about the company.


That’s not behavioral, it’s an initial screen for fit. It. Makes the relationship at least a bit more balanced from a time investment perspective.

It seems most places do this though.


I get the impression avoiding take homes is becoming standard advice. I hang out in a few communities of engineers (alumni groups etc.) and it's gotten to the point that many people see requiring a take home as a red flag. Not just because of how companies abuse it, but because if basically everyone the company has hired put up with a take home there's a strong chance you don't hire good engineers.


That's the reason I advise against them.

Unless you are the top company in your bracket, your completion rate will also not be so great for a take-home (people have limited time: they'll sort by how good your company is). Sure they'll do Google's take-home but local business that does local comp/prevailing wages? It's at the bottom of the pile. By the time they get to it, the best candidates will already have an offer from a better company.

That's why I prefer to bring people on-site as soon as possible, and do a little whiteboard (the point being that the candidate must... know how to code!). It's an artificial challenge that's really a pretext to have a technical discussion.

Only time I advise to use take-homes is when you are dealing with a massive number of applicants from institutions that have a poor signal to noise ratio. Bootcamps come to mind. But then you end up with candidates that have the homework done by someone else and it ends up not being a good filter either.


1. On-site

Are the best developer candidates looking for on-site roles? I don’t know any that prefer an office setting.

2. Whiteboards

Yet another way to test developers on stuff that is unrelated to the work. I don’t even like writing with my hands all that much because I’m on a keyboard all day.

I get into a whiteboard and I am being tested on writing skills, how visual my thought process is, how my handwriting is, how comfortable I am in a public speaking position…all great qualities, but all entirely unrelated to programming in a team.

Is whiteboarding even a realistic exercise in typical multi-office or remote companies? If I’m presenting to coworkers I’m probably on zoom in my development tools right? That’s not a whiteboard.


- writing skills - visualization / explanation skills - speaking to a small group of peers

all of these are _highly_ relevant to programming in a team beyond the senior level


Visualization maybe, but writing by hand? Until recently (when I started working on an art-related project) writing by hand alone took more concentration than solving actual tasks, not to mention writing in a fashion so someone can read it. I simply didn't write by hand for the last 15+ years.


after many unceremonious rejections, the first company that wanted an optional onsite interview hired me, after I went onsite.

so, people want to hire me when they meet me in person, and I can give more inputs of rapport. data point of 1.


Whiteboards are skewered just as much on HN. Basically every engineer on here is a 10x and every company should be grateful to even be considered.

Apparently we should just review github OSS contributions and how many unit tests they write.


> Apparently we should just review github OSS contributions and how many unit tests they write.

No, but speaking from experience - also don't reach out to senior, employed devs for a role, then waste their time with whiteboard leetcode quizzes to "make sure they aren't fakers or coasting, we have sooo many applicants".

In those cases, a conversation is really more valuable, we aren't interns.


I gladly participate in whiteboard interviews to ensure I will not have senior+ colleagues who are unable to solve them. The fact that intern candidates can solve them is all the more reason to filter out experienced candidates who can't


Completely disagree. I've observed other people run whiteboarding session with candidates and "solving them" usually means, "in the way I want" and if you've only just met then thats only going to happen through hints.

The "context" (hints) they give different candidates can vary massively, perhaps by the end you've realised that your question is ambiguous, or you like the personality of this other candidate.

I've worked with folks who will definitely give attractive women more guidance throughout so they arrive at the "correct" solution and not see what they are doing.


A good whiteboard problem is geared towards understanding how the candidate thinks and how well they can communicate ideas rather than total correctness. I’ve been the interviewer on a few whiteboard problems, all of which I’ve prefaced with something like, “we want to see how you solve problems and communicate ideas, there are no specific answers we’re looking for. You do not need to go into great detail or even know exactly how to build what you’re suggesting, just how it fits in to the system you’re designing.” Also when I’m designing a rubric for these interviews, the technical competency maybe represents 15% of the total score, 25% for problem solving/adaptability, and 65% communication.


The whiteboard is standard across all companies, I know how to prepare, and the company has to be at least invested enough to assign an interviewer to it. So I would choose that.


I don’t think whiteboards are bad. People greatly exaggerate their difficulty


I got denied three times in the final interview with different FAANG companies for not doing whiteboards fast enough. Not getting them right; I mean not doing them fast enough. I’ve passed some whiteboards too in other interviews so it’s not that I can’t do them. There’s a lot of variation here.


Yep. They evaluate the ability to make sound decisions while in tense situations. I don't want to hire people who implode with a bit of pressure.


"The ability to remember rote solutions under pressure." Not a need that comes up in actual software development.


I don't want to hire people who implode with a bit of pressure.

The pressure of whiteboard interviews (mostly negative; of the form "yeah this problem is basically trivial, but I'm not used to coding while talking to a stranger, plus the time requirements just don't smell right and certainly do not reflect real work conditions") are so infinitely removed from ...

The pressure at real jobs (mostly positive: "the problem is somewhat non-trivial, and I know time is kind of tight; but I'm in the flow and in my preferred environment; and at least I know the requirements are legit; so can I triage some good result out of this before the end of the day?").

At least for any job one would want to have.

If this distinction is not intuitively obvious to you - or are quick to characterize any negative reaction to your interview as a matter of the candidate being basically a total limp-wrist at your craft, ready to "implode" at the slightest difficulty -- then really, you have no business constructing interview challenges for people. Not challenges with any kind of a time limit, anyway (these being generally unnecessary, when we think about it).


One simple question I ask is to count the words in a string. It's a warmup question and a way to get the candidate comfortable with the format of the interview and the optimal workflow (discuss requirements, propose an implementation, flesh it out in pseudo-code, translate pseudo-code to something that looks like code).

You certainly don't need rote for that, the algorithm is trivial for anyone who did well in their introduction to programming class.

Either the candidate "gets it" in about 10-15 minutes, or struggles for 60 minutes.


> Not a need that comes up in actual software development.

I'm not sure. Ability to remain calm and code when shit's on fire can be fairly useful skill at some places (the smaller the company, the more valuable it can be). I've did that, coding emergency patches - e.g. to cache some hot paths when we've got unusual traffic that revealed bottlenecks that we've never hit (and noticed) before. Though I'd say it's a fairly different mental experience than coding at an interview.

Also, actually writing code on a whiteboard is a weird skill outside of academic roles. But most places I've seen asked to sketch some crude pseudocode or - better - draw wavy lines and boxes that represent processes and entities, then talk about it (and I think this is fine, whiteboard here is just a convenient tool).

And, of course, requiring to remember some algorithms from memory is a rarely useful skill. I've coded even in quite spartan conditions (right in a server room, with an ancient CRT monitor and keyboard, plain vim running on a 80x25 text console) but even then I had one or two Internet-connected devices that I could've used to look things up. If a candidate is allowed (or, better, even encouraged) to search for anything they want - it's all good, of course.

Outside of education, no one bothers to remember actual algorithms unless they've used them recently - everyone remembers names and key properties, and optionally remember some core principles (and then either just use them because they're already in a library or popular package, or go look for the implementation details).


> Though I'd say it's a fairly different mental experience than coding at an interview.

This is the key. Not all pressure is the same.

Shit on fire at work is not actually personal pressure, it's pressure on the company. And to be clear, shit on fire at work tends to be a metaphorical exaggeration, where an emergency code patch isn't going to save the company from imminent bankruptcy. You're still going to have a job the next day, still going to have a roof over your head and food on the table. And your coworkers already know you and respect your work (if you're competent).

This is vastly different from being unemployed, suffering from profound financial worries about your future, and having complete strangers looking over your shoulder, prematurely judging you in the span of a few minutes, where these few minutes can determine your personal fate.

I've done quite a few emergency fixes over my career, but exactly zero of them were lose-the-job emergencies. Whereas every job interview is a lose-the-job emergency.


YES EXACTLY.

I keep myself in the interviewing loop at my company because I actually enjoyed the process when I joined. It was conversations over “executable” solutions. Now, we kick out so many candidates that can’t get a coded, running solution in the 45 minute slot.

I have tried so many times to explain the WILD asymmetry going on. That for us, we just have to interview the next candidate. To them, they have the enormous pressure of “if I can’t do this exercise then I have no income, no health insurance, no security.”

That’s unfair.


Is it whiteboarding in general, or being asked to write realistic code on the whiteboard?

Sketch out rough program structure or system design or whatever

Vs

Write valid Python to solve some leetcode question


Amazing to me the general attitude here towards take-homes.

I don't know, perhaps it's because I've progressed every interview I've had the opportunity to do a take-home on (and I struggle with most leet-code style whiteboarding interviews) but I've always seen take-homes as an audition for your software development style -- what you deliver in a take-home assessment is what they will use to judge how you'll deliver on the job, plain and simple.

I've only done take homes for companies that I've already passed initial interviews for and generally like the people I've talked to and have favorable impressions of the culture/workplace thus far. I wouldn't do a take-home before that, but I'd do a take-home any day before a leet code grinding thing.


> I've only done take homes for companies that I've already passed initial interviews for and generally like the people I've talked to and have favorable impressions of the culture/workplace thus far

So your experience has been quite specific. A lot of companies think that it's fine to send you a take-home assignment as the first step. Some more do it after just a 10 minute phone call.

I'm not saying you're wrong, just that your amazement can easily be explained by the fact that you've done it in particular circumstances which is probably not the experience of most people.


The trouble is I've never seen it be a replacement for whiteboarding. You do the stupid takehome and then you still have to do the regular interview anyway.


Why don't they judge me based on my 4 year long degree which is only done to rubber stamp that I can do the job.

Do you also expect a Med graduate to see some patients in the hospital for free before giving them a job?


> Why don't they judge me based on my 4 year long degree which is only done to rubber stamp that I can do the job.

If you spend enough time hiring you learn that a 4 year degree is a very poor signal for "can be a productive engineer". Communication, raw coding, and soft skills aside I'm still regularly shocked by how weak the CS skills of many fresh grads are, it seems a double digit percentage never really learn big O well enough to use it day to day. When you're paying big salaries you can't afford not to verify this stuff.

> Do you also expect a Med graduate to see some patients in the hospital for free before giving them a job?

I mean, this is literally a requirement for graduating most pre-med programs.

Getting more at your point - in many skilled white collar professions you need to take regular tests to maintain "certification". The bodies that issue these certifications are trusted enough to maintain standards that it's possible to get hired by maintaining the certification. SWEs have no similar institution, and given how fast the field moves it's not clear we're ready to establish one.


As much as I want to agree with you and I do in principle. My reason for not trusting a degree is because I've first-hand seen degrees being given out like candy. I've seen companies hire degreed individuals and then spend a year training them in a bootcamp before they're remotely productive.

I've seen individuals that can't speak English, can't spell, use punctuation and write a coherent sentence given a degree. I don't want to bring race into this, but it's very much an important issue here in South African universities: there is a huge shortage of black developers. So universities have immense pressure to rubber-stamp pass the black individuals lest they be seen as the "problem" or "racist".


Yeah.... are you forgetting about residency? Because that's basically the equivalent of having to do doctor level work at a drastically reduced compensation.

If the computer science industry was more well regulated along the lines of traditional engineering by requiring a PE exam, then we'd have to do less upfront whiteboarding and leetcode interviewing to verify that an applicants abilities actually reflect their academic credentials.


4 year degree unfortunately is basically a neutral signal. Maybe if it is top scores from a top tier CS college it might be enough to be a positive signal in itself.


Would you be happy to work with any and all your peers who have the same degree?


Had one company in local city. They managed to interview almost every Engineer in the market that had skill set. Rejected all of them.

Few months later I got a call asking to interview again as they realized their standards were too high. Not a chance.


> their standards were too high

They could simply be wrong. I've seen people demand the wrong solution. I've seen tests with bad instructions.

Most people are not good at producing test


You had 45 minutes to write a connect 4 game from scratch.


requiring leetcode despite 20 years of experience, code samples you can show, and a CS degree you can show, is a red flag to me.


We applied some advice that I saw on a HN thread years ago (but sadly do not have the link handy) and it has worked well: we pay our candidates for their time. The take home concept really is insulting when a candidate’s time isn’t valued. Since we incur a cost so we’re serious about knowing for sure that we want to interview the candidate.

I wish people were less surprised when we tell them they deserve to be paid for their work, but companies have been advantaged in the hiring process for so long (excepting COVID) that disregard for people’s time is embedded in corporate culture.

However, to the point people have made about not getting comprehensive feedback: it’s just not worth it. Corporate counsel will basically refuse to let you tell a candidate why they were rejected for fear of any inadvertent lawsuit exposure. This is also deeply embedded in corporate culture and it’s hard to imagine it changing.


I’m sure there is a dollar figure. But I can’t imagine your company would be willing to pay me enough to do a take home test to pique my interest in going through the trouble. How much are you paying? I could throw a rock and a hit a consulting opportunity for $200/hour.


> I could throw a rock and a hit a consulting opportunity for $200/hour.

then why are you interviewing?


I’m not interviewing. I would only do consulting as a Plan B while looking for a permanent job. I work in the cloud consulting department at $BigTech now. I get the same amount deposited in my bank account every month whether I have 10 hours of billable work or 40. I also get a known amount of RSUs deposited in my brokerage account every six months.


There are more benefits to a stable job than money.


Yet most people, if potential income was comparable, would prefer to be their own boss vs having a job


For context, I am a full time, base pay + RSU earning consultant in the Professional Services department at AWS. I would much rather have the consistency of a pay check going into my account every month and stock (RSUs) being deposited into my brokerage account twice a year than hustling for a contract work - even if I got paid 50% more.

Besides, I don’t have to prove my competence with every client. The mere fact that I have an email address that ends with @amazon.com automatically gives me credibility.


Not just takehomes but also applies to things like automated coding challenges as an initial step before I've talked with someone.

Twice I've aced these challenge with time to spare and was then rejected or ghosted.

It's the same idea; don't invest time into a company's interview loop unless you know they're investing something too.


> Takehomes have been ruined as a concept by companies that spam them out before applying a first pass.

I had this experience a few weeks ago with Wolverine Trading.

It seemed clear that they didn't respect my time. Perhaps I dodged a bullet.


> Not once have I “passed” a take home interview

I have had the exact same experience. They have always been a complete waste of time for me and I have since adopted a policy that any takehome challenges = automatic close the loop for me. It's not worth explaining why either. If this is your process and you find it appropriate to offer it as a first option, then it's just not a good fit.


Yeah, we should just refuse to give away well as accept take homes when interviewing. There was the company where I and some other experience people decided to give someone a take home. And we regretted it because I think it took more than we thought. The one person that did well on it turned us down anyway.


Oh, don't worry. An actual engineer doesn't need to look at your work. From now on, companies will simply give your answer to ChatGPT and ask it what it thinks.

Not joking. You hated interviews before? Get ready to rock.


Maybe a trick here, at least with online marked ones, is screen record a take home end to end with the result. Then say - use that result or GTFO. A bit like how they let you drive a car after passing a test once.


As someone whose taken and authored many: the whole point is often to have different levels of completion. There may technically be a "complete" point - often that isn't that hard to achieve. Usually there are several other possibly-non-obvious improvements the tester is looking for. Whether these improvements are obvious to you or not is a sign of experience.


If I'm not given that guidance directly nor given a chance to clarify it as I do the solution (as with normal interviews), I don't really want to participate in the guessing game about what they want, if missing that guess results in all the work being wasted.

As others have mentioned, I find the "limit yourself to 2-4 hours" BS. I can stand up a new http server in 2 hours to implement some API, sure. Adding unit tests, integration tests, a readme, build instructions/scripts, containerizing it, adding detailed comments, linting... generally not possible to do in 4 hours unless you have everything set up before then (I can't use my normal dev tools to do takehomes because they're through my employer). IRL you would generally scope this as a week-long task even for experienced employees.

Maybe if I were unemployed and desperate for a job I'd suffer it, but most places I'm interested in working at ask whiteboard/design problems anyway.


If it feels like a guessing game it's probably not obvious to you what the improvement is. They usually stick out like a sore thumb for the candidates that they're seeking if they're written well.


"If they're written well" is key here.

Anyways, just show me the code and we'll discuss it together, don't waste my time making improvement to toy code that will never get used.


> Usually there are several other possibly-non-obvious improvements the tester is looking for. Whether these improvements are obvious to you or not is a sign of experience.

There is a risk here for something that even really strong intermediate programmers fall into where they take a very prescriptive approach to programming so they tend to not understand things built from first principles and also tend to overvalue things like future-proofing and abstraction. Since you don't mention any specifics it's easy for people to fill in the blanks here and assume that at least a good portion of the things you refer to are those kinds of things.


> Whether these improvements are obvious to you or not is a sign of experience.

In my experience as an interviewer fishing for specific answers is what makes you miss the best engineers.

I used to do this during my peak Dunning-Kruger phase and gave relatively poor marks to an guy who later proved to surpass both me and the rest of the team in every aspect that mattered - even though originally he barely made it through the interview process.

Chiefly, he didn't do things the way we expected because he prioritized.


I definitely wouldn't fish for answers. The point is that they are obvious to an engineer with a certain skillset. Doesn't have anything to do with prioritization.


> The point is that they are obvious to an engineer with a certain skillset.

Are they always worth fixing, regardless how many there are?

Also, you had plenty of time to prepare these and think this through. Someone seeing this for the first time may assume Chesterton's Fence and not change anything despite noticing - assuming it's less important than breaking whatever reason there might have been for doing things this way.

What I'm getting at is that you're not necessarily getting the best engineers, just ones who think like the current you.

It's a mistake and it's likely already costing you.


Not really, the improvements don’t break the solution in any way. There’s no “Chestersons Fence”. They’re much simpler than you seem to be assuming.


If that's the case then they're not worth putting in an assignment - an automatically graded quiz would suffice.


Er, no, it wouldn’t - the point is that you’re taking something that resembles a real world spec and interpreting it into code. You’d be surprised at how many people stumble on this despite how basic the solutions are.


(an example)

I gave out one that was sort roman numbers.

I gave out a file that was 1000 lines long. It may contain duplicates. The numbers are all valid, but are not necessarily reduced (e.g. the number 4 may be represented as IV or IIII).

Print out the reduced numbers in sorted order. If both IV and IIII exist in the file, then IV should be printed out twice (in the proper spot of the overall sorted order).

I timed myself doing this from scratch. It took about 1 hour (7:38 pm to 8:48 pm with some thinking and cleaning sporadically afterwards - https://github.com/shagie/RomanSort/commits/master its 6c79654 that has it acceptably working)

One candidate did it very similar to what I had.

Another candidate didn't have runnable code. Because it was only to take about an hour, and their interview was Monday at 10 am, they didn't start it until they got into their place of current employment that morning (they had a full week to work on it). Neither I nor my manager was impressed with that one.

Another candidate put them into an array (because arrays were faster than lists to access), and then wrote a Comparator to take two roman numeral strings and called Arrays.parallelSort() (because that was faster too), and took the String representation of the array and did a replaceAll on the delimiter with a new line and printed that out. It worked, but the code was messy.


I'm not seeing how your example includes an "obvious improvement" that "only the best engineers would do". I would use arrays over lists but I have a ton of low-level experience so I tend to care about performance, but I wouldn't necessarily expect the "best" engineers to do so unless the prompt says "Please take special care to turn in the fastest solution".

There's a couple obvious tests (written unit tests) that I would expect the candidates to think of for their roman number sorting function, and I usually graded the take homes better when there was 1) evidence of testing 2) evidence of smart, appropriate edge case testing, but not having time to write polished tests in 1h isn't necessarily a red flag.

I'm really struggling to find how simple examples like this can demonstrate anything like GP mentioned.


I will only do take home if I feel like the company is serious. If they are sending me a take home without meeting with the hiring manager, it goes straight to the bin.


Interviewed with a company recently and they gave me a take home assessment after my initial round with the hiring manager. This being my first take home assessment ever, I excitedly sat down to solve it. I took about a couple of days (had other things to do) and sent it over. I even wrote some basic tests despite the instructions saying that they were okay to skip (wanted to impress my interviewers).

Got rejected a week after submission with no explanation. It was a really good problem and I had fun solving it and it satisfied all the requirements correctly. When, disappointed, I asked them for feedback, the recruiter simply said that they "felt that it fell under their expected level" and that "The team has a very high bar set for this role and have made it a point to make sure people who join the team can contribute immediately which unfortunately means tough decisions like this have to be made". They didn't share any specific feedback on my code despite my following up for more specifics. Really pissed me off.


At some point, it is just a numbers game. If they only have 1 position available and 5 people complete the take home test flawlessly, 4 people are going to be rejected for no good reason except that they have only one position available.

As a wise man said, it is possible to make no mistakes and still lose. That is not a weakness, that is life.


I don’t believe in no win scenarios.


Unfortunately no-win scenarios believe in you.


it's not no win. it's single winner take all


>can contribute immediately which unfortunately means tough decisions

You probably dodged a stressful position. Companies or teams that can’t allocate time to onboard talent are likely poorly run and can’t afford to invest in people. “Contributing immediately” is a fair indicator of that problem.


Poor onboarding is one of the most baffling common ailments in this industry.

There are so many hard problems in this industry. Onboarding is not one of them.

Typically what I have seen is that management is so busy striving for short-term deadlines etc. that they "can't spare" engineer-hours to help the new engineer properly. Heck, I've been one of the senior engineers that has not had sufficient time to spare for the new people - not by my choice. In other situations we've had new engineers inserted into our teams with zero notice -- so we've not even had the time to prepare some "day 1 wins" for the new person.

It seems incredibly obvious to me that great onboarding is a true force multiplier.

Last place I work paid a lot of fake lip service to the importance of onboarding. But the reality is that we were judged strictly by the numbers in terms of commits, Jira tickets closed, LOC, etc.


In my current job, I had an onboarding phase of 1 year. Had 3 months in the previous one. Guess I am lucky.


You probably work for a service based IT company where they kept you on bench.


I work for product companies in R&D.


> You probably dodged a stressful position.

Word - I never considered this possibility. But now that you mention it, the hiring manager I spoke to had this smug swagger about him and a condescending sneer about my current company and open source in general. Not good vibes at all. As a member of his team, I might have placed myself in a position that is bad for my mental health. Thank you for sharing this perspective, really helped.


I don't think contributing immediately and allocating time to onboarding are mutually exclusive. I think ideally a well-structured onboarding process has you contributing (in small ways) as soon as possible. It doesn't necessarily have to mean you're just thrown in the deep end.


I've had this experience before.

Believe it or not, I've also gotten takehomes which took multiple hours, involved some non-trivial data retrieval and management, and gotten rejected with less effort than it took them to prepare the ask.

Later I had a sinking suspicion that it was not a real position, what it was was three brogrammers who found a way to get unemployed rubes to write code for them, for free, so they could deliver it themselves without having the requisite talent.

Are you positive that didn't happen to you too?


That is really terrible - sorry that happened to you.

The company I got the take home test from is a well known and established leader in its domain - not anything like the one you described above.


Totally agreed. After an initial 30 min interview, I then spent something like 12 hours working on an atypical (for me) take home project. Actually really loved the challenge and learned a lot along the way and was excited to discuss it and learn more.

Got a "Thanks, you're not moving on" message and that was it. Man, the rejection is totally fine, but I really do wish they could've spent a few min just explaining some details. I've gotten more thoughtful rejections from just cover letters and resumes, not to mention public PRs in strangers' repos.

After spending so much time, it just feels like a betrayal of some unspoken developer ethos (vs say talking to a generic HR screener). If you're going to make someone code for you for hours and then dismiss their work, please at least tell them why in just a few sentences. You don't need to comment line by line in a code review, just general thoughts like "better code org" or "poor architecture and readability" or "better tests would've caught this major bug" or whatever.

In my case this was a small company I was super excited to work for, and waited more than a year to finally have a chance to apply for (once my current job ended). It was definitely disheartening and makes it hard to want to try again with them in the future.

But, you know, the other side of the coin is that maybe they're just getting swamped with so many applications they can't take the time to thoughtfully answer each one. I imagine I'm competing against a horde of more qualified ex FAANGers right now and maybe they're too busy trying to decide between the top 3 or 4 vs the long tail of hundreds of us who failed the take home. Who knows...


> the other side of the coin is that maybe they're just getting swamped with so many applications they can't take the time to thoughtfully answer each one

Maybe. At the point in the process where you're seeing someone's project, you've already spent time probably phone screening them, sending them the project, hopefully reviewing it. It takes relatively little time to write a quick sentence or two in review of the assessment. It would be nice even if they didn't look at the project and just said "sorry, we didn't have time to look at your project because we're moving forward with other candidates".


If they're swamped, and they're asking all the ones they're swamped with to do a project that takes hours, that's abuse.

Why abuse? Well, let's say they have three solid candidates. They ask each to do an assignment. Would I do a 6-hour project for a 33% chance at a job? Maybe, especially if I really wanted that particular job.

Now let's say they have 30 candidates that they ask. Would I do 6 hours of throwaway work for a 3% chance at a job? No - not knowingly. If I'm an average candidate, I'm going to have to do that 30 times to land a job. That's 180 hours, or more than four full-time weeks of throwaway work. That's an abusive process.

Could I spend just as much time interviewing? I could, but there's a difference. If I'm interviewing with you, you're there talking to me. You can't waste my time without wasting your own. Whereas with a take-home assignment, you can waste my time but waste little or no of your own. As a result, interviewers (usually) pay some attention to not doing needless interviews, but pay less attention to not asking for needless take-home assignments.


I see your point, that there's a significant information and time asymmetry there in favor of the employer.

That said, though, I also think the take-homes are a much more enjoyable use of my time than running through interview gauntlets with people who don't even work in the same team or in tech at all (recruiters, HR screeners, upper level bosses, whatever). At least code presents a somewhat objective metric I can be measured against, as opposed to random people's opinions about me after talking for ten minutes. Those sorts of interviews feel more like theater than assessments. I'm good at them, but I don't feel like they're a good way to gauge a candidate's effectiveness, including my own.

With the best take-home I did (not this 12-hour one I was just talking about), we met right afterward to discuss the pros and cons of my assignment, what went well, what could use more care, etc. It was more of a discussion, like "Why did you do X this way" and "Did you consider Y when you chose this?", followed by a segue into how the take-home relates to the actual products they were building, and a high-level discussion about how I'd implement similar patterns in a production app. I did end up getting that job and loving it. The process never felt exploitative, and the team ended up being amazing. I'm still thankful for the person who gave me that take-home to prove myself, vs the hours of pointless recruiter chats before the assignment.

But yes, they shouldn't be 6-hour take-homes... that's too much to ask of an unpaid candidate. A reasonable take-home should take 2-3 hours max for a candidate of median skill, IMO. As an aside, my 12-hour one was my own fault for not having worked on that specific problem enough (web animations in SVG or canvas), but I was happy to be figuring out a new challenge after my last job got too easy and boring.


> It takes relatively little time to write a quick sentence or two in review of the assessment

Not just that: At every job where I interviewed candidates, HR expects some kind of feedback beyond just "hire"/"no hire". At least some bullet points of highs and lows, red flags, etc.

If nothing else, this helps protect the company from false allegations of illegal discrimination by having documentation for why a particular candidate was rejected. It also helps recruiters to know if they're finding candidates who are "close to what we want" or "not even in the ballpark".

Now, I know that some interviewers can be fairly blunt in their feedback, so you'd not necessarily want to just copy-paste it to the candidate, but the point is: that feedback already exists. The recruiter/HR just needs to (maybe) sand off some of the rough edges.


That's fair enough.

Well, it's a good lesson to have gone through though. If I'm ever in a situation where I am asked to evaluate others in a similar fashion, I'll be sure to leave detailed feedback (if they want it), now that I know what it feels like.


It's been surprising to me to read through the comments posted here and seeing how most people seem to really despise take-home assignments during the interview process.

Speaking as someone who absolutely hates live-coding something during an interview (unless, maybe, by chance it's a practical exercise ... something like "build up this simple CRUD web service" ... not "solve this leetcode problem" ... but this is pretty uncommon in my experience, unfortunately), I rather like take-home assignments. As long as they don't take any more than 1-3 hours. I've been in situations where I've declined a take-home assignment that was given to me where I estimated it was going to take 10-20+ hours. No idea why any company would think that is reasonable.

For myself, what I hate is when the take-home assignment comes first. Like, before you talk with anyone at all, or maybe immediately after you did the 15 minute HR/recruiter screen. If I get a take-home assignment at that point, I decline.

There's no denying the fact that a take-home assignment is a large investment by the candidate on this random chance to get the job. I'm not willing to make that investment unless I've gotten a chance to talk to at least _some_ of the people I'd be working with at a company to better gauge what the situation is and whether I think it's a good fit for me. Even a 15-minute HR/recruiter screen is not going to do that. So yeah, if they just throw the take-home assignment at me right off the bat ... yeah, no thanks.

This all being said, yes, it is frustrating though to have to go through a bunch of these and get rejected with no feedback, or just ghosted, yeah.


Take home tests are bad. Fizzbuzz questions are bad. Multiple onsite interview rounds are bad. Behavioral questions are bad. Leetcode style questions are bad. Collaborative coding tests are bad.

If you ask the majority of job seekers here, the only correct way to hire is to hand out 300K/yr offers after a casual 30 minute chat.


The actual best way is to only have a small set of senior engineers who also are very good at talking about programming, and are good with people and interviewing specifically and have a history of trust and good judgment do all of the interviews where they have a lengthy conversation about past experience, technical topics, and maybe code a little together if they think they need to and just give them total authority to make the call. It might not be that scalable but it would work the best.


This has empirically resulted in bad hiring, at multiple companies I’ve worked at. The best predictor of senior performance on the job has been the take home test or coding challenge. We frequently have seniors who suggest that the technical discussion would be enough for senior roles. We always have to let them down a bit when we tell them that the discussion is less predictive than they’ve built it up to be in their mind.


How do you know which candidates are worth your senior engineers spending all that time with?


I'm not sure. I think that's a problem to figure out. What I want to advocate for is mostly that talent scouting should be taken seriously not farmed out by scheduling random employees who aren't good at it and then collating the partial opinions of 10 people who don't really have enough time with the person to have a total view.


I agree in theory.

But engineers with good tech skills and good people are already likely to be the most valuable and time-constrained resources in the business. So taking 2 of them out to have a 2 hour interview with, say, 100 candidates is 400 hours. Even assuming they’d be willing to do that, that’s about 2 months worth of development from your best engineers.

This is why companies have processes like screening with a non-tech person and tech tests. It filters that 100 down to, say, 5 which is more feasible to put in front of those top engineers.

It hurts not to be selected for that 5, to be told you’re not good enough. Companies should do a better job at that. If they can’t give feedback because of the lawyers, they should warn you of that in advance, to set your expectations. They should be responsive and transparent through the whole process. What they can’t do - seemingly to the chagrin of many engineers here - is offer you a highly-paid job merely because you think you deserve it. I can see people on this post that likely have appalling people skills based on their comment, arguing that, since they didn’t get an offer, the company’s process was wrong. Perhaps those commenters need to look in a mirror to find the problem.

Recruiters might miss some top talent, but that’s the price they pay. All other solutions also have costs.


As they say on twitter, “this but unironically”.

My wife is a lawyer. They don’t send her a take home test, or ask her about her side project cases, or whiteboard bar exam questions. Somehow it manages to work.


> Somehow it manages to work.

It manages to work because expensive law schools and the state Bar do all of those things on behalf of her law firm. Meanwhile you can call yourself a programmer after taking a 30 minute online tutorial and writing a hello world script – and that is a good thing. We do not need gatekeeping in this profession.


We have certifications too - Azure, AWS, security ones, etc.

But a person with 15 certifications still gets asked to do a coding test


And those certifications mean absolutely nothing. There have been brain dumps for certifications for years.

Even if you don’t use brain dumps, it’s easy to memorize enough from studying something like ACloudGuru.

I got my first (of nine) AWS certifications (AWS Solution Architect) without ever logging into the console. But the only reason I get any of the certifications were to know what I don’t know and as a guided learning path.

For the first one, I was already a dev lead and wanted to know enough to know what I needed to experiment with when the company was trying to “move to the cloud”

My next 5, I was already the de facto “cloud architect” at a startup responsible for “application modernization”

And my last three (that I did within three months of each other), I was (and still am) working at AWS in the Professional Services department.

I can assure you that I don’t know but about 30%-40% of what those nine certifications cover well enough to hold an intelligent conversation with a customer and I only know the 30-40% from hands on experience.


I think the only benefit for me is that they hammered IAM concepts into my head, as they are a huge mess on AWS.


Then that sounds like we need more accurate certifications.


Have you taken any of those certificates? They can be passed from reading exam dumps and without proven hands on experience mean nothing.


And to be fair, I get it. Industry is too diverse to have one end all be all test. An AWS certificate won't mean much for my domain in games (unless you maybe work on online infrastructure for an ongoing service game).

But at the same time I wouldn't mind some fundamental license to optionally bypass many of these common tests.


Works fine as long as you went to a top-3 law school or graduated near the top of your class in a T14 school... I've explicitly seen lawyers lay that out as the criteria they hire by.

I'd rather deal with obnoxious interviews any day.


For your typical workhorse lawyer, yes.

But the legal profession, in terms of fairness for new grads, is perhaps the worst of all professions, I'm told. Basically, if you go to a top school you make 3x the average graduate based on that alone. While there is a decent correlation between school and talent, it is by no means strong enough to justify the phenomenon. Certainly it is not like this in other developed countries.

Computer Science has unfortunately been headed in that direction but at least an A+ graduate at a state school has a chance at getting the better offers.


There's no perfect way, but fizzbuzz is perfectly acceptable to me (do you really want to work somewhere with employees who couldn't pass it?), and leetcode is too, as long as it doesn't rely on some trick that selects for memorizers over problem solvers.

I don't know why HN is so against algorithms problems. It is not something I have ever needed to "grind" because I understand ds&a well. While the more exotic ds&a you won't use on the job, and IMO shouldn't be tested for, IME the most complicated datastructures you generally need for both algorithms problems and on the job are hashmaps and hashsets.


>I don't know why HN is so against algorithms problems. It

Probably because they use the ones you mention. They think an ideal engineer should be able to cold solve some level 5 leetcode problem using a Trie with some trick edge case to perfection in 90 minutes as if they are some sort of mathlete. It's not very realistic to what you do day to day and required dedicated studying simply to pass the test, not to advance myself as an engineer.

If they used level 1 questions as a fizz buzz esque filter for string manipulation, iterator tactics, and maps/sets, then I'd have no issue. I'd get why you may test a student for a junior position harder but a senior dev should have actual professional experiences to dig into.

If you can't trust them to code then maybe you should in fact do the FAANG method and describe what exact concepts you want prospective hires to be able to talk about and work with. Haven't seen that outside of FAANG tho.


> "I don't know why HN is so against algorithms problems."

Half of developers are below the median and, to paraphrase Upton Sinclair, "It is difficult to get a man to support something, when his salary depends on his not supporting it."


The best way to evaluate someone is to work with them for a year or two, yes. Everything else is full of an incredible amount of noise.


I agree with the consensus about behavioral questions, the multiple in multiple onsite interviews, and most leetcode. Fizzbuzz is great and take-homes are widely varying in quality.


> after a casual 30 minute chat.

which you have to pay them for


To give a different opinion, any company that doesn't do a take-home gets lower priority during my job search. My favorite interview was HR screen -> 30 minute chat with another programmer -> take-home -> final interview that included a take-home review.

I didn't end up getting the job but it was easily the best and least anxiety inducing interview I've ever been through.


I don't think that's a "different opinion" at all! :-) I look at it the same way basically.


Your timeline is best case scenario though. Your example timeline could happen and then they bring you onsite and ask you to do an in-person challenge as well. Now you’ve done all of that and still get ghosted.


Well, in that case, I assume you wouldn’t be interested at working for any on the top paying companies in the industry. None of them require “take home tests”.


You're right in that assumption but not because you putting words in my mouth. I said companies that don't do take-homes get lower priority during scheduling.


Why would I even schedule to work at a company that requires me to take more time than less?


>As long as they don't take any more than 1-3 hours.

Well, that's the issue. It may be because I'm a bad programmer, but I can't think of a take home that took me less than 3 hours. The take home I did for my first job took some 20 hours over 4 days, and I exaggerated and said it took 12 (which didn't garner a response so I'm guessing that wasn't an unusual answer). I could do that while I'm a student, I can't do that again as a working professional without wasting an entire weekend after a week of full time work.

I still hate leetcode more, but at least there you have an explicit timer.

>For myself, what I hate is when the take-home assignment comes first. Like, before you talk with anyone at all, or maybe immediately after you did the 15 minute HR/recruiter screen.

I've never had a test come later. 15 minute interview call to make sure the high level details are correct (pay, location, physical office vs. Remote, etc), and then I am sent an interview test.

Granted, my last 2 jobs did not employ take home tests, so I know I'm not forced to do them. But given my experiences I understand why others would be opposed to them. It sounds like you would be opposed to my experiences as well.


> I've never had a test come later. 15 minute interview call to make sure the high level details are correct (pay, location, physical office vs. Remote, etc), and then I am sent an interview test.

Depends on the company. My personal experience has luckily been (so far) that more companies I've interviewed with who do take-homes at all have given them later on in the process.

I suspect with the current job market situation given all the recent layoffs that things will probably start to shift for those companies that do take-homes, where they'll probably start giving them to candidates at the beginning of the interview process, as they'll probably see it as an easy filter for themselves. Ugh.


I’d rather do neither. That’s the main reason I have never done in person or take home coding tests in the 20+ years I’ve been conducting interviews. So far we haven’t hired a dud.


I bet while you “weren’t doing in person coding tests” as a developer you also weren’t making the type of compensation that the people who were “grinding leetCode and working for a FAANG” (tm) r/cscareerquestions.

I use to brag like you, about “not doing take home test” for 25 years. Then I landed at BigTech and saw that returning interns were making about what I made two years earlier at 45.

I’m not complaining, my goal had been to get into $BigTech in 2020 and relocate when my youngest (step)son graduated I did so without a coding interview and without relocating by pivoting to “cloud consulting/application modernization” (cloud + enterprise application architecture/development). But I tell my younger relatives to practice coding interviews and go for the most compensation possible.


If you base your success on money, sure. Maybe tests are a good filter.

Personally, I'm seeking something more furfilling and the difference between 200k and 300k doesn't matter if I don't find the work furfilling. 300k to 400k would be negligible for any material possessions I'd want.

But to each their own.


Yes, I have this insatiable addiction for food and shelter. The best way I know how to support my addiction is to trade labor for money.


Which as we all know is impossible to do without FAANG money.


Sure it is, I had my 3200 square foot home built in 2016 in the northern burbs of Atlanta when I was making $135K.

But why given the choice (theoretically) would I give up making $ALotMore just by practicing for coding interviews?

Like I said, I personally didn’t have to thanks to years of industry experience, knowing “cloud”, and having soft skills. But why would anyone without the path dependencies I had (ie children in school that I didn’t want to relocate) hold their nose up at doing what it takes to make $160K+ straight out of college and a quarter million+ a year by the time they are 25?


>But why given the choice (theoretically) would I give up making $ALotMore just by practicing for coding interviews?

Because I am content in my current state of life? Because the type of work being offered is not engaging to me and I have the luxury to choose? Because I'm not trying to speedrun out of the job market and retire at 40?

I'm making more than $135k, but once I get a decent car and pay off my house... what next?

- Paid off student loans

- Already maxing out my 401k for retirement as well as investing in an IRA account

- I of course have hobbies and disposable goods to buy, but It's not $5000/month worth of stuff.

- I have a decent savings right now, and I imagine by the time I pay off my house I will have a very hefty buffer as well as some more of that going into stocks

The only consideration is potentially for kids, but that's not even in the cards right now. Maybe I'd consider traveling, but I've never had those dreams where I'm an adventurer, nor one where I retire in some vacation resort. I'll cross them bridge when I get there, and it's not like I don't have the savings buffer if I need a short term transition phase for $ALotMore down the line should the need arise.

As is it now, if I got a million dollars in 10 years after I paid off my house/car my first impulse would be... going to art school. Not investing in some more stocks or starting a busines or whatnot which tends to be popular here, I'd just continue to develop myself in skills I never had a chance to. And a million dollars would take that from being a part time side hobby to something I can focus on full time for a few years.

----

so yea, given all that talk, guess my career in tech lol.


Or take my path, find a job that balances work and life priorities, doesn't require you to stress out on grinding rote memorized leet code problems, and work on a side business in your less stressed free time so that you can start your own company.

FAANG companies already have far too much power and influence, I'm ideologically opposed to giving them even more by subordinating myself to them for the almighty dollar.


I mean I agree with you on this, but the idea that you can gauge a person's skillset through conversation (gasp!) and talking about your craft and details about past projects, etc seems to be totally lost on people today.


I mean, it goes both ways. I feel that I am a fairly proficient engineer (built something with 10M+ downloads) but for a long period of my life I would do fine in every part of the interview except the part where you just chit chat a bit. I'd try to tell interviewers about myself and the smiles would just fall off their faces.

It took me a long time to understand that these sorts of conversations had their own "rules" to them, which I had to follow if I wanted to do well. These rules, of course, have virtually nothing to do with programming aptitude or ability, and seem to me, somewhat cynically, to be another way by which interviewers can allow their own biases to enter into the interview process. For instance, one time I got rejected because I seemed "too excited" about my personal side projects; it was deemed that I wouldn't be as excited about the work that I'd do at the company. Of course this is nonsense; I'm now happily employed and pretty excited about my work. I have plenty other examples of me saying reasonable things in interviews and being rejected for that reason.

There's really no silver bullet here. Getting rejected is always going to piss people off.


See, I take a bit of a different view in the example you've given. Like, if I got rejected because I seemed "too excited about my personal side projects" I'd come away from that thinking "if that's really their take away, I'm kinda glad I don't work with them!"

You're right that the conversational interviews (just like any social gathering, really) have their own rules. But I think the most important thing you can do during those interviews is to just be yourself. After all, you want them to get to know you, just as you want to get to know them, right? How else can you each be sure that you're a good fit for each other? If they reject you for something you said that is true but that they just didn't like (e.g. a difference of opinion on something), or they nitpicked some little thing you said even though the rest of the conversation went smoothly ... well, in my opinion, you're better off.


Normally I'd agree, but I got passed up by a large number of fairly reasonable-seeming companies for arbitrary reasons like this. If it's just one or two, sure, maybe I'm better off. But after that it starts to have a real impact; it becomes harder to negotiate, maybe one of those companies would have been just fine anyways, etc...


> gauge a person's skillset through conversation (gasp!)

I can't claim GP's 100% hit rate, but I think I can get to about 90% this way.

My most recent error is galling though, and absolutely would have been caught with a lightweight coding test.

Some people are really good talkers.


Seems fairly easily solved by letting that person go. Easily for the company at least.


I did over 120 combined interviews/phone screens/take home tests,etc for about 55 different companies between 2020-2022. Imo, unless it is for a big tech company, interviewing is just not worth it. Sadly I get too nervous to even try out for any of the big tech companies. Plus I'm white and in my 40's. Companies now really want people out of high school/code camp that can program in python or javascript and pay $25 an hour. If a smallish company is handing out take home tests they are only to waste your time. Most of the companies I interviewed for were looking for 3-5 new devs. In some extreme cases I was told they were looking for 10-30 devs in the next 3 months. Here we are nearly 2 years later and I browse the company linkedin page and the employees listed are still the same ones. They never hired anyone. I depleted my $60k life savings in my 18 months of unemployment. If it wasn't for my faith in a "God" and my family, I would have just killed myself via a stent of extreme drug use and homelessness. I haven't drank or smoked since high school. I did get a couple of brief contract jobs but was quickly fired/laid off. But that little bit of money on top of the stimulus payments are what got me into 2023. Then found a backend dev job for a porn site. There was no interview, I was the only one that applied. I now work with the worst code and the laziest people. But it is the coolest team. There are no "PC' police or HR nazis here. Money pours in. I work like 10 hours a week and feel like I'm in heaven.

Oh, but onto your questions. So yes it is frustrating. And as far as feedback. I stopped asking for feedback because it was always maligned. Nobody will really know why you didn't get hired, remember, these companies aren't really hiring unless you are young and cheap. In most cases my feedback sounded like it was almost for someone else. Like "needed more linux experience (I haven't touched a windows or mac OS since 2005). Or my favorite, after talking about a data warehouse I built to house 5TB of data and 15 billion rows, and all the different schemas I migrated through, their reason was they wanted someone with "more database experience".


120 interviews over the 2021 hiring boom and got nothing? I must say, this is quite out of line with most developer experiences in that time period. What area do you work in?


"these companies aren't really hiring unless you are young and cheap"

Nah, I think there are more factors at work (Also, cheap is relative, therefore dependent on other factors, and therefore somewhat vague)-- its a multivariate situation where you, and the many factors you bring, flow into a process and the many factors it brings. If enough of those factors line up enough-- you get hired.

If it takes 18 months to get hired, I'd analyze the factors you bring. Such as: Web portfolio & Online presence. Network. Resume posted on multiple sites. Appearance & Demeanor during interviews. Skills/Years Exp/Qualifications relative to the role's stated requirements. Etc.


> There are no "PC' police or HR nazis here.

I'm guessing this is not an insignificant part of it.


> these companies aren't really hiring unless you are young and cheap.

I landed my first job at BigTech at 46.

I changed jobs 6x between the time I was 34 until I was 46.

I doubt that I’ve done more than 40 interviews over 25 years between 8 jobs.


  > these companies aren't really hiring unless you are young and cheap
is this because its expected that younger workers work longer hours?


Or they're just generally cheaper


Yep, been burned by this quite a bit couple of years ago. Since then, I've refused to do any asymmetric interview processes. Surprisingly, a lot of companies are willing to skip that part and go straight to the "on-site" interviews.

If more engineers start to refuse these kinds of "dance, monkey, dance" processes, they'll start phasing them out.


I once rage quitted a process. We had a few interviews, and then difficult take home test à la leet code (I spent 4-5 hours on them).

Then the hiring manager came back, "all looks good! I need you to interview with the team, the CEO, the other team" and what not.

I declined and said that I'm not spending 10 hrs in a process you asked me for (as in I didn't apply, they reached out to me). I got an angry email saying "what do you mean? We have only met with you for two hours?". I pointed out that it takes time to do the tests, then nothing more than crickets from the company. Not even a thanks for your time.


How do you phrase the rejection to maximize the chance of proceeding to the next interview stage?


You don’t try to. You just say are not doing take home in a firm and polite way expecting things to end there and sometimes they think it’s still worth interviewing you.


I explain about the asymmetric risk part and just say it's an option if they pay for the time. Many companies are fine with this as hiring is expensive.


100%

I recently completed a take home challenge for a full-stack engineer role. Given a week and told I could request more time if necessary. Instructions specifically said "We're not looking for a pretty UI, we prioritize component structure and functionality." and "Use any framework you'd like." The challenge was to build a back end with four endpoints for CRUD operations on a resource representing an application for car insurance. The front end was a single form for updating and submitting the application.

So naturally I whipped up a solution in less than 24 hours with simple pre-built components that both looked great and were functional (Mantine UI + Firebase)

I submit the challenge and check the logs every day to see if they'd run the application. A week passes before I hear anything back from them, logs still showing that the front end was never visited, none of the CRUD endpoints ever submitted to. "We reviewed your submission, thank you for your time, best of luck in your search"

Naturally, I respond confused about the claim to have reviewed my submission, wondering how they managed to test the functionality without visiting the site or making any submissions. I asked if I had misunderstood the challenge, asked if there was any feedback about how my submission fell short of their expectations.

"We reviewed your submission based on the code you submitted and came to the conclusion not to move forward."

...yea long story short, I'm never spending time or money on a take-home challenge ever again.


If you are a company using this technique for hiring – and you're not providing feedback – STOP.

If you and your team cannot suss out someone's capabilities from their CV, portfolio, interview, and references then YOU SHOULD NOT INVOLVED IN THE HIRING PROCESS.

It's a shit-test that you believe helps filter out under-qualified/unmotivated candidates, but it actually has the opposite effect:

Experienced candidates (you know, the ones you actually want at your company) just move on to lower-friction interviews where they don't have to start jumping through nonsensical one-way value exchanges on Day 0.


But I can have my cake and eat it, too! I'll find the one who is stupid enough to do these things, but smart enough to build my industry-disrupting product!


Resumes and references have no value. I can put whatever I want on my resume. I can give you any phone number I want for my reference.


If you're comfortable lying about your experience and references, you're probably comfortable with cheating on a take-home assessment.


Tell me about it!!! blame the lawyers.

I'm on the other side and having to reject dozens and dozens of candidates after they spent 30+ minutes on these takehomes, and can't tell them why.

My only solace is that 30mins on a takehome beats an hour in a live interview. (my other solace is that we try to make the questions interesting and non-trivial, and we encourage candidates to use ChatGPT and then edit the answers, since the questions are just a bit too tricky for LLMs)

(hiring DevOps/SRE for wide area distributed compute: https://www.joinmassive.com/jobs?gh_jid=4236238005 )


Although I complained about the lack of feedback in another thread here[1], I was pretty surprised by the number of people who hated take-homes and refused to do them at all.

Why is that? Personally, even though I spent two days working on it and ultimately failing it, it was one of my favorite interview processes ever... way more efficient use of time for both parties, IMHO, and a better way to measure the actual skill set. And it was fast! I did the assignment in a couple days and then heard back from them in another day. That's way better than the other interview gauntlets I've had to run, which usually included 3-4 rounds of talking to a bunch of people, with weeks or sometimes months in between, and most of those folks are people I will never see again after the interview cycle.

And there's also not the absurd pressure and ambiguities of real-time whiteboarding or live coding, which is not at all representative of how people normally work (with access to the internet, their preferred workstation and IDE, etc.). Having gone through one of those sessions, THAT feels way more like a "dance, monkey, dance" bullet dodged vs the casual take-home.

By contrast, a take-home gives me a "just pretend it's a normal work day and show us how you work" situation and I can focus on the problem at hand instead of the politics of hiring. Isn't that a good thing...? What am I missing?

[1]: My complaint was about the lack of feedback after a take-home, not the take-home itself (which was actually illuminating and fun): https://news.ycombinator.com/item?id=36446949


The problem with take-home assignments, I suspect, is that nobody reads them. You don't read somebody's github portfolio, you don't really read their resume, and you certainly don't read their exquisitely written, commented, documented, and tested take-home project. If you even skimmed it, you're not going to sit down and summarize your impression.

I'm not saying that all interviewers are lazy or bad, but most of the ones I've worked with are. Hell, maybe even I am. Time spent interviewing is time not spent working on things that you might actually be incentivized to do well.


Time spent on interviews is time invested.

Now, it is up to you to ensure that you give an accurate filter to HR, so you can do focused investments.

And interviews investment is like financial ones. You invest in multiple promising parties and expect one to outperform the others enough to make a great overall return.

Long story short, hiring is your most important activity. As if you don't hire, or worse, hire badly, you won't have the firepower when you'll need it.

And if you won't need the firepower, why are you interviewing candidates?


I'm taking the "just following orders" perspective of an engineer in a large organization. I've heard that in FAANG, giving interviews was an unspoken prerequisite to promotion -- that's at least incentive to _do_ interviews.

In the places where I've worked, I'm not sure there was much incentive to do interviews, and there was _no_ incentive to do them well. So, few interviews were done well.

Maybe somebody further up the ownership ladder would recognize that a company is just a group of people, and hiring well makes the company. But those owners do not participate in the interviews.

edit: Those owners delegate hiring to others, reasonably, but every layer of delegation changes the incentives.


Personally, I have more enjoyable things to do with my time than spend a day doing something no one else will spend more than 5 minutes on before deleting it.

I’d rather spend a day preparing a D&D game for 6 people who will have fun for 4 hours. Once the game is over, we’ll have memories of a time we spent together.

Alternatively, I’ve spent a lot of time playing with Home Assistant. The end result has been automation I use on a daily basis.

That’s why I don’t like them.


I've never seen a takehome that wasn't just one step out of 5 or 6 calls/panels. I've gone through 4 or 5 at this point and they've been anywhere from 30 minutes to 6 hours for a single step (one takehome I had was fizzbuzz, another was super complicated) instead of a 45min-to-1-hr phone call.

A "takehome and debrief" call wouldn't be too bad as an alternative to a full day of panels, but I've never personally come across it.


If you want two days of my time I’m going to invoice you for $4,000, a fair rate for a short no notice job.

I don’t have two days to give away for free to potentially multiple prospective employers. Why? That’s none of anybody’s business.


You probably value your time more :) As someone currently unemployed, my time isn't worth much, and I look forward to having something productive to actually work on.

Even when I was employed full time, my skill set definitely wasn't worth anywhere near $2000 a day lol.


> the absurd pressure and ambiguities of real-time whiteboarding or live coding, which is not at all representative of how people normally work

How do you qualify “build us a full system with tests and everything in 6hr plx”?


That sounds like a good one to nope out of lol


I largely agree that take home assignments are better than the alternative.


I love it when people who are smarter and more qualified than me refuse to do take home assignments. Gives me a better chance to get the job!


I have gotten feedback on 3 take home projects so far, and in my experience the experience is much worse than getting no feedback at all.

The responses are arbitrary, and mostly based on the reviewers preferences.

And then knowing that what they think you did wrong makes no sense, you still have no recourse.


From my experience as an employer / owner, this is a complex issue.

In the past, when I was a middle-of-the-totem-pole decision maker, we administered a take home. I wasn't really empowered to give feedback, but as far as I know nobody did. The best take home test had a joke in the comments. We were a game company, so it made sense for us - you have to care about entertainment.

Today, as the ultimate decision maker, I ask only to see pre-existing code with git blame turned on, and I review it on screen with the candidate. In this context the candidate receives feedback, and it's almost entirely focused on style because IMO that is the single greatest predictor of actual coding experience.

Let's discuss the underlying issue here: ineligibility, of like 30-50% of candidates. In this context, take homes make sense, because they occur in the absence of any other legitimate reason you should talk to someone.

For example, here are my tech company LinkedIn Talent Solutions screening questions:

Have you completed the following level of education: Bachelor's Degree? What is your level of proficiency in English? During our call, will you be able to share recent source code, wholly authored by you, of a game, application or website you worked on?

Out of 20 candidates, 12 could answer Yes to all three. What exactly should we do with the remaining 8? I can see how a take home can address the ineligibility.


Why would you expect anyone currently employed to be able to share code they've written? Presumably, their employer would not permit that.

Oh, they should also be writing code in their free time, as a hobby? Well, maybe. Or maybe they have other hobbies, or other responsibilities. Doesn't mean they don't do a good job when they're on the clock.


> Doesn't mean they don't do a good job when they're on the clock.

While this is possible, I’ve found the people that do more of it to be generally better.


> I ask only to see pre-existing code with git blame turned on, and I review it on screen with the candidate

What code are you reviewing with them? Code from their current job?


I think the real issue is the misapplication of take-homes.

If after 3-4 rounds of systems interviewing, the company came back and said: "Hey we like you, and we're interested in making an offer, but we want to check your skills..." Yeah, I'd do the assessment. Especially if I did 1 round of 45m coding. Without that, the company can't verify that I have "FizzBuzz" level skills, and that I didn't cheat on the assessment.

When it is upfront, it is rarely worth it on either side. Companies lose a large fraction of the engineers applying into their funnel, and honestly as an engineer, it is rarely worth my time. I can spend that time interviewing at other companies that respect my time.

As with many things in life: It is when, and how things are done. Not what things are done that often matters.


Yeah, this is also my issue. You get a "take-home" after a short 15 minute intro call where some HR person basically just repeat what's on their homepage and job listing and where you repeat what's on your CV (I have a theory that HR people can't read, all observed evidence points in that direction anyway) and then you're given an 8 hour take-home. Sometimes you don't even get the intro call and just an email.

Even just a "hey, we're genuinely interested and you made it to the shortlist" or something to that effect would just make a world of difference. Now I have no idea if I'm on any sort of shortlist or if they're just shotgunning this to everyone. I tried asking a few times, but I always got such vagueries in return that I might as well not have asked, so I stopped bothering.

Would also help if people made the tests at least a little bit interesting. Now it's either "do this boring CRUD thing you've done 1000 times already" or "solve several NP-hard problems that have been subject of Turing awards within an hour".

The last time I spent quite a bit of time on one of these things because it was interesting, and even put the result on GitHub (with no mention that it was for a take-home test, it "just" looks like any other project, also slightly modified to fit my needs). Then again, what's "interesting" for one person is "boring" for another.


I've handed fully containerized code, with 100% code coverage. (Separate entry point from default to run the tests.) Then been told no.

I suspect the people weren't used to having someone hand in a basically 100% guaranteed to work program ;).

(I do know 100% code coverage doesn't mean working. But the docker container helps a good bit in making sure the code will run in the environment I wrote it for.)


Every time I've reviewed these sort of code tests I've never even run the code. I just looked if the approach looked alright, and to be honest I don't even care if it gives a slightly wrong answer because everyone can make an error; this is why we got code reviews and such. It's like physics and math classes at school; at least at my school an error in arithmetic was only a small minus if your general approach was okay (one reason I'm not in favour of computerized testing/ratings, because they can only look at the answer).

I don't know what your specific code test was about so I can't really judge, but generally I've considered overcomplicated solutions to be a minus. "I asked you to write a simple CLI and I get some container environment" is not something I would consider positive anyway.


You think simple CLI.

I think: I'm writing this in Python, most Python setups are a giant mess. By putting it in a container, I can guarantee it'll run right on their system. This is LESS complex.

If my employer considers "docker run" when given full instructions complex. I guess I'm not a fit.


As an example of an effective take-home project, I was applying for a contract helping implement features for an open source project. Normally I don't like take-home project, but this project was interesting, so I was willing to give it a shot. It was basically implementing a 20-line function, but in order to implement it you had to be good enough at code spelunking to find the appropriate place to put the function, and then figure out the build system to build and debug the implementation. The implementation itself was fairly straightforward, maybe used a set or a map or something. When I was finished they talked with me about the implementation, and asked how I tested it. After I finished I realized that it was an effective interview project (even though I was annoyed at having to spend extra time), because it was designed to show if I could navigate an unfamiliar codebase and build system, which is an important skill, especially for an open source project. (I did get the contract, and it was definitely an interesting project to work with.)


I agree. I invested a few hours in a take-home once, while I was interviewing for a FT, high-dollar, in-office, position with my current employer. I think the only feedback I received after I submitted it was, "we've got two other candidates and we're moving forward with them and not you."

It turns out that that position was eliminated within 12 months. If I'd been hired then I would be unemployed by now. So I'm amazed and grateful for my current position, with an incredible flexible schedule, 100% WFH, and all the rest. Sometimes things work out for the best when we least expect it.


I only saw this mentioned in 1 other top-level comment -- ChatGPT (and yes, sorry, you may be sick of hearing about it).

There's a perspective starting to come up, ie ChatGPT is ruining take-homes. This tweet[1] is one example from the hiring side, but I wouldn't be surprised if it ends up flooding their inboxes and your submission suddenly is competing for time with a bunch of bots.

[1] https://twitter.com/djsmith42/status/1661156114883567616


The cynical side of me wants to yell "good". But leveling it out a bit: if your test can be cheated by a context-less machine given the right prompt, is it really an effective test? Are you okay with just hiring someone who has so far, at best (if we assume a far future) code as well as said context-less machine?


If a developer can complete a take home test with ChatGPT, can they also do their regular job with ChatGPT?


Take home assignments are poor indicators of someone's actual software engineering ability. Unless you think of software engineers as some nameless, faceless, machines that you feed requirements into and get code out of.

ChatGPT just highlights how poor of a job takehome assignments do at evaluating software engineering talent because it literally is a nameless, faceless, machine that you feed requirements into and get code out of.

In a live interview with a competent interviewer they're watching to see your thought process, how well you articulate your thought process, how you problem solve, etc. Very little of the interview or their feedback about it should be focused on the syntax of your code.

In addition to being disrespectful of a candidate's time, takehome interviews lose all those important signals.


Hard disagree.

Following your logic, individual contributors working remotely are not software engineers, which is ludicrous.

The thought process or tools they used to reach a solution is not as relevant as the solution itself. Interviewers shouldn't just review syntax, but efficiency, comprehension, algorithms, creativity, and many other attributes the code itself tells you. After all, these are things we spend most of our time evaluating in code reviews.

Communication is also key, which can be evaluated in a live review of the code. Programmers are usually happy to talk about their code, and here you can assess not only if they actually wrote it, but what tradeoffs they made, what difficulties they ran into, how they would further improve it, etc. This is also a chance for the interviewee to evaluate the company's code review process; what they focus on, how _they_ communicate, etc. It's incredibly valuable for both sides.

What live coding assesses most is the ability to think on your feet, and performance under the spotlight. This is an unnatural state for programming, which requires deep and creative thinking, often done best in isolation. Not to mention that it's heavily biased against individuals with any form of performance or social anxiety, which sends all the wrong signals you don't want when evaluating someone's programming abilities.

AI tools are simply an evolution of the software engineer's toolbelt, and if they're allowed on the job itself, they should also be allowed in the take home assignment.

It's a mistake to create fake working conditions for potential hires that don't exist for actual employees. At that point you're evaluating based on some fabricated environment, and won't be certain what it's like to work with this person, which is the only thing that ultimately matters.


> Following your logic, individual contributors working remotely are not software engineers, which is ludicrous.

Do you just send requirements to your remote engineers then get code back without ever talking to them?

That seems like a really bad way to make software.


Of course not. Communication is still important, and you discuss and plan features that need to be implemented. What you don't do is hover over their shoulder and monitor how they solve problems.


Take-home tests tend to be simplified and well-defined questions. Not like real life and definitely something ChatGPT would do better in.


I've been wondering if I should just use chatgpt or copilot for take home assessments now.


I strongly suspect you are reading more into the review process than exists. There probably isn't any feedback worth having.

I bet the hiring company got a bunch of "fine" or "ok" project submissions from candidates and then made somewhat arbitrary choices on who they would move forward with.


As an interviewee, I don’t really feel great about take home exercises. If the exercise is really tight in scope & it’s more of a validation of whether one can code at all, that’s not so bad, but coding exercises are almost never like that.

Coding exercises I’d be ok with:

- fix all the bugs in this repo to get the tests to pass

- implement a feature to show basic understanding of a concept

- do some quiz or puzzle to show you can achieve a discreet objective

- refactor some code to clean up a mess

- show you can use their SDK in a sane way

Ways to evaluate a coding exercise I can support:

- if you can complete this exercise, that’s all we care about

- we’ll use this exercise to implement another feature in a pair programming exercise

- using it as an introduction to the type of work one might be doing

- communicating that we like to give candidates a chance to code without the pressure of someone staring at them in a live interview

- promising an up or down vote on every exercise submitted

- doesn’t require back and forth with a someone to ask a bunch of questions, mostly because it really drags out the interview process

Basically if a take home exercise anticipates all the common objections to a take home exercise, tries to address those expectations, and has an attitude of “this sucks for us as much as it sucks for you”, I’m happy to jump through reasonable hoops.


> - we’ll use this exercise to implement another feature in a pair programming exercise

One of my take homes had a follow up like this and I did enjoy it. The interview went well but then I got rejected with very confusing feedback. It's hard to win as an interviewee.


Interviewing is a grind, even under the best of circumstances.


I run the recruitment process for our small firm. Here is our process that has minimized frustrations like the link OP describes:

Every candidate that we are interested in from the resume pool gets a screening call from an engineer. If they pass it, they get the option of their technical test being take home or over interview.

If they do the take home version, we have a marking rubric for the features that we are looking for in a solution along with sample solutions at the different levels. Anyone who does the take home assessment gets feedback about their submission based on that rubric so that the feedback is consistent across the multiple interviewers.

In some cases, we also share our solution if the candidate had follow up questions on the feedback that was provided.


My email interview, and job i got later at fastmail was for an Ops/Sysadmin job. The todo list was simple, go write a basic logrotate script. I was hired and worked there for a few years. I don't hate take homes. I don't think they're as much of an indicator of quality or a company litmus test as people here think it is


There may be a bias of opinions here of people who otherwise are totally qualified for the job that they are applying for bemoaning all the hoops that they have to jump through to show that they are qualified.

I’d say about a good one in eight people who have passed the phone screen end up with shockingly dismal results in the technical screen. There really is a bimodal distribution of skill in the industry. The goal of these assessments isn’t to see how good folks are but to weed out those who are absolutely terrible.


Right. I usually just offer up my personal repos (not on git hub). Proof in the pudding i am a truly middling programmer but certainly not faking my middleness.


Yeah just don’t do these. Refuse. Usually it’s a bad sign. And I’ve only done one such interview where the time commitment was what they foreshadowed (about an hour), and then gotten the job offer. Every other time it was an email rejection with 0 feedback. Not worth it.


These seem to be the standard for FE roles as far as I can tell. Probably 10/14 interview processes I've been in have said they have taken home assessments as some part of the process.


You could maybe offer a personal project repo or code example instead? Iv’ve walked through passion projects with interviewers before and found them insightful.


I haven't had time in the past couple of years to keep up with side projects.


What's the contract between you and the company exactly? Yes, they have no obligation towards you whatsoever and they simply are pushing the border by probing how servile are you. Make a favour to the community and refuse to do take home assignments.


I see what you're saying but I'm unfortunately in a position that I need to exit my current job ASAP. I don't feel secure enough to ask to skip takehome assignments.


> I don't feel secure enough to ask to skip takehome assignments.

But you feel secure to have your time and self confidence burned? Because you will be ghosted or sent generic rejection email.


These sorts of assignments are terrible on many levels and, in my opinion, tells you something important about the company and what it will be like working there.

If I'm presented with one, I take it as an immediate sign that I'd be a poor fit there and move on.


I just spent a couple of days on one, albeit longer than it could have been because I getting back into the language and ecosystem after an absence.

No feedback at all, or even any aknowledgement that they'd looked at it, until a couple of weeks later with a request for a tech inteview.

I just refuse to do others, sorry. If they want a hands-on test, they can pay me to fix a ticket, or even just pay me to do a test project. It's a small sum for them - for me it's multiple hours, over multiple interviews, all for free, it's basically a full-time unpaid job, just BS.


Everyone does a few of these and gets burned by the no replies. These days I'll point to an open source project I wrote. If that is not good enough then they weren't interested in judging my coding style / sample and more interested in something unrelated


Last two jobs I interviewed at give me take-home tests that took 8 hours to complete each.

...but both paid me a great amount of money to complete them so I’m not complaining at all.

Paid take-homes are definitely my favorite way to interview. The process doesn’t scale past 100 or so hires but if your company hasn’t hit that point yet I recommend it.


I did get a $30 gift card for one take home assessment which was a nice token. Better than nothing I suppose. They also had a timed coderpad which was nice in the sense I knew all other candidates would be evaluated the same.

But in the end, they rejected my project though with no feedback. Can't win.


I was ghosted by Tenable after spending about 48 hours completing their CTF challenges, so I just posted the interview questions/challenges and my solutions to my GitHub.

https://github.com/jiva/tenable_zero_day_assessment


It looks like you made the best of a frustrating situation and, at the very least, have an excellent piece for your portfolio.

With the rise in number of new security engineers all competing for few "security research" jobs (security research/hacking is the "I want to be a game developer" of security), you start getting into these convoluted hiring processes. Unlike standard software engineering, there aren't even remotely enough positions to accommodate everyone, so the bar can get absurdly high.

Honestly, if the team is asking CTF questions, they clearly want hires with previous CTF experience and should just do targeted hiring from the top teams at different conferences.

At least send people a free t-shirt if they complete the challenge.


> With the rise in number of new security engineers every year all competing for few "security research" jobs (security research/hacking is the "I want to be a game developer" of security)

I’ll believe it, curious what other options there are for all those other new “security engineers”. Compliance work?


If you're new, it's the same advice as any other field. Find a way to stand out. Build a portfolio, have great grades, come from a good university program, ping contacts from your alumni network, do bug bounties, find and fix issues in open-source, etc.


I would suspect this is more likely than not (at least in the US) due to the problem of legal liability and shielding the company from lawsuits alleging unfair hiring practices if detailed feedback to the interviewee were given.


People always claim this with no evidence whatsoever.

Please cite the relevant court cases.

The irony of these claims is that the possibility of lawsuits doesn't actually stop companies from doing terrible, lawsuit-worthy things. They do such things all the time.

Occam's razor suggests the explanation is simply that companies don't care about job applicants. And this is demonstrated in many ways.


People do sue for not getting hired.

https://www.eeoc.gov/newsroom/ruby-tuesday-pay-45000-settle-...


Yes, age discrimination is illegal.

If your feedback is "You're too old", then you probably shouldn't give that feedback to the applicant. But you also shouldn't discriminate in the first place.

If not giving feedback is an excuse to cover up illegal behavior, that's not a good excuse.


Legal does not have the time to screen 100% of feedback sent out to job seekers, and there’s plenty that a regular employee can say or write in casual conversation that can be used as fodder in a lawsuit. Hence the reason for these blanket policies. This isn’t some conspiracy theory but the reality of how corporations work.


> there’s plenty that a regular employee can say or write in casual conversation that can be used as fodder in a lawsuit

Again, unless you're talking about illegal discrimination, I'd like to see the relevant court cases cited for this "fodder".

If companies are covering up illegal discrimination, it's not a conspiracy theory, it's simply a conspiracy.

Moreover, people are claiming or speculating that there are blanket company policies against job applicant feedback. I have yet to hear anyone confirm this from the inside. Are there also blanket company policies that you must "ghost" job applicants, as many companies do? And what's the "legal" reason for that?


Not sure which part you disagree with? If it is about the existence of such policies, go talk to any engineer from Google or Microsoft or Meta or Apple or a hundred other companies down the list. There are written rules and trainings about interview conduct at every one of these companies, and they all cover feedback - what you are or aren’t allowed to say, how much you can share. And at every one of these companies the interviewer doesn’t even have direct contact with the candidate after the interview is done. The results are instead relayed by HR.


> If it is about the existence of such policies, go talk to any engineer from Google or Microsoft or Meta or Apple or a hundred other companies down the list.

1) I don't think Google/Microsoft/Meta/Apple are the ones giving take home assessments.

2) I've talked to many such engineers.

> There are written rules and trainings about interview conduct at every one of these companies, and they all cover feedback - what you are or aren’t allowed to say, how much you can share.

Ok, so what are the rules? You're claiming knowledge here, so please demonstrate it.

> At every one of these companies the interviewer doesn’t even have direct contact with the candidate after the interview is done.

This varies quite a bit. Also, many smaller companies don't even have an HR department. In any case, when you're doing a live interview, you tend to get some real-time feedback, whether explicit or implicit. It's completely untrue to suggest that interviewers get no feedback. The OP was complaining specifically about take home assessments, where you can't give live feedback.


[flagged]


> "after you stepped on his toe, did you apologize?" "Yes, so you acknowledge fault?" or "No? so you didn't even care?"

Sorry, but I don't take imaginary inventions of your own mind as real citations.


[flagged]


> as I said, you've never actually been in any court proceedings

Untrue, and I'm done talking to you.


It varies a lot, but seems worse lately.

Not to "1-up" you, but I once interviewed at a company where the interviewers were not only not allowed to tell me what I got right or wrong of their questions (where there was a right/wrong answer), but were strongly discouraged from any sort of non-verbal cues about it.

I was told later this was to prevent interviewer "A" from giving me any answers that "B" might later ask. Insanity.

Someone much later said this was their "Kobyashi Maru" interview. I didn't pass it; probably for the best.


The kobyashi maru scenario is to see how you act in a no-win situation. Kirk passed by cheating.


The greatest trick the devil ever pulled was convincing otherwise smart people that job interviews are quantitative, unbiased processes rather than emotional, biased crapshoots.


Nurse here. A couple years ago, went for an interview at a hospital. Had to complete an exam in <1h. Some of the questions included drawing the anatomy of a heart, medication calculation, etc. I was never told if I passed. Went for a4-persons interview (they were 1h late...), and was offered the job (which I declined). Is it really necessary to assess someone using an exam when they have a professional license, references, and experience? Oh well.


> Is it really necessary to assess someone using an exam when they have a professional license, references, and experience?

Yes.


Based on my extensive 10 years of interviewing experience and the sentiment in this thread, I think if take-homes are part of the process you probably should just not apply unless it's paid.


I have interacted with more than a handful of job postings where the invitation to create a take-home deliverable was clearly a) an attempt at labor theft or b) a manager using the process to catch new buzzwords and troll for ideas to take to their team, which is still a form of labor theft.

At this point I view the practice as a signal the position wouldn't be a good fit for either party.


What if you add a license to your code deliverable, mentioning it is only to be used for interviewing purposes?


I mean sure, why not; but unless they want to open source your interview code, that license doesn’t really have teeth. Just another reason to decline take-homes, IMO.


Then just send a link to this

https://youtu.be/SCaAetNzXIc ("you are an idiot" video)


Took a "2 hour" take home project after several rounds of interviews. Actually took an entire work day. They checked it and discussed it with me. Next step was the recruiter telling me they are afraid I'm not a good cultural fit. Never again.

Oh, and the project smelled like something they actually needed in prod.


I did one where it took them a week to get back to me with, "not enough documentation" when documentation wasn't even a requirement and I did do documentation- just not enough. I suspect they just didn't like my choice of Perl- don't give me a choice then :)


Some companies can't even be bothered to respond _during_ the process, let alone with feedback after!

A couple of years ago a company send me a detailed take-home project that was supposed to take about a week with an engineer assigned to answer any questions I had, and by the day before it was due, neither the engineer or the recruiter hadn't bothered to respond to a single one of my three questions I replied with over the next few days, including a basic yes-or-no clarification about whether one of the libraries they listed as okay to use included one of its optional but extremely common extensions. When I replied on final time on the thread saying that spending any more time on the project would be a waste of my time and that I was no longer interested, the recruiter finally responded a day later saying "sorry, that engineer was busy" and adding a new engineer to the thread.

It was a waste of time for me, but I also realized that I probably dodged a massive bullet by not working there. An engineer not proactively telling the recruiter they won't have time due to unexpected work or a recruiter not noticing the lack of response from the engineer assigned to their candidate might be an individual failure, but both happening without anyone involved seeming to notice or care is a pretty good sign of dysfunction at the organizational level.


The process I follow:

1. 30m phone screen with the hiring manager with standard interview questions and explain the process.

2a. Ask candidate for code example. If they have OSS work or personal projects we can look at that first instead of take home. Very few people take this path which is surprising.

2b. Take home GitHub exercise. Fork a repo for the person and have them read the readme and open a pull request.

3. 15m code review from hiring manager. Filter out very poor communicators or people that don’t know the language or know it at the right level.

4. Dole out PR review to a lead engineer. PR review for day to day work is extremely high priority. Fitting in a take home usually can get picked up in a day too.

Easier said than done. I’m sure all candidates that are passed on are frustrated and would like more feedback. Sometimes they ask for followups with more but I push back if I think what was given is fair already.

The hardest part to me as a hiring manager is that hiring is a winner-takes-all. If someone has a good submission but there are two more people in the pipeline what do I do?

It’s also hard to do everything perfectly and timely for a candidate given all the other priorities going on.

I hope some candidates take comfort in how the process reflects the company. If you’re unhappy with the process you get you might be unhappy working with those people.

But if you’re unemployed you might be happier employed than not…


It's pretty insulting to be frank.

As are interviews of any kind without feedback. I'm not referring to holistic feedback (let alone an explanation for why they don't move forward). I mean the kind where they grill you with question after question (often poorly phrased and sometimes insultingly basic) and don't provide any indication at all as to whether your answers are good or bad or not.

They just keep going on, poker-faced - saying, in effect: "Don't speak unless spoken to".


> I mean the kind where they grill you with question after question (often poorly phrased and sometimes insultingly basic)

I mean you'd be surprised at the number of people that can't write a for-loop ...


The point is that for whatever level of fizz-buzzing they feel they need to - far too often these companies can't fizz-buzz themselves when it comes to basic interview hygiene. You know, like maybe reading the candidate's resume and cover letter (or talking to the recruiter to whom it was explained "I haven't worked with language X at all, is that OK?") before asking them to code over the phone for language X, for example.

Or even post a job req that accurately reflects the true needs of the position in the first place. Or (getting back to the subject of this thread) when giving take-homes, provide reasonable instructions as to what is expected and by when (and under some reasonable time frame), etc -- see for example this chestnut, posted just now:

https://news.ycombinator.com/item?id=36452104

A two-way street, and a bit of common sense. That's all I'm asking for.


I would only take-home for interns from college, if anyone. High quality experienced hires will not do this because the market is such that no one subs take-homes for interviews; it's always in addition. They will simply say no, and you'll lose them. This is a classic example of a test that has adverse selection. You think by increasing the difficulty of hiring that you get better people, but actually you get worse people because the better people do not need to do this.

This also means something about the person running the test, though. They are not experienced enough in the engineering hiring pipe to know what to do (i.e. they are unable to differentiate good engineers from bad engineers) and either do not know or do not care about the adverse selection. This means their team is likely to be mostly mediocre engineers. You should pick it only if you need a job.

Essentially, the interview process is part of how you find out whether the team you're going to be working with is good. If you believe a certain process would not result in being able to differentiate a high-quality eng from a low-quality eng, and they're running that process: you should increase your likelihood that that firm contains many low-quality eng.


More frustrating is when you complete the assignment (all tasks) but still fail because an unspecified task was not completed.

My biggest learning about software interviewing is spotting immaturity. I have it down to three points:

1) The only purpose to software is automation. It’s not data, entertainment, or anything else. If an internal team is not aggressively applying automation like static analysis or automated tests to their code they are not mature and I will not work there.

2) Does the team or employer understand the software platform they are hiring for? Software platforms are things like web browser, a game engine, terminal emulator, and so forth. I most commonly see over confidence about this when the employer is fully reliant on some framework to do everything. That is serious code smell because any deviation or problem outside the framework results in serious emotional problems. I do tire of emotional junior developers trying to define my job as some sort of infantile pacifier.

3) Can the team communicate in writing? I mean specifically the other developers.

These failures are what I see in 95%+ of software employers. I am willing to take a massive pay reduction to join an employer that doesn’t fail from such immaturity.


This strange take-home policy presumably originates in the presumptuous view that an interview is a one way process - the company is auditioning you only - and persists by assent of participating engineers who are happy to concede the point rather than demanding the obvious fix for the issue - payment for the suggested time.

I have no idea why anybody invests hours of their time in these situations without some guaranteed return on it.


The expected return might still be positive, even if it's not 100% likely.


This has been the story of my professional career.

For example at one place here, Centro Ático of Pontificia Universidad Javeriana, they assigned people to create a whole website (several pages pulling media) with plain HTML+CSS+Javascript, which I did. I don't remember how much time they gave but it took days. Even I pulled an all nighter on that because it was that demanding and I had a full time job at that moment.

After that, they selected the ones they liked the most and called them for a presential interview. They called me, I had to make up an excuse at work so I could go to that.

Ghosted. Never heard from them afterwards.

Same with many other places where they make you feel at home and the communication seems to be great - most recently, at "RebelMouse". With the Silicon Valley Bank thing they told it wasn't a sure thing they would hire someone. I did the test anyway. After several weeks they told me they finally were reviewing my test. But more weeks passed and never heard anything. Asked them about that, even saying something like "I don't think you're the kind of company that ghosts people!"... But turns out they are.


I recently had a bad experience with an interview project with CodeClimate. I put real effort into their assignment and submitted it to them. I wrongly assumed that as a technical company, they'd have a reasonably respectful process in place for developers. I can accept that my solution somehow might not have been the absolute best implementation the world has ever seen, or they decided not to proceed with me for any reason, but about a month later I got a really impersonal generic rejection message with no indication that anybody even looked at my submission and no response to my very polite and professional followup.

The worst though is that some companies are allegedly (cough, Clipboard Health, cough) using job applications for free labor according to Glassdoor reviews.


The mistake here is that the take home assignment isn't paid.

Why do programmers accept to spend all this time on a process that might end up in a rejection? I don't mean just on take home assignments, but on the grueling in-person process that can take days or weeks as well. It's a collosal time sink with an unknown outcome.

The interviewers are paid for their time spent interviewing, so it's only fair that companies respect the time of the interviewee, and pay them as well, whether they pass or not.

We often forget that interviews are a two-way street. The person is interviewing the company, as much as they're interviewing them. The risk if it doesn't work out should be taken equally by both sides.

If a company doesn't respect your time, it's probably a signal that they will disrespect you in other ways as well once you start working there, so it's best to move on early.


While I largely agree with the sentiment: this kind of system is enormously open for abuse - it directly pays abusers. And being flooded by applications by abusers doesn't improve the overall state of things.

It's a nice dream, but I really don't think it's feasible outside extremely niche scenarios. And those can be (and often are) quite a bit more flexible in how things are run already.


Maybe, but the interview process doesn't formally start before an initial screening. Once both parties are willing to commit to the interview, their time should be equally compensated.


No HR function that's equipped with the typical level of risk aversion is going to allow feedback on why a candidate was not selected. There's simply no upside to it.


There's always an excuse like this and TBH I think it's lazy.


The answer? Or the behavior of the HR department?

Candidates are looking for feedback on what they can do better. HR doesn't want hiring managers to form opinions on that. HR wants (as near as possible) objective feedback on whether candidates meet a hiring bar, and for a hiring manager to select amongst those that do choose the best candidate. Ideally, they want you to make sure you're making serious consideration of candidates in under-represented groups in order to show commitment to DEI targets.

Oftentimes, the reasons why candidates don't get selected come down to things like "you exaggerated your involvement in a key project and we figured that out during the interview" or "you're fine, but we found someone we liked better".

How do you think honest feedback of "you should stop providing misleading descriptions of your work experience in your resume" goes over?


> they want you to make sure you're making serious consideration of candidates in under-represented groups in order to show commitment to DEI targets.

It's illegal to hire based on demographics. I'm sure lots of places do it. It's lazy to me for HR not to provide feedback to the candidate based on some "upside" or vague liability.

> How do you think honest feedback of "you should stop providing misleading descriptions of your work experience in your resume" goes over?

This sounds like a really contrived scenario. I've been the hiring manager for several positions and I've never seen enough evidence to call out a candidate for something like this. Anyways, if it were true, just find a nice way to say it: "we were unclear about your work experience and the details didn't match our job description".


I'd blame it on HR. I do the the technical part of the interview for my company. We have a very easy short take home project that's given out after the phone screen. I'm not a big fan of take home projects, but they are a good filter. I purposely made it representative of the type of problems we solve, but very simplified. A junior dev should finish it in under an hour. A senior dev in our field could probably code it up in 15 minutes. When I review the code, I have a small rubric and also attach a paragraph of comments to it before it goes back to HR. Apparently, that all gets filtered to a yes/no call back to the candidate. I didn't know this until I ran into a candidate and he said all he got was a "we're going to pass" call from our HR folks.


I'd blame it on the CTO, for not knowing or caring what HR is up to.


Sometimes I help to create take-home assessments and provide a feedback. It is not unusual for a candidate to submit something that is copy pasted from some introductory tutorial and / or completely ignoring most of the requirements. In cases like this I do not have much choice but to respond with a generic polite rejection. Honestly, I don't believe company should be responsible for providing any kind of feedback for your assessments - it's not a bootcamp. You are either competent and therefore suitable for the position or you are not, and it was your choice to apply. Moreover, you are not the only one applying and meticulously creating a report about what you have done good or bad, and where you can improve takes a substantial amount of time.


> In cases like this I do not have much choice but to respond with a generic polite rejection.

Well, you do have a choice. You could tell them that they were rejected for copy/pasting or because the requirements were not met. You don't have to do into more detail than that.


> Moreover, you are not the only one applying and meticulously creating a report about what you have done good or bad, and where you can improve takes a substantial amount of time.

The report can be 1-2 sentences. It doesn't need to be meticulous or a "boot camp".


Do you encourage candidates to re-apply in the future or do they re-apply on their own?

If-so, you're doing yourself a favor by providing feedback so that they might improve by the next time they interview.


I think this is just symptomatic of how hiring for technical and non-technical roles alike is somewhat broken.

Both companies and candidates have realised that the signal:noise ratio is against them.

Companies solve for this (badly) by cranking up the funnel, but then are poorly resourced in handling that many suitable and unsuitable applicants.

Candidates solve for it with recruiters, keyword-packed CVs etc.

If we’re honest, hiring just sucks at the moment.

Perhaps we need a better way of ensuring job specs are correctly self-discriminating between good enough and not-yet—and can drive useful feedback for all, and a better way of handling that first pass that ATS-screening so that take-homes can serve a deeper purpose and be resourced correctly.


You can weed out a bunch of people by talking about the nature of the work…

“To be honest, a lot of the work is refactoring 20 year old technical debt. It’s tedious work for anyone that doesn’t enjoy that, but the work pace of the work is reasonable & we understand that it takes time to confidently make changes to our barely tested spaghetti code. If that sounds like your kind of work, we want to talk to you!”


And wouldn’t _that_ be amazing?

I think if you approach job ads with the understanding that they want as many applicants as possible then you start to look at what they _do_ say very differently.


Hah, this is something close to my heart.

A friend and I tried to start a company around this, basically resolving three big issues we saw with take-homes:

1) No feedback. If you spend a couple hours on something you should get meaningful feedback.

2) No clear criteria. Is the hiring company going to slam you on correct syntax? If so, you should know that upfront!

3) Time guidelines that had no enforcement and thus led to an arms race where candidates would spend ever-increasing amounts of time and skew the standard of what "good" is.

So we fixed those things:

1) We provided the feedback, not the hiring companies, so legal liability was non-existent for the hiring companies. We double-blinded the process as much as we could (evaluators didn't know who the candidate was and vice-versa).

2) We told candidates upfront what they'd be evaluated on. Not down to the level of "you must implement this problem using a max heap", but we would say something along the lines of "The company is looking for an academic algorithmic solution to this problem" or similar. We would then only allow evaluators to evaluate them on these axes and nothing else.

3) We also strictly enforced time limits by basically telling candidates "hey you'll have 2 hours to submit from the time you hit start and see the prompt, so please make sure you have two hours from when you hit start." -- not ideal, obviously, but the best we could come up with to resolve #3 above.

As you can probably imagine, the market just wasn't really there for this. I think candidates generally enjoyed it in comparison to the vague, unending slog that most take-homes are but:

1) The value prop just wasn't really there for most companies: They mostly use these types of evaluations on more junior candidates, and unfortunately the hiring market for junior candidates is highly skewed towards the employer.

2) More surprisingly, we realized the time their current engineers and managers spent evaluating these takehomes just wasn't really a consideration for them. We tried to frame it in terms of "here's how much it costs you to evaluate these take-homes wrt time spent vs. us", but it was a difficult sell regardless.

We actually had the most success evaluating candidates from more non-traditional backgrounds upfront ourselves and then charging a placement fee if they were hired, but we ultimately didn't really want to continue that.


> More surprisingly, we realized the time their current engineers and managers spent evaluating these takehomes just wasn't really a consideration for them. We tried to frame it in terms of "here's how much it costs you to evaluate these take-homes wrt time spent vs. us", but it was a difficult sell regardless.

employers don’t know, or seem to care, how much it costs them to replace and/or onboard somebody. usually because they have no idea how they’re off/on boarding hires.

not too surprising that they wouldn’t care about the cost of one component of the hiring process.


Side note: If anyone is interested in this space, highly recommend you check out https://www.woventeams.com/


i've done lots of data science take homes, and the vagueness is a huge killer. they'll hand you a dataset and give you an open ended prompt to 'analyze it'.

do they just want to see that yoiu know how to load a csv file and make a barplot? or do they want a showcase of advance statistical modeling to see where your ceiling is?


If you want feedback - go with third party recruiters, they are eager to collect feedback themselves and the hiring managers share with them more openly than with a random candidate. Of course, there are issues with this way of job search (recruiters pressuring you to take lowball offers, double submitting, putting a negative weight on your application with their fee, generally being unable to work with companies that do not work with such recruiters, etc) so it might be not worth it. After all, the only feedback that really matters is the number on your paystub.


What possible feedback are people looking for after an interview, though? I've done 100s of interviews throughout my time at FAANGs. The bulk of the feedback generated is just low-value snap judgements made by people who really don't want to be there. It's a dice roll. You either land in a loop that (a) has more people that like you rather than dislike you, and (b) asks questions you know how to answer, and, by far most importantly, that you answer in the way that lines up with their biases, or you're dropped as an incompetent rube.

One of my coworkers at Amazon was leaving for greener pastures. Before she left, for the lols she showed me the feedback she left when she interviewed me. It was something absolutely brutal along the lines of "doesn't appear to be able to code at all" (I was offended upon seeing this "feedback"!). My point being: interviewing is stupid, and, generally speaking, there is no valuable feedback to be had from the process.


I agree, the feedback often makes no sense (this is probably why most companies do not give it to candidates), but if you want it - you can get it from a third party recruiter anyways.


If anyone is interested we created a platform to solve these exact problems.

www.geektastic.com

We provide take-home assessments as a service for hiring teams. We have a team who carry out reviews on candidates' code challenge submissions in their spare time (paid of course). Their focus is to provide detail to the candidate that the hiring team hasn't got time to do. Our bar to join the review team is set very high and we are assessing their ability to provide well balanced feedback.

Every submission gets a detailed line by line review, with detailed explanation from our review team explaining the pros and cons of the submission. We're looking at things like code quality, solution design, problem solving skills and test coverage.

We also provide the ability for the candidate to write comments on the review to try and avoid the usual one way street that happens with lots of technical assessments. I have seen these comments get a candidate a next stage interview when their initial submission was deemed not to be a 'pass'

Each challenge has a set of review guidelines that aligns of team of reviewers to standardise reviews.


This is expected so don't feel too bad about it and move on to the next interview.

Recruiting is one of those things that is beyond broken and no one knows or cares how to fix.

I interviewed for all types of companies, big and small, from Google to Amazon to small startups and local university spin offs.

Overall the experience sucked, before, during and after the interviewing process.

I remember at Amazon, we (candidates) being welcomed by HR staff wearing shorts at a business center in Philadelphia.

Although this may not look like a big deal and in the spirit of a tech company, it really tells you about the judgment some of these people have and those who hire them.

If they don't care about throwing on some minimal business-casual clothes to meet up with grown-up professionals who took the time to flight from all over the country to be there, how will you expect they will give a damn about anything else during the process?

This is an extreme case for sure but I have noticed the same unprofessionalism at all levels, not only dressing codes, anything from being late for the interview, to last second changes poorly communicated to the candidates, interviewers looking at their phones or laptops while you are talking to them to even missing appointments altogether.

I did have some good experiences, one small startup out of MIT labs aced the process so well that I borrowed some of their methods for interviewing myself at my current company.

But in general, the recruiting/hiring process is unprofessional everywhere and I'm just glad I probably will never have to go through that manure again.


Fortunately for employers, there's no shortage of applicants willing to take these insults. This is unfortunate for applicants who respect themselves, their own time.


Hiring managers should realize that only the most desperate candidates will agree to do take home assessments, or even apply to companies with such a process.


Last time I went through the hiring process, I asked my recruiterto prioritize companies that had homeworks instead of companies that expected whiteboard or on the spot coding in an unfamiliar environment with multiple people looking over my shoulder.

Leetcode grinding for a whiteboard is a worthless skill outside of the interview. I'd rather code up something. If the project is over burdensome you can always pass.

The best homework I got was a "working" project that purposefully had bad practices scattered throughout. You could elect to fix the mistakes or just document them for the interview.


So you would rather do live coding session on a problem that is presented to you on the spot?


I will never agree to a take home assignment unless the company agrees to a debrief.

A (reputable) company asked for a take home assignment after a "group interview" as a college hire, and as a college hire I didn't know to tell them to pound sand.

Same experience as yours, I spent twice as long as I was told it would take, then was rejected with feedback that was generic and very subjective.

Admittedly my career went in a different direction (company was realtime ads, I went down the distributed systems rabbit hole) but I still disagree with their feedback. Given the chance to do the assignment again (assuming, for the sake of argument, that I wouldn't laugh in their face) I would come up with a very different solution, but still one that I think would yield similar wishy-washy but negative feedback.

Moral of this story is to never accept take home assignments, and don't get discouraged if you get poor feedback from one because it's essentially a distilled version of everything shitty about onboarding & code reviews, rolled up into a single package


If it's so insulting, stop subjecting yourself to it. Let me guess, white boarding is even more insulting, because don't they understand a developer of your experience can't be bothered to prepare for it?

No one owes you anything. If these companies display poor behavior during the interview process, add them to a list, name and shame, whatever you think is appropriate and move on.


Absent unusual circumstances (I did actually do one recently) the answer when asked to do anything alone is no, with an explanation that such tests do not respect my time and a hint that asking for such commitments might be discriminating against people whose protected class life situation prevents them from giving away hours of their time for free.


Not one of the comments has mentioned a main reason why feedback isn't shared is certain people don't take critiques very well, or may think the reason for rejection is benign that it's not enough reason to reject.

The company doesn't want to spend time getting into an argument with the person over it or potential legal issues arising from such feedback.


Then they can interview candidates without the takehome bullshit.

Companies refuse to even read a resume/cv these days. They make you sign up for an account specifically with their company and jump through all their hoops and type into fields exactly the information covered in the resume/cv/cover letter. They have you write out answers to questions they later ask you again in person.

Of companies don't want to spend the time that's fine, but they shouldn't expect prospective employees to waste exponentially more time. And arguably the job hunter is in the more desperate position, as evidenced by the fact we are willing to jump through all of the aforementioned hoops. Unpaid. For a chance at something. For which you may receive no feedback.

Fuck that it should be illegal to do so unless they compensate us.


I agree with the entirety with what you've said. Take home or not, most are reluctant to give specific feedback in general because they don't want to argue about their feedback to the interviewee and there is a potential for legal exposure if the interviewee thinks the feedback was unfair relative to some other group and wants to take action on that.


> The company doesn't want to spend time getting into an argument with the person over it

I don’t want to either, but if the feedback they send is clearly bonkers then what am I supposed to do?

If you have a dialogue in the first place you can correct their misconceptions immediately, but if they use take home the only time you can do this is after the fact.


It's only bonkers if the actual feedback is deemed so by you, and so they don't give any to avoid any possibility of a response that they must respond to


I have family and work responsibilities that take up more time than I already have. There is no possible way I can carve out multiple hours to do an "assignment", especially one that has such a low probability of working out in my favor.


Only once I did a take home and I passed & got an offer. Even though it was a mildy challenging I hated it.

Tbh if I didn't have knowledge on the framework I know I wouldn't have been able to complete it. However I found it disheartening the fact that they just checked that the app worked, didn't have any bugs but didn't even cared to ask around if I actually did it.

These days I can't even imagine how easy it is to cheat with ChatGPT. Last guys I have interviewed I tell them to download a repo with a sample app and poke around with questions. I can quickly tell if someone is BSing and how deep they actually know about the language/framework. I've found this to be better than live coding or take home.

On the other topic, remember that the hiring side doesn't have an incentive to tell you how well or how much you suck at coding, interviewing etc.


Yep. It is now my policy to not do any take home interviews. You should do the same. They don’t respect your time.


We had take-home tests at my previous place, and the main thing we tried to communicate throughout was respect. Evidence suggests we managed it in most cases, as we regularly had candidates thanking us for a good test after rejection, and on multiple cases I saw/heard rejected candidates encourage others to apply.

How did we do it? Limited time, feedback, and benefit of the doubt.

We put a time limit on the exercise. It was a 1.5-2 hour exercise, with a 3 hour cap. Some felt this added some stress, but many appreciated that it didn't take up their whole weekend or multiple evenings. In the few instances that candidates overran this massively (~8 hours) without notifying us in advance, we saw no difference in their scores.

As for feedback, we were scoring against a rubric that was quite detailed across multiple different categories. This forced us as reviewers to break down and justify, not just say "I don't like it". And because we ended up with 5-10 bullet points of things we did or didn't like, there was useful feedback we could give. There was also the added benefit that we all knew the problem inside and out, and so could drill down to the essence of the solution quite easily.

And lastly, the benefit of the doubt. No software engineer writes perfect code every time, and code review is an important part of the job, so if the feedback boiled down to "I don't like it but it's close", or if it was something that with regular code review would have managed to pass the bar, then that's what we'd do. We'd schedule another round to discuss the code with the candidate, see how they responded to the feedback, and hopefully move them on to the next round.

Why doesn't everywhere do this? It's actually pretty hard. You need a recruiting team who are very on top of the logistics and running the process, and you need reviewers who know the problem well, are bought in to giving a great candidate experience, and who are good enough to be able to judge others. In my experience most places don't have much of this.


I might be jaded after many years in the industry but to me not getting interview feedback is the default state of things. I don’t know why anyone would expect it. If a company provides it - great, it just means they haven’t yet run into a lawsuit and HR hasn’t put a blanket ban in place.


Name and shame on Glassdoor. I will skip companies that I can tell are a garbage fire of hiring or working


Maybe offtopic: are take home projects or whiteboard excersises still needed for senior and staff positions? I had the impression that for anything below senior, yes these things were required, but for anything equal or above senior, the focus was mainly: systems design and communication skills.


When I was interviewing a year ago. They were required at all but one role I interviewed at. And I told them to test me anyways, despite them wanting to waive.

If you are interviewing as a software engineer, you should be able to write FizzBuzz. Yes, it is stressful. Yes, I hate leetcode too. But, a company has every right to check that you at least know the basic skill you say you do.


FizzBuzz, yes. A whole greenfield web app that would take this company several weeks to deliver? No.


(laughing) There are limits.


I've seen both. In my experience it's been rarer to see a process with no coding involved, but it happens here and there.


It's been a requirement for the majority of interview processes I've been in for senior and staff positions.


Happened to me too once, and I agree: it's just insulting and disrespectful. It was also an ssignment with a hard time limit in my case, something that I will likely never again agree to (unless I get offered insane amounts of money or it is somehow my dream job but...).


Condolences. You can think of it as "bullet dodged", but that's not much consolation when it keeps happening.

Take-homes are much better than Leetcode interviews, but smashing yourself with a hammer in the thumb is much better than in the face.

I think some companies do an initial take-home "screen" just to filter to the candidates who are interested/desperate enough to invest in the "proof-of-work".

To reduce the jerky blow-off rate, are you getting at least a solid call with a manager or engineer (not just a recruiter phone screen) as a sign of their investment, before you invest in the take-home?

Balanced dynamics and mutual investment are good.


The feedback will just be some stupid bullshit that's complete subjective opinion with no use to you in future endeavors anyway. My code is apparently both "sloppy" and "overengineered" which you'd think are kind of opposite complaints.


Take homes, whiteboards, leet codes, iq tests, and every other random assignment in the interview process do not matter if the company has unrealistic expectations and looks for a 100x engineer in them. Most of the companies fail to find anyone and keep searching forever wasting their internal resources.

That’s why i really like my current company where we don’t set unrealistic expectations to candidates, we let them solve some basic coding task live and mostly talk to people. If the person can write code, a basic sql and clicks with everyone, we hire them. And our tech recruiter does wonders properly filtering cvs and recognizing fakers by just talking them. That’s probably the most important stage in the pipeline.


Did you sign an NDA about the assessment? Share your work with the world and ask for its feedback.


Yeah, this happened to me this week. Honestly, I wouldn't worry about it. It sucks that they don't want to see more but it could be any number of things that they isn't working out for them. This experience you have doing the takehome, it's definitely going to help you out in the future in your next interview. Just think of each interview as another opportunity to learn and grow your interviewing skills :)

But really, I think that interviewers should be gracious about takehomes. And if they're not, that gives you and indication of what kind of place it is to be.


Yea using it as a learning opportunity is useful.


Has anyone ever gotten a decent job from an interview process like this one?


TBH my past 2 jobs requested a take home assessment. 1 was a great job, 1 has been a nightmare.


I refuse to do takehomes as any part of the interview. My principle is out of fairness and mutual respect of the value of time: For every hour I spend interviewing, someone from the company must spend 1 hour too.

EDIT: Take-home-styled coding exercise in a pair programming setup is fine --- I've taken a few of those and they were actually quite pleasant. The task is not the issue --- respect is.


I think the biggest reason we don’t get feedback on take-home work is that the company people have decided to not move forward with you, and any additional time spent on you is time not spent hiring the person they want. You’re turning in homework for a course the instructor has just kicked you out of, asking if they’ll grade it anyway.

Adjacent to that, they probably don’t want to open themselves up to an argument about your work, while you try to convince them to give you another chance.

That doesn’t make it any less frustrating though.


For every 9 leetcode interview companies, there is 1 that doesn't do that and would rather pair program or work on an actual ticket together for half a morning.

Adjust your sights on those and it's lovely. More often than not those companies are great to work for across all metrics.

Most of these leetcode interview companies just follow what Google does for that reason alone. And Google does it because they have to cull 20,000 applicants somehow in the first wave. They just _can't_ interview that many people from the jump.


I try my hardest to design practical interviews when hiring. It takes time, but the results are worth it. For example, if you're an Ops style person, you would be SSHing onto a server or troubleshooting a configuration management system.

Platforms like Woven do this automatically, but they miss the mark. A good half (if not more) of the value is observing the candidate working through the problem and responding to collaborative feedback, not in actually solving the technical issue in question.


Not a take home assessment but after I went in for a half day in-person interview at one company, I was totally ghosted by everyone, including the recruiter. They didn’t bother to send a rejection email and didn’t bother to respond to any of my emails.

I used up vacation time to meet with them, I would expect some type of acknowledgement regardless of the outcome. I didn’t even care if they gave me feedback. Just give me a “yes” or “no”.

The company was Splunk.


The most bonkers I've started on was for Teleport/Gravitational. Like a 20 hour endeavor with very, very lopsided time investment. The feedback was mostly glib and seemingly from people overworked just trying to check the "I left feedback" box. Passed through two milestones but ultimately the situation soured on me so I bowed out.

Check out the Glassdoor interview reviews to feel better about pretty much any other take-home haha.


Yeah I refuse these take-home tests. I counter with "put me in the room with one of their engineers and let's solve a real problem". I also judge them by how much skin they put in the game too.

Ive done the "2-4h take home test" once. It was blatantly clear they wanted a day or 2 of free labor, with no real feedback. So, no. I'm not doing that. You want me to do a day's labor? Pay my standard rates.


Are you really going to be happy with 5 minutes of effort from a recruiter? If you got 5 minutes of their effort, then you’d ask for 15 minutes from an engineer.

You want these jobs more than they care about you as a candidate. If one of the companies that sent you a rejection had instead hired you, would you be insulted about how they treated the other candidates? No, you’d probably take the job and be happy you got it.

It’s a game and you’re each playing by your own rules.


Vague feedback is just as bad. I once “failed” a take–home assessment and was told that my solution “wasn’t object–oriented enough”. Bullet dodged, I’d say!


I have even been waterboarded by a corpo once, on site, and given no feedback.

Yes, it was close to where I lived so I didn't lose so much time or money but I would at least expect some kind of response.

As for take home assignments, your mileage may vary but up until recently I was receiving feedback from most companies. As time goes by though, I see more and more providing just a generic template response, if any.


It's a really bad smell if they don't give you feedback - is that a company that can provide quality code reviews to its staff, or not.


I hate take homes. I think my current company has a better approach by having new candidates do a code review of a MR we prepared. The MR is full of bugs and problems. Mixed code styles, weird commit messages, etc. And it’s not about finding every single one, but to see what the candidate is looking for to identify bad code and to see if they can review code.


The reasoning for not giving you feedback is:

"Why should we give you feedback and basically prep and improve you for your next interview at a different company?" If you are given feedback, the company is basically doing the prep work for the next company at their own expense. There is no ROI in giving you feedback if you did not make it.

It's harsh, but true.


It's the right thing to do to treat people with respect and not come across as a trash company full of assholes. Candidates talk too.


There is no right or wrong here. You have an expectation for feedback, that is true and fair, and they have no obligation to give it to you. Would it be nice? Sure, in your eyes, probably not worth it in theirs. Is either of you in the wrong, no. Managing expectations is always advisable.

If you go on a date with someone and they decide not to go out on another date with you, they have zero obligation to tell you why they won't be doing the second date. As nice as it would be for you to know why.


I got some very thoughtful and actionable feedback after an interview at IBM and it certainly made me think better of them.


If you are interviewing for a senior-ish position in some specific domain, and it has a take-home, and it recommends a time limit, and you blow past that time limit because you aren't familiar with the domain, then either:

- You aren't familiar with the domain and its working as intended, you're just bad at time management.

- You're not even familiar enough with the domain to know you're doing a bad job.

- You learned the whole domain and it was a good use of time.

In any case, slogging through a take-home you're not qualified for should warrant an introspection on your part. It should still warrant a response from the company if you submit it, and it sounds like certain companies have a ghosting problem. But it's not surprising that people who spend 10 hours on something that should take 2 hours are perhaps not coming up with good solutions and not qualified for the job they're interviewing for.


I don’t do them period. If you’re going to take my time, burn an engineer hour as well to show i’m a serious candidate.


I refuse all homework assignments as a matter of principle. Instead I link them to by github which proves beyond doubt that I'm a very strong developer. If they are not happy with that for some reason (e.g insinuating that I'm a "diva") it's not a good employer for me anyway.


On my last gig we were specifically forbidden from giving feedback because of nebulous "liability" reasons.


honestly if they dont pay for it or dont provide constructive feedback, upload it to github and link in your resume


Just say no to takehome assignments.

You can apologise and say "sorry but other companies have ruined it for you".


I take this as a litmus test and refuse to interview with a company that has pedantic hiring practices.


This is the correct course of action. Your labor isn't free. A company can afford to pay you as a consultant if they want to try before they buy.


Ask for feedback in writing before doing the assignment. If they agree great. If they don’t then walk.


That's a good idea. I'm going to try it out.


I agree 100% Politely decline and move on. Those 2-4hrs can be better spent applying to other positions. Think of all the other openings you could apply to in that amount of time? 25-30? A conservative 10, with a good cover letter.

Good luck with your search!


Simply say, I’ll agree to do this as long as I’m provided feedback with what the wrong/right answers are, and I certainly understand that your decision is final.

This will tell you a lot about the company and if you should be working there.


At least you got an assignment. On a few of my last attempts I have been rejected earlier on with little feedback. I sometimes get some generic stuff about YoE, meaning some Principal engineer at a FAANG also applied.


This could be an opportunity for a business to review the take-home assignment.

The idea is, if you submit to company A, they rejected without any feedback, let's other companies know about it, and both of you is in a win-win position.


Glassdoor - interviews section


The fact that roles called “senior” and “staff” are requiring take home tests, also says a lot about the meaninglessness of titles outside of the major tech companies.

How much are these “senior” and “staff” jobs paying?


A software labor union with licensing would save a lot of trouble.


Tell them a firm No to any of this nonsense.


I had a terrible experience with Blinkist for this. Absolutely no feedback, just a generic rejection message


This could be an interesting product idea.

Link the assignment for the applicant to the dev team to anonymously help with.


I honestly can't believe anyone would want to work for a company that gives you unpaid homework.


There is an asymmetrical power balance here. And there are lots of unknowns. You are often working with a lot of constraints and in the moment, you'll make concessions. So I don't blame the candidates.


Take-home should be made illegal. It offloads the cost of hiring on the candidates.


There are a lot of companies that give take home projects only to get free work. I always pass.


Don't do homework. Hirers will learn to respect the marketplace.


i still prefer this over leetcode which requires a much larger investment

also put a copyright/license on the code you submit. publish it if you don’t get the job.


imho. those unpaid (!) take-home assignments are nothing but a BIG red flag to just avoid such companies if possible.


I think take-home assignments are great, but companies using them should be willing to put in the time to properly review and give feedback. And - spoiler alert - that takes a whole lot more effort than companies like fly.io are putting in, as evidenced from the comments on here.

The reason take-home assignments can work well in my opinion is that it allows the candidate to show off how they perform in _real world_ scenarios - as opposed to live coding computer science toy problems. Interviewers can assess what code looks like that resembles the code candidates would likely write on the job.

What companies like fly.io are doing wrong: if you ask candidates to put in multiple hours writing code for free, the _least_ you can do is give them an interview to give the candidate feedback on their assignment, even if you're going to reject them. In case of rejection, it doesn't have to take more than 15 minutes of time from the interviewers (which is far less than the hours the candidate put in) and the candidate walks away with (1) valuable feedback that can help them improve and (2) a positive view on the company and their hiring processes.

What many companies don't understand is that your goal should be for candidates you reject, to still become "promoters" of your company.

Here is how we do it at Source.ag:

1. If the resume is promising, we always do a first screening interview with the candidate _before_ we even consider them for a take-home assignment

2. The candidate gets the take-home assignment after passing the interview, works on it and sends us back their solution

3. We schedule a technical interview with the candidate. This interview can basically have 3 formats:

a. the assignment was bad: we tell the candidate at the beginning of the interview we don't intend to continue with them and give them some personal feedback on the assignment

b. the assignment was great: we take time to ask the candidate about their solution, to elaborate on their tech choices etc. and generally use the interview to understand if their "talk" is just as good as their "walk"

c. the assignment was ok-ish: we use the interview to figure out what the candidate's engineering knowledge is and to what extend it complements what we got from the assignment. It could be that a candidate didn't perform super well on the assignment, but when talking to them we discover there is enough skill, knowledge and team-fit to work with. This happens mostly for junior-medior candidates, not for seniors.

This is a time investment for us as a company, but has gotten us good hiring results and a positive reputation when it comes to hiring practices.


Brevan Howard do this. Avoid them


It's unethical


Should be illegal


You're so 2013... Nobody takes home assessments now. Why bother, there are

1) plenty of companies who do not require that 2) those who require do not respect those who spend time on these assessments 3) even if you post about this everywhere, there is a very small chance you can change anything

There are other job search hacks out there that work x100 better than assessments. Can't share them for obvious reasons


> plenty of companies who do not require that

I'd love a list.

> even if you post about this everywhere, there is a very small chance you can change anything

What's your point? Don't post?


I am happy to hear you'd love a list, I can compile the list for you for the little price (1/2 of my hour pay), reach out to my email in profile.


No thanks




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: