Hacker News new | past | comments | ask | show | jobs | submit login

We used this a couple times at Sandstorm back in the day. At the end of the interview the candidate would play Factorio cooperatively with the team for a while.

I think it is remarkably effective at identifying the kinds of skills and personality traits that a software engineer actually needs to have in day-to-day work. You can find out if someone is self-directed, how fast they work, whether they produce clean designs or spaghetti code, whether they are good at cooperating or tend to go off on their own, etc. Some people will just sit and watch and do nothing unless instructed... that's bad. Some people will build stuff, but with obvious efficiency flaws and "bugs"... also bad. Some people identify what needs doing and get it done effectively but without trying to be perfect... that's good. Factorio essentially compresses real-world work patterns into a shorter time period, giving you the chance to see how someone works in the space of a couple hours. I don't know anything else that does that.

But, obviously, it's problematic. A person who has played the game before will have a big advantage. A person who doesn't play video games at all will have a big disadvantage. For these reasons, I don't think I could seriously recommend this approach be adopted more widely.

But then, I don't know of any approach to interviewing that I think is fair. Everything has problems:

* Regular interviews bias towards people that are charismatic, not effective.

* Coding interviews bias towards people who can code under pressure with someone looking over their shoulder, which is almost nothing like real-world coding.

* Puzzle-y algorithms questions identify people good at clever solutions but that's hardly what most people need to do day-to-day when coding.

* Take-home assignments bias towards people who have lots of free time, and still don't really tell you how that person works with others.

* Looking at GitHub history biases against people whose lives are too busy to code in their free time (e.g. maybe they have kids) and who weren't lucky enough to have a previous job that touched a lot of open source.

* Looking at past work experience misses brilliant junior programmers coming out of college.

* Don't even think about using ML for this, that doesn't solve biases, it just hides them in a black box.

It seems like there's no right answer here.




I've honestly simplified my approach to interviewing over the last few years and I'm pretty happy with the results.

I just have a conversation and ask them to tell me about a favorite code project they've worked on. Could be school project, previous job, whatever.

And just let them talk about it. I'll prod for more detail if I need to give a little nudge. Ask about problems they solved when doing it, etc.

The _only_ thing that I'm looking for in this process is whether or not the person is truly interested what they're doing. This comes through when talking about something they're excited to have worked on, seemingly small problems that they had to really work to solve, etc. Geeking out, if you will.

The single most important trait I've seen in programmers that I've worked with over the years is "level of interest". There's too much that you have to learn on a daily basis to be successful in this field. If you're not interested in learning new things and finding solutions to the problems that you encounter, you're not going to make it very far.

Beyond that and basic competence in the core skills that the job requires, I don't look for much more.

Thus far, everybody I've ever hired who has passed this little conversational test has been an excellent team member.


Same here. There's no silver bullet question, and no high pressure situation with me.

My approach has just been to have a conversation, and try to move it along naturally. What do you think of concepts? What useful STL did they add? Why did you go with this design? Ask me something.

If the person doesn't really have the level of interest required, they will run out of things to say. Like a cave left unexplored. The good candidates will go down all the caves with you, even if they don't quite know what's there.

In particular, the times I've had dud hires it was not possible to know from the standard fare of coding questions and IQ questions. When I was doing that obviously the people I hired passed. But for instance one guy really didn't want to be a programmer, he just knew how to answer the questions. He ended up stalling on his first assignment, and eventually decided to change careers. Another dud also lacked motivation, perhaps because he was asked to do something not quite in his area to begin with. He ended up not doing anything at all.

Apart from those two, dozens of others have turned out ok. I never ran into that guy who couldn't do FizzBuzz, despite not asking. I also don't think it was all that bad to let go of the duds within a quarter, they never got so ingrained in the team that we depended on them, and they could plausibly move on to another job without mentioning us on their CV.


I’ve been job searching lately and it is brutal. I’m fairly social normally but interviews terrify me. I freeze up, lose my train of thought and I don’t think my enthusiasm for my work really comes though. It’s probably not as bad as I imagine but it’s really frustrating.


If you will permit me some trite but genuine advice: I have been interviewing and hiring people for well over a decade and it's not much more fun as the interviewer. Just pretend you are having a conversation with a friend about a project you really enjoy. It's kind of like worrying about how you will perform in a race. The training is already done when you show up to the starting line. The race is the easy part. You know what you know, just demonstrate it. Let the chips fall where they may, as they say. If you fall short, so be it. It's kind of the Zen approach to interviews. Don't try to hard, just do what you know how to do and find that place where you can be genuine.


This.

I was a hiring manager for 25 years, and this was exactly what I did.

I made very, very few mistakes, and actually had some pleasant surprises.


> I made very, very few mistakes

How do you know? How often did you follow up on rejections to see whether they should have been hires?


Actually, for the company that I worked at, and the level of folks we hired, I often heard about how they went.

The only one that made me sad, was a chap who used to show up on the Adobe Photoshop splash page. We offered, he declined.

In the aggregate, I think things turned out OK for all of us. There are some folks that I have to admit, probably did best by not accepting us.


>There are some folks that I have to admit, probably did best by not accepting us.

Yeah, I hear that. I remember interviewing someone who was _good_, but not at this narrow thing that we were hiring for. She seemed SO hopeful that we would give her an offer, I wanted to tell her directly "this job is going to waste 6 mo of your life and offer you nothing I return"


My most fond recollection was a young chap that was a "second choice." The first declined, and this guy had very little of the exact skills we needed, but he had a real eagerness.

He turned out to be an amazing hire. I think that he far outstripped the first choice, and he has gone on to do quite well, after leaving our company.

I like to think that I ran a fairly good shop. We kept people for decades.

When they finally rolled up our shop (after 27 years), the person with the least seniority had ten years.


Same here. Motivation and will to learn beats almost everything in a ever changing world of IT.

Look for curiosity, interests and blend them with your product.


Hire quickly and fire quickly with a generous severance. That’s the only method that I’ve seen works effectively. Netflix does more with 2000 engineers that companies with 10-20x that.

This also lets you take chances on engineers that weren’t “perfect” but had a lot of positives to offer.


The Netflix model makes a ton of sense, but doesn't apply to most other companies, bc Netflix is a unique case. They pay top of market, all cash. They have massive name recognition and obvious longevity. Probably most uniquely - they only hire senior people who have a track record of experience they can look at.

When you're a startup that nobody's heard of, half your engineers are going to be junior, and half your comp is stock options that will probably never be liquid, the offer doesn't make any sense. Are you just going to give offers to 20 random new college grads and coding bootcamp alums planning on firing the 50-75% that don't cut it? Good luck ever getting junior people to accept offers again. Do you think someone with 10+ years of experience is going to take a pay cut to take the job with a random startup with a decent chance of getting let go if the manager isn't feeling it after a few weeks?

With 15 years of experience and a family to provide for, there's no world where I'd consider that type of hiring/firing culture for anything other than a proven brand name with top of market comp, environment, and coworkers. Did an interview loop w/ Netflix a couple years ago, seems like an amazing place to work, ultimately went a different direction for non-culture related reasons.


Start ups need different kind of people than big companies. With a startup the biggest mistake someone can make is not moving fast enough, and the startup dies. In a big company the code base always gets too big, so a person who works slower, but is able to improve the code base while shrinking it is worth more than a person who finishes fast, but blows up or slows down the code base. Interviews have to search for the right style of people.


> hire quickly and fire quickly

This kind of mentality is not conducive to a well-functioning nation with people who have dependents and need stability.


I would reverse it.

The nation is not conducive to business if it can't provide healthcare and safety nets for those without work or in between jobs. In the US we are held hostage with healthcare based on our employers - if you're even lucky enough to have a stable job with benefits.


"Hire quickly and fire quickly" has other issues than healthcare and safety nets though, moving across a country and spending weeks trying to find a place to live (possibly losing money on rent because you can't find anything to buy yet) only to have to do it again in a few weeks' time is difficult.

Less of an issue if you're full remote, or you provide good accommodations, I guess.


Not to mention that most apartments have mininum lease durations, and penalties for breaking said leases. So if you move for a new job, sign a one year lease (because that's all that's available, or you can't afford to pay extra for shorter terms), and then a month later you're handed a box and a security escort...


Indeed. And of course the biggest advantage of them all: living in a big city. One of the main reasons for the salary difference: it's a lot easier to get raises if there are 10 companies offering the same job


Show me one coutry who pays unemployed benefit at the level of the job you need to leave because of that insane hire fast fire fast.


In France your jobseeker's allowance is 80% of the average salary of your last 12 working months. Whether your last employer was for 10 years or 10 days. It starts slowly ramping down after the second year of searching I think


French unemployment has been well above US unemployment since the 70s. This isn’t a free lunch. If the state makes employing people more expensive employers will employ fewer people.


Given sufficiently generous unemployment benefits (ie. mincome), I fail to see how that's a bad thing.


Because most people who do work are being productive, doing useful things, serving their fellows. And being unemployed if you’re of working age and without some other productive role is awful.

Unemployment is bad for society generally and for the unemployed. Maybe one day we’ll get to fully automated luxury gay space communism but France is not there. Those extra unemployed people are having their talents and productivity wasted by economic mismanagement.


To say nothing of the universities.


Past patterns can be an interesting contrast. During WWII generals were hired and fired quickly. Because the practice was pervasive there was no big stigma to being fired and there were always alternative positions to try. It also helped that there was a general structure which was not explicit or required but was broadly exhibited. That is, new hires were expected to show basic proficiency within two weeks and to make a significant contribution with in six weeks. Since a relatively quick test of skills was built in to the management structure there was less pressure on initial evaluation.


I definitely think that if every employer used Netflix's strategy there would be issues, but if the job market had more churn stability would take on a different meaning. Imagine that it was expected to change jobs frequently if you or your employer was unhappy; I'd bet lots of people would be way happier.

The problem is of course that even if you have every reason to think that some other place would hire you, you need a degree of slack to begin with. There aren't any easy answers there, but that's where I'd say an expanded social safety net should come in.


Saying that it isn’t conducive to an entire nation is a bit of an over-dramatization, don’t you think?

I think a generous severance makes up for that. Plus people would know what they are getting into, just like Netflix. If you don’t want the risk/return you don’t have to apply or accept a job there.


> Saying that it isn’t conducive to an entire nation is a bit of an over-dramatization, don’t you think?

I don't think so.

> I think a generous severance makes up for that.

That's your opinion. But what you think is generous isn't necessarily what others think is generous.

> Plus people would know what they are getting into, just like Netflix.

I disagree.

> If you don’t want the risk/return you don’t have to apply or accept a job there.

The risk/return needs to be clear though and hirefast/firefast isn't always clear.


These arguments are silly. If you’re competent you’ve got nothing to worry about. If not, then something to work on.

Incredible hiring risk aversion is what makes finding a job for everyone a shitshow. Definitely not an efficient market as it stands.

If it weren’t for an easy to hire organization that gave me a chance, I’d likely be homeless.


If the nation is well-functioning, getting fired isn’t the end of the world. If your job is independant of your healthcare and you have several years of allowance to seek another one it’s fine to try out a few jobs before finding what sticks


Only when the pay doesn't compensate for the instability, which in Netflix's case, is a lot. You could work two months and have enough saved to last you an year.


It would absolutely work -- assuming everyone has access to healthcare (preferably universal coverage).

Unfortunately this cuts at the moat that big corporations have set up not to mention health insurance industry and big pharma. Which is why it hasn't happened yet here (though it has in many other countries).


Everyone has access to healthcare in the US, under ACA and Medicare.


Why? COBRA covers 3months of health insurance; ACA covers more, and unemployment insurance and severance pay gives you months to look for another.

The only major problem would be relocation, so I'd suggest working a few months at a new job before relocating for it.


You have to pay the employer's share for COBRA, which is usually thousands of dollars per person per month. Right when you've lost income from the job.


It isn't a company's responsibility to create a well-functioning nation.


Says who, Milton Friedman?

Surely companies have some minimal responsibility to act ethically, and surely behaving in a way that causes social dysfunction is not ethical.


Netflix doesn’t act unethically. They are upfront about how they hire and fire and people select into applying there and jobs there based on that. Same as CostCo has much higher standards for employees than Walmart and much higher benefits. If all employers were like Costco lots of people would be unemployed who currently have jobs. Having an employment model that is not perfectly average is not unethical.


It's the responsibility of the government to create a well-functioning nation, and to use corporations to this end. But to suggest that people or the corporations have a direct responsibility, is to shift the responsibility away from those who are on 'our side', and to those who are either neutral or its primary victims.


Surely every stakeholder has some degree of responsibility in the proper functioning of any system.

By your reasoning, I am free to degrade public infrastructure such as roads because it is not my responsibility to build them.

Equivalently: I am morally justified in committing crimes since I am not a police officer.


>Surely every stakeholder has some degree of responsibility in the proper functioning of any system.

A civilization enforces responsibility through the law.

>I am free to degrade public infrastructure such as roads because it is not my responsibility to build them.

Yes. And either the government should set laws to prevent you from misusing the infrastructure, and/or the people should shun your company for being a bad actor. The former being consistent for infrastructure that is owned and maintained by the tax-funded government.

>I am morally justified in committing crimes since I am not a police officer.

Morality and legality are different things. Neither people nor corporations should be judged by the government through the abstract, subjective morality of individuals, only by agreed upon, written law, and at its loosest which is a court's interpretation of the law.

You as an individual can hold anyone responsible for anything under any criteria, but then you should state you're speaking for yourself as per your own moral code. As a society, we have rules for that.

If you want to make your moral code the law, then that should also be explicitly mentioned as such instead of speaking as if it's the status quo.


>Yes. And either the government should set laws to prevent you from misusing the infrastructure, and/or the people should shun your company for being a bad actor.

It seems we agree. Similarly, governments should encode the civic responsibility of businesses into law.

Whether or not they have actually done that is a different debate.


I have a responsibility not to violate the law or vandalize the roads, but I don't think I have a responsibility to drive extra carefully or patch up potholes as I see them.

To me, this seems like a very cynical scheme. We tell "normal people" that they, personally, ought to BUY AMERICAN to prevent jobs from moving abroad. When they eventually do, we can tell them they should've done more and that it was their fault. It shifts the liability on to the victims, rather than the perpetrators.

But the responsibility of ensuring that a country has a good trade deficit is upon the government, if anyone! There's no serious way I can prevent it, so the only real point of saying it's my responsibility is to diminish the government's responsibility for failing to prevent it.

To suggest that the companies have a moral obligation, is to suggest that an abstract body corporate can have a conscience. If it could, there would be no need to ban slavery - just expect that the honest businessmen of America cease to traffic in them.


>I don't think I have a responsibility to drive extra carefully or patch up potholes as I see them.

We agree. We also agree that businesses do not have the responsibility of fixing society.

However, they do have the responsibility not to degrade it.


When you add in "responsibility" like that, it becomes very vague. What does it mean for a company to have morally transgressed? Does it mean that it has violated its conscience, or does it just mean that it's an unfortunate behavior we would do best to punish?

Companies are inert objects. They do not have consciences. I can't say that a company does an evil thing, any more than I can blame a rock someone else threw for hitting my head. It hurts, but I can't very well blame the rock for it. Companies are just like driftwood in the ocean; their motion is defined solely by the forces acting upon them.

What is left, then, is to suggest that society should go after the companies that don't do what we want. But this can never be done by individuals. It has to be done by unions, governments, or something of the sort. It cannot be done by the companies themselves, and it cannot be done by individuals.

When you suggest that companies are doing something wrong, and not the government for failing to regulate them, this is tantamount to suggesting that the responsibility for punishing them actually lay on the individual. But to do so makes the primary victim into the perpetrator of the aggression against him. It is a terrible thing to do.


In the SF bay area for these roles the job market is much more fluid than most places


This approaches biases against those whose residency status is dependent on employment (e.g. H1Bs).

There’s no silver bullet when it comes to hiring.


Which is fine because H1Bs shouldn't be used for most positions. If something actually requires an H1B candidate then you use a different process.


H1Bs are one example, but isn't living in a high-rent area another? In either case, you can't maintain your physical position for long without continuous employment.


H1B is only meant to fill positions that companies "say" they can't fill with american citizens. It has just been abused to hire people cheaply.

Let's focus on solving problems in america for american citizens first, then we can worry about everyone else.


You probably don't need Hadoop, and you probably don't need 10x engineers.

> This also lets you take chances on engineers that weren’t “perfect” but had a lot of positives to offer.

This is not true. An AS/HFA/Autistic person wouldn't survive very long at Netflix, because they biologically struggle with the goal-setting/performance review game. These individuals may have talents that are biologically unavailable to neurotypical people.

The Netflix approach selects for a very specific type of individual: an extreme degree of rote knowledge and the ability to effortlessly apply that knowledge. There are people who acquire this knowledge and talent via unconventional means, and Netflix is able to grab those people up.


I can personally confirm that there's at least one autistic person in a senior engineering role at Netflix.


n=1.

In addition, that individual is likely wasting mental resources gaming performance reviews, or their manager has identified their struggle and is protecting them from the ceremony. Systematically, Netflix is not equipped to deal with them.


Everyone either wastes mental resources gaming performance reviews or ignores them as a sideshow to actually doing a good job. The better the performance review the closer these two approaches are. Unless you have experience of Netflix’s culture and management you are less qualified to opine on whether Netflix is equipped to deal with autistic employees than someone who knows an autistic Netflix employee who is successfully working there.


> Everyone either wastes mental resources gaming performance reviews or ignores them as a sideshow to actually doing a good job.

You can find out that this is a false equivalency in DSM IV. Spectrum individuals struggle with certain executive functions, and performance reviews align incredibly well with those metrics.

You've claimed the anagolous equivalent of "everyone gets tired climbing stairs, paraplegics are no different."

> opine

I hate to tu quoque, but I'm one of those individuals and I do struggle with performance reviews - as would most (not all) adult diagnosed individuals. Literally page 5 of my AS/HFA workbook. I'm lucky to be managed by someone who gives me qualitative reviews, instead of quantitative ones.


I have also been diagnosed. Autistic individuals aren’t paraplegics in this analogy. They are dramatically shorter than average. Working under a disadvantage is not the same as being unable to do something.

It’s not like the population of people who take performance reviews seriously rather than as bullshit hoops that have to be endured or gamed is large.


I have two questions here:

- what is the shortest time they keep you before firing you? 1 week? six weeks? 2 months...?

- I suppose that the level of internal documentation must be stellar. Can someone comment on this?


You can only do this if you have the name recogition and compensation to attract enough candidates who are willing to take this risk. Plus, if you have long on-boarding processes this could become quite expensive.


Yeah. I used to know someone who worked at a call center once. The onboard training was like 40+ hours. They had a problem with people who only got hired so they could be paid to watch videos before quitting on the first day, but there wasn’t anything they could do about it.


That has major issues with immigrants, who wont apply in a fire quickly scenario.


I had a pleasant interview experience one time, that avoided a number of those problems.

I was asked to bring in my laptop with some code I had written. I did, we talked about it. The interviewer asked me to add a simple feature. It's code I was familiar with (as you would be after working for a company for a bit). So no stress. No gotchas.

No time wasted with yet another technical test with a throwaway app. Setting up a new project is always a bit of a faff.

Admittedly not everyone will have some code ready, but then a normal take home test would be the fallback.


> person who doesn't play video games at all will have a big disadvantage

My thoughts exactly while reading the article.


We used it at Cloudflare a few times too (as suggested by former Sandstorm people). I still remember getting the purchase of the game approved by engineering leadership. Was pretty humorous where it went from wanting to understand to whatever approved in like 30 seconds.

I agree with your assessment of the pros/cons of the various interview methods. I agree that there is no right answer.


Huh I actually had no idea it was tried at Cloudflare!


There really isn't. At the end of the day every company I have been in has: the asshole, the smart guy, the idiot, the guy everyone is scared of, the guy that can't be fired ("the originals"). Then you just kind of multiply them randomly depending on team size.

These hiring articles are a huge pointless waste of time.


Makes me feel lucky to be in my job, and a jolt of gratitude every-so-often is a welcome thing, so thank you.

I work with 8 other senior or senior-ish engineers, we are all basically taking a slight paycut (not too much, maybe 80-90% of rates we might get elsewhere) to work for a non-profit making interesting stuff for kids, and everyone is genuinely nice and fairly hard-working.

I wonder if the "taking a paycut to do something actually interesting and worthwhile" is part of it.


Hi, could you share a little of what you do? It didn't even cross my mind I can work for a non-profit, still get paid (not top dollar) and have a good community impact (kids in your case). Thanks!


I work for a grant-funded educational software company that makes science simulations for K through college, but mostly middle school and high school.


My company had all of those rolled into one. Thankfully he burned himself out with the technical debt that he had created and quit. He was smart and could probably beat me at hacker rank type challenges. I know to focus on maintainability over cleverness.


It seems like there's no right answer here.

I think you've done a good job identifying the most common options. Since none has proven decisively better than any other at aiding hiring decisions, how about letting the candidate decide which sort of interview to participate in?


> Some people will just sit and watch and do nothing unless instructed... that's bad. Some people will build stuff, but with obvious efficiency flaws and "bugs"... also bad. Some people identify what needs doing and get it done effectively but without trying to be perfect... that's good.

Some personalities are better suited for programming, although I think hard work can overcome anything.

I remember an old blog post [0] of mine that used Hammerstein-Equord personality quadrants of smart/industrious, smart/lazy, stupid/industrious, stupid/lazy and describing it for how that effects tech team planning.

[0] http://www.prepend.com/2009/05/programmer-classifications-by...


> Some personalities are better suited for programming, although I think hard work can overcome anything.

I am not sure about the hard work part. Some people don't seem to think the right way for programming.

I studied a one year IT masters conversion course back in the dot com boom, and chose the best / hardest course I could find. The first 3 weeks of the course was an intensive Java programming course. There was an exam at the end of this. They suggested that anyone who didn't pass this exam, don't do the rest of the course, but they didn't actually force them off. I was friends with at least two people that failed that exam, but continued anyway. They struggled and I don't think they passed the final exam despite plenty of hard work.


I think hard work and studying is probably better than natural aptitude and no hard work.

I’ve worked with people who didn’t care at all, it was a job, but they studied and worked hard and were pretty good.

Of course, I’d rather work with someone who had both.


so, you've pointed 7 flawed existing approaches as the reason to come up with and use an 8th one. At least those 7 have actual connection to programming. Your 8th one has only imaginary indirect connection only through your internal interpretation. It is just like those box-building/etc. games they play at Scrum training which have nothing in common with reality.

And many flaws in the 7 aren't intrinsic flaws, instead it is flaws of the wrong "one size fits all" application:

> Looking at past work experience misses brilliant junior programmers coming out of college.

yeah, don't use that as negative for junior programmers coming out of college.

>Take-home assignments bias towards people who have lots of free time, and still don't really tell you how that person works with others.

the same, if candidate prefers that way - let them, if not - use other approach

etc.


> A person who doesn't play video games at all will have a big disadvantage.

Even more so for a person who can't play video games, e.g. blind people. Many of the other interview techniques, for all their faults, at least don't (inadvertently) discriminate against people with disabilities.


> A person who has played the game before will have a big advantage. A person who doesn't play video games at all will have a big disadvantage.

There's also a "similar game" advantage. For instance, a person who played games like OpenTTD before will have a much easier time with Factorio rail signals, since the two kinds of rail signals in Factorio work very similarly to two of the several kinds of rail signals found in OpenTTD.


> Coding interviews bias towards people who can code under pressure with someone looking over their shoulder, which is almost nothing like real-world coding.

I don't understand this. Why instead play a game, which, as you said, could have disadvantages because there are people that do play and people who've never played. I guess the thinking goes that by easing into a game you can see clear behaviors of people. But you can also try to ease them into coding while you are looking over their shoulder. The interviews I do, and I've seen done at my previous company and the one I'm currently at, it is similar to pair programming, and yes, there is people that is super nervous at the beginning, and that seem to need a lot of hand holding at the beginning, but after coding and having a few small wins start to ease at the test and then they can surprise you with very nice coding or interactions on solutions.


> I don't know of any approach to interviewing that I think is fair.

That is true, because it was my motivation to build a wizard for behavioural interviews designed to remove biases and be objective. In practice, most of the time people are not even sure what they are looking for - I know it when I see it - and default to gut hiring. This leads to teams made of people that think like you, so you're basically paying 2 salaries for one brain.

My formula for removing biases is in 2 steps: first define explicitly the behavioural traits you desire, and second, use falsifiable questions to determine the truth.

Because we're here to help each other, if this is relevant to you, I'd be happy to share this solution with you.


> if this is relevant to you, I'd be happy to share this solution with you

Yes, please!


I hope it is ok to share this here: https://freyasense.com/recruiting/

Give me a sign and I would be happy to give you a tour


A reminder that the goal of an interview process is not to be fair. This is not a sports event or a court case.

Mostly your goals align with "being fair", but not always, and that's OK.


I agree, so upvoted. The most important question is: "Would I like to work with this person?". That is unfortunately unfair.


> A reminder that the goal of an interview process is not to be fair. This is not a sports event or a court case.

Given the power that companies have over individual workers, I'd say that a job interview should be treated as seriously as a court case. So yes, making it fair should be an explicit goal.


A company's goal isn't to hire every good potential employee, it is to hire a number of good employees, and avoid hiring bad ones. If Factorio wizards make good engineers, who cares if only 1/10 good engineers have even played?


Who cares if they are all white 23yo frat bros? who cares if no woman can get hired so long as the company makes hires it likes. Or gay, or Jewish, or whatever busy is in your selection.

I'm not asserting there's a specific bias in this particular interview choice, but I am asserting that if you were taking a fairly narrow part of the population you better be working damned hard that you don't have hiring biases that are not relevant to job performance


I think the answer is that it takes time to get to know someone and their skills. There is no quick simple test. You can give them multiple tests over time if you want.

I think that if you have money for recruiting then you can screen with an initial test, then pay them for a few hours or days of additional tests. And then ideally there is a contracting period that is an additional test but on a real project.


> Regular interviews bias towards people that are charismatic, not effective.

A cooperative multiplayer game is also going to be a similar soft skills check that favors people that can communicate not just well, but also charismatically.

That said, a possible advantage to doing that in a game, as opposed to a regular soft skills interview (or worse trying to force such soft skill checks into the pressure cooker of a whiteboard interview or puzzle questions) is that you maybe provide an environment where the interviewee can be dopamine hacked into believing they can relax and you maybe get a glimpse into how the person's soft skills check out on the other side of that boundary where interviewee knows that they need to be "on" and acting.

I'm not actually sure if that is good or bad, but a lot of my takeaway from this article and other comments in these threads is it sort of feels like there's a lot of excuses for "pick a game that seems technically similar", but most of the "good feedback" that seems to be coming back from these experiences resembles what history tells us about regular interviews versus technical interviews: soft skills matter a lot more than what you can gauge technically in an interview. Most technical interviews are just soft skills interviews under the pressure cooker of taking a non-standardized pop quiz your professor decided you need to take because you showed up to class in only your underwear (and also you are flying for some reason in this nightmare). Cooperative games possibly present the opposite of the pressure cooker: dopamine hacking the interviewee an excuse to let the pressure seem like it is off while you are still doing that same sort of soft skills check.

(Aside: problem solving is classically a "soft skill". "Behavioral interviews" were as much about problem solving decades before the software industry invented the "technical" interview.)

(Further aside: This sort of dopamine hacking is very familiar to anyone that's seen Fraternity/Sorority rushes. Soft skills are hugely important for "group fit"/"team fit" in college social groups as they are in employment groups, and seeing how potential new members interact in "fun"/"game" situations often tells you more about the person than if you sat down and interviewed them one-on-one in a suit and tie or equivalent and they know they have to be "on". Rushes include both of those things for many reasons.)


> * Don't even think about using ML for this, that doesn't solve biases, it just hides them in a black box.

Can you clarify: what is "ML" here? "Machine learning", ML the language, MatLab, "machine language" (a.k.a. assembly), Matt LeBlanc? I'm out of ideas...


Machine learning


You really hit the nail on the head here. I couldn't agree more.

However, I think playing this game would be a fun and interesting interview.


> But then, I don't know of any approach to interviewing that I think is fair

The companies that think deeply about interviews are aware that their interview is biased, and they do so intentionally. At a certain point watching 5 good engineers not meet the bar is better than having 10 bad engineers get through.

Maybe the 'right answer' is to let your needs drive the bias of your interview. Do you have a position that needs to deal with immense pressure with management breathing down their neck? Bias your selection with whiteboard interviews. Are you looking for a principal engineer that can drive adaptation and change in a large org? Bias your selection process to get that.

tl;dr Maybe the issue is the one size fits all thinking to the tech interview.


While I get some of this sentiment, wont some sort of social deduction game like 'Throne of Lies' be better at identifying potentially good team members?.


I'm not familiar with that game, but the point of using Factorio is it is a game where you have to build complex systems and maintain them, which has many parallels with real-world software engineering. "Throne of Lies" doesn't look like that kind of game.


Throne of lies and social deduction games like them are about finding out peoples real intentions from what they say and do, also aspects like using and understanding subtle signalling to collaborate with ones team.


In my software engineering work I do a lot more “writing documentation to be as clear as possible” than “social deduction and subtle signalling” ^^;;


In fact i would hate to work at a place where you have to do social deduction




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: