
Microsoft changed how it interviews software developers - pplonski86
https://www.businessinsider.fr/us/microsoft-new-developer-interview-process-2018-12
======
solatic
Dogfooding their process is going to give them adverse results here: Microsoft
is paying them to take all the time they want, but good candidates won't be so
willing to spend so much time. Already, they're up to a full day of
interviews, which is ridiculous. One of the worst parts of current FAANG
interviewing is how long it takes to get to the day of interviews, needing to
take off from work to attend the interview day, and then getting rejected
nonetheless. Not only does this process ask for too much commitment on the
part of candidates, it selects for candidates for whom such a commitment is
less relatively costly - i.e. people who are currently unemployed, students,
etc, and while there are many such people with great talent and potential,
respectively, they're also less likely to be strong candidates than people who
are currently employed.

One of the key issues in recruiting is how to make hiring decisions quickly.
Good candidates don't have the patience for your internal politics to resolve
over a month or two to figure out whether they're getting an offer or not;
they're just going to interview with other places in the meantime who will
beat you to the punch.

Take a resume off the pile. Email or text to schedule a call. During the call,
talk about relevant experience / past projects, what each side is looking for,
etc. Make a decision during the call whether you want to invite the person for
a two-hour interview, and schedule it during the call, first offering an
evening time slot for the interview. In the first hour, talk about culture,
problem-solving and teamwork approaches, engineering attitudes. In the second
hour, pull out an actual current problem and solve it collaboratively. If all
goes well, send an offer, and offer to set up a dinner to talk about /
negotiate the offer.

It should not take more than a few hours to make a decision on someone; it
should not require a candidate to take paid time off. Your hiring pipeline's
average lead time should be measured in days.

~~~
kluyg
Yeah, if only it was all unicorns and rainbows. What you describe is how I
imagine they interview for director / VP positions. With hundreds (thousands?)
of applications per day for software engineering positions, all with perfect
resumes, what you describe just doesn’t scale. Software engineers do 1-2
interviews per week each, it’s part of the job. I might be a minority, but I
would refuse to spend two of my evenings per week interviewing people. I’d
rather spend time with my family instead. I think asking candidates to get a
day off for interviews, which happens like once per year or even less often
for each candidate is more reasonable than asking your employees to work
evenings each week.

~~~
solatic
> Software engineers do 1-2 interviews per week each, it’s part of the job. I
> might be a minority, but I would refuse to spend two of my evenings per week
> interviewing people.

Why are software engineers doing interviews at all? What makes software
engineers qualified to interview people (especially when you consider the need
to avoid hiring bias, problematic statements by interviewers that expose the
company to legal suit, etc.)?

I'm not saying that companies should go to the other extreme and have all
interviews conducted by HR - that presents its own set of problems in that HR
doesn't know how to evaluate candidates for technical skill. But expecting
hiring managers to open up a few evenings per week in the irregular and
relatively uncommon circumstance (if it's common, then you have problems with
churn on that team, and you should consider firing the manager) in which
positions open up on their team to evaluate potential direct reports is not
even remotely unreasonable.

~~~
em-bee
i am right now preparing to fill two positions, and i am looking to interview
at least half a dozen people maybe more. due to my schedule i can't really
interview more than one per day, so this is going to take a week or two.

i do want the opinions of the people already on my team, because a big part of
the interview process is to find out how well they work together. for me
hiring is a team decision, not a top down mandate. so yes, all of my software
engineers will be doing interviews. there is no way around it.

on the other hand i agree with you. these rounds of interviews should not
happen to often. if our team grows, i expect to add two more positions every
quarter. (i don't know how realistic this is, but this just to give you an
idea of how much interviewing workload i expect.)

------
JustSomeNobody
> But the aha moment for me was that not everyone does well in those fast-
> paced brainstorming sessions. A lot of people (including me) prefer to sit
> with a cup of coffee and some data and try to think things through.

This is me. Hard to get that across in an interview, but once people work with
me, they're cool with me coming back an hour later in an email with some
thoughts on the last meeting topic. They know and respect that that's the way
I work[0].

[0] I'm not the only one on the team who does this, so that helps.

~~~
thewhitetulip
I had a different experience. They gave me a lot of time but either I don't
know python at all, or they didn't.

they asked me to build something. I used dictionaries and wrote the code in
ten minutes.

Both interviewers wouldn't believe that the code could solve the problem.

They later told me that dictionaries are not used in practical applications.
Never understood why.

Are dicts not used in "real life"?

~~~
bshacklett
I'm not the best with Python, but if someone gave me that answer I'd feel like
I avoided a land mine. Dicts, and analogous data types, are incredibly useful.

~~~
thewhitetulip
Yes! I avoided the landmine. I can't imagine how I'd have felt working with a
woman who interviews for Python role and doesn't know or understamd
dictionaries!!

Plus they were giving me (x%) a raise, and I got (x-5)% in my current company.
So thank God I didn't join there. Contacts are more important than salary.

------
mancerayder
Great, now can Netflix, Google and Facebook do this, too? Not because I want
to work in these places, but because they influence everyone else and as a
senior engineer in the systems space I feel I shouldn't need to study days or
weeks for fizzbuzz sorting algorithms questions that are designed to test comp
sci recent grads. I have a proven career, and was never suddenly stumped in a
project due to not being able to.. sort letters in some obscure order based on
arbitrary rules. How about we look at that, and compare the candidate's past
experience to the current needs and interview based on that -- You DID look at
the CV before bringing the candidate in, didn't you?

~~~
kronin
I've interviewed and been interviewing for over a decade. I have had the
displeasure of interviewing many people who have very impressive resumes yet
when asked how they would judge their skillset in an area, and being told from
them that they would say "expert", being shown exactly the opposite when it
came time to answer some questions in that area.

One data point... Senior engineer that founded the local java user group and
wrote a published book on the java language. Couldn't code a simple reverse
string algorithm, nor explain whether, after with help devising an answer, the
method was thread-safe.

Another data point... A self-described expert in SQL with it all over their
resume. Couldn't write a simple join, inner outer or other.

Weed-out questions exist because our industry is flooded with people who don't
know, nor care to know, their craft. And there are plenty of employers who
continue to hire these people, and promote them. I interviewed numerous
"architects" (with development experience all over their resume) who couldn't
come up with a naive solution, let alone the optimal, to a very simple
problem. Want to fix the interview process in our industry? Figure out a way
to weed out the liars? I'm all for it. I would rather not have to do basic
algorithm questions, but that's the current environment.

~~~
borgfriend
Why do you assume that the person is lying? Why do you start your relationship
with your new co-worker on distrust? There is a way to weed out the liars -
you hire them and then when it does not work out you fire them. You would be
astonished how little people lie on their CV.

They may exaggerate claims - yes, that is why you gave them a call in the
first place. But then again you exaggerated your claims in the job posting of
which skills are required for the job.

The one thing that I noticed is that only the US companies have this awkward
parallel reality 'technical-interview' hiring process. In other countries like
Germany or France the interview process is mostly focused on the person and
the desire to work for the company and the willingness to learn the skills
needed for the job. Just because you fail to answer immediately to a random
question it does not mean you are unqualified for the job.

When I conduct an interview I try to only ask open questions like: * What is
your favorite IDE? - Whatever the person answers, tells a lot about how he
works, what tools he uses, and if he is aware of the common tools * What
feature of the new Version of <Programming Language> do you like the most? -
A) you may learn about some weird feature, b) you can discuss how that feature
will improve code * Do you prefer JavaScript or TypeScript (dynamic vs typed
language)?

There is no right answer. It is all about how the candidate argues his answer.
You learn more about the person and if you can deal with him on a day to day
basis (and if he has the brains to think about things).

~~~
kronin
> There is a way to weed out the liars - you hire them and then when it does
> not work out you fire them.

In my experience, firing someone takes multiple months. Months in which they
are mutating the codebase (with review, but it's a time sink that shouldn't
exist for a decent senior), influencing other engineers, etc.

Passing on a good candidate is cheaper in the long run than hiring a bad
candidate. Or said another way, false negatives are preferred over false
positives.

> You would be astonished how little people lie on their CV.

This has not been my experience. From something as white-lieish as "I designed
and implemented" vs. "I was a peer on the team that did it" to flat out
fabrications. You learn alot by simply asking "what was your role on the
project" and asking follow-ups to dig into their contributions.

~~~
C1sc0cat
I am assuming the you mean the USA I would suspect not months in the case on
NCI Non Culpable Incompetence.

Don't most companies have a probation period these days.

------
jimbobimbo
The original article [1] describes new process in the context of PM
interviews, which are different from software developer interview, no matter
how technical PM is.

The changes that I particularly like are: improvement in coordination between
interviewers themselves, and stop sharing feedback between themselves until
the very end. The latter really affects how next interviewers view the
interviewee.

[1] [https://blog.usejournal.com/rethinking-how-we-interview-
in-m...](https://blog.usejournal.com/rethinking-how-we-interview-in-
microsofts-developer-division-8f404cfd075a)

~~~
JamesBarney
After reading about wisdom of the crowds, we started doing this with
estimates. We always break up the project into small tasks together then
seperate and estimate them individually.

We've found we're a lot more accurate because we don't influence each other,
as well as don't go into autopilot on an estimate assuming someone else has it
under control, or someone just deferring to the more Sr. Dev when the more Sr.
Dev made a mistake.

I could see how this would also work well with interviews.

------
chasing
> Under the new process, Microsoft shares the interview questions in advance
> so that candidates can prepare. During the interview itself, a candidate
> might run through a real scenario or problem the team is trying to solve.

LOVE IT.

I think best when I have a chance to actually think, not when I’m
participating in some odd job interview game show.

------
whack
Here's what they are doing, from the linked blog post:

> _Our dev teams had taken to working with candidates to solve a bug or
> feature as part of the interview process. It was a collaborative effort with
> the candidate and the team working together to solve a real problem._

Sounds like a great idea.

Except it's so variable. What if you get lucky and you get a easy bug, whereas
someone else gets something much harder. Let's standardize that by giving
everyone the same task to work on.

And what if we want to hire people who aren't fluent in your project's
programming language? That would be a big handicap for them. Let's deal with
that by having minimally sized "projects" in all the languages, and allowing
the candidate to choose their preferred language.

And we don't want the interviews to be biased against people who don't already
have domain knowledge in a specific field. So let's make the task generic
enough to be approachable by any good programmer.

Congrats, if you did all of the above, you've reinvented leetcode style
interviews.

There's some cool stuff that MS has put into practice. I like the focus on
reading/understanding existing code, instead of just writing code. I also like
the relaxed pacing and "open book" approach.

But for the most part, I don't think this is really as revolutionary as people
think it is.

~~~
avisser
> taken to working with candidates to solve a bug or feature as part of the
> interview process

This is unethical. How is this anything other than unpaid labor?

~~~
TheSoftwareGuy
A little bit of unpaid labor is honestly fine. Especially since I can’t
imagine this being easy to exploit in a way that really impacts a company’s
bottom line.

It’s like getting a free sample of food at the store, before committing to a
full purchase

~~~
bicubic
Imagine being a carpenter applying for a job and being asked to make a small
bedside table for free. So the employer can sample your skills before
committing to a full purchase.

Absurd.

~~~
simonswords82
If you ever sat on the hiring side and had just one new hire not work out, you
would understand your throw away statement is actually more absurd than asking
for free work. Let's flip the scenario and go your way instead...

Imagine being a programmer applying for a job and not being asked to
demonstrate your skills at interview. The interview goes well, and you're
offered the job.

You give your 4 weeks notice on your existing job, and turn up to your new
role. You sit down, it's day one, and you're presented with a small sprint of
bugs to work through to get warmed up on the project you're assigned.

Your heart sinks.

You realise that you are totally out of your depth. The solution you're
working on is hard, the documentation insanely complex, and you feel more and
more despondent as the week goes on.

By the end of the week your colleagues and managers have started to pick up on
your struggles. Despite their effort to help you can't make meaningful
progress. To cut a long story short you're let go, and now you're unemployed.

-THE END-

As an employer and owner of a software company, it's my job to assess the
skills of somebody as quickly and efficiently as possible. If getting them to
"build a small bedside table for free" gets both me, my colleagues, and the
candidate attending the interview comfortable that we're offering a job to
somebody who can succeed at my company, then so be it.

~~~
bicubic
I'm more often on the hiring side of the table than on the other. Probation
periods exist specifically to address the issue of hiring false positives, and
they work fine.

Your story is only possible if the interview process failed miserably. If your
process sucks and results in a lot of false positives, that's your problem.
Don't use it as justification for milking candidates for unpaid work towards
actual deliverables.

~~~
simonswords82
One must not use probation as a get out of jail free card. We used to think
the same way you do, we ultimately realised we were wrong.

Probation exists to be used only extreme circumstances, not as a catch all for
"hey, sorry we made the wrong hire". It's more expensive to hire a candidate
and kick them out on probation than to avoid making the wrong hire. This is
another reason my approach works - we don't "milk" free labour, we don't need
it. Our interview ensures we make the right hire first time, every time and
part of that process is some "real" work.

Our interview process is awesome. Refined over a period of 10 years.
Candidates learn something, we learn something, we collaborate with them. We
often have candidates, even those who do not receive a job offer, compliment
us on our process.

I'll write our interview process up one day and share it on HN so others can
benefit.

------
sys_64738
When I interviewed at Redmond 15 years ago, it was the recruiter who asked me
a bunch of techie questions then decided to bring me out for a whole day of
onsite interviews. During that I recall it was only a director of SW who asked
me some mentally crushing questions about subtleties in how the Windows kernel
fiddles with files. As I said, I was a UNIX developer who didn't know those
bottlenecks in the Windows kernel. Still he persisted. He did take me for a
spin in his awesome BMW sports car after though!

My thoughts on Redmond were that it rains all the time and the inside of the
Microsoft buildings is pretty depressing.

~~~
im_cynical
I've had something similar happen to me once. I'm DevOps, very systems
focused. I was leaving Cisco and got on an interview panel with 4 network
engineers at some place.

I caveated I know networking better than most but I'm by no stretch a network
engineer. They agreed and explicitly stated the position was on the systems
side and it was fine.

The interview was only an hour and half. About 30 minutes into they ran out of
questions and started to get bored. They tell they're all studying for their
CCIE's and decided to start borrowing questions from the final exam.

They decided to play "lets stump the Cisco guy". And did they? YES! Did they
stop? NO! This was 7+ years ago and I'll never forget how uncomfortable and
bad that interview was. They managed to dredge up all these arcane, stupid
questions, some of which bore no practical function.. and listened to me "I
dont know" for a solid hour

I'm sure it left them with a bad impression of me, and it certainly left me
not wanting to work there.

------
metta2uall
Good riddance to all those mostly useless brain-teaser questions.. They
require countless hours of practice for most people to get good at them - and
I find it semi-tragic that many highly intelligent people are wasting their
time on that instead of on complex real-world problems (not to mention on
learning things that would make them happier and more well-rounded).

------
HillaryBriss
> _... Second, we run through a real problem the team is trying to solve —
> improving satisfaction, increasing retention, boosting usage of a service or
> feature. The fact that it’s a real problem that we’re working on helps
> foster a collaborative conversation._

maybe I'm reading too much into this but ... for some reason, none of these
questions are highly programming-centric (e.g. implement a sort algorithm,
build a B-Tree, or even just implement the singleton pattern in C#, or
whatever). all of the questions are human behavior-centric. and I wonder if
maybe Microsoft has concluded that the really important challenges are not
technical challenges, but marketing ones.

~~~
bagacrap
I don't think those human-centered metrics can be classified as marketing. Is
marketing the business of satisfying customers?

~~~
HillaryBriss
maybe 'product development' ?

------
lawnchair_larry
I noticed that the two people quoted are both PMs. So does this have any
relevance for software engineers?

~~~
DylanDmitri
Yes, these are software engineering interviews. It's the PMs who are designing
and refining the process though.

~~~
kettlecorn
It seems to me that it should mostly be software engineers designing the
software engineering interview process.

~~~
cududa
At Microsoft many senior devs eventually go into the PM track (while still
actively writing code) because there’s more salary there

~~~
lawnchair_larry
No there is not.

------
sxp
[https://blog.usejournal.com/rethinking-how-we-interview-
in-m...](https://blog.usejournal.com/rethinking-how-we-interview-in-
microsofts-developer-division-8f404cfd075a) is the original post by a PM at
Microsoft. It doesn't have any useful information about a pure technical
interview for software engineering.

>(Note: I started at Microsoft when we were still asking questions about why
manhole covers were round, how many ping pong balls would fill a 747, and how
to reverse a linked list. In 20 years here, I’ve yet to have to write the code
to reverse a linked list (copy-paste anyone?) or fill a 747 with any kind of
ball.)

The manhole cover question is actually a valid industrial design or UX
question even if it's not appropriate to ask for software interviews. The
linked-list question is the equivalent of Fizzbuzz for data structures. And
the 747 question is a valid Fermi problem. The exact number doesn't matter but
the steps required to estimate the answer are similar to other estimation
problems when mapping out a business strategy and trying to determine if it's
profitable. Or maybe the company is interested in salvaging sunken planes:
[https://www.iusmentis.com/patents/priorart/donaldduck/](https://www.iusmentis.com/patents/priorart/donaldduck/)

~~~
WrtCdEvrydy
Isn't the round manhole covers just to make sure they can't fall in?

~~~
antaviana
According to the Wikipedia [1]

Reasons for the shape might include:

* A round manhole cover cannot fall through its circular opening, whereas a square manhole cover might fall in if it were inserted diagonally in the hole. The existence of a "lip" holding up the lid means that the underlying hole is smaller than the cover, so that other shapes might suffice. (A Reuleaux triangle or other curve of constant width would also serve this purpose, but round covers are much easier to manufacture.)

* Round tubes are the strongest and most material-efficient shape against the compression of the earth around them.

* A round manhole cover of a given diameter has a smaller surface area than a square cover of the same width, thus less material is needed to cast the manhole cover, meaning lower cost.

* The bearing surfaces of manhole frames and covers are machined to assure flatness and prevent them from becoming dislodged by traffic. Round castings are much easier to machine using a lathe.

* Circular covers do not need to be rotated to align with the manhole.

* A round manhole cover can be more easily moved by being rolled.

* A round manhole cover can be easily locked in place with a quarter turn (as is done in countries like France), which makes them hard to open without a special tool. Lockable covers do not have to be made as heavy, because traffic passing over them cannot lift them up by suction.

[1]
[https://en.wikipedia.org/wiki/Manhole_cover](https://en.wikipedia.org/wiki/Manhole_cover)

~~~
derefr
Don't forget also: round manhole covers are a Schelling point.

Making a manhole cover round is one of the first intuitive shapes you'd think
to make them (along with squares and rectangles); and so you'd guess, as a
construction company gearing up to _sell the concept_ of concrete manholes
with removable iron-plate covers to a municipality (back in the 1800s or
whenever we first got them) that what the municipality will ask you to make is
round manhole covers. So you provisionally set up your tooling for round
manhole covers, and build a few for the demo. And the municipality assumes
you're the expert about good manhole-cover shapes, and so just goes with the
flow, rather than trying to figure out the best shape themselves. So you end
up with "the first intuitive thing that worked" spreading via network/"system
compatibility" effects.

Same with road widths; same with screw drives.

~~~
BurningFrog
Thanks for explaining why roads are round!

~~~
derefr
More like, why we got four-way intersections for so long before anyone tried
roundabouts. It was intuitive to just drive one road through another, and then
we added traffic laws and signalling on top to try to make it work, rather
than anyone sitting down at the start to try to work out what intersection
shape would optimize for traffic flow and [non-]accident rate.

~~~
BurningFrog
Ok, that's actually an interesting (schelling) point!

------
calewis
Although I’m a product person (CPO) I’ve hired (alongside a CTO) maybe 100+
engineers in my time and I’m hoping my perspective is useful/helpful.

The process that my CTO and I have found works best is as follows;

A 20min phone interview (to screen for BS’ers).

A technical test that’s based on the sort of problems they will be solving in
their job. How would you do X in language Y, no boiler plate code, and it
doesn’t have to be perfect. (So no fizz buzz, conways game of life etc.) We
also stipulate to spend no more than 2 hours on it, and if their successful
they get that time back. We even offer a discount (on the thing we sell) to
unsuccessful people who don’t make it. The test serves as not only a good high
level demonstration of ones skils but, more importantly, gives us all
something to talk around. It also shows us how they approach key things like
testing, deployment, maintainability etc etc.

Then there is a 2 hours face to face interview. Ahead of this the test is peer
reviewed by the engineering team (or part of it) The interview is made up of
one one hour technical section with the CTO/Senior, and one with a
designer/product manager. A joint decision is made.

Then, if they are successful, we have a probation period where we (the
candidate and us) work out if it’s all going as we’d hoped. If not, we part
ways. You most likely wouldn’t marry someone before a first date. A job is the
same. You both need time to see if you like each other and your profile pics
match real life.

It’s not perfect, but no system is. Sure we’ve hired a few people we’ve had to
let go, but for the most part it’s a great system. We’ve managed to hire not
just great engineers but also build a team and culture that fits and works.

Hiring is time consuming, hard and at times emotional. There is no shortcut.

I’ve interviewed and worked at big tech companies and they all think there is
some secret sauce, a magic code to hiring. That is, if you ask super hard
questions only the “best” make it through - utter tosh. Some of the most
amazing engineers I’ve worked with would fail these type of stupid questions
(for a while load of reasons) but quietly write amazing code.

Also, and more importantly imo, to be an amazing engineer you need to be great
(equally) at teamwork and communication, not just be super book smart.

I read a while back (or imagined it?) that MSFT only hired people with
Msc’s/PHd’s and that it created a probelem where there wasn’t any intellectual
diversity. These tests cause this too. You get cookie cutter people who are
smart but often can’t work in teams and can’t communicate well.

------
pavel_lishin
> _He thinks it would be useful to figure out if there ways to either shorten
> the process entirely, or at least spread it out over multiple days and gives
> candidates more time to reflect and think._

Please don't. I can take one day off work for a full-day interview; I'm not
going to burn through four fucking vacation days for you to vacillate over
whether or not I'm a "culture fit" or whatever.

------
dietr1ch
IMO those interviews test whether you are smart "enough" and capable of
collaborating to solve a problem. There's not that much knowledge required and
good answers tend to be quite succint.

Depending on what you do, maybe most of the time interview-like problems won't
rise up, but not everyone is ok taking the risk of poor algorithm design on
production. While it won't happen that a single engineer will need to do
everything without any design or code reviews, if you keep "lowering" the bar
you risk breaking the review failsafe. Maybe scaling the review process can
enable teams having more engineers working "safely" as the hard problems will
still be reviewed by the "elite" engineers those companies try hard to get.

I hope this takes some pressure away on pretending to have "real" interviews
needlessly. There's too many problems to be solved out there to be overly
picky on who can work on them.

It would be nice to hear back from MS after they a few years.

------
naasking
Sounds like a good approach. It's nice to see some meaningful progress in
interview processes. Kudos to Microsoft.

------
suyash
Article used incorrect title, the original blog posts describes the interview
Microsoft has for technical PM and not software developers. I bet that
developer interview hasn't changed much.

------
calewis
I think a dumbest question thread might be quite amusing?

I’ll go first, because I’m quite sure I’ll win...

This was for a front-end mid weight software engineer position. The candidate
was smart and a 100% hire (I’ve since hired him 3x in different companies)

Me; “great thanks for your time and we will be in touch later today”

Co-interviewer; “I have one more question”

Candidate; “sure, go ahead”

Co-interviewer; “Imagine your uncle has just died and left you his pizzeria in
his will, what would you do?”

Me; “what?”

Candidate; “what...? Errr... sell it because I’m a software engineer, not a
pizza chef”

------
qls
Good to see an actual change vs. SWEs that suck at LeetCode-style questions
complain about these algorithmic questions...

It's ironic considering how MSFT pioneered the original brain teasers/problem
solving questions.

------
revskill
The quality of interview depends on quality of questions. A qualified question
is one which has many solutions, then the interviewee could pick the best
solution on his own.

------
danieltillett
Has anyone looked at how g loaded these interview quizzes and brain teasers
are?

------
29athrowaway
You can afford to be selective when you have more candidates than open
positions. Also you want to secure the top talent out there.

------
tootahe45
I wonder the effectiveness of the MS hiring process every time I use a
Microsoft product. Teaching interviewees how to test their software locally
for at least 10 mins before deploying it to hundreds of millions of people
would be huge.

