
Hired – Technical Interview Score - guessmyname
https://refdash.com/hired
======
bit_logic
Have we reached the tipping point yet? How much worse does the technical
interview process need to get before the industry finally decides enough is
enough?

When this all began, it was with the justification that it's not about getting
the right answer, it's about seeing how you think and approach a problem.

Companies don't even pretend this is the case anymore. It's now just expected
rote memorization of algorithm patterns, talk about it with the right terms
and key phrases, still pretend like the candidate had some brilliant "aha"
moment, when both sides know it's bullshit, and write the solution on the
board as fast as possible without errors.

A completely pointless process that's now heavily pushed by an entire industry
of tech interview prep (websites, books, interview coaches, etc.) When I now
read a discussion about tech interviews, I wonder how many are industry shills
who want to keep pushing the narrative that the process is great, but you just
have to keep practicing and studying more, using the right materials of
course.

There's also the side effect of ageism. Who other than college grads or
seniors in college has the free time to study for this process? Those with
families and full time jobs are of course going to have a hard time. Even if
they manage to find some free time here and there, how can they compete with
someone in college with no responsibilities who has time to hundreds of
problems? The simple answer is they can't. The college grad will always look
smarter and faster in the interviews.

~~~
bit_logic
Some comments here are asking what would be a better process. I think a big
step is just stopping with the requirement to get the optimal solution. If I
was doing these interviews, brute force is fine. If they can come up with a
brute force and write code for it, they completely pass my coding bar. Some
may say that's a low bar, but it's not. Because expecting anything more is
going into rote memorization territory of algorithms. It's also not realistic.
In the real world, brute force is always the first solution and it's the right
first choice for a number of reasons:

\- Brute force is often simple to understand. That directly translates into
maintainable code.

\- The business constraints on the input size may make the brute force an
acceptable solution.

\- If you need a better optimized algorithm, brute force is a great place to
start. It's basically your test case generator for verifying that your more
complex algorithm is right.

After they get brute force code written, the coding part is over and the rest
of the interview is about discussing why it's brute force and what we could do
about it. But no more coding is expected. It's a chance to be creative. If
they want to talk about improving the algorithm that's fine, let's draw some
diagrams. They want to throw more hardware at it? Sure, let's talk about how
to scale that. Maybe they've actually seen a business problem before that was
similar? Great, let's discuss how you solved it.

~~~
dopamean
I really like this comment a lot because it mirrors my experience with brute
force type solutions to problems. It's pretty much what I always start with
because on my first pass I need to prove that the problem can be solved in the
first place. Then that gives me a baseline where I feel comfortable making
changes and trying to optimize. Like you said, very often because of business
constraints the optimization only needs to go so far and so an "optimal
solution" may not necessarily be the one that is fastest in benchmarks. It may
be the one I can get out today that solves the business problem now.

------
alexandercrohde
I don't know what to make of this. Looks like a potential headache. As an
engineer in an interview, we never get told what we're being scored on in
advance (naming, speed, coming up with wonky edge-cases, scalability to
1million times workload, versatility, mentioning a bunch of relevant buzzwords
(bloom filters!), confidence?).

This is one reason why most interviewing is a subjective clusterfuck (not just
software, I suspect). I'd bet really good money that numerous companies would
reject Linus, Carmack, etc.

Hence the giving of graphs and other media that are meant to project an idea
of objectivity while reducing an engineer to a few quantities turns my
stomach.

~~~
ng12
> This is one reason why most interviewing is a subjective clusterfuck (not
> just software, I suspect).

This is why I get a laugh out of people complaining about whiteboard-style
interviews. Far and away tech still has one of the most intensely meritocratic
hiring processes. Not perfect, sure, but better than almost every other
industry. The majority of high-skilled jobs rely heavily on pedigree,
education, social standing, references, sociability, etc. I'd much rather
reject Linus for flubbing a BFS traversal than hire him for attending my Alma
Matter and knowing a lot about golf.

~~~
dragonwriter
> Almost every other industry relies heavily on pedigree, education, social
> standing, references, sociability, etc.

Almost every industry, _including_ tech, relies heavily on those things as
pre-interview filters. Tech is not a meritocratic exception here.

~~~
ng12
How many traders at Goldman Sachs don't have degrees from
Penn/Harvard/Yale/etc? I bet not very many. Yet some of the best developers I
know are self-taught or went to no-name schools.

My SO works in media and I can assure you the hiring process there is
completely, thoroughly, perniciously subjective. Experiencing her job search
really gave me appreciation for the fact that taking a test is 80% of my
interview process.

~~~
vonmoltke
> How many traders at Goldman Sachs don't have degrees from
> Penn/Harvard/Yale/etc?

I just did a quick check in LinkedIn, choosing four people at GS with "Trader"
in their title. The four schools I got: UNC Chapel Hill; Duke; Williams
College; Amherst College. I find that distribution hard to correlate with your
statement.

~~~
southphillyman
Aren't these all expensive private "elite non Ivy" schools? Maybe OP was being
too restrictive with the 3 listed schools but I think the general point being
implied is correct.

~~~
hyperpape
Chapel Hill is a relatively prestigious public school. The other three are
prestigious private schools.

------
throw401
The company I work for has turned into an Agile church. My productivity
dropped with more than 50%. Most of my colleagues have left already, now I'm
stuck with Agile masters, coaches and all the misery they come up with.

I know I need to look for a new job, but I hate the hiring process sooooo much
that I keep procrastinating.

I'm a senior full-stack, have written loads of production software over 20
years of experience. Who the hack from refdash is going to assess me with some
stupid algorithm question? Can he build shit himself? If so, he knows this
whole process is irrelevant.

Sometimes I really want to quit this business, it has destroyed a lot of fun
and passion for coding already..

~~~
lj3
Ditto. I'm taking steps to move into the business end of things.

------
wellpast
How well can you decouple?

That's all I need to know about you. If you can do this well, I (or you) can
fix all your bugs quickly. I (or you) can fix (most of) your performance
issues quickly. If you can decouple, you can isolate whatever other areas of
ignorance you have with ease.

How well can you decouple? Is it instinctual for you to riddle off code
architectures and solutions that are decoupled? It takes time + a value system
to acquire this skillset (so that it's instinctual) but once you have it, you
can move mountains with code.

The sad thing is our industry doesn't create the conditions for acquiring this
skillset. (And rarely, like in this scoresheet, do you see it measured in
interviews.)

The usual refrain is "I didn't decouple here or there because, um, it was
faster to take on the technical debt."

No, you didn't decouple there nor there because you didn't spend several years
falling on your face _trying_ to decouple and _failing_. Because you were too
busy getting paid to crank out _whatever works_ in the shortest possible time,
all the while saying "I didn't decouple here or there because it was faster to
take on the technical debt." You didn't decouple there because you don't know
_how_ to decouple there.

~~~
nnd
What do you mean by decoupling?

~~~
wellpast
Aka, separation of concerns. There's a lot of literature on this.

But the key point is that it's an objective measure. Given two equally
functional solutions to the _same_ problem, one will be objectively more,
less, or equally coupled than the other. It's a demonstrable, measurable skill
set; the relative coupling of two separate solutions is generally speaking not
some subjective measure.

------
rafiki6
What has this proven exactly? Has an study been done to show a strong
correlation between these test scores and ACTUAL programmer productivity? What
two metrics are you correlating here? All I see is a score. Is this score
valuable? I see nothing outside of pure speculation or opinion to suggest
that's the case in the link. And anyone who wants to rebut and say, "that's an
impossible task" or "but we know what it takes to be a good programmer" is
full of it. We don't. We actually have no real, documented objective way of
determining what a good programmer is. My definition is creates a program, to
spec that is acceptably performant. Even that has tons of subjectivity and
bias to it. Guess what, everyone has a different definition or bar. And
frankly they are all full of our inherent biases.

~~~
lolc
While I don't know whether this has been studied, I would expect the interview
results to correlate well with years of experience for example.

I think what these interviews test best is your ability to interact with your
peers. Candidates who interface badly with other people would have trouble
getting a good score. So yes some brilliant people will bomb those interviews.
And some talkers will manage to pass one interviewer.

In total I would expect those interviews to indicate a minimum of professional
knowledge in combination with good communication skills. That is valuable data
without it being objective in an absolute sense.

~~~
sulam
I'm not convinced they correlate with years of experience. In fact they may
well inversely correlate. I give a relaxed algorithm design screen to
candidates (of which roughly 70% pass -- I'm a soft touch) that include a
problem that is mildly academic in nature. New grads are the ones that tell me
they already know it and I have to move on to the next problem (which,
amusingly, they never have experience with). Experience people often have a
vague recollection of what it does, but not enough of a recollection to
prevent me from learning something from the discussion about how to get it
done.

------
CobrastanJorji
What degree of control do I have over the sharing of my interview score? If I
sign up for an interview with Refdash and get an outcome of "1.0/4, do not
hire," is that information going to be sent to every employer Refdash works
with?

Individual interview results for a single candidate vary wildly from day to
day. I wouldn't want one bad day to sink my chances with a large number of
companies at once; I'd much rather interview with each company separately if
that were the case. Using this company would just be far too risky, even if I
was 90% sure I'd ace the interview.

On the other hand, if my score was entirely private, and I needed to actively
grant each company that was interested permission to view it, I would view
this as a useful tool. A bad score would provide useful personal feedback for
improving my interview skills in the future, and I'd be happy to send along a
great score to potential employers.

~~~
kearneyandy
The interview data is only ever shown anonymously to companies, even if you
apply to a company with it. Only when the candidate and company agree to do an
onsite are names revealed.

Basically it's a good chance to practice and get feedback in the worst case,
and a way to save some time and get your foot in the door in the best case.

disclaimer: I work for refdash

~~~
CobrastanJorji
Could I ask you to add that to your FAQ? Knowing that the information is
anonymous would be super great.

It'd be even better if I could TAKE the interview anonymously. Then I wouldn't
need to trust Refdash on whether it was recording my information. Bonus: it's
much easier to limit bias when you don't know any demographic information
about the interviewee.

I understand that the downside here is that I could potentially take the
interview many times until I generated a positive result. But maybe there's a
happy middle ground somewhere.

~~~
kearneyandy
Thanks for the suggestion, just added it.

Interviews are somewhat anonymous, the interviewer only gets your first name
(added that too). We've discussed making it even more anonymous (voice
modulation, hiding video, etc.) but it seems like this has been studied before
and did not have much of an effect.

[http://blog.interviewing.io/we-built-voice-modulation-to-
mas...](http://blog.interviewing.io/we-built-voice-modulation-to-mask-gender-
in-technical-interviews-heres-what-happened/)

------
notmuchtotell
Clearly the person didn't run the code. The factorial function returns an int.
The factorial of 13 is already too big for a 32-bit int. The factorial of 21
is bigger than a 64-bit int.

~~~
kyberias
I think the code people have to write in interviews is not supposed to
work/scale in the real world. The purpose usually is to test whether the
candidate can implement a correct algorithm. Even though this is Java (?),
it's used more like pseudo code.

A follow up question could be: "Do you see some problems with running this
with big numbers? How would you scale it?"

~~~
logfromblammo
A double only has 52 bits of precision, but in a factorial, every second
number in the multiplication has at least one factor of two in it, so you
could get to 22! that way without any loss of precision.

Beyond that, "it depends" on what you're doing with the results. It's very
likely that the developers' time would be better spent removing the factorial
from the calculation that uses it, than on making the factorial calculation
better at calculating factorials.

Besides that, since you can only meaningfully do 22 different factorials
within the bounds of a 64-bit architecture anyway, just pre-calculate the
results and make it a lookup table. Returns in O(c).

~~~
nwellinghoff
What do you mean you can "only meaningfully do 22 different factorials within
the bounds of a 64-bit architecture anyway"?

~~~
logfromblammo
The 52-bit mantissa of a 64-bit double is only sufficient to represent all the
significant figures of 22! and a 64-bit integer can only represent 20!, so if
you want larger factorials with full precision, you need to use more bytes for
your numeric representation.

Not many real-world applications require precision greater than a double, but
pure mathematics can calculate millions of [decimal] digits of an irrational
number just for amusement. A cryptography application, for instance, would
need full precision, whereas an engineering problem might be perfectly fine
with fewer significant digits.

------
pvelagal
I am really curious to know why there are businesses such as hired.com build
around "interview" tools that rate software engineers only ? What tools are
available to evaluate other professions : 1) Doctors ? 2) Engineers from other
disciplines such as Chemical engineers, mechanical, civil , electrical
engineers ? 3) Lawyers ? 4) VPs / CEOs 5) Financial analysts ?

~~~
daleco
I moved from SE to UX because I needed a change from coding 8 hours a day.
That's one of the field where I could easily transfer my skills.

The interview process usually requires a portfolio, 2 phone interviews and one
onsite interview. I had to talk about previous projects and what was my
problem solving process, followed by a case to solve. This was refreshing and
way less stressful, a walk in the park compared to the white board interview.

I understand the challenges that hiring managers face, but who has time to
spend a day at 5-10 companies to switch job... A lot of companies still offer
only 10-15 days vacation a year...

------
pklausler
Don't just whine about bad programming interview practices. Suggest something
better that wouldn't have more false positive hires (i.e., candidate can't
program, but can pass interview), and maybe fewer false negatives (candidate
can program, but gets filtered out anyway).

This is a hard problem. Kindly think about it from the perspective of a
technical interviewer who really has no idea whether the candidate on the
other side of the table can program a computer or not and has to make a
hire/no-hire call in 45 minutes.

------
pfarnsworth
I have never seen a correlation between passing standard coding interviews
(whiteboarding, algorithms, etc) and how they perform as a candidate 1 year
after hiring.

Does anyone have any data that surfaces this information? All these companies
like Triplebyte do is standardize incoming interviews, but what's the point if
it has no bearing on whether or not the candidate is actually successful.

~~~
alexandercrohde
Well the other half of the problem is that I think most companies are so
disorganized that even if an engineer does great work the company might not
know within a year.

They may look at tickets completed, "buzz" around the individual, being
assigned to a project that was highly profitable, staying late a lot, or any
arbitrary thing they want to.

------
nnd
I've tried Refdash twice as a candidate. Frist time I used, I couldn't connect
with the interviewer because of the issues with VOIP, the second time it went
smoother and the interviewer appeared to be doing a great job at providing
written feedback after the interview.

On the downside, scheduling seems to be awkward: you have to manually select
your availability, instead of picking a timeslot, then the interview is
scheduled for you. Personally, I prefer to select my own timeslots like with
interviewing.io, unfortunately, the latter ironically is almost always out of
availability.

------
zappo2938
In 4 - 6 weeks I'll be interviewing again. If anyone asks me to implement an
algorithm during an interview, I'm going to search for the answer and use it.
I don't have time to waste nor do I want to work for a company that wastes
time which is money. How can they afford me if they keep wasting it?

------
goodoldboys
I'm assuming this is a fake interview to show off the interview feedback for
your platform? If so, I think it looks great and is a huge part of what's
missing from the current "standard" engineer interview. Receiving feedback
like this would be incredibly helpful in figuring out what I'd need to work on
to pass technical screens.

The fact that there's very little proven correlation between being able to
jump through these hoops and being a good programmer is another story
altogether, but I like this!

~~~
kearneyandy
Not OP, but I work at refdash

This is a fake feedback report for an interview, to show what companies and
candidates can expect.

Thanks for your comment. I do think it's important particularly from a
feedback perspective. Since companies don't give feedback, this is one of the
first things to attempt to help candidates learn about what is being evaluated
and how to improve.

I'm hoping this is a first step to finding out a better way to correlate those
things, for now I think that things that help candidates find jobs are a good
step :)

------
nxsynonym
Would an industry-approved certification be a viable replacement for on the
spot technical interviews?

Somebody feel free to change my view - but I don't see why there can't be a
standardized certification (could be broken up into specific areas of
tech/comp sci/programming) that proves the applicant is competent in the
industry standards, and just focus on the personal/soft skills/culture fit of
the interview.

~~~
mkozlows
Certs are worthless. Generally, the most terrible programmers out there are
loaded down with certs.

~~~
geebee
Largely agreed about certs. But the actuarial exams aren't worthless. We may
need to look to a new model.

------
wbillingsley
Always embarrassing when after deciding academic results aren't a good enough
indicator, the showcase achievement for "a better process" is a single
tutorial question from an undergraduate class (or less than a thousandth the
evaluation we do of students while they study)...

Welcome to the Cult of Hiring, I guess.

------
kearneyandy
Just for some context about this document, this is a fake interview with fake
interview feedback.

It is meant to highlight some of the criteria which Refdash uses to evaluate
candidates and how companies and candidates can expect to receive that
feedback.

disclaimer: I work for Refdash

------
nawitus
This seems to have a rather strong focus on speed.

~~~
drstewart
Algorithmic complexities in general, which in the day-to-day life of a
developer are rarely considerations. I can probably count on one hand the
number of times I ran into a problem that required the type of algorithm
design needed in these interviews, yet they form the basis of hiring in many
places. Crazy town.

------
_arvin
_crowd boos_

------
graphememes
This is horrible.

