
Hackermeter (YC S13) Wants To Kill Your Résumé And Replace It With A High Score - rjvir
http://techcrunch.com/2013/08/12/hackermeter-wants-to-kill-your-resume-and-replace-it-with-a-high-score/
======
pmorici
It seems like all "code exercise" sites like this have the same annoying
problems which I'll generally describe as lack of attention to detail in the
exercise descriptions.

Take their Fibonacci exercise as an example. In the first section they say,
"The first line of the input will be an integer N (1 <= N <= 100)" which
specifies how many test cases follow. Then in the next section they say, "The
first line of the input will be an integer N (1 <= N <= 10000), specifying the
number of test cases"

So besides the fact that they aren't even consistent in specifying the
expected range of N it isn't even clear why you would need to specify the
number of test cases in the first place vs just reading one from each line
until EOF. That makes you think maybe they want you to put in some basic error
checking which again since they aren't consistent with N turns into a trial
and error exercise if it even matters at all.

There is also no indication about the version of interpreters or compilers
being used to check the submissions. If I choose to write my solution in
python is that python 2 or 3?

Combine all that with an implied scoring system based on speed or number of
tries and you have to wonder if this is really measuring anything relevant or
if it is just filtering for people who's default assumptions happen to be the
same as the person who wrote the lame exercise description.

~~~
zenbowman
I had to seriously do a double-take, I thought this was straight trolling, not
a real company.

I doubt any serious developers would consider doing this, there's already a
way to demonstrate your programming capabilities in the real world, either by
contributing to an existing open source project or maintaining your own
projects on github.

------
vabmit
A number of start-ups have either already failed with a very similar business
model or failed to go exponential growth. There was a good "Lessons Learned"
post by one of them recently here on hacker news that got comments by both
hiring managers and hackers. Why the rehash, again? What is this start-up
really doing differently?

I personally wouldn't be motivated to use it either to find work or to hire. I
think, when hiring, you really need to do a face to face interview. I've found
resumes work well enough for pre-screening that it's not a pain point for me.
And, it's not like there are so many candidates out there looking for work
that it's impractical to do face to face, or at least telephone/skype,
interviews with all the good candidates. I learned the lesson that face to
face hiring is needed by hiring someone based on a co-worker's recommendation
(even though he did badly in an interview, I looked passed it based on the
recommendation). It's so costly and painful to make a hiring mistake that
there's no way I would deviate from the current model and try a new method or
service.

~~~
toyg
_> it's not like there are so many candidates out there looking for work that
it's impractical to do face to face_

That really depends on your target market. For the most popular technologies
(.Net, Java etc) in the most popular areas (London, Glasgow etc) for low-to-
medium positions, you'll get tons of decent candidates.

------
throwaway420
I wish this company well, and perhaps they'll be successful, but I don't think
they've nailed down some ultimate formula for ranking and hiring programmers.
This system sounds very easy for low-quality programmers with lots of time on
their hands to manipulate - you sign up with a throwaway account with some
fake name, get the list of questions, take your time searching for the answers
online, and then sign up with your real information and have a carefully
prepared way of typing out answers that get you high scores. If this achieves
any measure of traction, you can bet that this will happen a lot.

This isn't an easy problem to solve and is something that companies have been
struggling with for a long time, so no disrespect intended. But based on my
understanding of how this works I don't think it will be very likely to yield
great results.

~~~
xianshou
Thanks for the critiques, and yes, it's not an easy problem.

There are countermeasures. For instance, who said we'll let you see the whole
question bank? And if you type a close enough approximation, out come the
plagiarism checkers.

But when somebody's gone to the trouble to make a database online of every
problem, I'm happy, because if that's so we must be successful enough that we
have plenty more resources to game their gaming.

------
michaelrbock
Some direct competitors:

\- [https://www.hackerrank.com/](https://www.hackerrank.com/) (aka
[https://www.interviewstreet.com](https://www.interviewstreet.com) a YC
company)

\- [https://codeeval.com/](https://codeeval.com/)

\- [https://coderwall.com/](https://coderwall.com/)

\- [https://www.employiq.com/](https://www.employiq.com/)

\- [https://www.mindsumo.com/](https://www.mindsumo.com/) (college student
only)

\- [http://www.codewars.com/](http://www.codewars.com/) (in beta)

~~~
yogo
Isn't topcoder still the king of this kind of thing. I don't think they
particularly market it as being geared at hiring but companies definitely hire
from there (I know UBS likes to) and your score does reflect on your ability,
at least as far as being able to code competitively.

Edit: hyborg beat me to it :)

------
jacquesm
The best way to find good people is not by passing people as a value but by
passing them through reference.

~~~
xianshou
I'm one of the co-founders and I'm _still_ saving that quote.

As for the answer to the implicit question...the best people are passed by
reference when they're already recognized. We're aiming to help people show
that their skills have value when they don't have a reference.

The number is just, well, a pointer to a broader portfolio - and we'll only be
making that portfolio more expressive over time.

~~~
rob_b
Depending on your industry and location, recognition doesn't always come easy.
Considering that a majority of employers will have a non-technical HR employee
only makes things worse. I think that there is definitely a spot in the market
for a company that can successfully break this barrier and improve the
marketability and recognition of job seekers. The key is that a service like
this isn't the be-all and end-all, but instead should serve as a supplemental
indicator of a candidate's ability.

------
austinl
As a younger developer, I struggle to imagine our senior developers - who have
been doing complicated things for 5-10+ years - ever stoop to a number to
quantify their experience. I suppose that senior developers are probably not
the target audience, but I would never want a company to expect this, or a
score to define my experiences as a developer.

A technical screening can be handled in a ten-minute phone call for free -
with the dignity of the applicant intact.

~~~
saidajigumi
As an experienced developer, I struggle to imagine any of the talented younger
developers I've worked with being well-represented by a number that quantifies
their experience.

I take issue with any hiring system that attempts to narrow the field of view
of candidates. This includes any single-number "scoring" system, as well as
hiring cultures that focus on single methods (e.g. "puzzle questions") to the
near exclusion of other hiring criteria. If your organization is doing any of
the above, it's shooting itself in the collective foot.

~~~
jacquesm
The problem is not the talented younger developers. It's the not-so-talented
ones. At least they keep us old guys in business to clean up the mess
afterwards but this is quite a problem in some fields. The talented younger
developers are doing quite well by themselves. I get to meet them in small
groups every now and then and some of what I find blows me away.

~~~
cpncrunch
I can't really see this being any use at all for finding a really talented,
experienced developer. Any half-assed developer should be able to do these
little puzzles with a little research.

The really hackerish stuff can't be tested for in a format like this.

------
tsumnia
Best of luck, I recently received an email that WorkforPie.com was shutting
down ([http://thenextweb.com/insider/2013/08/02/developer-hiring-
st...](http://thenextweb.com/insider/2013/08/02/developer-hiring-startup-work-
for-pie-to-shut-down-on-august-31-points-users-to-coderwall-and-pitchbox/))
and they had the same sort of model.

I get we always complain about the developer interview process, but are these
business models actually solving the problem?

------
eksith
"...Lack of coder personality" That's more that just a mere gripe.

Many a job has been won or lost based on the character of a prospect. In fact,
I've lost count of how many times I've seen otherwise very technically skilled
people lose out to basic inter-personal communication. Above all else, a
seeming lack of tact.

~~~
xianshou
Indeed. Perhaps an option to include a video and explain an interesting
problem? Definitely something we're considering.

~~~
eksith
I like that. An "in your own words" monolog can go a long way to help
recruiters understand the type of person they will be dealing with.

That also means the recruiter(s) need to understand the questions they're
asking. Technical merit is all well and good, but how well can a recruiter
process the answers they're getting? This can't be a bullet point
questionnaire with blind matching of answer "C" to question "1". Sometimes
solutions aren't so black and white.

------
com2kid
Coding is a small part of a developer's job. Given a simple task, a programmer
can code to it. A software engineer can create a proper design that expands
out to solving a wide range of problems, a design that is easy to maintain and
flexible enough to deal with problems.

I don't want to see someone's first solution to a problem. I want to know the
solution they came up with after 5 mental iterations and a few hallway
discussions with peers.

~~~
e12e
I wonder if the better approach (for a company like this one) would be to hire
4-5 good programmers, and 4-5 _bad_ programmers, and let candidates write the
tests; assign the candidates that most successfully identify the on-staff good
coders from the bad ones the highest score?

It sounds about as silly as this scoring does in the first place anyway...

------
toyg
This feels more like a candidate for acquisition and integration in some HR
platform, rather than a standalone effort with a real future.

------
jsnk
Only people like software engineers would stoop so low to subject themselves
to measures like these.

Aren't technical interviews (which are just politically correct way to test
IQ) enough of a humiliation one has to go through?

~~~
egaoga
Let me rephrase what you said in a better perspective. I don't think _anybody_
would subject themselves to unnecessary testing, regardless of the profession
or industry. If this becomes an industry norm, then tough cookie for us
engineers.

Interviews are a necessary and evolutionary process of finding good employees,
and we know that so that's why we subject ourselves to them. It's not a form
of humiliation because it's the norm. How else would a manager figure out if
you got the skills and character to do the job? It's not a perfect process by
any means, but it definitely helps.

Here's where this startup tries to take things to a different level. I get the
idea behind it, but I cannot fathom why anybody would want to subject
themselves to unnecessary testing. What's the ROI? Why give out personal
information and have it stored on some database for some undetermined,
possibly indefinite amount of time (forever? 1 year? 5 years?)? Am I going to
be paid to do this? Can I no longer get a job if I don't do this? If this is
purely voluntary, I have absolutely no incentive of taking the test.

Perhaps these questions should be something the founders should consider.

------
prawn
In South Australia (and potentially elsewhere) university admissions for
medicine have started to involve something of a personality test rather than
purely academic aptitude. The workplace, and especially dealing with patients
(colleagues or clients in this case?), is more than a calculated diagnosis.

------
josephscott
I was disappointed to see that they limit the password length to 30
characters. Perhaps one of the exercises should be to hash passwords that are
longer than 30 characters.

Bcrypt will support up to 72 characters. If you are going to limit the
password length I'd expect it to be something closer to 72 than to 30.

------
ig1
It seems like as time goes on YC is funding far more startups that are direct
competitors with each other. Hackermeter's pitch looks almost identical to
that of InterviewStreet (YC S11).

~~~
dman
Makes sense from an investment perspective. For the verticals that they care
about this allows them to cut down on their exposure to execution risk.

------
jongraehl
Sounds fun, but everyone solves the same problems (ok, ideally not: "sets of
custom-built, rapid fire coding challenges built by each employer. You can use
Hackermeter’s pre-provided challenges, or build your own")? How long until
copied solutions corrupt the scores? Better check performance on a held-out
set before interviewing/hiring.

Let's not get into plagiarism-detection arms race; consider automatic
refactorings / pretty printers with variations, and human input to vary
variable names.

------
djunod
Oh Yay, another useless measurement of a programmer's abilities, very much as
useless as the puzzle interview made popular by Microsoft and then Google. So
you can measure somebody in an artificial sprint environment that has nothing
to do with actually delivering a product. The real measurement is how somebody
does delivering actual products to market.

~~~
walshemj
Not going to help you with problems like when your boss says oh we have
brought a mega expensive bit of kit 4 to 5 times my annual salary.

Just hook it up to the spare PDP we have and go and see Bob and work out how
to use it to analyze the data from his experiment tracking high speed droplets
droplets in 3dspace.

BTW I had to write a driver to network with a rtos (RT11) just to talk to the
device. I was 22 at the time and joined the company straight from high school.

------
jazzychad
Please make your pages load faster. A 1.1MB header image which blocks other
assets from loading doesn't do well for you.

~~~
xianshou
><

Thanks. Will be fixed.

------
mansam
Hey xianshou, I've been trying to figure out how to privately message you guys
about this, but your info@hackermeter.com email doesn't seem to be working.
Your online code editor is leaking environment variables pretty severely;
stuff like API keys.

~~~
xianshou
Thanks. info@ is fixed, so you can send messages there now. Please send the
leaks and we'll plug them.

~~~
mansam
Sent.

------
ucarion
I don't know if this is common knowledge or something, but how are all these
companies implementing a secure way to execute arbitrary code? Sounds like a
non-trivial thing to get right.

~~~
yogo
Usually there is some kind of sandboxing in place. That way you can limit what
the code is allowed to execute. For e.g. in PHP you would probably use
Runkit[1] for this.

1\.
[http://php.net/manual/en/book.runkit.php](http://php.net/manual/en/book.runkit.php)

------
chadscira
Check out [http://codewars.com](http://codewars.com) (only supports JS, and
Ruby). They offer problems with a voting system for solutions.

------
swanson
Perfect opportunity to let me Auth with GitHub _wink wink_

------
dman
Will they be normalizing high scores against Fabrice Bellard?

------
pkinsky
Have you considered adding Scala? You're already running jvms for Java, and
you could do incremental compilation with SBT to keep everything responsive.

------
wavesounds
Is this mainly for hiring people overseas? Otherwise couldn't you just pull
the candidate in for an interview and look over their shoulder?

------
hedonist
"I am not a number, I am a free man!"

~~~
hexasquid
I am NaN!

