
Propellerhead Software - Application Programmer Test - Arjuna
http://www.propellerheads.se/company/employment/index.cfm?fuseaction=display_test
======
ekidd
This test would be _much_ more useful if they put all the errors into a broken
string class, and asked you to find the bugs. Glancing at a broken class and
seeing all the standard C++ bugs "pop out" is a useful skill for a C++
programmer. But instead of defining an actual, useful class with a bunch of
bugs, they use horrendously nonsensical and broken code fragments. For
example, I can't even figure out what the GrowArray function is supposed to
do.

It's sort of like memorizing a chess board: A chess expert can memorize an
actual game-in-progress quickly. But if you just place a bunch of random
pieces around a chess board, a chess expert can't memorize it any better than
anybody else.

The later examples with memchr are less painful, because memchr actually has
well-specified behavior. But even there, the code (most likely) dereferences a
pointer one past the end of an array, which is (1) invalid C++ and (2) not on
their list of possible bugs. Granted, memchr is an internal compiler function,
so it's allowed to rely on undefined behavior. But if this is a multithreaded
system, that code potentially corrupts the heap's bookkeeping data structures
while memchr is running.

Just _looking_ at the first part of this test is like listening to somebody
drag their fingernails across a blackboard. I'm sure that they're a lovely
company, but _ugh_. To paraphrase Wolfgang Pauli, that code's so bad it's not
even buggy—it's just semi-coherent nonsense.

Another interesting aspect of this test: Both the code examples and the bugs
are pre-STL, pre-boost, and pre-TR1. For example, there's no question asking
why you can't put a std::auto_ptr into an STL collection. And a huge fraction
of the bugs in the test could be avoided by making rudimentary use of modern
C++ APIs. Granted, some companies have legitimate reasons to avoid std::vector
and std::tr1::shared_ptr. But avoiding those classes makes C++ vastly more
error prone, which is usually a poor tradeoff in the modern world.

~~~
davisp
Its been awhile since I C++'ed heavily, but I think this isn't quite right:

"But even there, the code (most likely) dereferences a pointer one past the
end of an array, which is (1) invalid C++ and (2) not on their list of
possible bugs."

There's a weirdo thing with array allocation in C++ at least which makes it
valid to use arrays with all of the std:: algorithms that take a start and end
parameter. Unfortunately I can't remember it well enough to find a good
reference but as I recall it was more than just a pointer comparison as the
std:: algorithms wanted to be able to dereference it occasionally.

Then again its been about four years since I've touched C++ so some of these
details have leaked out and gotten mixed up.

~~~
ekidd
_There's a weirdo thing with array allocation in C++ at least which makes it
valid to use arrays with all of the std:: algorithms that take a start and end
parameter._

The actual rule is pretty subtle, and I don't have a copy of the C++ standard
in front of me. But let's see if I can remember the precise rule.

Given an array _arr_ with length _len_ , you're allowed to create the pointer
_arr+len_. This is the _end_ pointer you see everywhere in modern C++ APIs.
However, if you try to deference _arr+len_ , or try to create the pointer
_arr+len+1_ , the result is undefined. Typically, the former might point to a
locked page, and the latter might overflow the pointer size. But if your
compiler applies very aggressive optimizations, even stranger stuff can happen
when you rely on undefined behavior:

[http://blog.llvm.org/2011/05/what-every-c-programmer-
should-...](http://blog.llvm.org/2011/05/what-every-c-programmer-should-
know_14.html)

In the _memchr_ example, the code potentially writes to _arr+len_ , which has
a decent chance of temporarily corrupting the implementation-dependent
internal headers of the next heap block. In the presence of threads, this
would very occasionally corrupt the internal state of _new_ and _delete_ , and
debugging it would be a stupendously painful experience.

------
cageface
These guys are behind what is easily the most robust and reliable audio app on
the market (Reason). Whatever their code practices are internally, they're
doing _something_ right.

~~~
eropple
Easily? Meeeeh. I really, really like Reason, but for me it lives as a bag on
the side of Ableton (Record doesn't cut it) and I've never had an issue with
Ableton either.

~~~
cageface
I love Live but Ableton themselves admitted that Live 8 wasn't as stable as it
should have been and stopped production on Live 9 to iron out the many, many
crash bugs in Live 8.

Reason may not be the most full-featured audio app but it's certainly the most
stable.

~~~
eropple
Really? Huh, I've never had a stability problem with it (even with my own
home-rolled VSTs). The more you know...

------
Deestan
Q1 and Q2 are horrible, but it picks up a bit after that.

Q1: I see a problem on all lines 1 through 6: The class doesn't _do_ anything.
There is also no documentation as to what it is intended to do. Here it
becomes, as leon_ also comments, a game of "guess what the test writer wanted
you to answer".

Q2: "It will work if..." is a meaningless phrase, because I am not told what
the code is supposed to do.

Q12: "How many books have you read this year?" It is immediately obvious that
"5 or more" is the answer that will give me the highest ranking from their
application sorting robot. Everyone is going to answer "5 or more". The honest
ones will then flip through the needed amount of books before the interview.
The question is as meaningless as "are you a good developer or a bad
developer?"

------
motters
The main problem with these trick programs is that as a software engineer of n
years I don't debug programs by staring at them for a long time or by
memorising a lot of tricks. I usually do it by running them and checking their
output, or by catching runtime errors in a debugger. Writing software is not a
one way process, it's a skill or an interactive process with the machine.

------
Vlasta
In my book their value "1) Good interface: safe to use - hard to misuse" is
the same as their non-value "3) Grand design for the future".

------
scottdw2
I think is may not be testing the right things.

For example, what if the test was in Visual Basic, and instead of asking
questions about non virtual destructors in virtual classes, it was asking
questions about uses of various database Apis. " Oops, you used the wrong enum
value in that function call."

Would that test yield top programmers, or just people who knew a database API
very well?

Also, it doesn't demonstrate the ability to solve problems by writing code. At
best it tests C++ code review skills.

------
tomjen3
Unfortunately what half of those questions tests are pointless - don't test
C++ minutia (especially not in 2011), test whether the candidate can code.

~~~
bandushrew
They weren't testing minutia, they were testing depth of understanding and
experience. I thought they did a decent job, too.

I would have preferred to be able to get a result without actually applying
for the job though...

------
ctz
Q10 is missing the correct answer: "someone inexplicably decided to implement
part of the C standard library in C++".

Not to mention the function's name will be mangled, making it unusable for C
users.

------
theDoug
Facebook has a (somewhat) similar process:
<http://www.facebook.com/careers/puzzles.php>

------
roel_v
This test sucks - I just spend 15 minutes and I don't even get a result!

------
leon_
Warning, some spoilers ...

I don't like this kind of tests. I always stumble upon questions where I
wonder if it's a trick question or just a typo/mistake by the test creator.
Example:

Function D will loop for ever if size_t is an unsigned type. Function E won't
even compile because of the comma and the missing * in the first line.
Function C is an edge case. The external effect is the same only in a single
threaded environment. If there's parallelism or interrupts the external effect
might not be the same for certain conditions.

So what do I do? Do I check the box or not? If I do, it might fire back
because "I didn't catch those obvious errors". If I don't then my application
might fall through the grid because "too many wrong checkmarks". Maybe the
recruiter is just a mindless HR bot and doesn't understand any of these
questions. So arguing would be pointless.

If you want to determine my skills, give me something to do. Say give me 6
hours to implement a server or something (not too big, not too small for 6
hours), then shuffle through my code and see if its quality pleases you. Being
bombarded with off-by-one trick questions just annoys me.

