Hacker News new | comments | show | ask | jobs | submit login
A Method I’ve Used to Eliminate Bad Tech Hires (mattermark.com)
1214 points by nickfrost on Nov 9, 2016 | hide | past | web | favorite | 499 comments



This. 100% this.

As many others are saying, I find that this is the most comfortable way for me as an interviewee to demonstrate that I do in fact know what I'm talking about. Put me in front of a room full of people and a whiteboard and throw something at me, and I'll freeze. Freezing in that situation and being a bad engineer are wholly separate things.

Anecdotally, when I get one of these problems, I spent 20 minutes reading the prompt, looking at resources and making an outline of my code. Then I step away for the afternoon and don't think about it. Let the subconscious internalize the problem for a while and when I come back, I've got a much better mindset to approach it. Solving new problems on the spot is just a skill I don't have. I prefer to think that this is what makes you valuable as an employee and engineer, solving a problem by the end of the day. Not while under scrutiny. Personally, my resume and experience don't get me far interviewing by the big companies, but on the occasions that I get take home code assignments, I tend to get pleasantly surprised and pleased reviewers and it takes me to the final rounds (where I face the dreaded white board and actually-there-is-a-correct-answer situationals).

My two cents, this just really resonated with me. Does anybody else happen to solve problems this way?


I don't care about getting paid. I don't care if I am doing it at home or in the interviewer's premises.

Don't let me connect to internet. I'll also leave my phone with you just to make sure I don't have access to the internet. I don't mind taking the time away from current employment to go through the process.

BUT PLEASE LEAVE ME ALONE THOUGH WHILE I AM AT IT. FOR GOD'S SAKE DON"T SIT NEXT TO ME AND TALK TO ME OR EXPECT ME TO TALK TO YOU WHILE I AM SOLVING THE PROBLEM !!!

Let's discuss the solution or the non-solution AFTER I have spent some alone time with the problem. I don't mind getting rejected AFTER I have made a serious attempt at it.

It is simply beyond me why people think it's all right for a complete stranger to sit next to a programmer while he is trying to solve a problem under the significant cognitive load of a new environment, new people possibly a laptop/keyboard he has never used.

This did sound like a rant :-/ but what I wanted to say was that the process in the article seems fair enough. Interviewees should be given the alone time to solve the problem.

Most people are decent and sociable enough that once they are one board and are comfortable and familiar with the rest of the team they will be able to function and be reasonable in pair programming envoirnment or other technical discussions as well. BUT not when they are in an interview. No matter how it is sugar coated, interview is where a programmer is getting JUDGED and that by default leads to pressure.


Sounds like I'm completely the opposite of you! I'm fine with someone sitting next to me - I love pair programming.

But cutting off internet access means no StackOverflow, no online documentation. The test should be very similar to normal working conditions.


>I love pair programming

Yes, but do you love pair programming:

-with a person you met less than 5 minutes ago?

-on a non-standard programming task you've been given less than a minute to think about?

-with a person who is focused on assessing you rather than assisting you?

-with your future job at stake and potentially your ability to ever interview at that company again at stake as well?

I enjoy pair programming too but I still choke on whiteboard programming questions during interviews. And this is after 20 years in the industry.


Interesting to note how different people are indeed.

I am fine with white board type situations, don't worry about writing code without IDEs etc (in my very first job vi editor was the only thing we had to write code so I still have instinctive memory to be resourceful without an IDE but I love IntelliJ for most things). Happy to discuss/draw out ideas/solutions on whiteboards.

But find it very seriously annoying talking or being expected to talk while I am writing code !

I think the internet-access requirement depends on the problem. If I am expected to solve a problem by first-principles on a core CS topic then I guess internet is of no use ? Online Java Docs for e.g. may not serve a purpose here.

If the problem requires using many different tools/APIs for the solution then I guess you are right, internet access adds value.


Indeed interesting how different people are.

Here, I'm like you. But it carries over to normal job - I work best when left alone. Especially when I have a tough problem to solve, just having people sitting close to me makes me frustrated, and if they're talking, then I won't be able to concentrate at all. It takes a lot of energy for me to be able to work in the presence of other people.


I understand where you are coming from...

I think somehow the world was agreeable enough to drink the Pair-Programming KoolAid. The consulting companies or body shops were happy since they could bill 2x the number of heads for the same amount of time. The "buy-side" of Pair Programming were happy because the KoolAid evidently worked.

Don't get me wrong, I am all in favor of getting my code eye-balled by anybody because I know I don't write perfect code 100% of the times and always open to critique and learning.

Here's a thought experiment. Have two teams A and B. A are pair programmers i.e. |A| = 2*|B|.

Now make team A don't do any unit testing, let them purely rely on pairing to ensure quality.

Enforce team B to do TDD to the best of their efforts.

I would be personally very interested in knowing where the consumer of the code and writers of the code have more confidence on code quality and functional completeness.

Open-plan offices are hip and in Europe maybe even a requirement due to space constraints but I'd personally choose a cubicle over higher pay.


I'd love to see that experiment, as I'm actually more inclined to believe in (occasional) pair programming (by the virtue of enforcing some focus) than in TDD.

> Open-plan offices are hip and in Europe maybe even a requirement due to space constraints but I'd personally choose a cubicle over higher pay.

Yeah. When I was still a high-schooler and later a university student, I used to laugh at cubicles. Oh, the corporate culture, oh, the rat race, etc. Now that I spent some time in this career, I would voluntarily go to cubicle and even take a pay cut. I now understand that cubicles were a good idea of solving the space/cost constraints.

Hell, every now and then I think about building my own cubicle out of pizza boxes, but I'm pretty sure the management wouldn't be happy about it. Over-the-ear headphones suffice for eliminating sound for now, but I still can't focus when I have people walking behind my back.


It's also interesting that the pair vs TDD divide is often based on the solution space.

Service/backend programmers often extol TDD because it is easy when working in a single language where your only IO is strings (i.e. Undergrad CS)

Front-end programmers may use TDD in certain established contexts, but pair may be more important when there are numerous integrations across unsupported toolchains. For example, when was the last time you used TDD with CSS? Oh? Too hard to verify that the CSS for a page makes it so that you can't click a button or print a page? Oh? Market bug isn't your responsibility?

Yeah, I see a lot of defensiveness when TDD is impossibly hard, so it's not "all that". But, use it if it makes sense. Know your tools and use the right tool for the job.


> I think somehow the world was agreeable enough to drink the Pair-Programming KoolAid.

Not sure what you mean. In 25 years, my latest job is the only one that pair, and even then it's only a few teams in a very large organization.

I don't think your AB test makes much sense. The two practices are orthogonal. Complementary, but orthogonal.

If I had to take one over the other, I'd to TDD, or at least strive for excellent test coverage over pairing, but they really don't have anything to do with each other.


> Not sure what you mean

Try enlisting services of a consulting company such as ThoughtWorks, Accenture etc. Talk to the people who are already using such consultancies.

Every single job interview I have had this year involved a pair programming session and lists pair-programming as a "culture-fit" requirement.

The very latest one I had to go to, I had already warned the recruiter that that is not my strong point but he still sent me in anyways !

>But they really don't have anything to do with each other

I was trying to propose that maybe enforced pair programming is actually of no real value or very little value it at all and hence pair-programming has nothing to do with anything actually.

I value formal code-reviews. I have ZERO love for being forced into pair-programming be it in a real job or during an interview.

Somehow pair-programming caught on because KoolAid happens.


Pair programming neatly sidesteps a number of problems, when done well. For a team, it helps to spread knowledge among the team, encourages people to follow the team discipline, gives a second pair of eyes on any bugs and gets people up and running faster.

done well is the tricky bit. You need people who get on ok with it, social skills need to be developed, and you need discipline around not committing code which hasn't at least been reviewed. Social skills is the hardest part IME: knowing you can ask for a break, some time to think, or to split for some research before resuming work as a pair.


What works for one person doesn't work for everybody. I feel that it does work for me - it keeps me focused on the problem at hand without getting distracted, over-engineering the solution, or yak-shaving.


Some of that really depends on the problem and the language. I program Java full time, yet I'd be useless with vi. I lean heavily on the IDE. I've done this a long time and I firmly believe that I should use the best tools available, not just a text editor or a TEXTAREA, even with syntax highlighting.

Having suffered through a couple code interviews in Java, I now pick Python when allowed to pick a language. It seems like the language for me that allows me the best chance of coding effectively without an IDE.

I just did a phone screen yesterday that was extremely painful. After 45 minutes of fumbling around the thing mercifully ended and I googled for a few solutions. According to what I read on stack overflow, the problem is unsolvable with the constraints given. I expect to get the "no thanks" email from the recruiter soon and I plan on asking him what the proposed solution to the problem was - not out of spite, but because I'm genuinely curious. Maybe I misunderstood the problem statement.


Just out of curiosity, between Eclipse and Netbeans which IDE do you prefer for Java and why?


I prefer IntelliJ, with Eclipse a distant second (I know I'm not the comment author, but I spend 40+ hours a week slanging Java)


Hey, you can take the words out of my mouth. I like IntelliJ as well.


Pair programming is not the same as writing code on an interview.


It's completely different in my opinion, the only similarity is that you have two people and one computer.

In pair programming you're both working together, complementing each other to tackle a particular problem or issue. In a good pair programming session we'll both work together to surpass an obstacle.

In an interview, one person is observing, testing and ultimately judging the other person, while actively ensuring they do not contribute to the solution of the problem so as not to skew results. Only one person is truly working towards the programming goal, and both participants know this.


Agree. Pairing up during the onsite follow up interview whilst you describe and extend your sample app is essential part of my recruitment preferences.

That is how I expect people to work once they are hired so I expect candidates to be comfortable with it. Many are not, including me 10 years ago. But most are these days.

I don't expect them to solve the problem perfectly during that time, or even at all. The important bit is to test how they try to solve it, and how they communicate during it. Using Dash, google, SO, etc if fine, asking for help is fine. Essentially portray how they would work if hired.

Many times this has quickly highlighted complete blaggers, or cowboys that should be rejected, or people caught like rabbit in head lights that are not right for us now (but maybe later).

Adding a requirement that completely blows most people's initial domain model is good way to see how people respond, even if the requirement is essentially impossible.


> Agree. Pairing up during the onsite follow up interview > whilst you describe and extend your sample app is > essential part of my recruitment preferences.

When you say "describe and extend your sample app", where does this sample app come from ? Do you give the candidate a chance to work or go through the sample app on his own for some time before you pair ?

If that is the case then yes, I guess that is probably the correct way to do it.

However of late what I have experienced in general is that I have to go in, meet up people for the very first time in my life, they give me a problem and a laptop/computer/pairing station and a written description of the problem which I get maybe five minutes to assess and understand and then... have a go at it, while the strange (sometimes grumpy) developer sits next to me. If I go quiet for 5 minutes I get the "what are you thinking ?" question and then I have to talk while I write code at the same time !

Maybe it works for some or most of the people out there.

Not for me.


'normal conditions' vary widely, normalisation to a lowest common denominator allows for easier comparison.

The other extreme end is the young students assumption, "you don't need to know everything, just where you can find it", but the ammount of stuff I have to look up repeatedly increases exponentially with, eg, every wiki-page I read and that just takes too long to the point of impossibility.


For any software development position, normal working conditions probably includes an internet connection.


like, directly plugged into the hypothalamus?


Why would you need Stack Overflow ? Seriously, is the ability to copy/query from Stack Overflow expected from 'programmers' these days ?!


That's a bit rough. I think you are missing the other value proposition for SO: in a world of poorly documented interfaces, it gives cogent, vetted examples of what works. And you still need to have expertise to know which solutions are crap vs gold.

In the 'old days' you had maybe one or two languages and the language was the thing (not vast libraries like Java or .NET .. oh excuse me, the guys who used to get teased for the size of the CMS VAX reference are barely a man page in comparison to modern libraries) -- and now we have a proliferation of languages. Any web dev these days has to know at least 5, maybe more. Any old-timer has had to learn dozens during their career.

So no, I'm not going to use my precious grey-storage to remember an arcane Perl invocation when I can find it easily on SO.

Perhaps I don't meet your definition of 'programmer', but I've seen many junior devs get trapped in SO 'solutions' and wonder how I can possibly glance through 30 solutions and categorize them: "crap, crap, doesn't know what they are talking about, cargo cult, ... ah, there it is.. that's what we should try."

Benefit of experience?


I have to agree. Not necessarily on SO in particular, but on having access to resources in general.

Maybe someone who's only ever done web stuffs in javascript and RoR and the one framework that is being used for the task at the company he's applying for, it's not necessary to check facts.

When you've done two dozen programming languages, everything including low level assembly hacking, embedded & kernel development, graphics programming, network services, traditional desktop software, mobile software, video games and front end web stuffs and you've used more libraries and frameworks than you can count, it is quite possible that you're more than competent enough as a programmer. Yet you might need a bit of a refresher on whatever tech you're being asked to use because chances are it's not the only thing you focused on last year, or maybe indeed you haven't used it in a year or three.

Of course it's not black and white; if a company needs a C expert right now, somebody who knows all the pitfalls and can point at security issues & UB, then maybe he should do well on a test for that, without a reference. Although even then I'd let him use a copy of the standard (or its draft).


I just wouldn't expect to be applying for something / trying to do a test task, where I didn't already have the knowledge.

Then again, I don't really use SO anyway - I know somepeople use it daily.


What does it mean "to have knowledge"? How much information qualifies you to be a knowledgeable? Plenty of us have a knowledge. Until, for one of your project you have to use Spring Boot and Hadoop at the same time. You know them well, you work with them already. But then you put both libs into one project and something throws an exception somewhere deep down in the unknown code. This is unexpected and does not come from your code. Good luck with searching for solution by yourself. Not that it's impossible, and you would learn something, sure, but troubleshooting takes a lot of time, you'd have to keep your current tasks aside and probability is quite high that someone had that problem before you.

IMHO, what makes you a good software engineer is not an encyclopedic facts in your head but general knowledge about principles, algorithms, patterns, etc. You don't have to remember all methods of all classes of standard library, for details you can check the web or use IDE. But if someone asks you about thread safe objects you have to know what does it mean. What may go wrong if you fail to make it so? Why properties must be immutable? And what the hell immutability is in the first place? All other things you should be able find in the online documentation / language reference.

What makes you a good software engineer is the ability to ask a meaningful questions. We have a powerful tools at our hands these days. Why wouldn't we use them for our benefit? You don't have to remember everything. It is there, within a hand's reach. Sometimes I have a feeling that asking Google the right question to solve your problem should be another skill tested during interviews.

What makes you a good software engineer is the ability to switch the tools and adapt quickly to new environment. Give him Internet, give him StackOverflow and ask him to solve a problem in new language, new framework. If he got the concept well, if he knows principles, that should be more than enough to jump in. Don't get me wrong here. I not going to underestimate guys with years of experience. I am not saying he will write flawless code. But he will solve the problem - that's a good start, isn't it?


I guess that I disagree with many people on HN (and they certainly like to express this with downvotes).

Fair enough.

Thank you for your verbose reply - I'll reflect on it.


Well, let's explain the downvotes from 1 POV. The "copy/paste" comment just willfully misinterprets how StackOverflow is used.

I think that we just don't believe you. Did you arrive at every job with perfect knowledge in the areas you'd be working on? Have you not had to learn anything on the job?

If that's the case, then you're vastly overqualified. I feel like I'm being utilized as an employee the best when my manager says: "Hey, there's something weird happening in area X" or "I want you to explore feature X" and I get to dive deep into something for a few days. Our strengths are our ability to learn.


I suspect you're right in many ways.

I mention it as a copy/paste mechanism as I am surrounded by people who use it for that very purpose.

I also did have the knowledge required for the job I applied for, and whilst I have learned a lot (off my own back) over the last 18+ years, very little of it is actually required, as demonstrated by those others who get given similar tasks that they can complete using Stack Overflow.

I'm definitely in the wrong place, and after a while I sense I lost a lot of perspective due to my peers / surroundings.

Personally, I never use downvotes for things I disagree with, but perhaps some people who are luckier to work in better scenarios believe that situations like mine don't exist - I don't wish anybody to walk in my shoes.

Thank you. I appreciate you taking the time to explain.


That sounds like a crappy job to be in, I'm sorry. I've worked around people who were... negative, and were really bad at explaining things to me, and also kind of hacky. It sucked and it drained my morale.

There are good places out there, _even_ if they're low-skilled environment. I think it's totally a cultural attitude, to be willing to take the time to learn, and to have a good attitude and effort about it. You can copy/paste from SO and still be using it as a learning tool.


I've used dozens of libraries professionally over the past year or so. Some of them I use every day, others I only need to touch once a week or once a month. Why would I be expected to memorise all of this when it takes only a few seconds to look it up?

I use Python, Clojure, ClojureScript, C++11, Javascript and bash regularly. I remember all of the standard library functions and third party libraries that I use a lot. But even after 15 years of using Python, I still have to look up, for example, the datetime module or what functions exist in the collections module. I still need to look up how to use the C++11 chrono library. I still haven't memorised the entire Clojure core functions (there are so many of them!) I still need to look up the DOM API for javascript (but then, I don't use JS that much).

I'm sure part of it is the context switch: if I were using one single language 100% of the time, perhaps I could memorise its core libraries. But why bother when it takes 3 seconds to look up and every professional job I've ever had its perfectly fine (expected even) to look up online documentation on the job.

So why not during an interview too? If your interview isn't testing me on what I'll be doing when I'm on the job for real, then whats the point? Testing my memory isn't a good reflection of my skill when working on real tasks.

Now, when I learned to program, I didn't have internet and worked completely from offline documentation (I learned Visual Basic, C++ and Python this way), so its not that I'm completely incapable of it, but it seems like a terrible use of my resources.


>Then again, I don't really use SO anyway - I know some people use it daily.

Out of curiosity, what do you do professionally? I'm impressed that you have so much domain knowledge memorized that you seldom ever need to look up anything.


StackOverflow often provides an answer to a specific trivial question faster then going through the docs. There is no point to memorize or read plenty of docs for trivial things such as 'sort array of objects by key', etc. I see no problem with this at all.


The OP's point was without internet access you can't find solutions made by others for problems you might not frequently run into in your daily job.

Example: you have a failing install of a package in a linux version you never used before where the solution would be to increase ulimit but you are applying for a frontend position so you don't even know what ulimit is, but by a quick search on google for the error message there is an article providing a 1 liner for this issue.


Basically, you just wrote nearly (exactly?) what I wanted to write. I'm good at algorithm problems. I LIKE algorithm problems, but having someone watching me on a whiteboard in a high pressure situation diminishes my abilities. At a minimum, it is a distraction. If it is an easy or familiar problem then it usually doesn't matter and we can discuss the solution and I might even want to talk about it. But if it is a tricky problem -- and a lot of tech companies intentionally give problems that are tricky or vague or superficially easy -- then I need to use my natural process for systematically and rigorously tackling tricky problems. I'll still probably finish it in a half hour and we can still talk about the solution and discuss improvements but let me work on it alone first.

One thing I've found from technical interviews is that, due to the pressure to speak and communicate and "show my thoughts", I'm more susceptible to falling prey to trick and vague questions that I would otherwise treat more carefully. This is consistent with scientific results on the way people respond the pressure situations[1]. They tend to fall back on what is familiar instead of trying approaches that may be better. Working alone, there is no fear of judgement if I need to try an unusual strategy[2] that may not work or if I need to backtrack. Let me dabble and doodle on my own sheet of paper and iterate rapidly over whatever comes to mind freely.

Furthermore, turning problem solving into a theatrical event unavoidably increases the cognitive load on an interviewee simply due to the fact that you have to make more decisions. You have the technical decisions like data structures and algorithms which are normal and expected, but on top of that, there are all these non-engineering decisions that you have to make because they want you to talk about how you're solving the problem in real time. Should you mention some fancy algorithm or approach that you are considering? You must choose your words carefully since anything you say can be used against you.

I'm not claiming that everyone works better alone or even that I don't like working with others. I just think that the interviewee should have the option of working on the problem alone for up to 30 minutes. As suggested by parent, we can discuss the solution or non-solution after I've become familiar and comfortable with the problem. One of the worst feelings is when I fail to solve a problem during an interview, only to have the solution come easily when I'm able to tackle the problem in a pressure-free environment. Expecting some kind of portal into someone's mind while they solve an unfamiliar and maybe tricky problem strikes me as overindulgent and voyeuristic. It is a flaw in the current technical interview process and the only way to mitigate its effects is to just get more comfortable interviewing.

[1] - https://www.psychologytoday.com/blog/choke/201106/flocking-t...

[2] - For tough problems, tend to use my own idiosyncratic variant of Z-Notation. I have a very high success rate solving algorithm puzzles when I use it but it is easy to forgot to use it under pressure of an interview.


> Furthermore, turning problem solving into a theatrical event

That's a very interesting perspective actually.

If this process prevails then only a certain personality types will be able to work at companies which conduct such interviews.

The danger is that when the biggest and most successful companies (who's founding members didn't go through the same process, on the contrary were most likely solitary coders) conduct such interviews, the smaller and actually very nice places to work tend to copy the same.

I am somewhat in favor of the likes of Google and Facebook who need to filter an ocean full of applicants following this process.

But I have seen smaller companies copying the same process assuming they will find the similar hires as G and Fb. The funniest/hilarious part is when the recruiters start saying "we have a very bar" with so much conviction. Yes, you are a startup, you have a "very high bar" and you need to hire people who will scour the internet and integrate the various bits they find because they need to take the product from A to B YESTERDAY !!!

Although having said that, I think I may be a bit different from what you described. I actually don't mind whiteboarding or coding with pen and paper and I noticed for me it is a lot easier to "talk my thoughts" while whiteboarding. On the contrary, I get seriously annoyed if I have to talk while actually writing code and if it is during a job interview, I just choke.

> Expecting some kind of portal into someone's mind while they solve an unfamiliar and maybe tricky problem strikes me as overindulgent and voyeuristic. It is a flaw in the current technical interview process and the only way to mitigate its effects is to just get more comfortable interviewing.

Yes. We are problem solvers after all. A way shall be found.


> actually-there-is-a-correct-answer situationals

Ugh. The worst step of interview process. Remember that guy who wrote a post about Google's hiring process, where they outsourced the asking of technical questions to untrained call-centre staff.

"Standard way to allocate memory in C?"

"malloc"

"Incorrect. Actually there's this cool library that does it for you with additional checks. Sorry, you have failed the interview"

"Wut?"

I've always found that those kind of interview processes put far too much stress on being correct to their "ideals" rather than be consistent with your own and your team's. I.e. I don't care if you name your column "domain_id", but for christ's sake name the other column "account_id".


> Remember that guy who wrote a post about Google's hiring process, where they outsourced the asking of technical questions to untrained call-centre staff.

Hey, I had this experience with Amazon when applying for a software related position based in Europe. For the entry screening I got a call from Indian phone number and a person with Indian accent was clearly going through a list of various quirks an features of a programming language. Wasn't so bad though... the second round with MBA product managers was much worse:)


FWIW, my experience of Google's hiring process wasn't like that at all. Sure, it was hard, and they did do whiteboarding problems in the interviews (all seven of them), which is in its own way terrible, but it was all done with people who really did know the stuff they were asking me about.

I didn't get the job, but it was an interesting experience. Certainly selects for a certain kind of engineer, and would not be my preference for how to hire people if I was setting up my own operation.


We tend to think in a one size fits all manner and in a team environment I find that counterproductive or even destructive.

Every team needs a couple of people who can solve problems on the spot. They're going to fix your production issues. You need a couple of people who want to know why all the time. Someone with a big picture view. Someone with good aesthetics. Someone schedule driven, and a process nerd.

But they don't have to be the same couple of people. Diversity is more than fine, it's essential to avoid echo chamber syndrome. Some of the worst projects I've worked on had little to no dissent in the ranks. The code was self consistent but entirely wrong (bad info architecture, slow, brittle, impossible to learn after the fact).

But everyone tries to hire people that think exactly like they do. If we weren't so bad at determining that we'd probably never make anything interesting at all.


This works, but make sure they spend the 2 hours in your office and not 12 hours at home solving a 2 hour problem..

The other problem is that when you pay them you generate all sorts of nightmaris legal liabilities.


> This works, but make sure they spend the 2 hours in your office and not 12 hours at home solving a 2 hour problem..

This goes against the core message of the article. The candidate is supposed to think about a problem on their own, and propose a well-reflected solution on Monday. That's pretty much the opposite of "sit down in this noisy corner in an unknown environment and give us something in two hours".

> The other problem is that when you pay them you generate all sorts of nightmaris legal liabilities.

If hiring a freelancer to do a little thing tends to generate all sorts of nightmarish legal liabilities, your company is either a government contractor or it has even bigger structural problems than hiring.


> If hiring a freelancer to do a little thing tends to generate all sorts of nightmarish legal liabilities, your company is either a government contractor or it has even bigger structural problems than hiring.

It might not be an issue for your company, but your candidate might have a non-compete (or other clause) preventing them from accepting compensation or even performing work for another company at all.

Things get trickier as soon as money is involved.


Any company trying to enforce such a non-compete against someone who just earned $200 would spend a lot more than that in legal fees. This feels like legal paranoia to me.


Furthermore, in German we have a saying: "Where there is no judge, there's no executioner". Maybe this attitude is more popular in Switzerland than in Germany, but the basic meaning is: don't sweat the legality of small exchanges that (a) don't affect anyone negatively and (b) aren't in some government agency's crosshair. That being said, your mileage in different legal systems may vary - in our civil law system it's probably more common for a judge to simply throw out such trifling matters and people are generally also way less inclined to start a lawsuit.


USA is kind of special regarding legality handling. An unhappy candidate my find legal arguments to complain. In Europe, states may be unhappy that work has been done without tax payment.

This payment looks like easy money for cheaters who would copy ready made solutions or ask friends for help. I'm not convinced, but this is indeed how things should be done in an ideal world.

I guess we could also ask candidates to pay. That is what is done in engineer schools in France. Not sure this is a good filtering method because of the Donner Kruger effect.


> In Europe, states may be unhappy that work has been done without tax payment.

Well, it's a good thing we have (now domestic only) banking secrecy in Switzerland, for exactly these reasons. You only get into trouble when your livelihood doesn't make sense from what you earn and own - as long as you pay what's generally considered a 'fair' amount of tax, no-one will blink an eye and come after your documentation. And even if for some reason the government finds out, all that will happen is a penalty tax ranging from 20 to 300% of the missed revenue, usually 100%. There is an explicit difference between tax fraud (document forgery, which could mean jail) and tax evasion (when you 'forgot' to declare something, which leads to above penalty.

Overall, the system is set up so that the government can't abuse its asymmetric power over individuals, instead keeping it as close as possible to an 'employee' of the people.


To me it feels like classic "A-ha! I discovered an edge case so the entire solution is invalid!". Just handle the edge case differently to how you handle everything else.


HR don't care they can afford the cost where as an individual employee is unlikely to be able.


Then the candidate says something like "Well, you know, I can't take any payment before I'm out of this job. Can't I just do it for "free", and payment come as a bonus on hiring?"


What if you don't get an offer or if you decline their offer?


It's a risk you take, or don't, it'll certainly depend on how compelling is the job.

But you are the one that placed yourself in a bad situation. They have a sane procedure that fully respects you, but isn't fine-tuned to your current problems. You can try negotiating their procedure too, but in their place (what I'm not, I'm in the "can't bill you" place right now), I'd refuse to.


I wouldn't do that. But then, I don't work for free.


There can be legal issues from the candidate's side too -- people on an H-1B visa in the US may not be able to accept payment for work outside of the company they've been hired by.


Actually its common for many if not all employers to ban outside work with out approval from your manager.


Which is bullshit and absolutely none of their business.


And also not very enforceable in some areas. I got legal advice on this from my own attorney some 20 years ago. I had created a piece of software on the side while working under a contract that said something similar. The software took off and I was worried about it. He advised me that worst case would be paying the company the money I earned while under that contract, but even that was a stretch.


In addition to being bullshit it is also a hint you should be moving on.


In the minority of cases where this is true, just handle it differently.


This works, but make sure they spend the 2 hours in your office and not 12 hours at home solving a 2 hour problem..

Yes. And to make sure they don't cheat on it, maybe don't let them use a computer or the internet. A whiteboard ought to be enough. And just to be really sure, make them explain what they're doing in real time as they do it.

That will do quite well, and offer a wonderful alternative to the standard, awful, whiteboard coding interview.


>This works, but make sure they spend the 2 hours in your office and not 12 hours at home solving a 2 hour problem..

Did you read the article?

That's what the article author is saying not to do, they prefer to see a well worked solution after you define a particular technology, showing they are adaptable and have thought it through properly.

Limiting them to your office, timeboxed, advantages people who spit out lines of code quickly, which is definitely not the right fit for a huge amount of coding positions, especially when you are emphasising logical choices and design decisions.


Please don't suggest someone hasn't read the article. It sounds hostile and doesn't really add anything to your comment.


It does add to the comment. It hopefully encourages people to actually read the article. People can not contribute to a discussion constructively if they do not read the subject of the discussion.


From the HN Guidelines:

> Please don't insinuate that someone hasn't read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that."


I know it is in the guidelines. I am saying that I disagree with that guideline as it does not accomplish the goal the guidelines are intended to accomplish.


2 hour deadline and forcing the task to be done at the office turns it to a different kind test. This becomes more like how do you perform under stress and how well can you propose things out of your head. These are valuable skills as well, but I think having the weekend to spend with the problem more accurately models the actual development work.


Pressure comes in many forms. Certainly, being able to handle pressure is a good thing. But how often is it needed and what kind of pressure is it? What kind of relief have you got?

As I understand it, the more common form of pressure would come from having to deliver things in a tight schedule, and perhaps in certain kinds of meetings and presentations. Not from being forced to work in a foreign environment with foreign computers & tools and unknown people hanging around, watching and JUDGING you. If a company puts an engineer in the latter position very often, it might be doing something rather wrong. Whereas in the former situation, employers can find relief in at least having colleagues they are familiar with and an environment they're used to & comfortable with.


Your point is valid, although technically it would be under pressure as opposed to stress. The difference might seem minor but it's rather significant, and it's a characteristic many employers would intentionally want to measure. I highly recommend the book Performing Under Pressure, by Weisinger & Pawliw-Fry.


> The other problem is that when you pay them you generate all sorts of nightmarish legal liabilities.

I was wondering about the tax and legal implications around paying someone for an interview challenge as well.

The overall idea seems fantastic, but paying or receiving money for an interview coding challenge is a little weird.


Its better than being asked to do it for free.

A company recently asked me to do a task they expected me to take ten hours, unpaid of course. I told them I thought it was kind of ridiculous.

"Well do what you can in two or three hours and if you really impress us with what you have done..."

The company really didn't impress me enough to bother.


Tax authorities usually don't care about a one-off small payment.


where I live you are allowed to gain up to 5k year for 'hobby' projects that you do on the side.


I was pretty happy to find this out recently. It was a non-tech person who brought it to my attention actually. It's definitely motivation to monetize some small projects I've had in mind for a while now.


We can treat that as a consultation gig which may lead to an offer. It is up to the candidate to declare this income to the tax authority.


Plus, what about candidate non-completes, doing a paid task for a competitor? (Most people aren't in California.)


What's the legal liability ? It's not illegal to do a small project for someone and get paid for it. If the interviewee is on a H1B or some visa like that then I can see some issues.


It varies a lot depending on where you are. In Australia, it's dead easy if the employee has a registered business number (ABN) to be able to contract as a 'sole trader', because they can just give you a tax invoice and declare the income separately when they do their tax return (it's just a separate section on the web form).

But probably only 10-20% of people will have set that up, and for people who don't have an ABN, you have to basically fill out all the forms you would to make them an employee (you need their tax file number, have to submit a form to the tax office, have to get their superannuation details so you can pay them the mandatory 9% additional retirement savings on top of what you pay them, etc.)...


Absolutely agree with the sentiment here. I have used this technique for a couple of years now, except that I kept the task to 20 minutes (and didn't pay people, since it's a short amount of time).

To save dragging people into an office for that time period, I set up a custom git server that marks the timestamps of their pulls and pushes. The instructions for the task are contained in the README of the repo, so as soon as they pull it they've started.

https://github.com/Prismatik/codescreen

It's critical to me that the limit is adhered to so that over time I can compare candidate responses on a level playing field.


Jesus, your solution is a nightmarish timed examination and as far as I can tell, the opposite sentiment expressed in the article.


20 minutes is way too little for a meaningful programming task that's worth presenting to the other devs (and that's the really vital part of this method in my opinion).

I prefer to see what people write in their normal way of working when they're not under extreme time pressure.


> I spent 20 minutes reading the prompt, looking at resources and making an outline of my code. Then I step away for the afternoon and don't think about it. Let the subconscious internalize the problem for a while and when I come back, I've got a much better mindset to approach it.

That's definitely a thing, and I believe it applies to some extent to everyone.

But here's the other side of the coin - the ability to operate efficiently when the world is burning. I've been in the industry for a while, and this sort of thing definitely happens. But it's very hard to simulate in an interview. You don't know who the adrenalin junkies are on your team until PagerDuty lights up glowing red across the board.

The only indication I've seen is the correlation with a preference for risk-taking activities, such as sky diving, etc. But it's quite circumstantial. And there's definitely also the true reverse of it: some of the car racers and bull fighters out there are not good team players in times of peace.

Well, if at least a few key players can handle pressure gracefully, I guess you'll be fine, since it's a skill rarely used. You don't need everyone to be able to switch to catastrophe mode at a moment's notice.


There's a big difference acting on instinct and heuristics, versus actually thinking witch is very slow.


I'm a subconscious problem solver also, I do it on purpose alot of the time, I can solve some things immediately but most of the time I get it later in the day when I go back to it or let the answer come to me.


Yes, me too. I often find far better solutions to problems after backing off and deliberately not thinking about the issue.


i do love this idea but not sure if doing it remotely is a good idea. We do this in house because I am not sure how to be certain the candidate did not get help. I mean i believe people are generally honest but there will be cases when they are not, and I'd rather not find out 3 months after hiring them.

Also there are many indicators along the way, i.e. which parts the candidate got stuck at, did it take them 1 or 3 hrs to get to the same point (i.e. basic wiring of DB calls). I think all that would be lost if they did it on their own time.


> I am not sure how to be certain the candidate did not get help

Pressing them hard for information on Monday will expose this.


Do employees at the place you work at never ask their colleagues for help?


This isn't a viable approach for much of the hiring pool, as a majority of working engineers have contracts which explicitly forbid work for contract for other employers.

As such, if you hire any of those, you're hiring somebody with a proven willingness to ignore their contract, which is a strong anti-pattern.


IANAL, but that's not what this is.

"Work for contract for other employers" could literally mean I can't help my neighbor mow his lawn in exchange for a beer. In this case what it really means is a non-trivial amount of work done for meaningful profit.

If I fix a friends computer in exchange for a nice meal, I am using my relevant skills to perform work, but I'm not actually violating a contract forbidding outside work in a way that could ever possibly result in damages or firing for cause.

For a potential employee to be so rule-stricken as to concern themselves with performing a coding interview, that would be the potential red flag for me. So this could actually be another positive outcome of this approach.


I fear deportation, being unable to ever enter - never mind work in this country again.

You can call that a red flag, but it is the reality I live in. Maybe most of your candidates don't have that problem, but I would definitely be eliminated.


I mean, you could always just ask that the payment be donated to a charity instead, right?

But since almost everything we do is in some way technically illegal, it is important for everyone to be able to function under that premise.

I mean, just to cut the check technically the employer would need a W-9, a work for hire contract, and who knows what else. Let's call the lawyer and spend $5k deciding whether we can do this, and then decide 6 months later it's too risky.... Or, we do it, get better hires, execute on our plans better, meet revenue targets, and succeed in the market, by not following all the rules, just the rules that matter.

Knowing the rules to follow and the rules to forget is the hardest part, but a crucial skill in life and business. The more you push the envelope, the faster you can run, and the more likely you are to combust. See Zenefits...


You are an edge case. There should be a workaround for you, but just because something doesn't work for you doesn't mean it doesn't work in general.


Ever hear of tortuous interference? When one party induces another to break their contract with a third party. For as realistic as your example is (no employer is going to sue because you mowed your neighbors lawn), plenty of employers can, have and will sue when they realize you have hired away one of "their" people, regardless of your rights and theirs.

Learning of your little deal to pay them for software development work will be the stick they use to hit you with that interference suit.


You perfectly verbalized what was bothering me about this reply


Certainly you don't want to hire people that are rule stricken. At the same time though, if someone signs a legal contract saying they won't do something and does it anyway, isn't that a red flag?


IANAL but I took business law 101, which taught me that courts almost universally lean strongly towards the employee when considering restraint clauses.

Moreover, if you have a current employer that is sufficiently disrespectful of their employees that they'd even consider enforcing an anti-moonlighting clause in this circumstance, with the preposterous assertion that it qualifies as employment - run away now.

The law is not a programming language.


If you're involved in the court system while looking for employment, you've already lost.


It is a requirement of my country's regulators that I ask for my employer's approval before I do any other work for compensation.


Damn, what country is that? Sounds like a very controlled, restrictive environment.


I worked at a top investment bank in NY which had/has that policy


Will they? I've seen contracts saying you can't work for competitors, but I'm not sure I've ever seen one that forbids all other employment.

In California, it's illegal to prevent employees from moonlighting: https://californiaemploymentlaw.foxrothschild.com/2015/03/ar...


Or a $200 donation to their fave charity


This is a great idea, no legal issues. The important thing is that since the candidate loses 2 hours of his time working on the problem, the employer has to lose something equivalent of that too.

I would be much more willing to do the project if I knew the employer would lose $200. Among other things, this shows that I'm valued.


That's a great idea! Not only does it just 'feel right', it also prevents some of the legal mess people were seeing with effectively moonlighting for the hiring company.


That is a really good idea.


It's possible to not pay them. Most candidates will do this for free, as it's far better than traditional ineffective hiring practices.

The thing is, there are always reasons why this method isn't viable. It's not till you try it that you notice huge gains.


That only makes sense if you're talking about people who are unemployed. As someone who is already employed full-time, I simply don't have the time to solve your problems for free just to see if you'd like to invite me over for an interview.

I avoid these problems like the plague and very much prefer to whiteboard. If you pay me consultant rates, I can more easily justify the effort.


Even when I was between jobs a couple months ago I neglected to complete a take-home interview exercise, because the opportunity cost still exists. The advantage to interviewing in person is that the conversation goes both ways. I was interviewing daily, so I still valued my spare time for personal activities.

Edit: I most likely would have completed it if I were offered some type of compensation, even below market. They claimed it was an exercise and they wouldn't use it, but this was a startup with about 10 employees, and the deliverable was definitely something they could have used. That alone rubbed me the wrong way.


>> much prefer to whiteboard

Who would have thought? Maybe - just maybe - not everyone has the same strengths and weaknesses. The real answer here is obvious: present the interviewee with options. Forcing all potential hires to whiteboard is a terrible idea. Forcing all potential hires to do a take-home project is a terrible idea. It's extremely short-sighted and a little pompous to assume that any single interview format one chooses is going to magically sort everyone into neat little buckets of "good" and "bad".

If you force the whiteboarding approach, you are only going to wind up hiring social butterflies who have absolutely no nerves standing up in front of complete strangers and having all the answers on the spot. If you force the take-home project approach, you're turning off a lot of people who can nail a first impression presenting themselves and their skillset in person.

People are different. Applying the same interview type to everyone is going to target a specific set of strengths, and a specific set of weaknesses. And frankly, no business should be composed entirely of one type of person. Some diversity does wonders when assessing the overall strength of a team.


> "If you force the take-home project approach, you're turning off a lot of people who can nail a first impression presenting themselves and their skillset in person."

Whiteboarding is a poor reflection of what people do day-to-day, and can be misleading if a candidate practices interview technique over coding experience. In other words, just because someone can regurgitate an algorithm on command, doesn't mean they'll be a good developer.

If all you want to optimise for is finding people who make a good first impression, by all means carry on with whiteboarding. However, if you want to find good developers then there is a one-size-fits-all solution, and that's to see how they develop code in the real world, which is exactly what the 'weekend project + appraisal' approach is a great fit for.


> The real answer here is obvious: present the interviewee with options.

If you're a company that does any kind of business with the federal government, that's simply not an option. You have to be super-consistent in terms of your interviewing and recruiting process across all candidates for a job req and can't do things for some candidates without doing it for all of them.


> I simply don't have the time to solve your problems for free just to see if you'd like to invite me over for an interview.

I think that would be misusing this approach.

We build recruitment software.

The recruitment process is a funnel. At each stage, the employer tries to weed out the bad candidates. Simple maths says they try and use cheaper tests early on in the process when there are more candidates.

For example, first stage is often eyeballing the resume and a quick search on GitHub/SO. 5 minutes, costs say $10, weeds out say 65% of candidates.

At the next stage, we use a more expensive test, since we have fewer candidates to filter.

This particular test sounds like it would sit after an initial phone screen and before a team interview in terms of cost.

Personally I would only use it after a first interview, or tell the candidate that you will decide immediately after completion of the first interview whether it was successful, and if so send them away to do this project afterwards,i.e. reduce double handling


I hear this argument quite often but my reaction tends to be...

...as someone that's fully employed I'm probably only looking at jobs I'm really interested in. I don't mind doing some free work for stuff I'm interested in. I also enjoy doing small unpaid side projects or programming puzzles in my free time...basically the same concept.

That being said, my guess is (ignoring legal implications) paying 200$ or whatever for it is actually more beneficial to the company than expecting you to do it for free so it seems like a nobrainer (once again assuming you can handle legal). Reciprocity and all.


Even if I'm interested in it, I'm not interested in doing free work for a for-profit company.


Do you also fundamentally oppose to contributing to FLOSS software because that's most certainly being used by for-profit companies. Or is that ok because others that don't profit from it can also use said software?

I do something that I enjoy and throw it away afterwards or whatever I tend to do with little programming puzzles and project Euler stuff or I do the same thing and a company profits off it. The profiting doesn't really hurt me so I don't mind it. I'd also argue that if the intention of a company is to profit off my interview code I'm hopefully filtering them out (as they tend to be pretty boring)...having people write code for you by pretending to offer a job is also all kinds of horrible strategically so I don't see the potential for a mass exploit.

I can understand your point of view but I hear the dreaded "but my time is too valuable" more often than "it's unfair that they profit of it". My argument was mostly that...if you want the job, your time probably isn't too valuable to try to get it :)


> As someone who is already employed full-time, I simply don't have the time to solve your problems for free just to see if you'd like to invite me over for an interview.

That means you don't care enough about changing jobs. There are people who will happily do that "interview" for free even when being employed full-time. The only relevant question for the company is whether there are enough qualified candidates willing to do the test for free.


So your filtering for the "desperate" job seekers


Job seekers who don't have their weekends full.


Most job seekers in our industry have jobs and lives outside of work


I re-read this later and wondering if that's actually true. Our industry is very "you don't have anything to do this weekend, right? right!" Anyways, it's true for me.


That couldn't be more wrong if you tried. Not to mention, you're assuming that people need the job. It's the other way around. The company needs the engineers. Much more than the engineers need the jobs.


I don't understand the paid part at all. If I'm working full time, $200 isn't going to swing me one way or the other on deciding if I want to do your homework assignment. And if you come up with a ten-hour problem, then we're talking more of a chunk of cash, but you're also asking me to burn a lot of free time.

I'd much rather travel and meet you in person and do stuff there, but if you don't have the budget for it, we could figure other stuff out (shared docs, skype, whatever, I've done a few of these on both ends with some luck).

"Since the candidate is getting paid" seems like far weaker motivation than "since the candidate wants the job you're offering," which is what's going to determine my level of effort.

The "not a real problem" thing is key, though. I've tried testing out a few different "this is one of our real problem" things but there's almost always more hidden business rules or assumptions in there than you think.


>I don't understand the paid part at all.

There's a concept around a lot of preliminary engagements between two entities called "skin in the game." The idea is that when something is completely free the other party will take advantage of it without any real serious intent to follow through on anything.

Mostly this seems a reaction to the homework-type assignments where candidates are expected to spend a lot of time on some interview assignment with very little real cost to the company--which raises the possibility that it's effectively a cattle call.


I get this in theory, but the numbers sound too low for that to make a difference still.

I could run 50 candidates through a $200 problem for 10K. That's still a pretty large mismatch between "amount of work done by the candidate" and "amount of work done by the interviewer," and is cost-of-doing-business money for recruiting for a lot of companies currently.

Compare that to the cost of me flying people out (which is still done in this approach), or even the cost of spending an hour of mine or someone good on my team's time on the phone with them.

I guess it's just down to the difference between trying to hire fresh-out-of-college (or still in) free-time-to-spare junior devs and experienced people. That's actually a topic I should write a blog about somewhere myself, one day - I pushed pretty hard at my current company for moving towards a different process for industry candidates, and are extremely pleased with some of the people it's helped us hire.


I get your point. But, in practice, there's a big difference in many situations between free and even a nominal sum.

I do agree that if people are flying around, that represents a fairly significant investment in any case. But in Silicon Valley, that often isn't true.

It's hopefully a case of coming up with processes that don't encourage people to waste others' time. But that also assumes some reasonable balance of power in the process.


Yeah, sorting out serious people from those wasting time is a hard one. Though I'm generally pretty sympathetic to employees/candidates on that one - am I wasting your time if I'm pretty sure I don't want to leave my current place, but want to keep a hand in and know what the market looks like? I would say know, but I know some people who would disagree, and have decent reasons for it.

Even after that, the "experience candidate phone call to see if they're worth flying in" part is definitely the one where I have the hardest time, being not in the Bay Area. I was lucky to find some great local people, where the cost was lower, because that's an expensive one to have to get good at fast. You lose a lot of nuance over the phone, though some other people in the company have been having very good luck with homework in this case - but as a candidate-choice option, where they could still go the traditional route if they wanted.


> Yeah, sorting out serious people from those wasting time is a hard one.

It's easier than it seems.

Just put a HackerRank test with a single question. "Print the numbers from 1 to 10 [inclusive, one number per line]".

20-30% of candidates won't attempt the test. 20-30% will be unable to give a decent solution.


The big expense for the company isn't the $200, it's the time. If the hiring manager is doing their job, and you have a few people in the physical interview (plus follow up, etc.), then that's the dominating cost.

If you're running 50 people through this, you're doing it wrong (unless you have 20+ positions).


>> but you're also asking me to burn a lot of free time.

I don't understand why you think improving your employment situation wouldn't be hard work.


I don't agree that it's a string anti-pattern whatsoever. It's a clear signal that the person can interpret the meaning of a rule and adhere to it in a way that makes sense for the person who made the rule as well as the contractor.


I think person who is either very well compensated for that or with slave mentality would sign such a contract. Either way I don't think those people are a target group of this post.


Non-competes are worthless clauses that should never have existed in the first place. I am perfectly ok with someone ignoring that blatantly anti-employee clause.


On top of this, many candidates have visa-related restrictions that would prevent them from doing this legally.


>> a majority of working engineers have contracts which explicitly forbid work for contract for other employers.

Well that was very, very dumb of them. Maybe we engineers ain't so smart after all.


Custom and practice - you need to read your company handbook /contract some time.


Work agreements are really easy to modify before you sign them.

And I don't have an employee handbook. I own my own business.


Only if your Beyoncé and can demand a bucket of kittens in you dressing room before a show - the rest of us less so?


If you can't solve a new problem "on the spot", or at least quickly, then one is at least not suited for SRE work.


Love the idea of paying people. That's great. However, consider the perspective of someone who already has a job in the industry. If every company did this, I wouldn't be able to interview at more than one per week. Not only that, but the Friday and Monday onsites mean I miss work on TWO separate days. This would be a non-starter for me if I had any decent pool of interest from other companies, although it's great for people trying to break into the industry.

Another problem, depending on your philosophy. Specifying the exact technologies someone has to use means that you've closed yourself off to generalists, unless they have time to learn a whole new stack that weekend. Not that it's rocket science to do so, but that turns a two hour project for me in say Angular into an eight hour project in your favored stack. I can always learn your stack on the job, as any good engineer can.


Yeah, it makes sense to me as a sort of moneyball approach, to cheaply hire skilled candidates who are not as good at the standard evaluation process, are likely to have few competing offers, and who are willing to do extra work on nights/weekends.


Agreed with this. I wouldn't take a 4-day weekend to do interviews and a homework assignment unless I was desperate or already out of a job.


From the perspective of someone who already has a job in the industry, I'm really not going to want to do more than one interview per week anyway.

This format also doesn't appear to want full days: it should be an hour or two Friday afternoon, plus a half of Monday. Two part-days isn't really that worse than a full day off (Mileage may obviously vary; particularly the mileage to get to the interview). The initial interview could also very easily be done remotely.

Even if it is a bit higher-impact, I would still prefer it simply for the greater quality of experience, and that's before throwing in the payment.


I feel like it's in the best interest of the employer to leave the choice of tools fairly open and see what the potential employee elects to use. For the example given in the article (a basic web application), I'd be very interested in whether the interviewee decides to go with a heavy stack, does it completely in vanilla JS, or something in-between.


I wonder about this. I'm pretty new to js frameworks and I see the advantage when things need to scale or get very complex, and I get that companies use these things. But for a simple assignment like this, is it better to use vanilla js and maybe handlebars, or "show off" that you can implement all the routes and whatnot in Angular, even though that's overkill? Or maybe the interviewee's justification of their choice teaches you that they understand the trade offs.


This is literally the only way I've gotten job offers. I'm a very competent programmer, but when I'm asked to solve some silly puzzle with someone staring over my shoulder, I freeze under pressure and can not think at all.

Luckily not everyone thinks solving leetcode problems is a good way to hire.


Exactly my case. My startup is ending and I'm looking for work. Wonder if there's a way to look up companies that hire by giving assignments.


Your next startup could be a job board that specializes in this. I'd subscribe


This article is written by (and marketing for) someone who created that job board.


Be persuasive, ask for an assignment. Everything is negotiable.


If the company is large enough there may be interview process reviews on glass door.


I've been in situations together with clients deploying a product, them presenting it for their managers, orpreparing for a big demonstration, and something goes wrong due to some unforeseen environmental factor you couldn't have imagined or predicted, and breaks in a way you don't expect, despite the weeks of testing you've done.

You've got your main client staring over your shoulder, arms crossed saying fix it, and every two minutes asking "is it fixed yet?"

You damn well better be able to code under pressure... And like a freaking ninja


I can present to audiences pretty easily, I actually enjoy it. I also don't mind at all having a client watch what I'm doing. And yet, while I have no trouble doing interviews - having worked as a freelancer that would have been pretty bad - the situations are not comparable even for me. I loathe interviews. Fortunately a lot of clients, and I've had several major companies, had people hiring me without any of the usual stuff based on a gut feeling, I had to answer silly irrelevant technical questions only twice. And once I even failed "explain a right join" - something surely everybody would be expected to know, and of course I do, but at that moment I blanked. I go the job anyway but it goes to show that even to me interviews are a very different and greater kind of stress than anything normal work can throw at me. Even if it looks the same, it isn't, not at all! The brain knows context.

An interview situation is much more adversarial: They are looking out for reasons not to hire you, you are competing with others for the job and only one can get it! No such consideration on the job, unless the work environment is completely and ridiculously broken. When are you ever in a work situation where several people work on competing solutions and everybody else but the person who made the winning one is fired? At work you are working together, and to solve a problem, not to week you out of the pool.


Most tech jobs aren't like that. If the job you're hiring for is one of the few that involves high-pressure coding in front of strangers, then by all means test for that.


If you are coding 'in front of customers on the spot' - you at the wrong company :)

Coding under pressure is quite different from coding on a white-board in an interview, often on an arcane CS problem.


> If you are coding 'in front of customers on the spot' - you at the wrong company :)

Or a procrastinator ;)

"Hi, one coffee please."

"Sure, just one minute. Ok, first I'll pip install stripe…"


>> despite the weeks of testing you've done

I have never in 12 professional years been given weeks to test anything. On only a couple of occasions have I been given 3-4 days to extensively test a project with multiple months of development time behind it.

Clearly you're talking about the project where the developer(s) told you very explicitly that it could not be done in less than 3 months. So you just assign the Jira ticket a 3 week deadline anyway. And then flip out when a half-assed project fails to ship. Welcome to the world of software development.


This is not a good way to hire people. I can do all those things, but I'm not normal. Some engineers do a lot better under stress with a system that they're very familiar with. Others do better under stress when they're working with people they know well. And still others are great engineers but aren't the person to be in front of the client under stress.

Congratulations that you can do all these things -- but unless 100% of the engineers in your company need to be like you, you might not want to use your list as hiring criteria.


"You damn well better be able to code under pressure... And like a freaking ninja"

Being able to 'code under the pressure of a deadline' is very different from 'solving a tricky problem with someone staring right at you'


Being able to code under pressure when the world is burning is also different to working under pressure of a deadline. Or rather, being on the last day of a 30 day deadline isn't the same kind of pressure as being told you have an hour to complete a complex task.

Not everyone needs to be able to work well when the world is burning. Its not part of many jobs. However, if it is part of your job (eg your the guy who has to fix stuff at 4am) then I see nothing wrong with asking candidates to do timed work in an interview. I just think most jobs don't or shouldn't require this.


This isn't the same kind of pressure, though. In this case you are familiar with the people, problem, code, etc.


In consulting you get in similar situations. In those cases it is not a "be able to code under pressure..." thing. Rather, a "know your code well" one. I have also been in a few demos where everything fails. As the manager sitting next to the client with your arms crossed, you have to know how to defuse the situation without having someone sweating their soul on a keyboard. Been on both sides.


This is why the demo should just be a pre-recorded video that you can narrate live. Seriously.


>clients

The type of situation you're describing is is why I avoid working for any sort of shop or agency.


if that's common then it's a reasonable criteria for the position, sure. That hardly ever comes up in my work environments, so I'd be happy with an engineer who is awesome 99.999% of the time when they aren't in this position.


If you can't code under pressure, you aren't a very competent programmer. Real world projects come with high pressure scenarios for software engineers. Code monkeys may be able to get away with low pressure, minimally impactful projects.


Upvoting you to help prevent your comment from reaching oblivion - I don't agree with you, but I think what you said is a common belief.

The thing is, it's a vastly different kind of pressure. When I am coding as part of a job - by myself or with others - there is no inner monologue in the corner of my brain doing something like this:

"Could do that or... no damn, that's O(n^2), damn I'm taking too long that one looks bored, if I instead put those keys in a hash... no, I'm sure there's a better way, running out of time though, is he looking at his phone now? Okay, what was the profile of merge sort again? ... oh, shit! She felt like she had to offer me a guiding hint, I'm doing it wrong!"

And it never stops going. It picks apart every choice I make, and by five minutes into the interview I come across as a sub-articulate mess.

In no other high-pressure situation does this happen to me. Hell, this past summer I actually spoke at a conference and I didn't have the kind of trouble I do in an interview.

Whiteboarding, paired coding, participating in architecture design - these things come easily. In the context of an interview, though, it goes out the window.


Have you done pair programming before? Especially as you become more senior, the pressure to not look like an idiot in front of junior devs adds up fast. You have to learn, quickly, how to get over that kind of inner monologue, and just get to work.

That kind of scenario happens to me daily.


I've never felt that pressure. Keep an air of confidence. Let them assume you know way more than they do about almost anything else. They'll feel like they're really shining in this moment, and impressing you, and growing in their role. Win win.


I don't understand the desire to make it seem like you know everything.

If you learn something from someone, is it that much trouble to give them credit for expanding your horizon?

If this mindset is recursive down your organization's hierarchy, doesn't it create a scenario where a legitimate critique never gets heard because people don't question the decision of a higher ranked employee because there is the impression that they know more than you?

Seems like you and your employer are worse off, but your ego remains intact.


I do pair fairly regularly -- for whatever reason, the concerns don't manifest the same way as in interviews. I can't think of anything in my experience (professional or personal) that's comparable.


Thanks for clearing that up, it's exactly the same for me. I find technical mock interviews help.


It's not like software development is the same as being a fighter pilot.

Code now or die isn't usually something most people face in their day job.


The Swordfish art of tech interviewing


According to Vim aficionados, the experiences are similar.


I can't code with people staring at me, talking to me, asking me questions. If you can more power to you, but if that reflects the nature of the job then they can take that one and shove it. It simply doesn't reflect reality.


How about when you're alone, it's 3am, you haven't slept much in the past three days, and you have a 10am deadline to meet?


You almost certainly tell the project manager that they have dramatically failed at their job. You go to bed and come back when you aren't going to make stupid errors.


That is a thing that happens, yes.

If that happens consistently for months at a time, then as said the PM is severely fucking up. And if that wasn't communicated as an expectation before compensation was negotiated, then the company is probably fucking up.

As the quip goes "If a race doesn't have an end, then it's a death march."


A) Been there, done that, told my boss to fuck off. I've been on week long coding binges where I maybe got a cumulative 10 hours of sleep. This is a fundamentally different kind of pressure on yourself than having someone watching over your should or giving you a stare down on Skype while badgering you the whole time.

B) If you're working a job like this you're working a shitty job and ought to quit. Unless you're maybe hacking the Gibson to stop some oil tankers from going belly up there is hardly a reason in our industry for this kind of shit.


Your job sounds terrible.


I can code under pressure.

But I will not work in a company where that's the norm.

I value my health more than the potential money from that stressful job.


How do these opinions even survive? Yikes


You're getting hammered for this, but for some jobs you're absolutely right. If you can't handle being put on the spot for a simple interview question, how are you going to go when a production line is down and it's costing your client $100k/hr, and the foreman's standing looking over your shoulder wanting to know when it'll be working?

Not all jobs give you all the time in the world on a test server without interruptions.


All these "yeah but imagine this horror scenario" (sorry to pick your example) descriptions sound like badly managed situations. Even in total failure situations like that...the foreman shouldn't be looking over your shoulder. The first step is to take a deep breath, assess the overall situation and get the programmers into their most comfortable mode to fix it. That can very well mean letting them think for 1h before starting doing anything. You can ignore that 100k price tag because that just leads to "just do something" thinking. It's very likely manically hacking away will actually end up making the situation worse and costing more.

Also...these situations can be trained for to certain degrees. There's well established precedents in crisis management. If you write code for a customer with a 100k/h downtime risk you should have done a very thorough risk analysis and have mitigation plans for most scenarios. The unknown unknowns that can happen are obviously the hard cases but you should be able to justify that these are going to be expensive no matter what. But even for these cases bricolage is trainable to a certain degree.

If you know there's 100k/h costs of failure you should also very specifically mention the fact that you are looking for these stress coping skills in the job description (and as such the interviewee should be prepared).


In my experience, high-pressure "fix it now" scenarios are about leaning on what you know about the system, interpreting metrics and log messages, understanding how it's going wrong, and deploying a (usually very simple) fix. Sometimes just blindly hitting the "rollback" button as a first step fixes it, too.

Performing well in this kind of scenario is much less about algorithms and data structures, and much more about systems thinking and the level of your understanding of your system.


Both of you have points, but I absolutely agree with this. Outside of "Healthcare.gov goes live tomorrow and turns out it's completely broken" scenarios I've never seen something that would require a ground-up design and build of a system under emergency production time constraints.

System knowledge and knowing which one bolt to replace or turn is almost always more effective.

PS: Not to mention, if we're getting into algorithmic complexity levels of engineering on a monkey patch to a production system, then so many alarm bells are already ringing.


See my longer comment above. Last paragraph:

An interview situation is much more adversarial: They are looking out for reasons not to hire you, you are competing with others for the job and only one can get it! No such consideration on the job, unless the work environment is completely and ridiculously broken. When are you ever in a work situation where several people work on competing solutions and everybody else but the person who made the winning one is fired? At work you are working together, and to solve a problem, not to week you out of the pool.


If you've done your job properly in the first place you have sufficient testing and redundancy in place that the situation you're referring to is pretty much impossible.

If there's ever going to be a situation where something costs $100k/hour if it's offline then you have 5 test servers and 2 or 3 load-balanced production servers so a failure doesn't take you offline, and you've written procedures and processes to handle critical problems long before they happen.

You definitely don't try to hotfix things while the foreman complains. That will make things worse.


That's a terrible analogy. In a production crisis, you have full knowledge of and access to the system being diagnosed, have all your standard tools and information sources available to you, and are able to enlist your co-workers to aid you. Been there, done that and it's's nothing like a interview whiteboarding session.


    For example, I may say please use functional reactive
    principals in JS to solve this problem, but the decision
    of using Kefir, Bacon, or RX is left up to them.
These days I'm utterly unable to tell whether someone talking about JS is being serious or if it's parody.


No, this is real...however, using reactive principals in JS is a bit esoteric and definitely not something that needs to be covered in an interview (IMHO). At my company when it comes to JS interviews we are more interested in proper separation of concerns, understanding scope, using proper data structures...you know, the basics.


This is a great way of hiring, but unfortunately it doesn't fit in all scenarios.

First, this model alone doesn't scale. Big companies interview hundreds of candidates every week. It's not possible to design and evaluate a weekend project for each one of them. This doesn't mean that they can't have projects, but early filters are needed. That leads to the classic phone and F2F interviews to rule out those who can't code. If you freeze in one of those because of the pressure, you're out.

Second, for simple developer positions, such weekend projects might be very good. But if you are trying to hire a senior developer who is going to solve problems at big scale, then the ability to create a simple movie catalog doesn't tell you anything. And there are no big scale weekend projects. You have to ask about algorithms, data structures and other CS topics. These things are important!

Bottom line, this is a nice hiring technique suitable for small companies at a small scale. But it might not be that useful for big companies.


Sorry but this is simply not true. After a while we decided to do exactly the thing that the article describes as our preliminary interviewing process at Amazon. We also had extensive onsite interviews, however we did not ask candidates to code at the whiteboard. Btw, when you work for such companies it is expected that you spend 10% of your time on hiring. Getting hiring right is one of the most important things that you need to do at the scale of Amazon, Google and all of these large companies. Having a single bad hire can bring down the performance of an entire team.


I'm curious if you have any statistics about the different interview styles/processes. Obviously it's hard to figure out the false negatives (good engineers you didn't hire) but how about false positives? Also, is homework assignment now a part of the standard interview process at Amazon or is this just your team/department?


Sorry but no stats. Amazon has different hiring procedures for different teams. This was for a particular team in one branch.


> This was for a particular team in one branch.

Which is a bummer. When I interviewed at Amazon a few years ago, it was a day of coding on the whiteboard. Turned out I didn't have the Java experience that group wanted, and I'm allergic to pager duty so that was a non-starter for me.


You are right about the scalability of the process, but not about the rest. It seems possible to design weekend projects that would be way better for evaluating senior engineers than asking them to do CS brain-teasers on whiteboards. CS topics are important, but the typical interview is not great at identifying who is good at using CS to solve real production problems. It seems like the typical interview doesn't have many false positives but a lot of false negatives.

The common approach for big companies is to declare that false positives are way worse than false negatives, but if the problem is that you can't find enough developers to fill your roles, it may be better to have false positives coupled with a trial period or something similar.


> It's not possible to design and evaluate a weekend project for each one of them

Can't you just design and evaluate one project that they can all do?


Even if you design only one project. You still have to evaluate them, have a discussion, etc. That takes time. It doesn't scale to hundreds of interviews per week.


It's absolutely doable to evaluate one hundred take home tests in a week. It is a full time job though.


That will only work briefly because of glassdoor interview posts. However that might only mean tweaking it slightly every week or two.


Glassdoor wouldn't really help someone cheat in this situation. It's not a quick answer, it's a code discussed with the interviewer after finishing it. It would be fairly easy to tell whether someone had copied their code.


I wish more companies would give example problems to solve in your own time. I did one for a fairly large Australian company, and they rejected it. By the time they got round to letting me know (very long turn around time) I had already found other employment. But still, their reasons for rejecting my solution - the output was not formatted to their liking, and they didn't seem to know what an anonymous function was - was hugely beneficial to me. I would not have been a good fit at all.


I don't mind homework, but companies need to start specifying their salary range up front. I've been having a lot of time wasted lately by getting through Tech Interview #1 or Homework #1, only to find out their max salary is laughable for San Francisco.

"You should just find that out upfront" I hear you saying. Well that is not always possible. Recruiters either don't know the range, or lie about not knowing, or you have to submit a big packet of info (cover letter, history, resume, homework) before you can even talk to a human about salary.


Recruiter here with a history of working as a developer

    > but companies need to start specifying their salary
    > range up front
The costs outweigh the benefits for employers (encouraging existing employees to ask for more money, losing candidates who will accept significantly less but wouldn't apply at a lower range, etc) -- so don't hold your breath.

    > I've been having a lot of time wasted lately by
    > getting through Tech Interview #1 or Homework #1, only
    > to find out their max salary is laughable for San
    > Francisco.
State it upfront when you submit your application, rather than "find that out":

"I am interested in roles paying above my current salary of $xxx,xxx only, and am looking for an improvement on that to reflect my increased experience and the risk of moving roles"

    > Recruiters either don't know the range
Yeah they do.

    > or lie about not knowing
"Could you tell me about offers you've previously made for this client?" They may also be lying about actually being authorized to recruit for the role(!!!) so consider asking them for a copy of the technical specification. If it's a larger company, ask what the salary banding and official job title for the range is, then head to Glassdoor.

    > you have to submit a big packet of info (cover letter,
    > history, resume, homework) before you can even talk to
    > a human about salary
I had a hunch that roles like this weren't worth applying for when I was a developer. As a recruiter, this has been almost completely confirmed.


> "I am interested in roles paying above my current salary of $xxx,xxx only, and am looking for an improvement on that to reflect my increased experience and the risk of moving roles"

Are you really advocating that people disclose their salary to a recruiter, voluntarily?

What terrible advice. "How to immediately throw away your best bargaining chip 101".


    > Are you really advocating that people disclose their
    > salary to a recruiter
Yes

    > What terrible advice
No

    > "How to immediately throw away your best bargaining
    > chip 101"
If they're an internal recruiter, they have a banding you can be paid in, and they don't really care where you fall in it. Decisions above that salary banding will need to be referred to the person whose budget you are.

If they're an external recruiter, they have no control at all and are incentivized to get you paid as much as they can, as they're paid on commission.


External recruiters get paid more for getting more candidates hired with the least effort per candidate -- they are not interested in spending a lot of time getting 10% more salary for a candidate, instead of getting another hire done.


Most external recruiters don't make much more than a couple of placements a month, so they are absolutely incentivized to amp their commission up as much as possible without losing the sale. Was this not the case when you were working as a recruiter?


As a hiring manager and candidate dealing with external recruiters, that's not what I saw happen.


How did you get people you didn't hire to share with you their actual salary preferences, and companies that didn't hire you their actual salary ranges? Sounds like that would be super useful info


This is outside the US, but I just ask at the first call: please give me a range on your expectations to see if it overlaps with ours and that it makes sense to move forward and avoid wasting time. Everyone answers, most giving a number directly and few counterasking for our range for the position, which of course I disclose at that point.


Do you mind not using multiline code block quotes that split sentences in half?

Not only is it obnoxious to read on mobile, as horizontal scrolling is necessary, but it also fundamentally doesn't make sense as you should be able to copy/paste whole sentences.

One might expect that as a recruiter/(ex-?)developer you'd realize how painful it is when you use this formatting.


Looks much better to me, including on mobile, and is the first complaint in the 2,020 days my account has been active here, whatever "one might expect". Additionally, it will look entirely natural to any of us who used email before it was mostly HTML.


Which mobile client and do you browse in landscape?

I agree with photogrammetry in that it's essentially unreadable on mobile portrait as you have to horizontally scroll each quote you want to read (then scroll back and again if it's multi-line).

Shorter >'s with italics are my preference.


Thanks for the downvote - good to know that you're not interested in improving the UX on HN.


And if they are an actual external company and not external recruiter?


I'm not entirely sure what you mean, but are you sure you don't mean an internal recruiter?


What changes if you drop just a handful of words so it starts "I am interested in roles paying above $xxx,xxx only"?

That might be my salary. It might be my floor for moving to a new, more expensive, area. It might be what I think my current salary should be.

Depending on how things like bonuses are done, base salary alone isn't super relevant anyway. There's a potentially very big difference between a company that says "we give bonuses of up to 20k per year" and "our bonus target for good performers at this level is 25%."

Maybe 9 times out of 10 the recruiter says "sorry, that's not going to happen." But you only want to find the ones who say "we can work with that" anyway.

This may not be the best way to absolutely maximize your total, but if you pick the right floor, you can save a lot of time up front. And the way to really maximize your offer isn't to hide your current salary, anyway. It's to take that first offer to another other company you're interviewing with, and see if they can beat it. That's your best bargaining chip.


"The costs outweigh the benefits for employers"

That's funny, I thought this was a employee's market... Better get with the program, bub.

I'm actually in favor of 100% transparent salaries. I've yet to hear a compelling argument against it that wasn't simply "it feels icky."


    > I'm actually in favor of 100% transparent salaries.
    > I've yet to hear a compelling argument against it
    > that wasn't simply "it feels icky."
I have no strong feeling at all on them, but generally when I find myself unable to make a strong case for or against a divisive issue, and can't think of any compelling arguments either way, it's indicative that I haven't researched it properly, or am simply discarding perspectives that don't agree with my existing biases.


> The costs outweigh the benefits for employers (encouraging existing employees to ask for more money, losing candidates who will accept significantly less but wouldn't apply at a lower range, etc) -- so don't hold your breath.

> State it upfront when you submit your application, rather than "find that out"

As an employee, I would say the costs of specifying my expected salary range upfront outweigh the benefits.


    > As an employee, I would say the costs of specifying
    > my expected salary range upfront outweigh the benefits
That's interesting. Having well over a decade of being an employee, having been a hiring manager, having been a freelancer, and having been a recruiter, I'd do it every time.


This is true, and one of the worst inefficiencies you encounter as a candidate (and probably as an employer, I don't know): There's so much song and dance that has to go on before you get to the meat of things: Compensation. Companies could easily solve this by posting clear salary ranges, but most just don't.

Another thing that companies could do better is think really hard about "required" and "desired" skills, and be truthful. Lots of companies list things as required which they're probably willing to bend on for the right candidate. All this does is scare off potential candidates (odd when you're complaining about that mythical shortage of engineers). On the other hand, recent events have encouraged me to not be afraid to apply for jobs that I'm unqualified for :) so maybe it's not such a big deal.


    > There's so much song and dance that has to go on
    > before you get to the meat of things
This can be an advantage of using an external recruiter, because they sure as shit know what their potential cut of the revenue will be.


100% This. I just wasted a bunch of time extracting an offer from one of AmaGooFaceFlix. Supposedly a top SV company with good compensation.

Got an awful offer to relocate to an area where 1/4 to 1/2 the house costs twice as much and I would no longer be able to work from home. I would also lose a significant portion of my life commuting. The offer was comparable to what I made working remote from Boston.

I don't mean comparable in the sense that they tacked on a CA bump to handle the cost. Comparable as in same money, but I would have to live in CA.

Ended up working remote for more somewhere else. Everyone's time wasted. I don't get it at all. It's not like they didn't know it was low.


Sounds like Amazon?


Re-read (it is an odd wording): "1/4 to 1/2 the house costs twice as much". He also said SV

Seattle is not even close to being as bad as silicon valley in housing costs.


My guess: Facebook. I talked to them. I flew out. I spent the entire day on their campus and ate a really nice lunch. I wasted my time with them. They were excited to hire me for about $45k/yr less than I was already making.

But don't worry, they'll make it up with about $10k/yr of stock grants which you don't get until you've worked there 4 years.

I picked another lesser-known company and am making - no shit - about $70k/yr more than Facebook was offering. With stock grants that also might be worth about $50k after 4 years, though admittedly, the stocks are a much bigger gamble in this case.


I don't publicly shame current or past employers as well as potential employers. It's why I listed multiple companies. You will have to reach out to me on some other medium to get details.


In addition to salary range up-front: find out up-front if they're actually hiring you, or want to do the "contract for a while, then maybe we'll hire you for real" thing.

I had to say no late in the process to several companies which offered me the contracting-to-maybe-hire dance.


I had an incredibly bad experience with this recently. They failed to do the end-of-contract review on time, extended the contract period numerous times after, and when they finally got around to the review said, "we're not a good fit." I actually liked the people and the job, but got the distinct impression it was just a team full of devs that wanted a devops to clean up their terrible infrastructure mess, future proof it, and then let them go.

Never again will I do contract-to-maybe-hire.


I agree, I always ask what the salary range is and if they are not willing to give it I tell them I am not willing to go any father in the interview process. They say then if they really want to interview you.


Most companies pay poorly.

If you are in the higher range of salary, you gotta find out which local employers/industries are worth talking to and limit your application to them. There probably aren't many.


Alternatively, you can specify your expectations up front. It doesn't have to be an exact number, just a hint -- e.g. "No talk if the salary isn't well north of 100K".


You're anchoring them to offer you 100k. Pretty sure that's not what you wanted to do ^^


"Well north of" are the key words here.


Have you tried hired.com ?


This is one of those "aha" solutions that people keep mentioning, but completely ignore how it would work in a non-ideal world.

• What 2 hour project can you assign that would let the candidate go into enough depth for you to judge their code, design decisions, architecture skills?

• If you give them more time (e.g. a weekend like in the article), is it still financially viable to the company, especially at $100/hr??

• How do you filter out the candidates that aren't serious about the job but just want to make a quick buck?

• What about all the candidates out there who you can't legally pay (e.g. workers on a visa)?

This works when you are a small company that needs to pick 2 out of 6 total applicants, but otherwise completely unfeasible.


Exactly. Their example:

> "Create a single page app that lets me enter movies in my home movie collection, store them in offline storage and search through them. I want to search by Genre, Title & Actors."

Significantly more than 2 hours, unless you are looking for a quick hack and for the candidate to repeatedly explain "I would have done it this other way if I had more time..."


That's only if you're expecting a candidate to provide a perfect solution; and don't make it clear that the time limit should inform their design and not just measure their speed of implementation.

Part of software development is tailoring a solution to the scope of the problem, and this includes taking into account the time and effort available to come up with a solution. Someone who looks at that problem statement and designs something that will take a week has missed a very important requirement.

You could reasonably scale up any problem to take an arbitrarily long amount of time, by adding onerous requirements (must have 7 9's of reliability). What I've found more rare and precious is the developer who can limit the scope so that items that are not up for negotiation like time (2 hours) & resources (1 developer) are respected.

I fully expect a developer to state their assumptions up front, so that I can judge their design based on the stated assumptions and not some unstated imaginary assumptions. It also lets me see how or if they prioritize.


I completely agree. I saw that and thought - um, that's not a 2 hour job unless you want a massively hacked together app. I might be able to do something modestly decent over a weekend, but 2 hours? Nope. You have to have ways of inputting/verifying data, tagging genres, etc. I've done a lot of similar code but that's why I know it's basic, but not quick. It would let me know that the job wasn't going to last because of the lack of knowledge in estimating software development.


> What 2 hour project can you assign that would let the candidate go into enough depth for you to judge their code, design decisions, architecture skills?

Let me answer that with an example. In my previous job, I had to implement a simple username/password based authentication API in the first interview. The problem was explained to me. A two page document with a written specification of the problem was also provided to me. After I read the document, I was asked how long I would need to solve the problem. I asked for 40 minutes. Then the interviewer left me alone in the room with a laptop. After 40 minutes, the interviewer returned and went over the entire code I had written, discussed the API design decisions I had taken such as the corner cases that were handled or not handled, exceptions that were thrown, how empty or null usernames or passwords were handled, what error messages were generated, how concurrency was handled, what performance problems could occur if thousands of requests invoke the API per second per thread, how the password was secured, etc.

After I got the job, I learnt that my colleagues had also gone through a similar coding interview with different problems of similar size and scope. Some people asked for 30 minutes while some asked for 90 minutes. Therefore, I believe that it is possible to have a tiny 30 minute to 90 minute project that lets a candidate demonstrate his or her software engineering skills.

I felt this method was very effective because it evaluated a couple of key skills that were important to do the work that we were doing there: time and effort estimation, API design, writing clean and correct code, robust error handling, and design decisions from performance and security perspective.

I didn't get paid for that 40 minutes but it was part of their half a day onsite interview process, so I did not have to spend any extra time apart from the time I had already scheduled for visiting their office. Also, they did provide me free lunch, snacks and drinks, so I didn't complain. I wish more employers did this.


What 2 hour project can you assign that would let the candidate go into enough depth for you to judge their code, design decisions, architecture skills?

Where I work, our take-home is a simplified version of a task that's common enough (in our field) that we built tooling for it. It's generic, it tests understanding of a few things, and we've gone through three iterations of our own tooling so we know there are multiple viable approaches.

As for judging decisions, that's why the interview that goes with it is a collaborative code review. You treat it the same as if you were reviewing code a co-worker had written; ask questions, discuss tradeoffs, etc. etc.


I read the money/difficulty calibration as "pick a problem that you think would take someone about 2 hours to implement and offer $200 for completion." Not "offer $100 per hour and then cough up $1600 when the following Monday the candidate swears that they spent two full days on it."


A key point of the specification for the solution is that the produced work //is not// a product. You are using the exercise as a test.

Maybe it would be better to explicitly state that the results of the work are to be released under CC-BY?


> What 2 hour project can you assign that would let the candidate go into enough depth for you to judge their code, design decisions, architecture skills?

Write a program that reads a text file, and output each unique word with the number of times it appeared.

The program must be done in a programming language among: C, C++, C#, Go, Java, PHP, Python, Ruby.


foreach word: ++(wordmap[word])

print wordmap

This is on the level of a pretty simple first phone screen problem.


Being on visa makes this payment thing harder unfortunately. Maybe the charity idea might work for those people.


• What 2 hour project can you assign that would let the candidate go into enough depth for you to judge their code, design decisions, architecture skills?

What whiteboard interview would?


Since this would be a 1099 contractor relationship and not an employee relationship, would there be any issue for Mattermark in this case? All they need is a TIN for their 1099 filing; it's up to the visa holder to know whether they are permitted to do contracting work.


> What 2 hour project can you assign that would let the candidate go into enough depth for you to judge their code, design decisions, architecture skills?

Well, the movie project suggested in the article was a pretty reasonable one.


If you write that in 2 hours, honestly I can't see that being code I'd hire someone for. You'd have to hack together garbage code and you'd look like someone who threw things together without thinking.


Alternatively, with that time limit it's just going to select for candidates who've been churning out Django/Rails/etc. webpages recently and have all the details committed to memory.

That feels like premature optimization to me. I can see how it'd find generally good web developers, but I can also see how it'd produce just as many false negatives as Google's interviews do. So is it really that much of a step forward?


I completely agree with that. I was thinking, 'yeah, you could hand this to a kid who only knows surface UI and he could crank this out using Rails or something, but it would just be the same garbage that you couldn't maintain later and it wouldn't prove much more than he knows how to write simple apps.' But then maybe that's all they want?

I dunno...I've worked with a lot of people who could code a simple app but completely fail when it comes to serious programming.


Hm - fair point. If I was to do something like this, I'd definitely treat it as a throwaway - I'd do some cursory data modeling, put some tables into a local MySQL DB, reverse engineer that with (say) Hibernate and then throw some Jersey services and some Ajax in front of it. I wouldn't give any consideration to scalability, or security, or maintainability... one would hope that a hiring committee would take that into account, but you're probably right, the people doing the evaluating would likely immediately forget that this was supposed to be thrown together in a couple of hours, people being people as they tend to be.

More

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: