The reason they are important is that they are part of every engineers day-to-day work. We use them to collaborate and design and understand.
Therefore, if a candidate can't whiteboard a design, solution, architecture, etc. in a manner that other engineers can understand it, they are going to struggle.
That being said, the way Google and other companies use whiteboards is an epic fail. Asking someone to code on a whiteboard is stupid. Asking them to solve algorithm puzzlers on a whiteboard is lame. This isn't how we use whiteboards every day so why should a candidate be expected to use them this way during an interview.
The employer (or school, etc) wants to make successful hires and to avoid bad ones. False positives are always a big problem. False negatives are usually much smaller one. A process is good if it produces good hires, and no bad ones.
In principle, a process that hires 3 good candidates from a pool containing 5 good candidates and lots of bad ones is good. Avoiding bad hires is paramount. Missing 2 good hires is not as bad as making one bad hire, not near as bad usually.
Most employers don't care about false negatives at all. If they do, they probably care most about missing really good outlier hires, not mistakenly selecting the 5th best candidate instead of the 4th. The only reason to care is fairness. People instintively care about fairness so this affects practices but it's not nearly like false positives.
Employers that need huge volume (eg, the army) are the only ones that care about false positives as much as candidates do.
The candidate (student etc.) generally feels the process is "broken" when it is unfair to them. Unfair means that a person could have done the job well, but cannot get hired. Ie, if the candidate is a "false negative" then the process is broken.
Anyway... When things become systemic, the "brokenness" becomes acute to a candidate. If whiteboards (your architecture version, Google's CS version or any other version) are prevelant and you suck at them... you can't get hired. There are people who suck at your test (whatever it is) but would be good at the job. Your proxy is a biased against some people, every proxy is.
Like I said, a lot of things are like this. One side is working the numbers game, the other side isn't. Insurance markets, college entrance, executive appointments, PayPal's fraudster detection (bastards!), credit scores, investment (even startup investment)...
The problem is that very few companies know how much false negatives are costing them, so it is very difficult to evaluate whether the false negatives are—cough—a negligible tradeoff against weeding out the false positives.
False positives are right in your face. But it is nearly impossible to systematically keep track of the candidates that aren't hired to see if they are successful elsewhere.
So people lean towards eliminating false positives at all costs. It's another variation on "Looking for the watch where the light is better."
It's another variation on "Looking for the watch where the light is better."
These situations aren't at all symmetric. A false positive always costs you (the company). A false negative is only a cost to you if you fail to hire well for that position.
Also the fact that a candidate you passed on does well elsewhere doesn't mean you were wrong to pass on them (it could, but it's not clear).
That applies in the case of caring more about false positives than false negatives, since nobody (or close enough) even knows how many false negatives they have.
Your statement that someone being successful elsewhere doesn't prove we have a false negative underscores the uncertainty involved. It doesn't invalidate what I'm saying in the least.
But if we want to be rigorous, we could do some direct testing. Let's say we give a telephone screening and four interviews to each candidate. Simpkifying somewhat, let's say that our policy is that a fail on any one eliminates the candidate.
If we hire lots of people, we can collect false positive and false negative stats by randomly passing someone who fails exactly one test.
If a lot of people who fail a particular test end up succeeding, we can infer that the test provides false negatives.
On the other hand, if everyone who gets the random "free pass" on one particular test ends up failing, we can presume it has low false negatives, and maybe abandon the random passes for that test.
The original joke/anectdote works because it leads to clearly suboptimal search - the guy knows his watch isn't near the streetlight, but he's looking there because he can see.
This is a very different situation. To abuse the analogy further, the guys best bet to find a watch seems to be just looking near the streetlight. After all, not only can you see better there, but most of the watches are near there too.
Your saying "false negative is only a cost to you if you fail to hire well for that position" could be easily disproved with examples of people, who were rejected and started their own competitive business (e.g. ). IMHO, one of the goals of having 10Ks of engineers on staff is to keep them from the competition. But since nobody is tracking this and the extreme cases like WhatsApp can be hand-waved away it's easy to pretend that the false negative's cost is miniscule.
The guy with the watch analogy only works if there is good reason to believe he’d be better off searching elsewhere. But we don’t appear to have that here....
Let's be generous and say the WhatsApp was a 10x unicorn so FB overpayed only 9B for it. Which makes 4.5B loss per false negative on each founder. If a false positive was, on average, 1M loss (which is an extremely generous in an organization such as FB in my recollection), it means you could have had 4000 false positives for each of these negatives and still come ahead.
And, it brings us back to the Type I and II errors - it's always cheaper to make one, which has a limited downside than the unknown (potentially very high one). Typical example given is that while you are on the way to an expensive cruise you realize that you might have forgotten to turn off the stove. If you go back you might miss your 10K cruise. If you cary on, you might lose you 1M house. On average, people who miss cruise in such circumstances fare much better than ones, who risk their houses.
I get what you guys are saying, there just doesn’t seem to be real evidence that is an issue nearly all cases. Whatsapp notwithstanding
So in order for it to make sense to spend that money, you want some evidence there will be a positive return. Far from contradicting my position, what limited data I have seen does not support that effort, at least in the general case. Certainly whatsapp and the like don’t speak to it - they are very much outliers.
By all means try it. If your false positive rate explodes it will be an expensive lesson, but that’s always a risk.
One last point, the idea that nobody thinks about this and try’s thinks is laughable , which is worth noting because some comments in this thread almost suggest that it’s true,.
With my example of sacrificing a 10K cruise for a chance to not lose a 1M home, I figure?
> Far from contradicting my position, what limited data I have seen does not support that effort, at least in the general case.
Indeed, the data does not contradict a statement "on average, false positives are 4.5B each", which nobody has argued. However, it does contradict the statement "false negatives have any cost only when you are unable to fill the position as the result", which you have made in this thread.
>One last point, the idea that nobody thinks about this and try’s thinks is laughable , which is worth noting because some comments in this thread almost suggest that it’s true,.
Again, it would be silly to say that nobody thinks about it. Evidently, a number of people think the same way you do.
At first, yes. But when one has had enough experience looking for scarce commodities (of whatever kind: genuinely attractive apartments, interesting music / restaurants, romantic partners, etc) one learns that the "where the light is" search technique only gets you so far.
Certainly in this scenario - the false positive rate is only important if the true positive rate is too low. It's not at all clear that is a factor.
That's the whole point -- in order to find evidence that it isn't working, you have to start looking at the false negatives --
or that is to say, where the "light is not".
LinkedIn could offer this as a service, if there was sufficient demand that they could monetise it. Every company will routinely check out every candidate’s profile anyway. They just need an automated way to check again in 6 months and see if everyone they reject is going to a competitor who is beating them in the market.
I would agree with you if the supply and demand weren't so skewed towards top developers. Do you think you are choosing between A and D people, or are you really choosing between C and D people and A and B people never walk through the door?
It's pretty hard for a hiring manager to be called out as too selective. However, if you let in a few bad apples then you look really bad.
The incentives are for hiring managers to be too careful.
It is, oddly enough, the case that if you are not hiring any bad engineers that you probably are being too selective.
Then again, probably one of the worst bad apple I have seen would ace whiteboard as he was quite good at trics. He just did not like the actual work involved in software development and tended not to do work he don't like except bare minimum.
Also you can fix some false negatives - allow people to interview again for other positions at the company. I've seen multiple people get hired on the 2nd or 3rd time around and ultimately be successful.
When we first got married my wife was really hestitant about me working on contract. Over the years, she's seen how much larger my checks are when I do.
Besides, there are some opportunities where companies just want someone to come in, fix a problem and then after that it's just boring maintenance work. If it's a contract, I can come in, fix the problem, and look for a more interesting full time job without being labeled a "job hopper". I can just put on my resume that the 6 month job was a contract.
I prefer contracting but I found that more companies are willing to pay me my minimum salary than the commensurate hourly rate I ask for to make up for the possible gap in employment, unpaid time off, and self employment taxes.
But, my wife has a stable government job with decent benefits that allows me to go back and forth between contracting and permanent and allows me to take risks working at companies that pique my interest without concern about the long term viability of my job.
People complain whiteboarding because they've had bad multiple bad experiences with hiring processes that involve whiteboarding and they came to a conclusion that whiteboarding itself is the problem.
Unfortunately, you can't fix a bad hiring process by simply changing the technique. Each technique has its own pros and cons, but no technique will guarantee success.
These are the most common problems I've seen with hiring processes:
1) The people doing the hiring don't really know what they're looking for. Instead, they go by a vague, handwavy notion of whether the candidate is "a fit" or not. You have to define what "a fit" means in such terms that all of the people involve understand it, agree on it and know how to test for it.
2) The people doing the hiring don't know how to test for their criteria. Whether you're using whiteboard or a take-home project or some other testing technique, you have to design the test so that it helps you observe the traits you're looking for.  And you have to actually train your people to interview candidates.
3) Lack of transparency. This is a huge problem and it infests all aspects of hiring. Some recruiters won't even tell you which company they're hiring for. A lot of companies misrepresent what you would be doing for them, sometimes intentionally, but usually not. Some companies won't give you perfectly reasonable information about the job.  And, thanks to the litigious nature of our society, no company -- anywhere, ever -- will give the candidate honest, detailed feedback of why they were rejected.
4) The company is unreasonably demanding when it comes to candidate's personal time. The typical two-stage process -- a phone screen followed by on-site interview round -- is demanding enough, because it involves more than just the interviews. The candidate has to spend time and effort on other aspects, too: multiple phone calls with the recruiter, preparation for the interviews, travel, and juggling all of this with their personal and professional lives. Some companies don't understand this, so they think there's nothing wrong with a process that involves a take-home project, a debriefing phone call, a face-to-face conversation to test for cultural fit and a full-day on-site interview round.
Most companies don't even know that their hiring process sucks. Of those who eventually realize something is wrong, many end up cargo-culting whatever they read or heard about "more successful" cases. As things are right now, the whole situation is eerily reminiscent of the fuss and hype the industry went through with software development processes. I wouldn't be surprised to see yet another wave of parasitic consultants pop up to prey on companies by flogging their hiring process improvements and best practices.
 This is where a lot of people's complaints go wrong. The most frequent grumble is: why does this company's interview process focus on algorithms and data structures, if you'll just end up coding some services in Java? If the company is doing things right, they're using these topics as a tool to observe and measure whatever it is they're looking for; most often, it's the candidate's understanding of computational complexity. Just because you won't be writing your own implementations of common data structures and algorithms, it doesn't mean you shouldn't know how to pick the one that won't blow up your memory or execution time.
 I had a case where I was interviewing with a company that gave me a print-out overview of their benefits, which included the price of each health plan, but not what it covered; when I inquired about it, they replied they wouldn't be able to give me that information until I signed the New Hire NDA.
I've started to turn this around and so far our senior candidates have loved the experience. (I only wish I had come up with the idea.)
Instead of asking to write code on a whiteboard and solve an arbitrary problem, couple of our engineers came up with an inverted problem. And ever since seeing it in practice, I've been trying to introduce it to every interview I can think of.
We write a piece of code on the whiteboard, and then ask the candidate to review it, as well as modify it to better suit the problem and context. This way the question turns into a discussion on design, constraint exploration, service expectations and externalities. Hell, sometimes it even goes into deep language design issues. Sure enough, it still measures a level coding skills but puts a much higher emphasis on understanding what the problem and its scope really is.
It's also a lot of fun.
I work at Google, opinions are my own.
I think your assumption that Google interviews based on what you will do on the job is wrong. The reality is that for most SWE positions, we can't assume you will know the things you'd be using for the job because of all the internal tools. Thus, algorithm interviews are more or less a proxy for some combination of general intelligence and coding questions.
I'm not saying it's perfect and there are many things I don't like about it, but I think it's important to know what the goals are when making a critique of whether the process meets those goals.
It's a proxy for one particular style of coding questions: leetcode. It's nothing more, nothing less.
Calling it a proxy for general intelligence or coding in general is a joke. That's propaganda passed around the company to convince people that the practice should be continued. It strokes the employees own egos to think they passed some magic intelligence test so it motivates them to keep repeating that it measures intelligence/skill.
My assumption is that there are a lot more people who are studying for these interviews than you think, which is why the proxy is failing (or producing homogeneous results)
On the other hand, it's okay to target sufficient but not necessary. False negatives can be fine, so long as you are finding enough true positives.
Even if you are systemically missing out on a sub-pool who are good, it's not necessarily a bad thing - particularly if you are realistic about its limitations. To my mind the real problem isn't that you might miss good people (you will) but that if your filter is too constrained the good people you find might be too homogeneous.
IMO the biggest problem is that small startups copy the Google's approach while their hiring environment is completely different.
As an aside, if you've never done systems design or pseudocode on a Smartboard, you're really missing out. Copy/paste, duplicating a slide, recalling earlier slides, etc. Man, that's the way to design a system with quick iteration but still no worry of losing your work.
When I interviewed at Google, they asked me if my whiteboard code would compile. I said, "give me a laptop and I'll tell you". I don't think they liked that answer. ;)
Some of the biggest productivity gains I've had over the years were related switching from "writing code down" to "thinking in code" mentality and abandoning linear, paper-like ways of thinking, which hindered me when I was in college.
For anyone who believes that paper and whiteboards are "good enough" for coding, I highly recommend watching some of Victor Bret's or Alan Kay's talks. Pretty much any of them. They consistently touch on the subject of dynamic media as a way to empower people to... well, think better for the lack of better terms.
Not when you are literally writing code on it.
And there's nothing wrong with whiteboarding pseudo-code.
Hey if they work for you, that's great.
But the simple fact is that they don't fit everyone's brain.
And that there are other tools available to accomplish precisely the same ends -- some of which have many advantages over whiteboards, in fact. Like these things known as "pen" and "paper", for example.
So this idea that they're part of "every engineer's" day-to-day work is, quite simply, disconnected from reality.
I do agree with you, however, that the problem lies not in the whiteboards themselves but in the way they've used. Which has truly been an epic fail.
> Like these things known as "pen" and "paper", for example.
When you need to share this pen and paper, you use a whiteboard. I don't understand this hate for whiteboards, which are a cross-industry cross-profession standard (including medical) when collaborating.
One of the more interesting things I have noticed is that happy engineers have a little free time each day. They aren't under crushing pressure at all times. To this end, it's very common for whiteboards to accumulate drawings and doodles. This is often turned into "the elaboration game" where one doodle is built upon (by a little or a lot) every now and then. This is probably because coding is problem solving, but also because it's a creative endeavor. Engineers who are happy tend to work around other happy engineers and those engineers are all creative with free time.
What's not to hate?
* unless both the board and markers are of very high quality, the boards tend to stain over time
* as do your fingers
* and the markers dry up, and the erasers get less useful as well (if they're available at all)
* you have to erase other people's crap (or wonder / ask if it's worth saving) first
* the dizziness that results having to turn back and forth between the board and your audience (that is if you don't end up simply talking into the whiteboard most of the time)
* and from stooping to pick up the pens / erasers / pen caps all the time
* and all of this, mixed with all those indelible memories of crappy interview experiences
I'll take pen and paper any day
which are a cross-industry cross-profession standard
Along with horrible interview experiences unfortunately.
Solved by getting a good office manager.
I'm not even joking. We have very large whiteboards in every room and in almost every space where people can wheel their desks in. Cleaners are happy to clean them up and make sure they are in good condition. One team recently complained that they may be too eager... (Our whiteboards are heavy glass, bolted to the wall. It probably helps.)
Office manager(s) on the other hand always keep their ears open and make sure the office has enough pristine marker packs and erasers available. Having a supply cabinet properly stocked is a cheap investment.
One of the most enjoyable things I can hear at the office is: "Let's hijack a whiteboard".
In theory, I guess.
But -- getting back to the original subject -- for some reason during interviews the odds that you'll get an inadequately stocked and clean whiteboard seems to be almost laughably high.
Consistent with the observation that in many places, the seem to almost make a point of the importance of positive candidate experience in so many other ways (from poorly prepared interviewers to inappropriate / irrelevant questions to stalling and ghosting, and on and on).
If that's the kind of whiteboarding you do, no problem. But, don't ask me to write a merge sort in C# (been there done that). If you ask me to sort a list in C#, I'm going to write:
sortedList = employeeList.OrderBy(e => e.LastName + " " + e.FirstName).ToList();
I get so hung up on this it’s changed how I ask the question. The last time I asked a sort question, all I wanted was a compound comparator. And even then I couched it as “we want the rest endpoint to support sort criteria so we can do a top five list, but to sell the idea, we are going to sort in the UI for now. Give me some code to do that.”
Generate Code -> create Relationsl Comparison and click on the properties that I want to use for comparison.
Every candidate I interview has the choice between coding on a whiteboarding, and coding in a text editor with rudimentary code formatting.
I really like it when people code in the text editor, because that's one less thing I have to transcribe in my interview evaluation.
Regardless of whether they will choose the whiteboard or the laptop, I have never, and will never mark people down for typos, confusing .length with .size, a missed semicolon, or any other petty bullshit that isn't pertinent to the problem. I'm not hiring a compiler.
AFAICT, the backlash is around interviews where candidates are asked to write code (or pseudocode) on a whiteboard. Being able to implement bubble sort on a whiteboard does not correlate very well with being able to deliver business value, in my experience.
To add to this, it's being able to write code on a whiteboard in a high pressure situation while talking about every step you're taking. Over time I've gotten more comfortable with this, but mainly by doing more interviews, not by getting better at my job.
I spent about 30 seconds looking at it, but at least the first chapter 1 seems to be about proper whiteboard techniques
To all the responders to my query, your answers are super helpful.
Being self-taught, there are just these few holes the internet doesn’t quite cover. But a little reaching out on HN goes a long way in filling the gaps.
Basically, if it is an idea, concept or design, draw it out. If it is functionally correct code that compiles, screen-cast and type it out.
The time before that was for a database migration where I mapped out each step that had to be done and in which order.
The point is not to have or not have the whiteboard during interview but how and what it is being used for.
The best way I found to interview candidates is to have few different discussions. One of the discussions I like to have is to try and solve some kind of problem that you could potentially have in real life. I am solving problems with people this way on a daily basis so this lets me observe candidate in a setting that is already familiar to me and hopefully to the candidate. The whiteboard is then used exactly as in real life -- as an tool to visualize and communicate ideas and hold it as a kind of cache and reminder for the duration of the discussion. The whiteboard is there during interview but I will not require candidate to use it. If I have trouble understanding the idea I may ask "ok, I can't visualize this, could you maybe draw it for me?"
Nobody programs on whiteboard in real life. If I need to program for presentation, for example to train my team on some topic, I will bring laptop.
The key about whiteboarding is it take away all the specific syntax and gets to the heart of the understanding the logic.
Thats what collaberation is for, i think many young engineers that only code with an ide are nervious whrn they dont have autocomplete available and they lose focus on the core logic.
Programmerd used to write on paper ... this entire process (coding without the ide/search engine) is lost.
But this is the part where the programmer shoes their strength in logic ... not syntax memory.
I feel like there's two angles to this; 1. understanding complex algorithms commonly used as tests. 2. understanding how to convey information on whiteboards efficiently, common paradigms, etc.
#2 is what I think I'd actually enjoy improving. #1 is just SAT style stuff that I infrequently use in my day job and serves as an annoyance more than anything.
I think this is largely the point.
They are terrible at compiling and syntax checking.
Theoretically you should already know whether someone can do the physical code part. That problem is largely solved for and don't require in-house interview.
In-house interview should be used for c̶o̶n̶f̶i̶r̶m̶a̶t̶i̶o̶n̶ b̶i̶a̶s̶ ̶f̶o̶r̶ a̶g̶e̶/̶r̶a̶c̶e̶/̶g̶e̶n̶d̶e̶r̶ determining if the candidate can think/design/solve for the domain space they're being hired into. A whiteboard is a really good tool for assisting in that.
They are optimizing completely different metrics than you do. They want crème de la crème developers that are perfectly trained not to think about certain classes of problems and waste time there but instantly come with acceptable answers, immediately recognizing the structure of the problem and having the ability to faultlessly transform it into a piece of code without a debugger. It allows the training from the best universities to shine.
The problem is when everybody starts doing the same, but without the prestige, salary, equity and lifestyle that Google provides.
Or without actually needing to problems that even remotely resemble the kinds of problems that Google (sometimes) solves.
I feel slowed down having to write out on a whiteboard when I could instead write some dummy code to get a point across. It just seems crazy to assume all developers everywhere must be good at white boarding.
I tend to move my hand faster than I can hand write, which is why I like typing for all forms of explanation.
Especially since I’m on a full remote team, whiteboarding would be really weird.
I'm sure a shared digital sketchpad would accomplish the same thing for remote workers.
What a cluster it's all becoming on the software engineering side.
And I need to know what the work exactly entails, and what is expected of me. Many interview questions are disjoint with what the job actually requires.
>Prove to me you can
I can show you my past experience, and I can do technical exercises. What I can't do is devote a ridiculous amount of time and effort to poor questions.
As someone who has hired a lot of devs over the years (almost exclusively for our offshore team in India, but sometimes in the EU), I can tell you that the number of candidates who embellish and down right lie about their capabilities and past experience is shocking.
Truth is, I hate both sides of the interview divide, as interviewer and interviewee - it's a horrible, inefficient, flawed process regardless which side you're on, and nobody has the answers.
People outside of India often embellish or exaggerate too though, even if not close to the degree that I've experienced in India (disclaimer: this is my own opinion based on my own experiences over 10 years or so. Obviously India has a lot of great, honest Engineers, our organisation never sees them though, largely because we don't pay well enough).
The reference knows the candidate and likely feels a much closer connection with them than they do with you. Assuming the candidate isn't an asshole, they will want to help them out by giving them a good recommendation. If the candidate was let go due to downsizing or such, the reference may even feel pressure to help the candidate find a new job.
So it's a great data point to have and as you say, it can help weed out the straight up liars.
But any 2 given candidates with 5 years at Acme Inc are not interchangeable. Some candidates are just better at programming (or some subset of it) than others, and I think employers try to find ways to predict that at interviews.
Wow, what an incredibly tone deaf response.
>Some companies simply do not understand how to properly engage in employee selection.
Maybe changing this prospective is the point of the "incessant whining."
>I need to know you can do the work, prove to me you can, I will then hire you and pay you to work.
I'd have to know how you are testing, but I'd bet dollars to doughnuts your method has no way of predicting output in any meaningful way.
>What a cluster it's all becoming on the software engineering side.
Absolutely. I would say it's endangering the overall quality of the industry. The saying goes, if you are a bad manager or a bad company the first people to leave are the A and B players. The people that will never leave are the C and D players. I suspect people who do all this trivia don't even know what an A or B player looks like.
It's really not though. Here's the bottom line: You want money from a company. The company doesn't owe you jack. shit. If they fucking whiteboard, and you want money from them, then do the whiteboard. Later, when you are employed by them, try to suggest they improve their process. If you have better options, take them.
> Maybe changing this prospective is the point of the "incessant whining."
Sure. But is that the companies #1 priority, or do they have other things to do, like make money and stay in business that take precedent? Anyways, the way you do this is from within or providing feedback and having conversations with recruiters -- you know, human interaction. This job board is the ultimate culmination of millennial passive-aggressive behavior.
I agree with your bottom line as well. Both are true! It's a negotiation, and it isn't zero-sum. Working is good for both the employee and the company. Neither is in a universally stronger negotiating position.
I'm not annoyed by bad interview processes and strong preference for false-negatives, I'm annoyed by that in conjunction with the claim that there is a shortage of competent workers. There is some lower-hanging fruit to take care of before that claim makes sense, and improving recruiting processes is among the lowest hanging (along with raising wages, obviously).
Unfortunately, unless the sides are favored to the labor market (in our cases it is not, and even if it was, there is always outsourcing), this equation is not equal.
Both are true, but it is an extremely unbalanced equation. At the end of the day, the company has the money and you have the skills. Unfortunately only one of those helps you pay the bills -- unless you can figure out a way to pay your rent in exchange for your skills.
> This job board is the ultimate culmination of millennial passive-aggressive behavior.
This job board just enables more people to pursue a course of action that you're already suggesting they do!
I am frustrated at times just as much as any other engineer by playing "monkey-see-monkey-do" in front of an audience in order to receive employment and I have indeed passed up job offers because I was turned off by the interview process (but again, that was my choice) but if you take a step back, look at the overall job market, and what other workers have to go through to get a job, we really do have it made, and this seems like a minor nuisance at best.
No, this is how change is driven. How do you think change happens? Getting a big committee together and getting buy in from all interested parties? That's not how it works. But maybe this is how it works, maybe this job board attracts a bunch of candidates who are tired of the status quo, and other candidates start telling the companies on other job boards that, sorry, but they're more interested in the companies that list on this particular job board, and those companies then see that the talent pool they're drawing from is smaller than it needs to be, and they decide they want to draw from the larger pool, so they change their practices, and then that snowballs, well maybe now you have industry-wide change.
I don't personally see this having much of an impact, but I think the people who started this are doing exactly what you're saying they should be doing, except that you're pillorying them for it because they aren't doing it through some impossible broad consensus approach.
> what other workers have to go through to get a job, we really do have it made, and this seems like a minor nuisance at best
It is true that we have it made, but I do not think it is true that we have it made in this sense. We have it made because our skills are very highly in demand and because of that, we are treated well and paid handsomely once we get a job. It is true that we should never ignore, down-play, or fail to appreciate this privilege, but that doesn't imply that we should not discuss or advocate for improvements where needed.
Where we don't have it made is in the predictability of employment once competence has been demonstrated. Other professions and trades usually have long-and-tedious up-front competence proving phases, but then they have a credential that will be respected by future employers when they have open positions. We are fairly unique in forcing people with long and distinguished careers to continually re-prove themselves. Our employers have open positions, but it takes a lot of pointless and duplicated busy-work for a competent person to slot into them.
I think what it comes down to is that our process is relatively mediocre, good, or even great for inexperienced unproven new entrants to the job market, but rather sub-par for experienced and proven folks. And that's why you see people looking for a better way.
Do mechanical engineers draw up a design for a valve or something at every job they apply for?
Another example it's related to is the XKCD standards comic. Rather than improve existing standards, the committee breaks off with dissenters, forms a new committee, and now you get "Situation: There are now 16 competing standards"
edit: Better Solution? Talk to recruiters? Let them know you won't consider places that force you to play "dance monkey". The letting them know part is key. More broadly: maybe unionize?
And the company wants to make money from me. In my market, there are a lot more open positions for senior devs than senior devs that are actively looking for jobs. I
You need that money more than they need you. There really are few exceptions to this rule -- AI being one of them.
Again I wish it wasn't like this, but it's the reality for the overwhelmingly majority of software developers in this country.
1. That you are in an area of the country with a lot of opportunities
2. That you kept your eye on the market and made sure that your skill set in demand
3. That you kept your resume updated
4. That you kept your network of recruiters, former coworkers, and references fresh.
If you have limited savings like the vast majority of Americans -- time is your enemy
With the unemployment rate of software developers being extremely low, the chances are that someone looking for a job that you are interviewing, already has a job. When I said my turn around from looking for a job to getting a job is about three weeks, I didn't mean to imply that I was a special snowflake.
A company I was working for went through a few rounds of layoffs before everyone was let go -- this was in 2011. Without fail, every developer and even L2 support person had another comparable job within a month.
If a company makes the hiring process too long, more than likely if you are in the right market and you are just a regular old "Full Stack Developer", by the time you go through the process, you've already received a couple of offers
Besides, if I'm looking for a job and I am unemployed, the time I am taking to do homework, I can be meeting with recruiters, preparing for interviews, brushing up on my skillset, etc...
> prove to me you can (do the work)
A take home exercise only proves they can get the work done elsewhere. It doesn't prove that they actually did it.
I could present you two options:
1) I just hire you after a 30 minute chat. 50% you don't work out, and we fire you after 30 days.
2) We have a longer interview. We whiteboard. We take home test. We whiteboard some more. 90% chance that you don't get fired after 30 days.
Which would you prefer? As a business owner, I am happy to just try people I don't know well - IFF they are willing to accept that there is a very good chance they are fired within the first 30 days.
There is such a huge range in developers. The haves vs the have-nots can be a 100x different productivity difference. I really want to try to tease that difference out. It is really really hard.
That is how it works in other fields. One of our managers asked his friend that owns a body shop how he finds good employees. He basically has a chat with them, feels them out, then hires them. If they don't work out he hires someone else. Perhaps it shouldn't be as stigmatic as "firing" after 30 days but you have instead hired someone on a 30 day contract with option to hire permanently. If they don't work out, that option is not exercised and that person is free to go try somewhere else. I can tell pretty quickly from chatting with a developer if they are a total fraud. I can't tell really quickly who will become a great dev, I just know when someone has the chops to have potential.
That's the way it used to be done and I think we should go back to doing that. Read the resume, ask some questions about the resume, 30 minutes to an hour full interview, done.
You get hired on, put on a probationary period, if they don't work out, fire them.
Personally, that's my preference. If I don't think I can do the job I'll know after a 30 minute chat.
In other words, I think there is far more than 50% chance that things work out after 30 days and far less than 50% chance that I impress 6 interviewers.
Interesting idea. If you fire fast enough it may not actually cost that much more money to a business to hire a bit more as long as you fire the bad ones fast.
But a 400% increase in that you find some sort of excuse not to hire me and I won't even get hired in the first place. Your percentages only make #2 more attractive if there's a more-or-less equal chance of being hired both ways.
I suspect many people would rather do 10 interviews and then get 1 job they keep for a year+ compared to 2 interviews and a 50% chance the job that hire them, fires them in 30 days. But I could be wrong, mostly basing on what I would want.
Why did the industry seem to forget this?
I would not leave a w2 job for a contract job. Nor would any good developer I know.
This is why the industry "forgets" it. It's not that attractive to a good candidate.
Wow, 8 jobs last year? Busy life?
If you want to do that by way of an open source portfolio, sure. But not everyone has open source contributions, so then I would need some other process for acquiring a work sample.
I have seen many people here say you should hire people on a contract basis for a few days/a week. I don't know what it's like doing that in the US, but for the UK market that sounds like a ton of administrative overhead for both parties to take on in order to establish if they are even a good fit.
In my experience, anyone we've hired WITH open source contributions has been 10x better than those without.
I like the format, but it obviously has other downsides, such as a higher up-front investment from the candidate.
That being said I found the format enjoyable last time I went through it. It was less stressful during the interview itself, and I felt more confident walking in. I would recommend it to companies who are considering it.
We use a whiteboard in our interviews for sketching UI, and sometimes drawing a database table. Generally candidates find this much easier than talking through their design, and we think that interview very closely represents what we do on a day to day basis.
If you're asking candidates to balance a binary search tree when no one in the history of your company has ever had to do that, you're asking the wrong questions.
We do a mock technical meeting with multiple members of our tech team and the candidate presenting options, talking through pros and cons, etc. Much of the interview is about assessing communication rather than technical skills.
The interview is basically a standard tech meeting that we hold regularly in the team to discuss designs of new projects.
In my opinion it is all about aligning your questions with the day to day task you expect from people. If the work is very algorithm heavy, by all means probe for that in the interview.
I base my interviews around one main question: assume the tech challenge you written beforehand is a prototype for a complete product/service. what are the steps necessary to get us there and beyond?
I do not care about the candidates tech preferences or the exact solution as long as the argument for it is coherent and not just based on heresay. Then I probe for a couple of things within their solution where I see potential weaknesses based on their argument, experience or code in relation what we look for. It is more about problem solving, decision making and communication than computer science.
Candidates are free to just talk trought it or use whiteboard/paper to draw it out. But if you choose the former you better not forgett what you said five minutes ago.
Obscure “I had to google it before asking you” problems are the problem not the whiteboard. I think whiteboards are good visual aides.
Sometimes I just ask people to draw a frog though.
I don't have any solutions only complaints and bitterness, sorry.
We only do them when we are pretty excited about a candidate but need more information to make a hiring decision.
It's cheap, really, considering the cost of sourcing candidates and the cost of making a bad hiring decision.
Why would I go through that trouble? I have 20+ years of experience that I can explain how I architected and developed from the website, to the middle tier, to the database to multiple VPCs on AWS and dev ops. I'm not going to do homework.
When I was first starting out I might, but living in a major metro area that's not on the west coast, there are more openings for senior devs/architects than people to fill them.
If a company wants to try me out, I'm more than willing to do a contract to perm, but my hourly rate is going to be enough to cover self employment taxes, make up for not getting paid for time off and at least two or three weeks between employment.
We have a take home test. It is an obviously fake problem, but we review it to the grade we would for our production code.
We don't find that candidates turn down taking the test - it's capped at 3 hours so that it can't take up lots of time. We do find that we have a low pass rate, and even among candidates with 10+ years experience, we still find candidates who write code that is just not good, and who haven't read the spec.
I'm not at all suggesting that you might be in this category, and if you can find jobs you want that don't require tests that's great.
However, there are people with plenty of experience, who can architect things brilliantly, and who can't write a ~300 line program in Python in a way that satisfies a spec or in a way that anyone would be able to maintain. That's important for many companies, and so I think tests can be one useful part of an interviewing process.
What does that number look like? 10k a week?
2080 - hours in a work year
-80 Paid Holidays
-120 Paid Time off
-120 Gap between employment (based on my history)
= 1760 work hours per year
FTE Salary * .028 (Employer Medicare Contribution) + $7967 (maximum social security amount - 128500*.062) = minimum salary
minimum salary/1720 hours = pay rate.
Take a reasonable salary for my market as $140,000, that is about $152000/1720 = $88.37/hour minimum.
Adjust the number based on market, skill set, and the amount of time you estimate between jobs. I'm also not taking into account health benefits that you would need to pay yourself because I'm covered under my wife, or 401K matching. I would add 4% to the hourly rate to cover 401K matching.
Is there a person who claims to be a coder that has configured and launched a multi-environment microservice with several attached artifacts like DBs and queues, that knows in depth what each is doing, that can't code? At that point you're just testing whether someone has memorized syntax details that they might usually get from an IDE or a 3 second google search in regular development. I have been writing production JS for years but still have to occasionally look up Array.slice vs Array.splice parameters, especially if I haven't done a lot of data munging in a while. That may indicate to a person giving an interview that somehow I'm "faking" it, when in reality its just a momentary stall-out in my brain's lookup tables.
Funny that you mention it. I know a person who can't shut up about microservices, Kubernetes and CI and can't code their way out of paper bag. They had no problem passing one of those interviews.
And sure rote API memorization isn't the point. But if a candidate can't code say bubblesort (or any kind of sort) in 30 minutes they are not very good at coding - and it doesn't matter if they can quote their architecture patterns by heart.
Thats bullshit. Sorry. Most developers will NEVER implement a generic sorting algorithm except in a code interview. You say you don't want rote API memorization, but you do want rote sort method implementation memorization. Yes you can be a system or tools developer and maybe have a higher chance of doing it, but its 99% likely even then that you just use a library for it for the language you're using, or copypastaing it from someone who's studied the specific implementation for your lang.
There's a number of people with CS or SW engineering degrees who can't actually code. Those can often quote you Gang of Four patterns, discuss broad slices of tech and draw architecture diagrams on the board. But still can't code.
It still seems strange to me to pay for throwaway work but I can also see how that’s really just my personal sentiment / bias and this is probably a pretty good and balanced approach.
This type of interview self-selects for people with fewer family/non-work obligations, who are willing and able to spend their free time still programming.
Are you also job hunting at work, doing interviews on your employers time, etc?
That guy already has a good job. Another company is trying to poach him. Why would he, or any other qualified and gainfully employed candidate go through that unnecessary homework bullshit for a company that's trying to recruit them? The company has its priorities misplaced. They should be trying to convince the candidate to join them, not putting up barriers to their most qualified candidates.
I mean this isn’t a lottery, and hopefully you’ve reasoned there will be a high probability of getting hired; say 50%. So then the expected value of your time is $6.25k/hr.
Go, take the moral high ground, and don’t do the homework. That helps rational minded folk achieve an even higher hit rate.
There are much better and less humiliating ways to get top dollar. Work your ass off and build reputation is the biggest one.
If you are a company and you have to call a recruiter, you're already behind. Putting people through that stuff just makes it worse.
I mean, even "discount" recruiting platforms like Hired and Vettery will cost 5-figures when we make a hire through them, and traditional recruiters often charge 20% or more of the new-hire's first year salary. A few hundred bucks to get some more information about a prospective coworker is cheap.
I said no thanks and moved on to other companies. Felt good. I just avoided working in a culture where there will be high pressure to always perform. :)
They can afford it, but still, it's far to say that the wasted effort isn't entirely one-sided. Hiring is an expensive and inefficient transaction.
Figuring out how to make this both more fair and more efficient is a really tough problem.
If you want to get that first house downpayment for the Bay Area, I'd say it's almost essential at this point.
Asking someone to solve an obscure mathematical problem (the Two Egg problem seems to be in fashion) doesn't do much except test whether the candidate has memorized the answer.
When not in an an interview I like playing with these kinds of problems, but if I look it up I'll probably find interview prep where it shows the answer as well.
Could you summarize just the problem please?
If an egg breaks when dropped from floor n, then it would also have broken from any floor above that. If an egg survives a fall, then it will survive any fall shorter than that.
The question is: What strategy should you adopt to minimize the number egg drops it takes to find the solution?. (And what is the worst case for the number of drops it will take?)"
The answer should be a form of binary search, but since you only have two eggs, you'd have to do a sequential search from the bottom floor. You have one chance at a mistake, so I would almost say start with the 2nd floor, but I know it will break on the first floor.
I don't see how that in any way gauges programming what so ever.
yeah, sorta. essentially you partition the 100 floors into some number of groups (i forget what partition length is optimal) and drop the first egg at each segment until it breaks, then go linearly from the previous one
Since you have 2 eggs you can start on the 33rd floor. It breaks you can use the 2nd egg from 1st to 32nd floor.
If it doesn't break, use the first egg on the 66th floor. If it does break, the answer is between 33 and 65, if it doesn't the answer is between 66 and 100. That cuts the maximal testing time time from splitting at 1/2 (1st egg at flr 50), to 1/3rd.
To optimize further, since in the bad case we'd have to search up to 34/35 floors, you can mathmatically find a number where you split the drops at an interval (IIRC the "real" optimal answer is 14, then 13, then 12, etc), which means you're doing less explicit checking.
At 14 - i, you have 10 drops to find a range of 13 - i floors in which the optimal path is found. So instead of a worst case of 35, you have a worst case of around 17.
I find the best way to approach this sort of problem is to first find one optimization, and then use the same process to find if there are further optimizations.
It does relate to programmer's ability to amortize worst case running time.
Once you realize that 1 egg requires linear search you just have to realize that with 2 eggs you have to decrease the level increase by 1 on each success.
Of course those must be some crazy genetically modified eggs.
I would guess the interviewer probably rates people who have memorized the solution far higher than people who have never encountered it and solved it with their brains.
Hmmm, that's an interesting question. It's the Price is Right closest without going over solver...
Practical answer, drop from the first floor and then the second. You're probably not going to make it to the third.
I don't want to put my actual answer in case others want to solve it. But I will say there's an obvious solution, but if you apply one of the hard problems of computer science you might eke out a bit of an edge.
EDIT: Actually I don't think that's right. I think the answer varies based on your expectations, and should be tuned after each run if you run the same test multiple times. I do have a good starting point in mind though.
After thinking about it on the ride home today, I was not quite on the right track before. Again don't want to post my solution because the entire point in asking was so that people could see the question and work it through themselves.
That's The Price Is Right, not Wheel of Fortune ;-)
I was definitely picturing The Price is Right at least.
As an employer I don't care how great your skills are, if you can't walk me through your thought process for 45 minutes you're not going to be overly successful. If you can't incorporate feed back or engage when pressed to change your design you're probably not going to be successful.
Do your recruiting right and you shouldn't need the white board to simulate whether the candidate can code at all (I think white-boarding with the intent of looking for named algorithms is a waste of everyone's time), instead you want to see if they can synthesize information, engage with other people, and otherwise display the soft skills that tend to be much more useful metrics for employee success than coding aptitude.
Granted, my industry is known for not having deep coding problems to solve, this strategy might not be as useful in verticals that require more technology chops.
The post you're responding to makes a great point that this isn't the normal working environment for 99% of actually-writing-code developers. I discuss and iterate designs with my coworkers/leads/managers/architects just fine, but a surprise algorithmic problem you have to simultaneously solve, draw on a whiteboard, and constantly present to an audience while being timed isn't something I have been great at. I bet many others aren't either. I got good at it after a few interviews last time I was looking for a job; but if I hit the interview trail again today, I'm confident that I would flounder in that setting for a while.
Imagine if, instead, the candidate worked out a project -- maybe at home, or maybe in an interview room for some time -- then you take time to review and understand their work and then go through this back-and-forth process of discussing their work. This would also benefit the candidate's understanding of how you collaborate to problem solve in your work environment.
The problem with this approach is that it requires the interviewer to actually invest time to understand the candidate's work.
Anyway, I hope you see why many see it as a problem that the process for a candidate to succeed at a job interview is to practice interviewing skills rather than demonstrating their engineer skills.
As an investor, I don't care how great your skills are, if you can't come up with your new business idea on the spot and walk me through your thought process as you're doing it, you probably suck.. right?
Unless you already have the answer, the first step is to come up with the solution. This could involve time spent alone just thinking, or brainstorming with coworkers, or researching the problem on the net, etcetra. What the right steps are depend on the problem, available resources, your background, even your preferred way of working.
The second step is to present and dissect the solution. I dare say this is the point at which the majority of engineers have no problem whatsoever talking about their thought process.
Do you work in an industry where engineers genuinely have to walk somebody through their thought process while they're trying to think up a solution?
If a problem comes up at a meeting and you don't have a solution on the spot, you take note and discuss it later or add it on the agenda for the next meeting. I think that's how people work.
I don't care about where, when, and how you came up with the idea. I care that you can present and discuss it once it's ready for that.
I think this is why many prefer the take-home test. They can focus on the solution. Then, at the interview, they can focus on the talk about the solution. It's also a great equalizer as e.g. people coming to work in a domain they have less experience with can take more time to research the solution space.
The complaints about homework taking up too much time may be valid, but I'd be most happy to trade 2 hours of interviews for about 75 minutes of homework and 45 minutes of interview.
But since we’ve gone there, many high power companies also have a lot of chuckleheads running around.
Is there any scientific evidence to support that, or is that just industry hokum?
And communism probably redistributes wealth if it is administered correctly.
If companies want to hire based on general intelligence (not personality, team work, communication and so on) then they should administer IQ tests, which are based on a century of rigorous research. That would at least give you a fighting chance of measuring G.
Why do you think white boarding can measure general intelligence?
Question: How do you validate that the posted company or job position doesn't need whiteboard? Is it after the fact, as in a disgruntled candidate complaining? I couldn't find any information about this on the site.
Yet a lot of people are never really taught to do it. Perhaps they pick up this fundamental skill just fine after a couple whiteboarding sessions in the company.
If a company is rejecting candidates for not having a skill that is really easy and quick to obtain, they'd better not be one of these companies that keep complaining about shortage of talent.
Yeah, this one is pretty bad. But let’s try to do better, not worse.
I don't agree with making candidates write algorithms they don't need to know in a format that they don't need to be able to write them, but I am in favour of testing candidates thoroughly.
Anyway, I hoped you could search only for jobs where this box was checked by the company, but apparently not...
EDIT: It does allow searching for minimal salary and remote jobs, among the wishes listed in a sibling comment.
EDIT: Ok, my mistake, my memory is failing me. "Doors that close" is used in a whole bunch of Joel blog posts, and "walls and doors" is used in the original Joel Test description.
But WaybackMachine seems to confirm that the official label from the "Joel Test" has always been "quiet working conditions" and does not appear to have been changed.
So unless Joel went back and rewrote the past, it might be our memories. I am fairly sure that he did have another blog post that stated explicitly that every developer at FogCreek had a door.
Even better, I would love a site that classifies job offers/companies by:
- Has an onboarding process for new hires
- Has quiet working conditions / open office
- Has clear requirements
- Salary range
- Allows remote working
- Type of interview and time to get a response
Unfortunately we haven't yet expanded much out of the Midwest, so it MAY not be of use to you yet. But check it if your interested.
I went there, typed in "developer" in the search box, and got dropped on a map zoomed in on the Kansas/Oklahoma border and a message saying my search didn't match anything. Took me a while to notice the "zoom out on the map" part. Not sure why I landed in the middle of nowhere in the US, I'm located in Europe. In any case, super unintuitive.
Also, this might be an American thing I'm missing, but which (if any) option of "Pod Office Layout" or "Closed (Cubicle) Office Layout" corresponds to "proper walls and a proper door"?
I'd also suggest adding a filter for minimum salary.
Hope this helps.
On Jobiki, we don't have job postings. Just companies and data about them. So the "minimum salary" and "developer" searches are more job related, while we are more company related. Either way, helpful feedback.
As for the "proper walls and a proper door" aspect, are there developer/traditional employee jobs in Europe that can have a real office? In the US, unless you are a manager, executive, very experienced employee, you don't usually get an actual office. Maybe its more common in Europe?
I have a real, quiet office with lots of daylight. That's actually one of the reasons I'm staying at this job. That, and the fact that I'm not subjected to low-status micromanagement techniques like Scrum. In short, I have working conditions befitting a qualified adult.
"Pod" seems to mean completely different things to different people.
"Pet friendly" workplaces tend to be noisy and discriminate against people with allergies, so some users will want to filter them out.
Even if you don't want to track current openings, what kind of jobs a company has is important information.
And good note on the "pet friendly" environment.
We in the processes of adding that information to outline what positions a company hires for. (Whether or not they are currently open or not) Well, more what departments that a company hires for. Going all the way down to job title seems a bit too granular, maybe I am wrong? But by department seems a happy medium?
Yes. Maybe not every single developer; there might be two or three or four to an office, but it's still an office with walls and a door. There are companies with more open spaces and twenty developers to a room, but the (much) smaller ones are the usual case from what I've seen. ("Europe" is a big place, of course.)