I have zero desire to play these silly games. Sorry, I'm much more than a 40 minute probability puzzle! The only way to win is stop playing. Give me a problem to take home, and think about. Or, how about asking me about my skills (and having a conversation about a piece of code). How about pair-programming? TDD abilities count for nothing, whaaaa?
These types of tests do little more than reinforce stale ideas and turn away unconventional well-rounded candidates. Used to play the game, now I win because I don't play.
"Having a conversation about a piece of code" sounds good, it proves you can read code and not just write it. Everything else you mention should have been taken care of on your resume or you wouldn't be in the interview, BUT plenty of people can talk a good game.
As an employer, do you really want to find out AFTER hiring that the candidate is incapable of understanding a problem based on an oral or written description; incapable of capturing requirements, asking the right questions, and forming an attack plan without thinking about it overnight?
Testing someone's ability to follow along and contribute in real-time is part of the test, IMO. It does penalize "deep thinkers" and still waters, though.
I've had candidates email me a supremely perfect solution the next day, and that's a positive data point too.
Interview questions should hinge on, like, the experience and competence that I already have from having worked a substantial period of time as a professional computer programmer. Not on how cleverly I do on BS.
And I actually generally do very well on this crap but I still hate being impelled to take part in this kind of circus. In fact, it's annoying that I jump to being a top candidate based on this stuff and only then does an employer come up with some mundane requirement that should have been the initial qualifier/disqualifier.
I don't think anyone would claim these things are crap or BS, I think it's that these silly problem are irrelevant. These questions are like high school or college coursework, artificial questions designed to reinforce the instruction, but not something that would be encountered in a business setting. Additionally, they tend to gloss over a person's experience.
I may not be able to solve your silly little stock trading problem by myself in a short amount of time, but I am sure that within the normal development environment, I could, especially since I would be working with a team. Or could I? Since this question doesn't look into my ability to communicate or work within a team, you don't know. Also, while I may not be able to solve the problem in 45 minutes, maybe I should talk about the time I dove into an application written in a foreign technology and managed to solve a critical production bug before it could cost the company millions of dollar. But again, you don't know that because you're obsessed with my ability to solve a high school computer science problem.
Different jobs require different skills. Having the ability to code is important, but not nearly as important as teamwork, communication, and the willingness to learn something new. That's the problem with these questions, they are nothing more than smoke screen.
Take 20 developers. If you give them a test with 200 difficult problems they would all score about the same. Instead, give them 4 difficult problems in a limited time frame. Now some of them will look much smarter than others. Only hire these candidates. Now you can believe you are part of a very selective group.
I think there's some truth to this. Companies should weigh past work more heavily, and let candidates do real work with the team as part of the interview process, IMO.
That being said, I think these whiteboard problems can be useful when you think of them as "trying out what it feels like to brainstorm with the candidate about how to solve a problem." That's how I think of them. There are plenty of people I just can't get on the same page with when trying to whiteboard together (different communication styles/speeds/whatever), and it's helpful to know that ahead of time.
The problem is these interviews aren't brainstorms. They're exams, where you don't even know the range of topics in advance. No matter much interviewers claim it's just a conversation, the reality is they have standard questions with specific optimal solutions, chosen for their ease of grading rather than their relevance or what they actually reveal.
When no one knows the answer and the object of the meeting is to find it, it might really be a brainstorm. When the interviewer knows the answer and expects the candidate to get it, where a real discussion isn't possible because suggesting things that aren't The Answer (or asking stupid questions) could significantly hurt your chances, it's a test.
There exist too-clever "gotcha" questions that are obvious if you've seen them before, and near impossible if you haven't. Those are terrible, and they fall to your "They're exams" criticism.
Other questions have layers, like an onion. Those are the good questions. They should divide the candidates into STRATA. I think the stock question is one of those.
Strata 1:
Everyone should be able to produce a brute-force, inefficient solution, and avoid the obvious+wrong solution. If not, they should have been weeded earlier by your phone screening.
Strata 2:
Better candidates should be able to produce a more efficient and readable solution, sometimes with prodding. Some candidates never get here, even with prodding ("Have you considered x,y,z?")
Strata 3:
Candidate skips straight to the correct solution, sometimes provides more than was required ("You shouldn't store data as keys; you probably want to return the time and buy/sell prices, not just the profit".) Interviewer needs to prod and ask questions here ("what's wrong with solution x? why did you do y? What if I want a list of possible trades?") to make sure it's not a memorized answer.
You shouldn't expect that, in ten minutes, your candidate can come up with the optimal solution that took you three hours. But they should be able to understand and recognize that solution if you lead them to it.
Anyone using these questions as pass/fail is doing it wrong, but that's not the question's fault.
Like one of their examples, "I have an array stockPricesYesterday where the keys are the number of minutes into the day (starting with midnight) and the values are the price of Apple stock at that time" My response would just be, "you generally shouldn't store data in keys". Their response, "Okay, just solve the problem." No real discussion.
I'd give you brownie points but still expect you to answer the question. These questions are not THAT hard - they are definitely fair game and help screen out the people who only know how to use APIs and not how to think about things at a lower level.
But I agree an array would be a more efficient data structure for this :)
Another problem I have is that I get terrible anxiety in the interviews and my brain shuts down. I've made a personal decision for my own well-being not to participate in these types of interviews, no matter the circumstances. At that point, it's in the company's court. I will happily demonstrate in many ways that I've solved these problems and more complex problems before. Including narrating my code, papers, articles, body of work, speaking of my aspirations as a data scientist / developer. Talk about learning JavaScript from Crockford and what I've learned. Opinions on DHH's Agile v4 and convention over configuration. The truth is, I research every problem I have (at least a little bit). I never sit down and blindly program or do math problems.
I'll never forget my former Bell Labs-theoretical statistican advisor (American. Stat Association decorated!) telling me about accidentally helping his high school-age daughter fail on a take-home probability problem for school.
You should look at smaller companies, most of them seem to have figured out the whole package counts. If you're a hacker like I am, it's a dream. Little bit of everything across your domain (but quality). Plus, you might even learn what's necessary to run the show!
That's where I'm currently looking. My biggest problem is identifying the companies. Being in the Bay Area, it's more a problem of how and where to search than limited number of companies, though.
I think I'd fit in with a company far enough along to have built its MVP (so they're not always in that perpetual crunch mode), but which has fewer than 10 engineers. I really value work-life balance, so I don't want to work insane amounts of overtime all the time, but I could totally live on a below market salary if the other aspects of the job were right.
I'm considering working with a recruiter. Anyone know any technical recruiters in the Bay Area who are worth working with?
Oh, and as always, my email is in my profile if anyone wants to take this discussion offline.
I'm from Central America. I went to SF and had a few interviews over a month. I failed in all those 40-minutes-solve-as-many-as-possible-fizzbuzz-problems because I wasn't fast enough.
That said, I've been working as a web developer for 11 years now, bulding CRM's, CMS', Crawlers, automation tools, and all kind of web based stuff companies needed.
Also, I never went to college, and have been working before finishing high school (15 years old).
So, while I lack a lot of computer science education, I consider myself competent at developing solutions, and have progressed with coding practices.
But because of the US interviewing style, which I had never gone throug before, I am not employable there. Interview Cake may be just what I need : )
It's tough out there. Most companies suck at finding candidates, and most candidates suck at finding employers. Very very few are doing the Matasano route of work samples and then measuring to make sure that this is actually working out.
We're a 70 employee B2B startup in NYC that uses something similar. We have a set of problems that we actually solved as we were building out the company - we now give them out as take home interview questions (different questions for different roles). Instead of brain teasers it's testing for real world coding abilities that apply to the job the person will be doing.
Assuming the code comes back ok we then use it as the basis for one of the interviews on site. I usually discuss design decisions and then make some changes to the requirements to see how they react.
We honestly haven't hired enough engineers to make any sort of statistical measure on it but anecdotally it seems to be effective. We've yet to end up with a false positive and I know that we've hired at least two engineers who are awesome that I don't think we would have hired based solely on an "algorithms at the whiteboard" interview. Plus our engineer-time-invested per declined candidate is way down. Rather than spending 60 - 90 minutes watching a candidate flounder at the whiteboard we can axe them in a 2 minute code review.
Before actively looking for my second job, I really had no idea how to interview folks effectively. After going through the process many times over the last month, I've come to the conclusion that there were essentially 3 good ways to evaluate me:
1) Give me a take home problem.
Granted, a couple of the companies that gave me these problems clearly didn't know how to interpret what I gave back to them. One of the comments I got back actually offered as a criticism that my code looked "too sparse." But if you actually have the ability to make sound judgments, this seems to me to be the ideal way to judge a candidate.
No one is coding over the phone (ever) or with someone watching over their shoulder even most of the time, so if you reject someone just because they can't perform under such pressure, you're going to have a lot of false negatives.
2) Ask a breadth of questions.
One of the other interviews which I excelled at was a series of 20 questions that asked me about things ranging from what happens when a browser makes a request to a server, to what a CDN was, to what the difference between UDP and TCP was. This was only a small part of the range of what was asked, but the fact that I was able to speak with confidence on most of the issues showed that I had a strong foundation of knowledge. They also asked my confidence in each area before asking me the questions, which I imagine helped them gauge whether I was down to earth enough to even recognize the things I didn't know.
3) Ask a depth question.
Drill down deep into a specific problem that the interviewee seems comfortable enough in. For this one, it could be entirely theoretical, such as how one might build a particular system to solve a problem. The key is that you want to dig into why they make any particular choice along the way.
You should look for someone willing to say "I don't know about X, I'd probably look for the reference to learn if there's a data structure to solve Y" if you end up getting too deep. That being said, not all personalities will understand that this is okay during an interview, as by the same token, a selection of interviewers won't understand that this is a sign of strength, not weakness.
It goes without saying that solving algorithm questions are only a part of the interview process. Candidates are also judged on culture fit and design skills.
A nebulous description of a vitally important quality - someone who your existing team won't hate. It's one-sided to claim that people only use it to unfairly reject candidates. Being an ahole is something I'd like to see candidates rejected for, but it won't show up in a technical assessment.
The idea that culture fit is just a catch-all for personal bias and -ism's recently occurred to me, and kind of freaks me out. Right now I think there are three things I'm most looking for in a quality Sr candidate: restless learning, pair programs well with my team (we do it on all production code), and focuses on solutions not problems. I'm afraid though that "culture fit", especially around pair programming is just a way for me to cop out on someone I don't like, but I can't prove it.
It's a slippery slope if you ask me. If you don't like them, you might not be able to work well with them. What's more unfair -- adding someone to the team who will cause problems because they're qualified -or- rejecting someone who is qualified because your 45 minute evaluation is, "they're not a good fit"? I don't know the answer. But I do know that it's a lot harder to fire a bad fit than to never hire them to begin with.
I expect this to bite me as ageism in ten years, when a 23 y/o Stanford grad will be aghast that I don't spend my evenings playing with MeteorRails.js.
'Culture fit' could be a vicious circle: The more mono-cultural your team becomes, the harder it gets to hire the most talented people. And if everybody just happens to be looking for the same 'culture,' companies that have fallen into that trap end up with higher recruiting and wage costs.
A bit of a nitpick, but the stock trading question should have stated up front that short-selling was not allowed or that the purchase order must precede the sell order. (I suspect someone with some familiarity with trading would have asked these questions in an actual interview for clarification)
Relying on a "hint" to figure out that you can only sell after you buy seems more like an additional constraint rather than a true hint.
I feel like completely specifying the problem does everyone in an actual engineering interview a disservice and isn't a good analogue to actual engineering problem-solving. That said, it might be acceptable or even encouraged for the purposes of this site.
In the case that you mention, if someone asked me "Can I short sell?" I'd be happy that they considered the order of events there and were trying to explore the problem. I'd certainly remember it in my interview feedback. My response as an interviewer would be more or less "That's an interesting thought. To simplify the problem, let's say we're risk averse and prefer a finite loss potential. Go ahead and just consider purchase before sale."
One of the biggest criteria I looked for from junior engineering candidates was that they didn't just jump into implementation and asked clarifying questions. That's hard to do with the sort of site this is so maybe in this case more complete specification is useful, but any way that a bit of ambiguity can be introduced means that you can have a meaningful conversation with an engineer about representation, the shape of the problem, the inputs, what the criteria for success are, what is truly necessary to solve a problem. I want working code in the end, but hearing the things a person does or does not consider about a problem is interesting and valuable to understand a potential teammate and how they think.
Indeed. Having had quite a bit of experience with ACM-type competitions, I believe it is super-important for technical coding questions not to leave room for misinterpretation (especially when the key point being tested relies on the accurate interpretation, like this case.) Being sloppy in the interview may lead a bad taste in the mouth of the best candidates, because they are the ones who will take the question most literally.
It's not an excuse to leave things like that vague so that a candidate would have to bring it up to you. If you want to ask open-ended questions, ask obviously open-ended questions; don't make an attempt to turn simple technical questions into open-ended ones.
Agreed. I feel that if a question is going to be about a "real world" situation like trading stocks, it should reflect real world constraints that should be stated up front.
I also found it curious that the example listed the stock price at 1 AM. It would be far more of a real world example to only consider times during the normal trading hours.
A question phrased about a particular real-world domain that doesn't makes sense within that domain just doesn't seem fair to ask.
But again, total nitpick on my part. The actual format in which the questions were presented was generally done well, and a nice refresh from the typical "interview prep" sites.
My first thought was that just getting a min and a max was the right answer, mainly because I having something hard-to-borrow is the edge case; and that the general case is that you can short.
Alice got in facebook just attended coding boot camp and did some algorithm practices. Why does facebook recruit a non experienced like Alice but turns off lots of talented (with a top school cs degree)/with decent working experiences? I am curious.
>>Why does facebook recruit a non experienced like Alice but turns off lots of talented (with a top school cs degree)/with decent working experiences?
I don't know what you're considering "talented", but Google seems to believe that your school/GPA has almost nothing to do with how well you'll perform on the job.
NOTE: Let's not confuse this with how likely you are to become another Zuckerberg. Starting a company is completely different from working at one. I don't think Zuckerberg could work at Google. He'd be bored out of his mind and/or argue with everyone, then quit in less than a year.
"GPAs don’t predict anything about who is going to be a successful employee. “One of the things we’ve seen from all our data crunching is that G.P.A.’s are worthless as a criteria for hiring, and test scores are worthless — no correlation at all except for brand-new college grads, where there’s a slight correlation,” Bock said. “Google famously used to ask everyone for a transcript and G.P.A.’s and test scores, but we don’t anymore, unless you’re just a few years out of school. We found that they don’t predict anything. What’s interesting is the proportion of people without any college education at Google has increased over time as well. So we have teams where you have 14 percent of the team made up of people who’ve never gone to college.”
The line, "this is not just another technical question database" is really effective. You anticipate the first possible criticism someone might have and dispel it.
I'll definitely give this a whirl soon, great work.
But it kind of IS and the fact that the quality and quantity of questions is far inferior to some of the other books on interview prep I've looked at in the past.
I'd say the step-by-step hint process is the secret sauce. It mimics an actual interview scenario (where a generous interviewer might be willing to help you along) while teaching by example some structured ways to break down the problem by walking through it step by step. Notably, most of the answers start with a brute-force solution and optimize from there, which (IMO) is an excellent way to approach these sorts of problems, especially in a stressful situation such as a real job interview.
Yeah, that page is there so people who have finished all questions can go back and review. You can basically get around the paywall using this link. I figure people who dig around to find a way around the paywall probably wouldn't have paid anyway :P
If I pay to get all the questions only to learn later that I could have accessed them all easily for free, I'm going to feel ripped off, even if it was only $5. Either make sure the questions are blocked off properly or let everyone have access to all of the questions. You can't have it both ways here.
If I pay for this Pepsi only to learn later that I could have easily stuck it in my coat for free, I'm going to feel ripped off, even if it was only $1.25. Either make sure someone's watching me at all times and patting down my coat at the door properly, or let everyone have Pepsis for free.
Doesn't seem quite as appropriate when the context is changed, does it?
Reading through this I see I was wrong. Thank you.
My reaction was provoked when I was considering paying for the rest of the questions, saw the comments and saw the link to all of them. I was too hasty in my response.
Interviewcake.com is awesome. At TechCareerHQ, we buy all our student memberships because of the quality of the questions and ability of students to focus on one question at a time. After using Interviewcake, our students are much more prepared for interviews.
I wonder if they'll eventually allow anyone to go to this site too be interviewed, live, by an actual engineer. For example, a prospect just shows up and jumps right into a technical interview with an interviewer from an interested company. It'd be good practice from the perspective of the engineer and act as potential job applicant from the perspective of the interviewer. Totally non-committal. I would imagine multiple prospects doing the interviews at the same time with the same interviewer.
Instead of scheduling an interview, you have prospective companies posting their interview schedules. For instance, "Hi, I'm from Google. I'll be hosting a technical interview at 330 PST. If you'd like to potentially work for Google, go to this link and answer my question in the time allotted. I'll be in the room to answer any questions." Of course, there's no whiteboarding involved but it might act as a pretty decent vetting process.
I love this idea and the way the problems are presented.
One suggestion (based on the one question I saw) is to also mention at least in passing, concerns with edge cases in the inputs. Python often lets you ignore these things but interviewers tend to be charmed by at least an attention to the possibility that your input is empty.
Also, it might be worthwhile to have some "homework" problems that don't have solutions that are variations of the main problem.
For example, the stock price problem is sometimes presented as changes in price rather than absolute price. This adds some subtle differences that might throw people off unless they've tried it.
I imagine similar variations exist for all the problems available.
Say at Microsoft interviewers will drill you on edge cases (but it really is common in most shops). That means all empty / invalid input as well as overflow, floating point errors etc. Some places care about reliability vs speed or elegance more than others.
Looks like the site has been improved considerably since I last checked it out, love the new content and context (especially the story about Alice in the about page).
This is a very good list - I was asked a bunch of these & thankfully I was able to solve them & land the job. It's significantly harder to solve these if you require immutability (which they did in my case, since we work with Scala)
For example, the stock price problem (https://www.interviewcake.com/question/stock-price ), if you insist the solution has no mutables, I really don't know how a Java/Python/C guy would approach it.
In Scala, you'd do
import math._
val s = List.fill[Int](100)((random*1000).toInt)
s./:((s.head,0)){(a,b)=>(min(b,a._1),max(a._2,b-min(b,a._1)))}._2
I can't really understand your version, I'm not familiar with scala. Is this somewhat equivalent? I tried to use Clojure's loop/recur but I'm too much of a noob. :(
Love to get some thoughts from people on the pricing. Today the first 3 questions are free, and it's $5 for access to the other 11. How would people feel about $10? $20? What if there were 30 questions, sorted into different topics?
I'd expect it to provide about the same value as a book, and be browseable; not something that is purely linear one after the other (e.g., I could skip to dynamic programming, graph algorithms, or whatever topics there happen to be). I think that a basic interview puzzle book should be in the range of $25-$40, and depending on that price point be from 150 to 300 pages, with roughly 50-100 problems.
Though, admittedly, I wouldn't pay for this at the moment because I'm happily employed. I did spend about $100 on books to brush up when I was interviewing in late 2011. I found actually the best practice for interviewing was going on interviews (I ended up interviewing at around dozen companies over the span of a month after I was laid off).
I feel as though to top something like "Cracking the Coding Interview", you're going to have to get close to the breadth they cover for the price they charge. You mention they 'charge' $25 which for sure is more than your $5; however, they've got loads and loads of questions for each data structure.
Currently, you're selling each question for about 35 cents. Although that's super cheap per question, $5 for 11 questions somehow doesn't seem like a great deal. However, I think people will be willing to pay the same for a larger database of questions e.g. I would be happy to pay $25 for 80 questions).
I paid the $5 because the sales pitch is so convincing -- "for the price of a burrito, you can get a job" (and so I did pay for this and I did get a new job!)
edit: back when I used this in Feb, I constantly got logged out and had to log back (was using G+ logins). Did this get fixed?
Only 14 questions in total? And they cover "trees, graphs, depth-first search, greedy algorithms, brute force algorithms, dynamic programming, memoization, divide and conquer, amortized analysis, and more?"
Parker (creator) does a wonderful job of laying out the different techniques to preform well on technical interviews. The format is easily understood and flow of questions is gradual enough to traverse.
Hey gameguy43,
Great site! Can you tell us about the tech you used to build it? I see Heroku... Is that Bootstrap too? What language/framework/database are you using? Thanks!
Hey there! Yep, bootstrap and angularjs in the front, Django in the back. Cloudfront CDN to handle the static assets (I've /really/ fallen in love with their product--so easy to set up). And of course, Stripe for payments.
What if the summing overflow? Why not just XOR all the numbers in the set with 1...n (assuming the question is there is only one duplicate in integers in range [1,n]).
I feel what you asked is a really nice follow up question, as tricks to not get a overflow could be highly non-trivial.
One can do addition mod n. As long as 2n doesn't overflow, you are good.
You can further improve it by not using any number larger than n. Say we want to sum a+b mod n, wlog assume a<b. If a+b<=n, we are good. If not, then b>n-a. Note a+b mod n = a+(n-a)+(b-(n-a)) mod n = (b-(n-a)) mod n. Now the calculation doesn't overflow.
> One can do addition mod n. As long as 2n doesn't overflow, you are good.
I wonder if the result will still be correct. It's possible to have multiple answers. e.g. (7 + 10) % 6 = 5. But both 5, 11, 17 % 5 == 5. Can you find which one is the actual sum though?
But x is not necessarily smaller than n though. For example:
Duplicate array: [1,2,2,3,4]. If you do mod 2 sum, then you have 1+0+0+1+0 = 2 mod 0 = 0. But another possible duplicate set can be: [1,2,3,4,4].
10 + x = 0 mod 2
(see 4%2 = 0 and 2%2 = 0? So x can be either 2 or 4.) If you let modulus number N be larger than n (e.g. n = 5 in my example, just let N = 6. Then 10 + x = 0 mod 6 will yield only one possible solution that is 2) then I think your solution will definitely work.
But I think the big problem is: you said x = (b - a) % n. That is true, but how do we get "a" (the sum of all unique numbers from 1 to n) without overflowing? The reason we do modulus summation is for avoiding overflow, granted that can get "b" easily, yet now we have to find "a" still, I find we are still stuck. Any suggestion?
Even if assuming everything will work out and my previous question can be responded, still the modulus function is WAY MORE expensive to compute than xor operation.
> If you let modulus number N be larger than n (e.g. n = 5 in my example, just let N = 6. Then 10 + x = 0 mod 6 will yield only one possible solution that is 2) then I think your solution will definitely work.
That was entirely what I was suggesting, but we don't need N>n. N=n would work. (btw, I defined n to be the size of the list -1, therefore in your example, the number is 4)
> But I think the big problem is: you said x = (b - a) % n. That is true, but how do we get "a" (the sum of all unique numbers from 1 to n) without overflowing? The reason we do modulus summation is for avoiding overflow, granted that can get "b" easily, yet now we have to find "a" still, I find we are still stuck. Any suggestion?
We can compute a using the same function that computes b. just sum all numbers from 1 to n mod n.
Here is a quick demonstration in Haskell.
> Even if assuming everything will work out and my previous question can be responded, still the modulus function is WAY MORE expensive to compute than xor operation.
True. I just want to show how to modify another solution so there is no overflow.
Subscriber here, I felt like I needed interview preparation to be able to go through this interview preparation. I think it would be helpful to list some resources for those less well-versed with these types of problems.
The problems are definitely a bit more advanced. I do try to make sure that any "vocab words" (data structures, algorithm types) can be clicked on for a deeper explanation. But if you haven't seen big-o before, for example, you'll be in the dark on a few things right now. I have some more "bottom-up" curriculum that I've developed from teaching at boot camps around San Francisco, and I'm still working on figuring out the best way to present it on the site.
Good execution on a simple premise. Another terrific resource that I've pointed a lot of friends to is Project Euler -- a little heavy on the math but a lot of interesting questions that are great prep for the Google (or Facebook, or Twitter, etc.) gauntlet.
Sorry the site's a little slow guys. I'm up to 100 dynos on Heroku but things are still a bit sluggish at times. Looks like I need to move more stuff over to the CDN? Thanks for the load testing!
For the indexing the web question (what data structure should an under-resourced web-crawler use to keep track of visited links?) wouldn't a Bloom filter work better than a trie?
I noticed on the 'Validate string with open/close is well formed' question an edge case that's not handled. If the input has a trailing closer char with an unmatched opener, an IndexError will be raised because the stack will be empty when you try to pop from it. (Unless your particular Stack class just returns None or something? But then you'll get a KeyError when trying to access that key in the dict.)
These types of tests do little more than reinforce stale ideas and turn away unconventional well-rounded candidates. Used to play the game, now I win because I don't play.