The problem is, talented people (some of them, not all of them) are not interested in ridiculous problems like this. It comes off as making the applicant jump through a hoop and watching them squirm for your own amusement, rather than as anything productive. No-one's interested in golf balls on buses, because filling buses with golf balls is a waste of time. Talking about it seems a waste of time.
And the question is stupid. Buses are different sizes and of different designs, you know. We have double-decker buses in the UK.
If you must use this class of problem, at least make it more related to your field or narrow down the kind of bus and what furniture is on it. And explain to the applicant why you would ask a question like this. "We want to see how you think" isn't an adequate explanation.
..and I will no longer be interested in the job if you think answering a question like that is a worthwhile use of my time, or yours.
It's trivially easy to figure out how to calculate a rough number. I'd go so far as to say blatantly obvious. It's also a waste of time to actually do it.
When it comes to the real world, an estimate that rough, based on no actual data (since the actual dimensions of a golf ball or the hypothetical, poorly defined bus are not available), is not something I'm ever comfortable acting on. In reality, you can always come up with better data than that.
Like FizzBuzz, you over-estimate people's ability to do this. I know many programmers who are able to knock out a Rails App without breaking a sweat, but ask them to think about throughput, or simultaneous connections, or machine load, or disk access, or anything else requiring ballpark estimates and general arithmetic and they're lost.
Sure, I know what I'm hiring you for today, but in a small company what you're doing today won't be what you're doing next month, unless you're so limited as to be unable to think outside your own existing speciality.
Ability to adapt is critical. By the time I've got you to interview I already know you can program, and most likely you've already shown an interest in my company. If you're unable to have a hack at something just because you think it's unrealistic, or poorly defined, or just uninteresting, that tells me something about you.
And to be fair, and in the interests of full disclosure, before I ask these questions I ask the candidate's opinion of them. In some cases that's already enough.
No-one is interested in the number you come up with. They're interested whether or not you have a process you use to come up with that number.
Ask some people how many golfballs fit in a bus. They'll be stumped. They'll have no idea how to proceed. They'll just stop, and look at you, and flounder.
Ask other people and they'll have a problem solving process. They'll define the problem; they'll gather information; they'll make assumptions (and note the dangers of assumptions) and then they'll start to work out a rough solution. The number that plops out the end is not relevant. Unless your process comes out with a number that is clearly nonsense ("5" or "27 billion") and you ignore that.
> No-one is interested in the number you come up with. They're interested whether or not you have a process you use to come up with that number.
Then ask, "How would you figure out how many golf balls fit in a bus?" This circuitous way around finding out what you want is anathema to most techie types anyway.
My problem with these brainteaser type questions is the interviewer (in my experience) NEVER has had the experience or know-how to interpret the answer; all they have been able to do is get a level of "ah, interesting" type "gut feel" from it. If I'm interviewing with someone who's got the background to be able to analyze what I'm actually thinking, fine, but to date, I've not seen it.
The most important takeaway from a face-to-face interview for me has always been to gauge a candidates general problem solving ability. When developing, you'll constantly be solving problems that are new to you, in areas you may not be familiar with.
The most important general problem solving skill is the ability to eliminate problems entirely so that you don't need to come up with a solution at all.
I think it's prudent to know the etymology of the word problem, whose origin is in "proballein", which literally means "thing thrown forward", much like a hot potato. And like a hot potato, the easiest solution is to throw it forward until it cools down. i.e. it comes to rest with someone who no longer sees it as a problem.
Handled elegantly and diplomatically, the "it's not my problem" approach to eliminating problems is a perfectly acceptable strategy, especially when you poorly understand the problem and the benefits of finding a solution. Eventually, if it is a problem worth solving, it will come to rest with the person who sees an opportunity in solving it and is equipped to do so. In fact, this is essentially the very basis of all rational economic activity.
Because we presume that an interviewee is desperate enough that they won't give the "honest" answer - well, I'll google that and see if I can't find someone else who has already found the answer.
Seriously, between Google and Wolfram Alpha, do you think that any of these inane questions can't be answered within a reasonable level of proximity?
I highlighted your question, right-clicked and chose to Google it, and it was answer number 3 on the returned results. I suspect Alpha would give more useful detail, but then I really think that wasn't what you were looking for.
Slightly more seriously, going to Google first for ANY such problem would highlight both what is already known about the issue, and profitable answers to look for further information.
> Seriously, between Google and Wolfram Alpha, do you think that any of these inane questions can't be answered within a reasonable level of proximity?
Using Google? 3 in 600 searches used the + operator, and of those 2 were wrong.
Saying "I would understand the problem, I would work out a suitable search query, I would use available resources, I would evaluate the responses I got; here's the result when I used a typical question like this".
> Ask some people how many golfballs fit in a bus. They'll be stumped. They'll have no idea how to proceed. They'll just stop, and look at you, and flounder.
Because, as ebbv said, there is absolutely no way they can get a decent level of accuracy that would be useful for any kind of purpose. Any estimate in such a situation would be thrown away as soon as Google and other resources are available. Therefore they effectively cannot answer the question, and the question is useless.
Can I just say "I assume it's about 2000." and be done with it? I mean, that's a good enough estimate when we're not actually going to be putting golf balls on buses and are just chatting about it.
The purpose[1] of an interview (for the interviewee) is to get the job.
The purpose of the question is[2] to gauge the interviewee's ability to solve problems.
With those constraints your approach would appear to be sub-optimal. Sometimes in life we have[3] to jump through hoops to get what we want.
[1] Primary purpose; there are other things, such as deciding whether the company is a terrible place or full of idiots or etc.
[2] Should be; I accept that maybe many people have heard about these questions and think they're the right thing to ask but have no idea of the purpose.
[3] Or we decide that hoop jumping is not for us and we reject all hoop jumping and politely decline the job.
It's a matter of putting on a show for the interviewer. But we don't know what kind of show. A show where we make assumptions about things? or ask questions? A show where we try to get as 'real-world' an answer as possible? or one where we assume things to make our lives easier? Why not just assume it's a toy bus and nothing can fit in it? Is that a clever and funny time-saving technique, or am I mocking the interviewer?
But yeah, the candidate has to do what he has to do if he needs a job. I think that's what you're saying anyway.
I agree with all your points, but also think one must be circumspect when applying this sort of question.
You're right, some people, when faced with a question like this, will flounder. Maybe sixty percent of the population. The rest will all solve it in a similar sort of way.
It's thus a bit of an "are you a complete idiot?" test, so by applying it you express an uncertainty over whether the person you're interviewing is a complete idiot. This is fine if you're interviewing fresh-faced undergraduates or something, not so good if you're interviewing someone with a long track record of success for a senior role.
I don't interview people all that often, and the circumstances are such that by the time someone gets in front of me I can take it for granted that they're not a complete idiot. Thus, I feel no need to apply this kind of test.
Why is it a waste of time if it gets you the job? The answer is "trivially easy" so it shouldn't take you long to waste your time on. I'd wager that you spent at least twice as much time writing up your response to this article than it would take to answer such an interview question. What's the true waste of time here?
>It's trivially easy to figure out how to calculate a rough number. I'd go so far as to say blatantly obvious. It's also a waste of time to actually do it.
Given any X, only a small minority of the people who believe they have the skills needed to do X competently are right. Dunning–Kruger as usual.
It should be trivially easy to figure out roughly how many golf balls fit in a double-decker bus. In fact, it shouldn't take more than a few minutes of figuring, so the time wasted ought to be minimal. If it does take longer than a few minutes, then the interviewer has learned something important about you, so it's not a waste of time then either.
How do you mean "subjective"? There is an objective answer to the question "how many golf balls can fit in a double decker bus" and while I suppose you could call it "trivial", being able to make that kind of estimate efficiently really is an important skill for programmers (and scientists, and engineers) to have.
That said, this is another FizzBuzz sort of test, and should not be made unnecessarily complicated. This is what you're trying to filter out: http://www.youtube.com/watch?v=Qhm7-LEBznk
There's so many different kinds of buses, and it clear what kind of level of accuracy is required. I can just say 'two thousand' and that's a good enough answer.
The problem is, that isn't good, as the interviewer wants to 'see how I think'. But he hasn't really given me a problem to think about, so I have to just put on a show for him. He hasn't seen how I think at all; he's just seen me putting on a show based on what I think he wants to hear.
There is a lot of variance in bus sizes, but not enough to put you off by more than an order of magnitude.
> and it clear what kind of level of accuracy is required.
Knowing what kind of accuracy is required is sort of part of the test. The parameters are vaguely specified, so you should know that you're looking at orders of magnitude.
>I can just say 'two thousand' and that's a good enough answer.
That would fit in a large duffle bag. If you told me that you thought you could only fit 2000 golf balls in a double decker bus, I would assume that you either put zero thought into answering the question, or that you did not have a good grasp of how to estimate things.
> The problem is, that isn't good, as the interviewer wants to 'see how I think'.
The interviewer wants to know that you can give sane estimates based on rough calculations. As the article explains, there are many, many situations where a programmer needs to be able to do this kind of thing. It's not about "seeing how you think". It's about seeing that you can think – at all – about estimation problems.
> But he hasn't really given me a problem to think about, so I have to just put on a show for him. He hasn't seen how I think at all; he's just seen me putting on a show based on what I think he wants to hear.
And if that show is "2000 balls", he has learned that he should not hire you.
> There is a lot of variance in bus sizes, but not enough to put you off by more than an order of magnitude.
I didn't know that 'within an order of magnitude' was an acceptable degree of accuracy.
> Knowing what kind of accuracy is required is sort of part of the test. The parameters are vaguely specified, so you should know that you're looking at orders of magnitude.
I don't see how I would know that. I would just assume it was a stupid question.
>> I can just say 'two thousand' and that's a good enough answer.
> That would fit in a large duffle bag. If you told me that you thought you could only fit 2000 golf balls in a double decker bus, I would assume that you either put zero thought into answering the question, or that you did not have a good grasp of how to estimate things.
You're right that I didn't put much thought into it. But perhaps that's all the thought this problem requires.
> The interviewer wants to know that you can give sane estimates based on rough calculations. As the article explains, there are many, many situations where a programmer needs to be able to do this kind of thing. It's not about "seeing how you think". It's about seeing that you can think – at all – about estimation problems.
There's nothing to think about. It's not a well-formed problem. The interviewer is going to see me putting on a show, and all they'll see is that I can act.
Sometimes as an engineer, you have to make reasonably accurate estimates based on very little data. If you think that concept is stupid or not worth thinking about, you shouldn't be in charge of designing stuff.
I don't mean this as an attack against you, but your response to this is exactly illustrative of why it's a useful interview question.
As I said in my original post, I'm never comfortable acting on as little data (basically none) as is in the original question.
If you are, then I would go so far as to say you are the type of engineer who ends up creating problems other, more thorough engineers have to solve later on down the road.
So you believe that you should exhaustively explore 100% of your options, collecting rigorous data about every possibility, no matter how numerous the options are, rather than narrowing down your possibilities early in the game and focusing on the best candidates? Now that is a waste of time.
Furthermore, sometimes your data is inaccurate, and that means that you have to build in error margins in your estimates. Doing that uses exactly the same skills as making rough estimates based on vague information.
There are just too many possibilities here. We don't even know if it is a double- or single-decker bus. If we worked for a company that actually did do this kind of thing (fill things with objects), we would not start making estimates and assumptions based on so little information.
There are good reasons to make assumptions early on in a project when narrowing down the solution space. You have 10,000 golf balls and you need to get them 5 miles south of here. You ask your coworker to look out the window at your company parking lot and tell you what vehicles you have at your disposal. A sedan, a double decker bus, an SUV, a small trailer, a motorcycle.
If you have reasonable estimation skills, then by the time you get out to the lot with a tape measure, you will have narrowed the search down to the sedan and the SUV. You don't need to bother measuring the double decker bus because you know that it's around 100 times as large as needed. You don't need to consider using the motorcycle because that is clearly far too small. If you collected data on those options, you'd be wasting time, because in the time it takes to collect the data, you could have already ruled them out.
"OK," you say, "but it doesn't hurt to be thorough." But see, it does hurt to be too thorough. Every second you waste on deciding not to use the bus is a second you could have spent making a better decision between the sedan and the SUV.
But it doesn't matter what the aim is. The point is that estimation based on very little data is important because real world situations call for narrowing down the solution space quickly.
I agree with the need to assess one's ability in Fermi-style problem solving, but I find fault with using non-sensical information for the topic. Seriously, golf balls on a bus? Why not ask about a relevant problem to be solved?
For me, an interview is a two-way street, and a company can most certainly put itself out of consideration with these types of questions. The biggest issue I find with these esoteric-problem-analagous-to-something-relevant types of interview questions is that they usually send a negative signal about the company. I immediately think of gamesmanship, trick questions, watching candidates squirm, or about a dozen other things that I find have nothing to do with the entire reason I might consider joining a company.
While the interviewer's intentions sound very noble, I find the means to the end more risky than useful.
I've found that usually once I get asked one of these, that's the end of the interview. Why? Because I epitomize "think outside the box". You see, there are a number of different bus styles - are we talking a BlueBird full length or their Handicapped Plus series - ways to fill the bus - after all if I am allowed to tamp and pack I'll get a lot more of them in and we need to account for that - and of course golf balls are not all the same size - though they are close but when you're talking literally thousands of them the differences add up.
Another favorite of mine is the "scored" bar of gold to pay someone's salary. First off, real gold is kind of soft. I guarantee that if I have the tools to build something I need to pay for, I can cut the bar of gold. Even if I didn't, gold can be dissolved by a number of different acids.
> I've found that usually once I get asked one of these, that's the end of the interview. Why? Because I epitomize "think outside the box". You see, there are a number of different bus styles [...]
That's IMHO the wrong approach. Everyone who's thought two minutes about this problem realizes these issues. None of these are a show-stopper to making a reasonable estimate. It would be completely acceptable for you to say 'Okay, I'll assume a sort of schoolbus like in the Simpsons, let's say 12 rows of seats, we'll need about 2m of vertical space plus some extra for when the driver goes over a speedbump so tall people don't hit their head', etc. The point is whether you can make a quick estimate, so that when in a real work situation you are asked to do something tricky, you can work out whether it's feasible or not.
How do you know that it's OK to make assumptions like that? How do you know that a quick estimate is acceptable? Is this level of accuracy acceptable? How do you know?
I can't get the point of this.
The interviewer wants to know how many golfballs fit on the bus, but he doesn't even know what kind of bus? He hasn't decided what things are on the bus, and how many of them, and whether there is a driver, and what kind of track he's going to be driving the bus on? And he probably doesn't know why the golfballs are being put on a bus at all? And he doesn't care as to what kind of accuracy this estimate is being made, as long as it just seems I've gone through some kind of motions?
This is ludicrous. It doesn't resemble any real-world task.
'We have an idea for a game where you walk around a real-looking representation of London, with realistic traffic and people on the side walk. We'd like to have this on the iPhone, PS Vita and Nintendo 3DS. We want you to write the engine for it.'
So, what do you do, start developing and hope for the best?
The fact that it's balls in a bus and not polygons on a screen is not a negative. It makes clear that the point is not to test your knowledge of the RAM of each platform, the point is to see if you can reason while facing unknowns.
> I can't get the point of this. The interviewer wants to know how many golfballs fit on the bus [...]
No, no, no. The point is to see if you give up (a large proportion do), or if you make bad assumptions (e.g. hand-wavy 'Lets say there's 1000 golf balls in a cubic meter').
The polygons onscreen question is a far better one, as it gives me a clue about the reason for asking, the kind of assumptions I can make, and the level of accuracy needed. The golf balls on bus example does not.
Really? Because the times I've tried to "seriously" answer such questions it became apparent the person asking was looking for "THE" answer. In fact I was halfway into the effect of having seat belts included when the interviewer started gently feeding me "the" answer they were looking for. Why? Because they felt that there really was a "right" answer that folks could agree upon. Not surprisingly, said company folded two years later as I gather their client didn't agree...
Ah, in that case, you're quite right to walk away! It's just meant to see if someone can 'put their arms around a problem', a rough way of weeding out people who can't make a move until they have every answer, a blunt tool.
Sounds like you used it appropriately as a blunt tool to weed out the company :)
Yes. And what's wrong with any of those things? It's a give and take - the interviewer will say "Ok, let's stipulate it is full length and you can't tamp them" and "Oh, that's interesting, assuming some sort of variance in size, estimate a range instead of a number", etc. That's one of the reasons made-up questions are better than "real" questions, the imaginative tit-for-tat is a good thing, and is harder with "real" questions.
Is the bus in space? Is it a toy bus? Is it a bus for aliens? Can I assume the bus is perfectly spherical? Is the bus a ghost? Do I have a ray gun that can shrink and enlarge things? Are the golf balls made out of water? Do kids keep coming along and stealing golf balls from the bus until it is perfectly full? etc.
Interesting, how would that affect the answer? No, it is a real bus, like the ones they use for humans in public transport. Interesting, if you're curious because aliens might be a different size, no, it is a bus for human-sized things. No. Nope. No, they are normal golf balls. Maybe, after we estimate the number, perhaps we can estimate the time-to-fill based on various rates of child-miscreant golf-ball theft. etc.
Is that a good thing or a bad thing? If you're really trying to show your outside-the-box thinking skills, you're gonna get as good as you give and it could be fun. If you're just being a jerk and trying to frustrate me then yeah, you should just politely stop the interview and go home.
Seriously, golf balls on a bus? Why not ask about a relevant problem to be solved?
Asking about an irrelevant problem makes it clear that this question is about your logical process and not about your knowledge related to this job. It's a way of isolating variables.
The actual golf ball/bus question is a bit too much of a cliche to actually use, though. I'd rather ask how many raccoons you could put in a battleship.
Oh, I get the point the interviewer wants to reach -- using size estimation questions to gain an understanding of one's logic process. But as a method of achieving that goal, I've found it's a poor instrument.
Suppose candidate #1 is unimaginative and cannot offer a reasonable response.
Suppose candidate #2 says "well, a golf ball is such and such size, and there is empty space around each ball, and the bus is such and such size....."
What, exactly, have we learned here? In my experience, nothing.
Having gone through this cycle as both a candidate as well as hiring manager (not my choice to use this line of questioning), the real-world results I've found is little to no correlation between candidate responses and accurate assessment of logic process. The false positive/negative outcomes were significantly higher, and the success rate for candidates into the positions we hired them for were mixed. Basically, these questions offered us nothing in finding candidates who would be successful in the position for which we were hiring.
If I were to ask a question such as this, it would be to gauge one's reaction to the question (assuming it's a stressor) and how they react. The response is nearly irrelevant. But I don't like the tone the question sets between myself and a candidate, so I just don't resort to this as a means to achieving that goal. There are better ways to get at that information.
What variables are you trying to isolate and why? And how does asking about golfballs on a bus or raccoons on a battleship effectively isolate those variables? What was the thought process used that led to that solution?
"Asking about an irrelevant problem makes it clear that this question is about your logical process and not about your knowledge related to this job."
1. But it doesn't make it clear (that my logical process is being tested). This problem fails in its purpose.
2. I don't have a set 'logical process'. I just try to do my job, and do what I need to do to solve problems.
3. You should at least try to come up with a reason why golf balls are being put on a bus. I don't know if I can engage the problem-solving part of my brain if there's no real problem I'm being presented with.
Well I hope you make it clear exactly why raccoons are being put on a battleship, so that the candidate has a clue about the kind of logic he's expected to use and the level of accuracy required.
I always assume the reason that you ask these questions is because you don't have anything interesting to talk about in the interview.
Interviews are as much a chance for a candidate to disqualify the interviewer/company because they asked stupid questions, as they are for the interviewer to ask stupid questions.
You got me all the way out here, spending my valuable time talking to you, and all you can come up with is a lame question about golf balls? Sell your company to me. Show me that you are excited about what you are doing. Talk to me about what problems you've been solving lately and try to see how I would go through the process of solving those problems. Sure there are NDAs and you don't want to violate that, but programming problems are programming problems, and if I am spending my time talking to you you can do me the favor of talking generally about what you are doing and making it interesting for me.
Interviews are as much a chance for a candidate to disqualify the interviewer/company because they asked stupid questions
Is this really true? Generally candidates know far more about the company than the company knows about the candidates. I think any notion that it's a meeting of equal interests is self-empowered fantasy.
"as they are for the interviewer to ask stupid questions.*
I ask "stupid" questions of candidates. One of the reasons I ask those questions is, quite honestly, because I want to exclude those people who have an attitude about "stupid" questions. Those who can't professionally humor simple questions are seldom the people anyone wants to work with.
And for the record the people who are really in a position to essentially set the terms we don't even interview (beyond meeting with senior management to give them the sales pitch).
Regarding the "Chance for a candidate to disqualify the company", yes you can know a lot about the company, but often you don't know how they work internally. An interview can give you a good idea of how the company works and that not something you really have chance of known, unless you know people how are or have been in the company.
I've never been to a job interview outside Denmark, and the bus with golf ball type questions isn't something I've ever encountered. What I have learned however is that if the interview is conducted by the HR department, something is wrong. HR can be present, but for developer interviews it should be your future boss and technical personal doing the interview.
Are you sure you aren't conflating "having an attitude about 'stupid' questions" with "simply rejecting 'stupid' questions"?
It's possible to reject stupid questions in a way that is diplomatic and non-douchy.
I'd think that being tolerant of accepting 'stupid' questions is as undesirable as having an attitude about them. The former demonstrates unproductive compliance, while the latter demonstrates unproductive social behavior. Surely there must be an optimum point where someone can behave in a way that demonstrates productive non-compliance, no?
It's possible to reject stupid questions in a way that is diplomatic and non-douchy.
Absolutely, and that's how professionals deal with other professionals. The comment that I responded to demonstrated significant attitude about what they called stupid questions (which are the Fermi problems).
An interview is not a skill discovery session -- ideally your resume and online presence (e.g. github projects) has demonstrated that, along with, if necessary, a practical test. An interview is getting to know how you think, how you communicate, how you deal with people, how you approach problems, etc. It is astonishing how many people don't understand this, and thus don't understand questions that aren't an autism-level focus on a person's specific skill gamut.
Can you tell me what you believe your biggest weakness is?
I don't have an attitude about answering 'stupid' questions if they are asked in an honest manner. I have answered and asked many 'stupid' questions in my lifetime and will answer/ask many more. Asking stupid questions is a part of learning. You are reading far too much into my use of the word stupid.
My point is that if I agree to an interview, I would much rather have an interesting technical discussion with one of your engineers than be asked brain-teasers by someone from HR.
I would politely answer the question and would not dismiss it. However I find the practice off putting in general.
Fermi problems are a game that give you an estimate of the person's thought process. Another way to gain an estimate of a person's thought process is to engage that person in intelligent conversation on just about any subject. I much prefer intelligent conversation, as it can actually be quite pleasant.
My biggest weakness?
The interviewer clearly is interested in something contrived and job related. But what are they looking for? Do I spin it and say how "my biggest strength can sometimes be a weakness when..." Am I honest and tell them what my actual job-related biggest weakness is?
If I just straight up tell you something like "I'm incredibly disorganized and never show up when you tell me to," I'm clearly not going to get the job. You don't get any points for honesty. Do you want something where the interviewee has spun his biggest strength? It is dishonest but a better answer.
Asking my biggest weakness is really just you asking me 'What do I want to hear.' Another game... I don't know you very well; how could I possibly answer it? So there it is, my biggest weakness is reading your mind. :)
Or maybe my biggest weakness is over-analysis. In what context of the words 'biggest' and 'weakness' would you like this question answered?
Size/importance/grandeur/impact?
Mental/medical/social/skill competency?
Literally speaking, it would probably be the oxygen supply on earth. It takes up quite a lot of volume, and if it is removed I'm a goner.
I often do ask at least one of these questions on an interview. I don’t do it because I care about a precise answer or whether you know the exact dimensions of a golf ball. I care simply because if you can’t do Fermi calculations, you can’t make long term architectural decisions. You’ll build a system which handles 2x today’s load very nicely and which I need to replace in 2-3 years, or you might overarchitect a system which can handle 1000x more load than I’ll ever need.
...why not just ask a question about building a real-time monitoring system itself and judge the responses there?
i would never pass this test. i would solve this problem using variables for dimensions of the bus and a constant "packing factor" (or assume the optimal cannonball packing and approximate the golf balls as spheres) and write an express formula for the solution. then i would substitute in various constants for the dimensions and adjust the packing factor to find a range of solutions.
the problem with the question is they want an intuitive solution. giving them this formula and evaluating for a range of variables would just piss them off, but it's the best way to approximate the answer of something that poorly defined.
> ...why not just ask a question about building a real-time monitoring system itself and judge the responses there?
agreed. these abstractions for their own sake don't help, make the candidate feel like they're jumping through hoops for no reason
and they're intrinsically less useful than simply talking about real problems you've faced; you can't follow up an answer about golf balls down various interesting avenues of discussion when a candidate touches on some other area, because your job is not packing golf balls into a bus.
i would never pass this test. i would solve this problem using variables for dimensions of the bus and a constant "packing factor" (or assume the optimal cannonball packing and approximate the golf balls as spheres) and write an express formula for the solution. then i would substitute in various constants for the dimensions and adjust the packing factor to find a range of solutions.
Oddly enough, this was exactly how I was taught to solve Fermi-like problems in my physics program. The point was not so much to be able to "intuit" the answer as to learn how to find approximate solutions with little or no real data. You could do this sometimes just with off-the-cuff estimates (as in the piano tuner example in https://en.wikipedia.org/wiki/Fermi_problem), but often enough we came up with approximate formulas and guessed at the constants.
I haven't been in enough startup job interviews to know if actually doing algebra would make them think less of me, but that would really seem pretty dumb.
that may be an unfair assumption on my part. i just assumed that when people ask these abstract riddles, they're really looking for some way of testing your intuition and not something this mundane and boring.
it really boils down to basic algebra with some constant fudge factors. meanwhile, cs people have a host of sophisticated tools to estimate asymptotic behavior for the worst, best, and average case, or error bounds on solutions to NP complete problems as a function of the number of iterations of approximation algorithms, or trade-offs when certain pre and post conditions on the data influence the run-time of canonical algorithms (think searching).
the basic toolset of computer scientists is so advanced i guess i'm shocked anyone would ask such a relatively simple problem when there are much better ones out there.
Warning: I don't actually do web startups myself, so this is based on an outsider's view of the field.
Computer science provides some really awesome ways to deal with algorithmic problems... but for most startup-type applications, those problems only come into play in a few places (if that). A cool time-management webapp might have an interesting algorithm at its core that uses machine learning to auto-decide what tasks are most important (making something ridiculous up), but the rest of the app is basically just a shiny interface to a database.
I really doubt the problems of scaling a CRUD webapp (at less than Facebook scale) lend themselves to good mathematical analysis. Even if they did, these apps involve lots of third-party layers glued together to put together interesting functionality, and those seem to change often enough that it's not worth even trying detailed analysis.
So instead they do Fermi-problem estimates to figure out their basic scaling relationships. Then you can decide how to plug things together, decide what your basic resource constraints are (compute, memory, disk, bandwidth...) and make it easy to add those resources when your customer base expands.
Edit to add: Thinking a little more, that's exactly when you'd want a set of approximate algebraic constraints, so you could decide how many customers-per-EC2-instance you could serve and how that number might change as instance count increased. Math, it's important.
at my large company i do this every day on the embedded system i work on, which luckily is not a CRUD app. i guess i would hope they would ask me about that XD
databases can get tricky. but you're right, if something complicated like a bloom filter is used to solve subset-query it's probably in an third-party API they grabbed somewhere that handles most of the difficulty.
The last thing I would think is desirable in a job candidate is their acceptance of problems that are poorly defined and that at best have dubious benefits.
Trying to come up with an approximate answer to a question like that without at least questioning the problem itself or at least trying to narrow it down to something well defined is not a desirable trait in someone on whom you are bestowing responsibility.
I would look more highly upon a job candidate that questioned why anyone would ever want to pack a bus with golf balls to see if there is even underlying benefit to actually doing so. I would expect the interviewer to be prepared to give a pretty good business reason for how packing a ball with golf balls will increase revenue or customer satisfaction.
that formula narrows it down pretty well once the constants are defined and takes about a minute to determine. with more time than is found in a typical job interview you could actually go much further.
the real criticism of the question is that the real skill they're trying to test is better determined by directly asking about Fermi calculations relating to computer software and hardware.
the golf ball question could be interesting in it's own right, but it's asked in a context where additional information to optimize the solution is not available and an optimized solution couldn't be determined due to time constraints. hence the formula sucks and you end up simplifying it with fudge factors that only demonstrate that you are able to use basic algebra to model the volume of a bus and a golf ball.
there are numerous practical reasons you would want to solve this problem. for example, say you work in an orange factory. you want to pack oranges of radius n. perhaps you would like to determine and minimize the amount of shrink wrap you have to stretch around them for transport, or keep track of the materials cost for accounting purposes.
that's essentially the same problem and uses the same formula, you're just solving for the dimensions of the container instead of the number of golf balls. you can save a lot of money minimizing the wrapping material if you pump out hundreds/thousands of packed oranges a day and choose an optimal packing.
alternatively, if you have to ship it in a non-optimal packing like boxes, perhaps it's good to calculate the materials cost to pack your product to estimate future expenses.
that article lists other problems where sphere packing is integral to the solution. just because an application doesn't seem obvious doesn't mean there isn't one.
if someone honestly expected me to challenge this interview question without understanding how it could be useful i would rather not work there. people have been studying this problem for practical reasons ever since they had to stack cannonballs for transport, it's not exactly a contrived example.
I understand the value of sphere packing (but IMHO ellipsoid packing is more interesting), but why not just ask the orange packing question instead of golf balls on a bus?
A question about orange packing in an orange factory simply does not elicit the reaction of "let's see if this monkey can dance" which one would have if asked about golf balls on a bus.
An intentionally absurd question, even if it is an abstraction of a non-absurd question, does not make that question any less absurd. If you want to abstract, abstract, but do not abstract then unproductively obfuscate, because when you do, you've taken a perfectly legitimate problem and made a trick question out of it.
I don't have a single clue how many fashion items are on sale at a given time, not because I'm incapable of estimating it (even though I've never heard of Fermi problems before), but simply because I have no field knowledge about online store business.
Give me a day to think/research if you really want to make sure I'm the right guy for the job, because in the real life, I have the luxury of not being forced to come up with an answer in 3 minutes in a job interview (that would very well change the course of my life).
With all respect, I think all your questions are irrelevant and are definitely not a good metrics for hiring/filtering out candidates. For all means ask them about clustered/non-clustered (to use MSSQL's terminology) indices or stuff like that, but don't put them in a position like this one (they don't know the answer, but given a few days they can give you a pretty accurate estimation).
Just another rant: I have a very big problem with such estimations. It's a bad analogy, I know, but I want to use it just for the heck of it:
Such of estimations (how many would view a newspapers webpage on a given time - completely glossing over its popularity: is it a good local newspaper, or one that is just going out of business? for all we know, a very good local newspaper might get 1/5th of page views that BBC gets (it has a vocal community that tend to comment on stories)) just remind me of those stupid VCs that helped create the Color iOS app. What was it? 100M dollars just for the domain (color.com) or something like that. I'm sure they've used the same kind of estimations: Facebook gets xxx hits per second. We'll be a premium social network and everyone would want to be on our network just like they did on Facebook when it was a harvard-only club, etc. so we could be a 5 billion dollar company in 9 months so a 100M dollar domain name is justifiable.
I don't expect you to know the answer. I expect you to tell me how you would research it.
An answer I like: "I'd search on amazon and a few other top fashion sites to get per-vendor count, as well as some of the lower end."
At that point I'd tell them that ShopBop and Net-a-porter are two typical bigger sites, and they each have about 20-50k items. I'd also point out there is a longer tail of much smaller sites.
If your answer is 3M or 20M to my 10M, I don't much care. This isn't a question with a definite answer, it's a question about whether you can think on your feet and get started on poorly defined problems.
With me, it's a good sign if I'm asking you the poorly defined questions with unclear answers. If I reach this point, I've already decided you probably know how to code, and I want to determine if you can figure out what to code.
> If I reach this point, I've already decided you probably know how to code, and I want to determine if you can figure out what to code.
Well, if that's the case, then it could be a good question.
However, you didn't state or emphasize that in your blog post and from what I've heard, many companies ask such unrelated questions first to filter out completely hopeless candidates and then get to the real interview (technical questions), so I assumes this is what you meant, and I don't owe you an apology ;-)
This is dumb for one simple reason, It divides candidates based on an arbitrary line that has only limited correlation to the desired split: Those who fit your job requirements vs. those who do not. Instead, ask them to solve a pertinent problem to your company. If they go down the wrong path, give them a hint as to the right track and see how well they can grasp the new method. Simple. OR even better ask the candidate to explain how they solved their own last 'interesting' problem.
You're asking the wrong question. If an interviewer asked me "How many golf balls fir on a bus?", the only sensible answer is "I don't know.", and if the interviewer would expect anything else, I'd call him an idiot.
The correct question is "Estimate how many golf balls fit on a bus." I.e. I'm not expecting the correct answer, I'm expecting an order-of-magnitude estimate.
This specific question is however inappropriate even with the estimate wording, since 1) busses differ (double decker, how many seats vs. standing space, ...), and 2) golf balls are round. This means that the simple estimate "how many 2 cubic inch things fit into a x * y * z box" (as attempted by another commenter below) would produce wildly incorrect results.
Not to put too fine a point on it, but if you made this comment during an interview, we wouldn't hire you unless looking for someone to sit alone in a cubical to write device drivers.
These type of questions belong in the "Weekly Puzzler" part of the Car Talk radio show.
Honestly, what's the deal with them? Do you have an urge to somehow feel smarter than the person you are interviewing?
Why don't you instead do the following:
Sit the programmer down and show him your source code. If he can make any sense out of your spaghetti, then hire him/her. That's all you are looking for, right? Someone who can understand what it is you are doing so he/she canhelp you to do it faster.
No other industry has to deal with this nonsense. Imagine if auto mechanics had to guess how many miles does the average Formula One engine piston travel in a scheduled race. We would be riding around in bycicles because no mechanic would take that.
Maybe I can add some context. I was the OP that started this whole discussion about posing math problems / Brainteasers at interviews.
Out of the blue I got a call from a recruiter saying that bank X would like to interview me for a quant role they have. They wanted to fast-track the process so they booked me for 2.5-3 hours of back-to-back interviews. I was not looking for a new job but I thought it might be good to talk to them and explore leaving academia.
The first interview was with some quants who work for the hiring manager. They tell me that they plan to have a technical talk with me and then follow that with some math problems and brainteasers. At that point I respectfully declined saying "I wasn't aware that I was going to be asked those types of questions and I don't really believe they are a good hiring tool". The guy tells me too bad, that's the way they hire now. I say fine and we carry on with the first part of this session and have a technical talk.
We then come to the point that they want to ask me the math problems. I again say that it's silly. They disagree. Then I say "For example, who's to say you're not going to ask me problem A". They laugh. "followed by problem B, C, D". They stop laughing. "Finishing with the problem about the expected number of tosses of a coin to HH". They look at each other. "and who's to say that I don't know the answers by heart. It's luck of the draw, maybe I know your questions and maybe I don't and how would it reflect on me if I don't know". The guy who had problems ready for me turned white. They quickly ended the interview in a cordial way at that point and seemed to scramble to find the next two guys to come and interview me. Probably because the interview finished 30 mins earlier.
What they didn't realize is that I love these types of problems and that it is a small community. I know a lot of people and remembered what questions they had asked one of my friends. When they walked in the room, the guy had the solutions written out on a piece of paper and with only a brief glance I knew that I was going to get asked the same ones.
He had paper with the solutions face down during the whole interview but when he left I have never seen anyone pickup a piece of paper in a more awkward way. He did the whole fold in half as you lift thing.
A bigger problem I have with these problems is that I do not feel motivated to solve or even think about such problems. My internal reaction therefore always is, why do I care how many golf balls fit, even if I do not say it out loud.
For whatever problem the interviewer poses me, I would like them to supply me the context around that problem so I can see what is the benefit of solving it. Solving it for the sake of the interview is not a valid answer for me.
There are a plenty of important problems to be solved for me to not waste a minute on a useless one.
I don't want people who solve problems because they're important, I want people who solve problems because they're intensely curious and just love solving problems.
Without knowing why the problem is important, you do not really understand the problem yourself. Understanding the problem better is often even more challenging than solving the problem (once understood). I have seen people go circles around trying to solve the problem at hand without reaching the solution that actually solves the problem, just because they did not understand the problem correctly to begin with.
There indeed are curious problems. Estimating number of golf balls that can fit into a bus is certainly not one of them. Try to come up with a curious problem, then we'll talk.
The author gives a reasonable example of the usefulness of this skill.
My problem with these questions is when they come out of a box delivered by Cargo Cult Airlines. If you or your organization don't actually use these techniques in the normal course of the job, then don't ask these questions.
"I just want to see how you think." OK, fair enough. After I've answered, describe how I think, and compare and contrast that with how other candidates think. My bet is that most people who ask this question have no idea how anyone thinks, including themselves. Most people.
Column 7 of Programming Pearls (which can be read online, http://netlib.bell-labs.com/cm/cs/pearls/) is an interesting exploration of the actual use of this kind of exercise.
When you get a question about golf balls, don't just declare "Bogus." Consider who's asking it, and whether they seem to know what they're doing with the question.
The question is valuable to the candidate, and not just to identify lame companies. Face it, most people just don't know how to interview, and if you decide based on this, you're being a lazy interviewee. In rare cases, the question might even expose a company with a clue.
It's pretty simple really. The answer does not have to be right. Just talk your thought process through... how big is a golf ball... maybe 2 cubic inches. I wonder how many cubic inches are on a bus... how big is the bus? Well, it doesn't matter... assuming the bus is 20 feet long and 7 feet wide, and 8 feet tall there would be X cubic feet in the bus some of which would be taken up by seats and other things so we'll say X minus that to get the total number of cubic feet that we can put the golf balls in. Now the question becomes, "How many 2 cubic inch things can you fit into X cubic feet?"
That's it really. Just think it out and solve it like you would any other problem. If you can't talk it out and sort of layout how you'd go about solving it, then you're likely not a good candidate for some jobs.
I think this is really what it is. There's this trend in the valley of everyone thinking they need ninja zombie pirate rockstar gurus who can solve P=NP and interviewing accordingly, when they're really writing simple CRUD apps and just need someone that knows how to do solid architecture and design.
Asking stupid questions like golfballs on a bus sends a signal of "we need geniuses for some reason" and is intended to make the candidate think "wow, the interview questions were so hard, everyone that works here must be really smart!"
It also means that the person asking the interview may not be the technical hiring manager. I expect such questions from someone in HR, but not from a technical hiring manager.
Someone in HR most like can't judge the answer to the question "How would you scale X, given Y and Z", nor could they come up with such a question that is appropriate for the position in question.
Such questions do more to isolate the variables, (1) the job to be performed, and (2) the interviewer's ability to judge the answer to a question relevant to the job. I can really only see such a question being asks by somebody that doesn't know why they are doing the hiring in the first place.
Judging by the responses in this thread, such a question is controversial, especially among the type of people that make up the HN community. That alone should be indicative that such a question may result in a high rate of false negatives. You probably lose more good candidates than gain.
I was thinking the same, but it'd be funnier to take it one step further:
"Short bus? Long bus? Double decker bus? Coach bus? VW Bus?" Since I'm not sure of the relevance of the question, I choose a Matchbox Bus. The answer is 0. 0 golfballs fit in a matchbox bus. Enough bikeshedding. Let's move along to questions that will more reliably assess skills I will be using on the job as a software engineer."
"Bob Slydell: I'd like to move us right along to a Peter Gibbons. Now we had a chance to meet this young man, and boy that's just a straight shooter with upper management written all over him."
All questions like this would tell me about a candidate is that they have a high tolerance for bikeshedding.
> "Short bus? Long bus? Double decker bus? Coach bus? VW Bus?" Since I'm not sure of the relevance of the question, I choose a Matchbox Bus. The answer is 0. 0 golfballs fit in a matchbox bus.
You never know how far they want you to go with their game. This could mark you as a genius, or as an ass.
the point of the article is that that detail doesn't matter. short vs long might double the answer, but orders of magnitude are the only interesting answer.
however, I feel that the abstraction away from something connected to the job you're hiring for is more harmful than useful - definitely ask those questions, but don't phrase it in such outlandish and clearly designed-for-interviews formats.
I've asked candidates to estimate the number of RSS feeds on the internet. I've found it a useful question which offers a lot of avenues of discussion and means to probe thought processes - and it's connected to my work.
Despite the questionable wisdom of it, right there in your answer is exactly why these questions get asked: you've just given the interviewer a mountain of information about you.
I've probably sat through 500 or more interviews over the last 15 years, and the percentage of candidates who would approach such a question with a blank, not incredulous, but blank look is utterly astounding.
Everybody knows there are different kinds of buses. It is the first thing that springs to everybody's mind when you ask "How many golfballs fit on a bus?".
Someone who didn't ask about what kind of bus is under the impression that, as it's just a toy problem, the kind of bus doesn't really matter, and they're just supposed to pick an average kind of bus and play along with the stupid game.
You're making a number of assumptions, most of which I believe are wrong.
a) No, it's not the first thing that springs to everybody's mind. Experience says it's the first thing that springs to a relatively small percentage of minds, actually.
b) No, someone who didn't ask about the kind of bus isn't necessarily under the impression that it's a toy problem. Again, experience says that for a large percentage of people the problem, or more specifically any problem of the ilk of this problem, is completely overwhelming.
> Experience says it's the first thing that springs to a relatively small percentage of minds, actually.
Can you explain in more detail about what experience lets you know this?
> No, someone who didn't ask about the kind of bus isn't necessarily under the impression that it's a toy problem. Again, experience says that for a large percentage of people the problem, or more specifically any problem of the ilk of this problem, is completely overwhelming.
Also I'd like to know how you know these people are 'overwhelmed' with the problem itself, rather than just with not understanding the point of it or not knowing whether its OK to make assumptions or ask questions about the type of the bus etc.
It's not about getting the "right" answer, it's about how you approach tackling the problem. If you talk through your solution as you do it, no one is going to question which assumption you make. The assumptions are baked into the answer, but part of the point of these kinds of questions is that they don't matter that much. Or more to the point, the uncertainty in your answer is on the order of the uncertainty of your assumptions, which hardly matter whether you assume a 25 ft bus or a 50 ft bus, if you think the precision of you answer justifies confidence better than a factor of 2, you don't understand the problem as well as you think you do.
And the seats really don't matter, volume-wise. Unless you're actually counting the golf balls, which, I agree, would be silly.
Instead of asking me about golf balls and bus, would you not have consideration for my intelligence and ask me a question that pertains to the problems your company faces, and see me analyse it in various contexts, leading to a richer interview? This way I will know something more about your company and its work too, and you will know how I think about problems. In your case, respect me enough to tell me your problem with number of clothes and such and test me against it. Golfballs are Golfballs, your company's problems are what I am hoping to solve. In that sense, you and your company will earn my respect if you treat my time and insight as something valuable. I will not solve golf ball problem for some myopic "insight" of questionable value.
If a company is solving interesting problems, I think that they won't even be thinking about irrelevant problems for an interview.
Assuming a company has interesting problems that they are solving, why would they willingly leave their interesting problems at the door instead of using those problems as the basis for their interview dialogue.
The best interviews are a dialogue (not an interrogation!) where both the interviewer and interviewee spend about equal time talking. Questions where the interviewer spends a few seconds asking the question and the interviewee spends several minutes answering is not demonstrative of real life collaboration, and at the day the purpose of an interview is to recruit a potential collaborator.
I don't see the problem with these questions. If you are really good it will take you 5 minutes to answer that question. That is it. And lamenting about interviewer waisting your time is ridiculous, you waist much more time every day on the stuff that is less useful. The point of the question is to quickly check if you can think logically that is it.
It isn't a well-formed question. There's so many different kinds of buses, there might be furniture, it might be toy bus, the doors might be open, it might be an alien bus, it might be full with people or superglue; Who knows? The question is so loosely-formed as to be nonsensical. You might as well ask "How many/much object(s) can fit (insert spatial relationship here) (a/an/the) thing(s)?"
Well to me it seems like a lot of questions in the real world are formed like this. I.e. if you are asked to find how many customers will buy a book of your website you make some reasonable assumptions like:
The world will not come to an end tomorrow
You don't ask a person what should be defined as a book or what should be defined as a buy
etc.
What I mean is that it is good to have the ability to research the common case. And imagine the question about balls was a real problem that your company faced, I don't think you would ask your colleagues all the questions that you pointed out.
But in this case (golf balls on a bus), I don't know what kind of assumptions I can and cannot make. It isn't a real-world problem, so I can't use my 'common sense' to reason about its constraints.
Hm. I've never faced any such problem. But then, I do uninteresting stuff like deciphering horribly documented APIs and writing desktop software.
If I go into an interview for a desktop software role and they start asking me Fermi questions, I'm probably just going to walk out. In fact, if they ask me questions that look like dynamic programming, I might walk out, too. Why? Because I know they aren't ever going to present me with such problems, and they're just cargo-culting their interviews.
If...they start asking me Fermi questions, I'm probably just going to walk out...dynamic programming, I might walk out, too...I do uninteresting stuff like deciphering horribly documented APIs and writing desktop software...
Maybe if you stop walking out of tough interviews you'll find a more challenging job.
...real life examples which have absolutely nothing to do with any of the jobs I've ever interviewed for. But hey, I'm sure you know about them better than I do.
And the question is stupid. Buses are different sizes and of different designs, you know. We have double-decker buses in the UK.
If you must use this class of problem, at least make it more related to your field or narrow down the kind of bus and what furniture is on it. And explain to the applicant why you would ask a question like this. "We want to see how you think" isn't an adequate explanation.