I don't really ask these kinds of questions, but I do kind of like them.. the giant puzzle/estimation types. Can someone who asks these regularly give some feedback on how well you think they work? I'm curious how much you end up learning from them.
That said, there are some doozies in this list. For example:
"On a scale of one to ten, rate me as an interviewer".
Um, okay but only if the next question is to rate you as an asshole.
In my experience these "Fermi questions" aren't that useful, the primary thing they tell you is whether the person has previously been exposed to the concept of "Fermi questions" or not.
I guess if the person being interviewed is 30+ and working in software development and hasn't been exposed to Fermi questions then maybe that's a red flag, but other than that I've seen no indication that answering these types of questions well correlates to real world performance.
I once interviewed at a place that had an interesting spin. They gave me three fermi questions, which I had 5 minutes total to estimate:
1. Mass of the earth in kg
2. US GDP
3. Number of carriages on the London Underground
I then had to give a 95% confidence interval for each one (i.e. an interval that I was 95% sure the true answer lay in) and then asked, for each estimate, which bet I wanted to take:
1. Bet that the answer is inside the interval at odds of 10 to 1 on.
2. Bet that the answer is outside the interval at odds of 30 to 1.
There's a lot of work that has to be done there, and potential for a very interesting discussion (this was for a trading/research role, rather than a software developer role).
Which would be more impressive: that I long ago memorized that the Earth's mass is about ~5.98E24 kg or that I could estimate it knowing Kepler's third law (T-squared = C * R-cubed) and orbital information about the Moon(28d@250,000miles), Earth (365d@1AU), and Mercury (88d@0.4AU), where 1AU=93million miles. Or that I could cross-check it by knowing the circumference of 25,000 miles to get the approximate volume, and knowing that the density is around 5tons/cubic meter? Or I could use the orbital numbers to estimate G then knowing that Earth's surface gravity is 9.81m/s/s then I can also get M_earth.
All of those involve memorized numbers, and most require physics knowledge. But I have a feeling that the more complicated ones would impress them more, and make them more likely to hire me. If that's the case, then my strategy is not to give the answer which I know, but to exaggerate the difficulty by hiding my actual knowledge.
GDP? It's about the size of the debt, which is something like 7 trillion, so I would have estimated $10 trillion. (Actually, the debt is $12, and the GDP is $15.) I might have double-checked my answer because I think the budget is about $4 trillion, and 20% of GDP giving $20 trillion. (Actually 3.54 trillion and 22.6% GDP).
How is this memorized knowledge useful? Especially since I don't know how the GDP is actually calculated? Nor the difference between GNP and GDP. So this mostly tests to see if I've paid attention to the economic news. It's almost like asking me if I know the hottest temperature ever recorded in Death Valley.
And as for the London Underground - too many variables for me to even estimate. Turns out to be 4134, and that what I thought was part of the Underground was actually rail services. If forced, I think would have said: 20 rail lines? (Actually, 11.) At rush hour, 10 cars separated by 5 minutes distance, for a total of about 90 minutes per line? Which gives 3600 cars. I probably would have rounded up to 4,000 for spares, and then been amazed at how close I was. (Ahh, I forgot the factor of two because trains are going in both directions. And I think the inter-train spacing is less than 5 minutes on some of the lines.)
But this only works because I've been to London and made those observations. Without that experience, I would be off by an order of magnitude.
So there you go - this proves that I have a degree in physics, that I follow the news, and that I've visited London. It helps with the job you applied to because ... ?
(And if you say it's because I can reason about these sorts of problems, then, see "have a degree in physics" and add "and also math.")
Almost all of the interesting stuff comes after you've made your estimates (although the process of getting to the estimates is not completely uninformative).
If the candidate performs well on this question, then they have shown:
1. They have the ability to reason about problems in a potentially new domain.
2. They have the ability to pare down a problem to what is important and what can be neglected.
3. They can make good order-of-magnitude estimates (very important for an equities trader).
4. They are aware of the typical size of their errors, or can estimate them.
5. The are aware of the typical sources of error.
6. They understand the notion of a confidence interval and can apply it to real-world situations.
7. They can reason about bets with different characteristics (e.g. different expected value, different skew characteristics) and choose a good portfolio of bets.
I think that knowing how GDP is calculated is useful information for someone working in finance. Especially someone who is going to have to trade through economic announcements. Would you hire a trader who didn't follow economic news?
PS re "shows you've visited London" - the interview was in London. I assume if it had been in New York, they would have asked about the New York metro.
I passed the first test on rote memorization. The second was nearly the same. Only the third would have given information about my ability to estimate, which isn't enough to give real information about those 7 bullet points. I remain doubtful about the utility of these tests.
There is nothing, for example, which probes my experience with statistics.
Here's a more in-depth test. We run an medical experiment on lab rats. The protocol says that we will experiment on 20 rats, and that a mortality rate above 30% is too high so we must stop the experiment. After 5 rats, 2 have died. Should we stop the experiment now? Why or why not?
One could look at that and say that a 2/5=40%>30% so of course it should be stopped - it's higher than the allowed rate.
A better response is to point out that if 10% is acceptable, then there's a 0.9^3 * 0.1^2 * C(5,3) = 7% chance that this is an expected statistical fluctuation, and that the experimental protocols should have given some guideline on how certain one needs to be in knowing if the mortality rate was too high.
An even better response would be to point out that this means the first three rats lived, and it was only rats 4 and 5 which died, as otherwise the experiment would have stopped earlier under this protocol. In that case the possible experimental states are: 1) first rat died, so the experiment stopped = p. 2) first lived, second died, stop = p. 3) first lived, second lived, third died, stop = p. 4) first, second, third lived, fourth died, fifth lived = (1-p). and 5) first, second, third lived, fourth and fifth died = p giving (assuming I recall how to do this correctly) p+p+p+(1-p)+p = 1.0 or an estimated mortality rate of p=0.3.
In other words, this experiment is allowed to keep on going under the given protocol.
But this sort of modeling awareness is not present in the three tests you mentioned, despite being directly relevant, I assume, to what a trader does.
Yes, this kind of statistical modelling is certainly very relevant. I don't want to give the impression that this was the only question I was asked, or that it covers everything you need to know about a candidate. The entire question and discussion took about 15 minutes. I was asked questions of a similar kind to the one you propose in the other 11.75 hours of interviews.
I watched a once-coworker give a 30 minute lecture to high school students on how to do these giant puzzle/estimation types. His trick was to take the geometric mean of reasonable min/max bounds at different stages of the estimation.
Example question: How many flights occur in America in a day?
You can start from the bottom, with an estimate of the number of people who fly, then the number that can fit in a plane, et cetera. Alternatively, you can start from the top at the number of airports and their flight capacity. With a bunch of sqrt(minbound*maxbound) and reasonable guessing, the students (via groupthink) ended up with two roughly matching estimates that were within an order of magnitude.
That said, there are some doozies in this list. For example: "On a scale of one to ten, rate me as an interviewer".
Um, okay but only if the next question is to rate you as an asshole.