The article implies that the interviewee assumes that the number is being chosen randomly, when Ballmer could actually be choosing adversarially.
However, if the interviewee assumes that Ballmer is being adversarial, then you can pick a different value as your initial guess, which causes the probabilities to shift. Even the OP assumes that the interviewee will start guessing with 50, but, because of the way binary search works, you can select an initial guess that is offset from 50 (with a randomized offset each time) to defeat trivial adversarial attacks that attempt to game the heuristic, while still mostly reaping the benefits of binary search.
I'd be interested to see someone do the analysis of what the optimal random-offset-selection algorithm would be to counter trivial adversarial choices.
It would not be shocking to find out a cocky interviewer posed a brainteaser while leaving out a fundamental assumption, then judged an answer as incorrect because it violated that unspoken assumption - I can imagine Ballmer saying "no actually, you have to start with a guess of 50, everyone knows that."
I worked with a guy like this. He told me this story to impress me with how incisive he is. Instead it told me he is an egomaniac. His story went something like this, I don't recall the exact details:
"I was interviewing a candidate who said he had experience programming on an IBM/370. So I asked him if you perform a character edit format instruction in EBCDIC mode with the leading zero specifier and the numeric value is too great to fit into the allocated field, after the instruction completes, what is the state of the program status word overflow field?" Then trounced the guy for not knowing. The thing is the guy asking the question happened to have worked on that instruction when he worked at Amdahl.
One thing to know is the IBM 360 and descendant family had a commercial instruction set option that, in a single instruction, could take a format value and generate a string output that followed some format specification, kind of like sprintf but with even more options.
> if you perform a character edit format instruction in EBCDIC mode with the leading zero specifier and the numeric value is too great to fit into the allocated field, after the instruction completes, what is the state of the program status word overflow field?
"Is the computer operating on American electricity, or European?"
Ah, the B type developer. Knows enough to find exciting and interesting problems but doesn’t know how to distinctly separate a type C (who can’t solve the problem at all) from a type A ( who knows the problem in and out and knows “it depends”). Not all that different to me from midlevel dev who learns about concurrency/metaprogramming/etc and starts using it as a tool for everything. Just enough to be dangerous.
Unless the interviewer has totally lost sight of the purpose of the interview, they’d recognise a candidate starting at an offset from 50 as an instant pass.
I think like 70% of interviews I ever had were like they were there to prove how smart they are and how stupid I am. I suppose most likely to make me feel stupid and accept lowball offer.
One of the worst interviews I ever had was just like that.
Often times, the "gotcha" part is just dumb and nonsensical, yet gives the interviewer(s) a sense of misguided (false) superiority, and wastes everyone's time. I would venture to say that 99% of the time it's complete un-indicative of how effective the candidate would be in the role.
Referring to my aforementioned bad interview - the question, after all the technical stuff had been cleared (this was for a junior frontend dev role) - they asked "imagine a car is broken and not running. how would you go about figuring out how to fix it?". Being someone whose brother and father enjoyed fixing cars, I asked every question about the problem with the car - how it was used, what sort of car it was, what the issue with the car was, what prior problems the car had, etc. I got a bunch of useless answers. After I exhausted all my questions, the interviewers told me I had failed. Why? One interview brought out a tiny hotwheels car with a missing wheel out of his pocket, and proclaimed to say "you didn't ask if it was a real car, it's a toy car, of fucking course it's not supposed to run like a real car!" while laughing hysterically. How on earth does that indicate if a junior frontend dev can do their job or not? Stupid.
I had an interview at a major tech company with a similar thing, for a more managerial role.
The question was to estimate how many vacuum cleaners there were in the city we were in.
Fine, I did some estimation of how many vacuum cleaners per household and per office, across how many households and offices. Standard stuff.
Then the guy starts laughing and saying I'd failed because I didn't include discarded vacuum cleaners in landfills. Or the vacuum suction devices they put in your mouth at the dentist's office. And so forth. And then had to spend the next five minutes listening to him "teach me" how not to make assumptions. So I acted all polite and tried to fake "oh gosh thank you so much for enlightening me!"
Shockingly, I got the job, which required unanimous approval from all interviewers. Never met him again, and to this day I still have no idea whether this was supposed to be a test of estimation (which was easy to pass), a test of not making assumptions (which is dumb, but OK fine I failed), or a test of being appropriately professional and smiling in the face of complete bullshit (which I'd say I passed with flying colors).
I mean, in my professional life I've certainly had my fair share of customers and managers and coworkers who spout bullshit and you really do just have to lie with a smile and say "oh my gosh you're so right thank you for explaining that, I appreciate you so much!" Where you need to make them feel smart.
On the other hand, I just don't think he was thinking that far ahead.
Is the landfill actually 'in' the city, or in a rural area outside the city limits?
I had a similar interview years ago - something like "how many windows are there on houses in our town?". Wasn't quite that, but I asked up front if "houses" meant just physical standalone houses, or if they meant living spaces, including apartments/dorms, etc. I got clarification, gave some estimate with some reasoning, and was then told I was the only person of the 8 they'd interviewed that had asked any clarifying question at all, which apparently impressed them enough to make an offer.
Yes, I have used questions like this before for junior roles and the notes for the interview were something like:
- Asked/did not ask clarifying questions
- Did/did not (or could not, on prompting) verbally walk through their reasoning
- Could/could not articulate which assumptions they felt were most important/why
Nothing about the actual content of the question itself, or if your answer was approximately correct (I usually did not know even the ballpark of the correct answer myself). I will say I did sometimes write down if candidates make comically bad assumptions. Like assuming the population of the USA was 1 Billion people. It's a fine line on what is "comically bad" but like, if you are interviewing for a startup of 20 people and you use $20B/year as the revenue assumption with no wink. That's a red flag. Lmao.
> Like assuming the population of the USA was 1 Billion people.
Is that comically bad? It's only like three times too much (if you're using the American, 10^9 billion), but we'd accept a much greater margin of error in some other assumptions (like in the classic 'how many Piano tuners are in the city', I think an assumption of 1-in-30 or 1-in-300 households having a piano (that would need tuning) all sound like they could be true!)
Totally my personal opinion, but not knowing the approximate population of the country you live in betrays a pretty serious lack of curiosity about the world. Obviously a judgement call on how far off is "far off", but it just gives a funny feeling, you know?
True, but in the context of making a ‘finger in the air’ calculation, 1 billion or 100 million (approximating to the nearest power of ten) should be “good enough’.
And if you’re interviewing, consider that for a decent proportion of people it may not be the country they live in (at least at the time you’re asking the question).
Yeah, I think there's a possible pitfall with using this as an interview technique though.
When you're in a real situation with a customer or user, you ask tons of questions. You use a lot of common sense to figure out what they really want, what's actually important, etc.
But often times these interview questions -- like how many vacuum cleaners in a city -- don't need any further questions asked. The idea that you'd count vacuum cleaners in landfills, or dental suction devices, is just silly. In real life, if someone wanted to know about the vacuum cleaners in landfills too, they'd tell you in the first place.
If an interviewer wants to see if someone can ask clarifying questions, they'd better come up with a scenario where it would make sense to ask them in a normal conversation. Scenarios that are genuinely ambiguous to anyone with common sense.
Otherwise interviews become this weird cargo-cult thing where you have to learn that interviewers present common-sense clear questions, but you have to ask silly clarification questions that you wouldn't in real life, just so somebody can check a box that you asked questions.
Yeah I mean the vacuum landfill thing is stupid. The point was never to try to trip people up, there's no wrong answer (or question), just find out if they could recognize ambuiguity. "Let's try to estimate the amount of pet food sold each year in the USA." -> "Dogs and cats too?", "In terms of dollars or pounds of food?" type of stuff. This was for analyst roles - basically your whole job would be something similar to this where you're asked ambiguous questions and you need to translate that into a semi-rigorous analysis, infer intent, etc. I hate to say it but if you didn't realize that going into the interview, you were probably not a good fit as an analyst in the first place!
I’ve seen customers wanting X and assuming that you should know that X comes with A and B because everyone in their business knows that. But you implement only X with A because you made your own assumptions and did not ask. So you missed the deadline and customer that’s it.
> How on earth does that indicate if a junior frontend dev can do their job or not?
Playing devil's advocate, maybe a junior frontend dev that doesn't trust that they understand what someone is asking for and pushes back on bits that should be obvious will perform better (in some contexts?) than one that doesn't.
For a junior role in particular, though, it really doesn't seem like that should be the threshold and it sounds like it was delivered poorly on top of that.
I hear what you’re saying, but really, this is a stupid question and a waste of everyone’s time. Instead you could give a real big report that has insufficient information or that contains a red herring to an improper assumption. The interviewer could then measure if the candidate properly pushed back, exposed an improper assumption and asked relevant clarifying questions. Like one would do at the actual job exhibiting the characteristics the question purports to measure.
The car example is just stupid. Could you image how idiotic you’d sound beginning by asking is it a real car? You’re basically accusing your interlocutor of operating in bad faith (which they were).
Not just that, but the person said they did ask the interviewer what kind of car it was and what it was used for, at which point any reasonable person would explain it was a toy!
That interview question is basically, "We lied about an imaginary situation and were disappointed you failed to accuse us of lying mid-job interview."
> really, this is a stupid question and a waste of everyone’s time
Yeah, I think I agree. I was just responding to (what I saw as) sentiment that it was entirely unrelated to performance, which seems to overstate the case.
> Could you image how idiotic you’d sound beginning by asking is it a real car?
Tbh, "willing to sound like an idiot to double check assumptions" is probably something worth selecting for! I don't think that saves this question, though.
It’s been a long long time since I was interviewed for a job, but I have conducted a lot of interviews since then and any signal that the candidate has engaged with the question and has interesting thoughts about it is a huge plus.
FWIW I would never ask these kinds of gotcha questions. I just give simple programming problems and talk through solutions with the candidates, and then throw in complications to the questions to make them more interesting and test more areas of the candidates knowledge and problem solving abilities. Yknow, like what happens on the job every day.
Good for you, as you look for someone to work with, not someone to cut down their offer.
I am basically doing the same as I also interview people - but I also check the market from time to time as I am not company owner.
But I basically don't care about the offer if company pays guy much or not it is not my money and I only win if I get a smart, nice person who knows his job to work with.
It goes both ways - one interview turned into an acronym gotcha session. I quickly figured out that was not a place I would want to work; another friend that ended up working their later confirmed my suspicions.
I strongly agree with the sibling comment that this is a perfectly valid interview question.
One of the biggest red flags in an interview is if I ask a question and the person doesn't know how to say 'I don't know', because it suggests there's a big risk that if I assign them a task in their day-to-day work, they won't tell me if they feel unprepared to tackle it. That's a far bigger issue than not knowing that traceroute uses variable TTLs to figure out the timing along the route.
The problem with "I don't know" is that it's really not culturally appropiate to say! Americans are supposed to be rugged, resourceful individualists and asking for help is frowned on because it shows weakness. Can't do that in an interview!
I don’t doubt this happens but seems like a poor example. She effectively rooted out a bullshitter. No worries if you don’t know how a tool works, but just say I don’t know. That answer was nonsensical.
The purpose of an interview is not to root out a bullshitter but to find someone you can work with. If that guy doesn't know about TCP/IP, move on to the next topic. If the guy has poor attitude, be polite. Just don't waste time with making fun of the candidate, it does not reflect well on interviewer and company.
For what we know, the guy may well have been employing "test-taking strategies", and he may have been led down the garden path by the interviewer.
There's far too many posts in Rachel's blog where she goes on about "the one", who knows much better already, and here she channels the asshat that she complains about when she encounters him at work.
Each router checks its table for the destination, and if it doesn't know it, queries the next upstream router, its default route, the next hop. Each router likely ultimately informs you of the hand-off via a packet of some sort, and your then traceroute sends a ping/ICMP to each hop to learn how far away they are.
He maybe could've been pushed to expand on what he did know in more detail, but it seems like she just threw out SNMP as misleading bait, and he maybe mixed up ICMP and SNMP. She is right to call herself a troll, but wow, that's crazy to say she caught the guy in a lie of insanity.
The stepwise increasing TTL is the fundamental mechanism that makes traceroute work. Any answer that omits this is so vacuously incomplete that it might as well be considered wrong.
When you think about it, the candidate isn't even that wrong. Back then, at university, a certain professor would explain the oral exam to the candidate at the beginning. He would explain that he would incrementally increase the difficulty and skip from area to area. The goal would be to find the limits of the student's knowledge, the student would walk away feeling terrible, and he, the professor, didn't enjoy the experience.
That's how it ought to be, but here? OK, candidate doesn't know ICMP well, next topic, no need to waste time and dig in.
Here's another unfavourable thought: some people with abusive childhoods react very badly to dominance displays, and here is Rachel engaging in just that. One wonders what had happened before.
Downvoted for jumping from legitimate criticism of her interview methodology to very personal and completely baseless accusations. This is not the internet I want to live in.
Storytime! In a previous workplace a disagreement over fire safety with escalated into uncalled-for and unwelcome dominance behaviour from my supervisor. All attempts to deescalate were rebuffed, and now there is litigation from multiple plaintiffs, this person took out her sociopathic tendencies on many people. With a minimum of professional detachment or a HR department with a clue the peace would have been kept. (Yes, a few months later the fire marshal issued a code violation, as predicted.) You may not wish to live in that internet, but we live in a world where sociopaths are overrepresented in leadership positions.
Something rubs me here just the wrong way. Rachel complains about ageism and contempt for women in tech, and with good reason, and then she takes it out on an overenthusiastic candidate who can't read the room.
I wasn't denying the existence of antisocial people, I was decrying the public shaming of someone you've never met as a victim of child abuse, and now also as a sociopath, based on only a small thing they wrote on the internet.
I assure you, I've met many people who treat their interviewees worse who are not sociopaths.
Yea, they’re pointless. The amount of time someone spends on a truly difficult and important problem is maybe 0.1% of their job. And usually it’s better to just call in a domain expert anyway if it’s something that important. The other 99.9% - do they show up on time and work hard, do they care about the company, do they fit in with the rest of the team, etc, mostly can’t be determined in a short interview anyway.
> you can select an initial guess that is offset from 50
Given that 7 guesses covers 128 numbers, you can offset by +/- 14 without actually affecting the "worst case" of the algorithm (i.e. provided you have at most 64 either side of your guess). As you say, randomly selecting this offset would neuter most adversarial examples (purposefully chosen to fall into the gaps of binary search) and would possibly completely remove the benefits from adversarial choice (though a tailored distribution on offset might be required there).
I might be confused, but don't 7 guesses actually cover 255 numbers? I think you have to count all nodes in the search tree, not only the leafs, because you can get the correct number before reaching a leaf node.
Or more generally k guesses cover 2^(k+1)-1 numbers,
e.g. with one guess you get the answers correct/high/low, which can cover 3 numbers)
Maybe there is a mistake in my thinking, because this would mean you can cover 127 numbers with 6 guesses so you could not lose the original game.
Edit: My mistake is that you still have to explicitly guess even if you know the precise answer already, so you cannot cover 3 numbers with 1 guess. This means 7 guesses cover 127 numbers.
That approach would still leave you weak to always picking 1 or 100.
Without proof, I believe the optimal guessing strategy would perform equal (on average) for every number, to not give the opponent any standout choice (common for optimal strategies, but not always the case). If my math serves me right, that would be an average of log2(100) = 6.64 guesses for any number, which would make you lose 0.64$ on average.
I don't think you have to put your random offset all in the first guess either. Maybe you could random offset +/- 7 on the first guess, +/- 3 or 4 on the next, something like that.
The way forward is to make Ballmer pay with time for screwing with you, which gets us into geopolitics, and then using the resulting MAD dynamics to make the game fair again. That's how adults with keys to the nukes do it :).
The ultimate conclusion of which is likely that both parties will decay to picking the secret value/first guess randomly (although I'm not sure if the optimal distribution is perfectly flat?), which is also something that we can model.
Seems like the distribution definitely won’t be flat since the guesser can randomly choose any of the numbers from 37 to 64 as a first guess without losing anything on the large side, so Ballmer starting with any of those increases his chance of having to pay out the $5. Likewise for other numbers there are nuances to what can be guessed.
But if you assume that the opponent knows that they shouldn't pick between 37-64, doesn't that change your odds?
The game theory here is similar to another quiz "Guess 1/3 of the average".
You are really trying to guess how deep the other has thought about the problem, so you can tell which strategy they settled on, and then you adapt your strategy based on that. Of course, it's a loop.
I have not really studied this but maybe choosing the guess randomly when the number of possibilities is even is already enough to counter an adversarial opponent. Note that 50 is not the only 'optimal' guess in the beginning. 51 is just as good.
No it doesn't, it's quite clear that Ballmer can be choosing adversarially. The point is that even if Ballmer chooses randomly and the interviewee plays optimally given this, the game still has a negative expectation value, and that is enough to be sure the game is a loser for the interviewee.
The post never answers the question "so what is the real expectation value", which is a more difficult question. But I think if the interviewee chooses a number randomly from 40-60 as the first guess and does a binary search from there, Ballmer can't really improve on choosing his initial number randomly.
I think you did the math wrong. The expected value for the guesser is $0.20 if Ballmer chooses randomly. I think Balmer is saying that he can beat you if he chooses adversarially and you choose the expected initial guesses.
I agree that if you choose your first guess somewhat randomly in the 40-60 range (maybe not a uniform distribution though) Balmer would be forced to choose randomly and you would be back at a positive $0.20 EV. For example, you could flip 6 coins and add the number of heads, then flip another coin to decide whether you add or subtract the number of heads from 50 for your starting guess. But I think you would need to randomize your later guesses a bit also.
Okay I did the simulation. I don't think this strategy actually works, but I initially thought it might like you did. One such nash equillibrium my sim found was having the Ballmer player mix between picking either end of the range (not always 1 or 100 but around those numbers). I have the Ballmer player winning with around $.85-$1.00 EV per round. The resulting player strategy was to also try to start their binary search at the extreme ends of the range and hope they guessed the right side. It's kind of like the soccer penalty kick dynamic between the shooter and goalie. Goalie wants to pick the same side, shooter wants opposite sides. But with 100 choices, the goal is too wide I think.
I now think that not constraining the players remaining choices to follow binary search pattern would completely change the resulting equilibrium and improve the results for the player. But that would be more computationally demanding to calculate because there's a strategy choice for every range of choices. And also I've avoided work for 2 hours by working on this so that's not great haha. I _am_ curious what not constraining the player to binary search would do though...
You have 31 positive payout guesses (1 $5, 2 $4, 4 $3, 8 $2 and 16 $1) leaving 69 other numbers with zero or negative payouts. You don't want to have gaps larger than three between your positive guesses, but there are 32 gaps for a total of 96 possibilities, or an excess of 27 over the numbers you need to cover.
It seems like a lot of possibilities and I think you can get away with a minimum gap size of one, but let's assume you do 5 3-gaps at 1, 25, 50, 75, and 100 and 2-gaps everywhere else. So start with 51, then 26 and 76. Then go up or down 12, then 6, then 3. If you have a gap of two you flip a coin, if a gap of three you pick the middle one.
Or if you have them write down the number and you think it has double-digits you could put your 4-gaps below 20. Start with 53 and go up or down 24, 12, 6, and 3 (unless it is below 20, then it is multiples of four.) 59 would pay you a dollar.
Your starting guess could be anywhere from 37 to 64 without paying out more than a dollar, but if you start with an extreme, then low odd numbers and high even numbers will have a negative payout. However, I think you can still randomize sufficiently starting with 38 and 63, e.g. 63-31-15-7-3-1.
One can make the case for a perfectly rational adversary who always avoids picking paying numbers in anticipation of the opponent to exclude paying numbers successively in their guesses. When the game is played with perfectly rational characters the picker is doomed to select one specific number and thus you always make the maximum amount. There are some variations of the binary search but that can also be worked around. If they are not cheating, that is.
Other commenters are wrong in saying that the payout is different for an adversarial choice.
The crux of the payout derivation is: we can only cover 1 number in step 1, 2 in step 2, 4 in step 3, 8 in step 4, and so on. You can choose your initial number in binary search randomly, and as long as you meet the above condition is met (# of possible numbers covered in each step), payout should be same as 0.2
If I 'know' that my opponent is adversarial, then I might assume that he's not picking from the set of 100 possible numbers, but actually from a smaller set of 'adversarial' numbers, like the set that will always take 6 or 7 guesses using the naive binary search approach, and I can adjust my strategy accordingly.
> I'd be interested to see someone do the analysis of what the optimal random-offset-selection algorithm would be to counter trivial adversarial choice
If you know your opponent picks a number uniformly from all numbers that lead to a maximum of guesses, the optimum strategy is a binary search between those numbers, making sure to pick one of those numbers at each turn.
The problem stays completely symmetric under this condition, so there would be two (maybe four due to edge conditions) optimal first guesses summing to 101.
In general, I think the trick still is a binary search where each guess splits the range of options in halves of equal expected/min/max cost (depending on whether you want to optimize for expected/min/max cost).
If Ballmer is being adversarial, he won’t pick the number at the start, and always win.
Of course you can set up the game such that Ballmer has to commit on a number at the start of the game (by sealing it in an envelope or whatever), but that wasn’t specified.
Without writing the number down, it's up to Ballmer to decide that aspect, because you cannot look into his brain, or prove that he didn't commit to a prior number. Therefore, it's fair game.
Genuinely asking - not directly to OP of course, wasn’t this how people were playing the game when you were kids? Not as rigorous, but you intuitively try offsets to get lucky and find the number in fewer tries?
I'm not really married to this idea, but my first reaction is that to assume a random number would be an invalid assumption.
The scenario is framed as a zero sum game: one of us wins. The question is, "should you play?"
In order to answer, you need to be able to determine whether or not there is an optimal strategy that is generally successful.That should include both the assumption that Ballmer has chosen a number adversarial weighed against the random choice.
I recently interviewed for a senior level role for a complex domain (payments), this is an area I have more than a decade of experience. The interviews went flawlessly because I know payments inside out, not just in US but in UK and most EU jurisdictions. The funny bit is that the role being senior, influencing, soft communication skills and managing conflict are even more important than the subject matter expertise and I nailed those areas as well (they threw an obnoxious senior manager that kept interrupting me as I calmly answered the questions, the follow up was that my performance was a masterclass in handling conflict). The final round was with a business person who fancied himself the defacto subject matter expert and kept throwing trivia questions about payments. His plan was to go through as much trivia as he could until he could find something to justify a no. His last question (he literally stopped as soon as he got his way after this question), the question was, have you got personal experience working on real-time payments? I do, in more than one countries (US introduced this very recently as part of fednow), he pushed me about the fednow and obviously this is so new that I only have read the specifications and evaluated a few vendors to decide whether to build or buy. He used this as justification to make a negative reommendation, claiming I don't have real-time payments experience.
Honestly, I don't want to work in an environment like that, it was a large US bank and where their biggest problems are not product innovation or focusing on customer but production failures! An area I have rescued several large companies in, apart from payments expertise and made sure I communicated this. But sometimes you get lucky and don't have to find out the hard way that this place is not pleasant.
> they threw an obnoxious senior manager that kept interrupting me as I calmly answered the questions
This is a red flag. To me this signals that a company not only has a toxic culture, but embraces it. Such places attract personalities who love conflict and once there are enough people, they set the culture.
What doesn't get said often is that conflict is a failure of leadership. Often all it takes to resolve conflict is for one very senior leader to snap their fingers and say, "Guys, I want you two to make this happen". But what happens is that leadership is either far too disconnected from the ground to align their teams, or they constitutionally advocate conflict within their teams in the name of competitiveness. Either way, such places can be hell to work in.
The way I read it:they inserted the manager as a litmus test AGAINST aggression / toxic culture. Kind of like when a psychology test is given, the __thing__ they're trying to measure is always one level removed / abstracted to avoid subjects gaming the system. I suppose deceptive practices in interviews don't bode well, but I could see the argument given the interviewee could be deceptive (something that this site complains about a lot with upper management / ChiefBullshittingOperatives etc.)
I think the point is that deliberately trying to piss someone off and annoy them is a super childish and ridiculous thing to do and is indicative of a place I wouldn't want to work. Interviewing is already stressful and terrible enough without deliberately being antagonized. Most people are not going to go off on someone doing this, they're just going to be turned off by the entire process and decline to go forward to the interview or hiring process.
I think a good comparison would be your romantic partner "testing" you by asking their friend to try to sleep with you and see if you try to go through with it. This is toxic, manipulative, sociopath level behavior.
Personally, I don't like this kind of thinking that it's a failure of leadership first and foremost. Yes, of course leadership can both work proactively to prevent conflict, as well as try to minimize/react to situations. But, what about the conflicting people? Shouldn't they (in most situations), bear the most responsibility to not end up/turn a situation into a conflict? Sometimes I get afraid of comments that (in my interpretation) imply that basically everything bad that happens is the fault of leadership (management). To me that breeds a culture where ICs are taught to not own their situation, which I believe is very very dangerous (to everyone involved).
As an IC leadership picks my coworkers and my projects for me. Why should I be made responsible for the consequences of their decisions? What do you think leadership does do if not build successful teams? If I am expected to get along with everyone on my team then I expect to be allowed to make hiring, firing, and prioritization decisions. At which point I’m now leadership and we don’t need dedicated leaders.
That SME guy sounds like an asshole, but I used to have an interview technique where I’d ask increasingly specific and low level questions about the candidates area of expertise until it got to the point where I’d be pretty confident they wouldn’t know the answer off the top of their head. I wasn’t adversarial or rude about it, I just wanted to find out if they were comfortable saying “I don’t know”, because not knowing something is an everyday part of technical work, but not being comfortable saying it can be big source of issues.
The candidates who were otherwise the most competent tended to be the most comfortable with the I don’t know answer. Getting defensive about it I always considered to be a red flag.
I've been on the other side of that table. The interviewer stated in advance that the questions would get harder until I couldn't answer anymore, and that's OK because he wanted to see where my knowledge stopped. That clarity made it much more fun than stressful. I felt alright saying "I think the answer is X, but it could possibly be Y, and here's what the different implications would be".
But for the luvagod, please state that up front. It wouldn't have been nearly so fun, or informational for the interviewer, if I'd felt like I was failing a quiz.
I like that idea, and I may need to steal it. I have this tendency of asking candidates questions, and if I feel like they’re demonstrating very strong knowledge then I may toss them a few extremely obscure questions for bonus points, but I never expect candidates to get them right.
But this up-front approach of setting expectations seems like a better way to go.
Please do. I think you'll get more signal, too. If I were worried that I've forgotten something very basic I'm expected to know, you're not going to learn much about me other than that I don't do my best work during interrogations. Tell my I'm not expected to know a thing, and then there's lots of room to talk about it, and I can show you that maybe I'm at least familiar with the ideas even if I've forgotten the particulars.
I do this, but with the goal to find a specific thing they don’t know in an area they do know. Then I want to see them work out what a reasonably possible answer would be
I'm 100% fine with that. I had one interview where we ended up talking about the best data structures for a visual editor to store text files in-memory. It wasn't related to my day job at all, but I walked away feeling like I'd learned something, and the interviewer got to watch me reason my way through unknown territory to see how I handle such things. That was fun. I have no idea if I got the "right" answer or not, but it was at least defensible, and I stumbled across some ideas that he seemed to find unexpected and interesting. I ended up getting the job.
What made it enjoyable was me knowing that I wasn't expected to know the gory details of how text editor internals work.
I always say I don’t know in interviews when I really don’t, rather than try to bluff. Some interviewers don’t like this though. As with the parent, perhaps that’s actually a good thing as you avoid having to work in a bad environment. Other times though, you may be being interviewed with a bad egg who you’ll never actually need to work with in the actual job.
I love it when interviewees say "I don't know", so long as they follow up with some sort of mental process explaining how they'd find out the answer/solution. So, "You know, I'm not exactly sure how the new payments API handles excessive requests, I've only glanced at the documentation. I can look at the docs more closely and get back to you.", or even "I don't know how the new payments API handles excessive requests, but honestly, if we've reached that point, I might wanna investigate that specific issue first, and try to figure out why we're sending so many requests." - either of those responses are great, IMHO. The response I'm NOT looking for is basically, "I don't know [shoulder shrug]."
Yeah this is basically how I see it, there’s a natural selection to it. If you practice deceit and politicking in interviews (and in the office), you’ll select yourself into, and only be able to succeed in organisations that value those things. If you practice honesty and candor in interviews, then you’ll expect the same (over time at least). In interviews I think you should just be guided by your genuine values and be yourself (well, whatever version of yourself you feel most comfortable bringing to the office every day). It probably doesn’t maximise job offer conversions, but in my experience it maximises being in working environments that I’m most likely to enjoy and fit into well.
Edit: By honesty in interviews, I mean to a point. There’s some things you absolutely should lie about in interviews (if you’re confident you can get away with it). For instance “what’s your current salary” is a great question to lie about, that they really have no business asking anyway.
My first interview at a FAANG company was so awesome. The interviewer said “I’m going to keep asking you questions till you can’t answer anymore. That way I learn the limit of your knowledge. If I can’t it’s because your knowledge in that area exceeds mine.”
This framing has helped me ever since. It helped me emotionally to recognize that finding the limits of one’s knowledge is not a bad thing, it helped me get the job, it helped me interview people, it helped me hire people who knew more than me.
FAANGs are notoriously famous for having one bad feedback rule. That is, if even one interviewer feels you didn't answer their question they reject you.
So it looks like all the top paying places interview this way.
Hard agree there. I'm quick to say "I don't know (yet)" because I don't want to waste everyone's time while I stumble through a bunch of made-up answers trying to sound smart. If I were punished for admitting I didn't know the details of something, I'd leave in a heartbeat.
I remember some experiences where an interviewer thinks they are doing a deep reach for something they think should be an "I don't know", but it happens to be something you do know. Sometimes they think you're bullshitting or arrogant for this.
Judging people and getting an accurate read on people is hard. Often people are overconfident in their ability to do it.
Remember that you are talking to humans, they are flexible. You were being adversarial. If you could explain them in advance what you were trying to "read between lines", I'm pretty sure most/all of them would have changed their answers. So what you were supposing that was unfixable/permanent, apparently is fixable within 1 minute (of explanation).
Interviewing is by no means the perfect way to assess a candidate, but ultimately that’s what the purpose is. If I just tell candidates what I want them to say upfront, then why even bother with it at all? I want to assess what qualities they have that I want/don’t want, and what qualities they don’t have, as best I can. They don’t need to be perfect. Skills can be trained, personality characteristic much less so. People are flexible enough that some of them could spend one hour flexibly pretending to be the candidate I’m looking for, but that’s not the purpose of the exercise for me.
What you should care is a behavior. You should be open about what is expected. People change behavior all the time depending on the situation/group/company/context.
People can change the behaviour for brief periods. But who they are day in day out is going to be pretty consistent. Telling people how I would want a good candidate to behave during an interview doesn’t help at all with candidate selection. An inclination towards saying things they think people want to hear is a characteristic I’d like to select out of my candidates as well, so perhaps I’ve been killing two birds with one stone here…
> People can change the behaviour for brief periods.
This is a false assumption. Especially generalizing the behavior in such an adversarial setup as a job interview to a regular day to day work/life.
> Telling people how I would want a good candidate to behave during an interview
You should tell them the rules of the game. The thing is, with interviews, there are already predefined assumptions, such as not knowing something takes a point from you, so people avoid this. In your case, you are altering these assumptions without disclosing it. So people might already had changed their behavior for the interview specifically - avoiding admitting not knowing something.
I have the same idea in interviews. they need to be able to admit when they don't know or need help depending on the level. However I thought about it and I think the continuous "why" comes off as sort of childish or low effort. I didn't want to drive off people that reasonably didn't want to work in a place with a toxic culture. My solution was to ask a question that was specific to the workplace but technical so that it would require more information to solve. I looked for answers along the lines of:
- I don't know
- I don't have enough information based on the question
- I would do it this way generally but this question requires employer specific information.
Not someone that just barreled forward and came up with a defacto answer as the solution. They had to give some sort of admission that they could not really solve the problem as is.
I usually had some overlapping technical expertise with the candidates I was interviewing, so my approach was to prepare a line of questions relating to some obscure or esoteric technical issue I’d dealt with in the past. Usually I’d get to an I don’t know pretty organically.
One time I had a candidate who didn’t not know about a single massively obscure thing I’d asked him. He was a DBA for a Chinese ISP that had more subscribers than we had total population in the markets we were operating in. That guy was probably the best hire I ever made. He was always in an incredibly genuine good mood, he was always happy to help everybody, and he’d help people learn how to solve problems rather than just doing the solving for them. Everybody on the team got smarter and more competent working with him, and he was so good at his job that he never even got behind on his own work due to helping other people all the time. I hope he’s still doing well now, before I left that company I managed to make sure he was being paid bucketloads of money (which wasn’t something he ever seemed to be seeking out independently, he was always just happy to come to work and do his job).
> because not knowing something is an everyday part of technical work, but not being comfortable saying it can be big source of issues.
I'm honestly never afraid to say those words, if someone doesn't want to hire me because I said it, I dodged a bullet. I'll go where the devs and leads are sensible people.
In an environment like that you'll be respected a lot more as a consultant and paid advisor, even if you provide generic and mediocre advice, than as an employee providing high quality expertise. Toxic management loves overpaid external consultants and advisors more than their own, much lesser paid internal staff.
I've experienced something very similar in the same field. It's honestly frustrating when you're fully prepared and qualified, yet the process feels more like a trivia game than a genuine evaluation of your skills and experience. As others have pointed out, this kind of behaviour is a clear sign of a toxic culture. What's even more absurd is that it should be the exact opposite. If they're looking to grow their team or replace someone, they should be seeking out someone who's even better than anyone they currently have.
When they nitpick or push for irrelevant details just to find a reason to say 'no,' it's a massive red flag. It shows they're not really interested in innovation or solving the real problems, like the endless production failures we've already helped other companies overcome. Honestly, in situations like that, the best thing you can do is apologise for wasting their time and walk away. But I get it when they have the job in your area, it's tempting to tolerate the nonsense. Still, it’s a good reminder that sometimes, dodging that bullet is actually a blessing in disguise, even though you are unemployed and running out of money, as I was at the time.
Absolutely yes. I like games. The purpose of games is to have fun. This seems like a fun game for like the first $20, a sum I can afford to play a fun game for 10 minutes.
Then at the end, I get to say "I once lost $20 to Steve Balmer playing binary search", which is a fun sentence I can dine out on, and is worth more than $20 to me.
I feel like perhaps this is why MS under Balmer lost relevance. Too busy looking at the technical and not the human.
Underrated comment. His point was to see how they approached the problem regardless of the answer, which is a much different criteria than having the right answer.
I don't know why you'd make this comment... I find it hard to believe you're actually stupid enough to not understand the implicit "(i.e. is your expected profit greater than 0)".
If you answered like this in an interview I would definitely not give you the job. I did actually interview someone once who was like this - "How would you do this?" "Well you shouldn't do it. I think you should do this other thing.". He did not get the job.
This is greyed out, but I tend to agree with the sentiment that there’s a right way and a wrong way to approach these “EV” questions. OP was a bit harsh with the stupid comment, and for SWEs EV understanding is not usually a critical thing, but ultimately you’re being asked about the probability and the ability to make good decisions. Trading firms make use of this when hiring traders (most famously Jane Street and also SIG); The thinking is that if someone makes bad decisions with toy games, and their thought process is not analytical, they’re going to make for a bad trader, not making good decisions with millions of dollars on the line. A good example of something that would rule out a trader is: You can flip a coin, if you win you get $1m, if you lose you lose $1m. Would you play? The EV is zero, but the question is about bankroll management and disaster avoidance. As an individual the downside risk of a $1m loss (usually) significantly outweighs the upside of a $1m gain.
I would be glad not to given a job then, if that's how you react to someone taking the quiz and giving it its own twist.
It's like interviewers don't understand the power dynamics going on during an interview: for the interviewer, this is just their job. Nothing will change for them whether they pass the candidate or not. This could be their 100th interview. On the other hand the interviewee is not paid to be there, they could be doing something else and they are stressed as who knows what kind of egocentric asshole might be interviewing them for a job they want.
If I get a candidate who can make a fun and clear headed remark in such a tense situation, I would read that as them being comfortable with problems and with stress. I would move forward and ask further questions to see their thought process but rejecting someone out of a single question is ludicrous.
Slowly and for the span of many years I've come to realize that binary search is an amazing problem solving tool, specially on systems that are too big and complex to debug.
For example, recently a colleague had a problem with a rendering tool for Figma, of which we don't have the source code. The tool would take too long exporting a specific design. The team mate tried changing things randomly for days to no avail. Each try would take hours and sometimes crashed the browser.
The solution I gave him was to remove half of the elements and check how that affects the exporting time. Then keep repeating for the groups that still failed. In a matter of hours he found the element that caused a seemingly infinite loop.
In the networking classes I took, we used binary search to determine where a problem was occurring, but with a slight twist: Each step away from the end device (e.g. workstation, etc.) take two steps upwards in the network. This broadened your scope easily but allowed for very fast refinement of "These are fine, but this is broken".
I recall a story about a private mailing list with ~1000 participants, where someone was leaking all the messages to the public. To quickly catch the responsible subscriber, the admin used binary search and selectively altered the messages by inserting an extra blank character somewhere.
This is where binary search is not necessary. The admin could have just sent out unique messages, by making binary variations at 10 locations in the email (or ternary variations at 7 locations, or quaternary variations at 5 locations, etc.). E.g. choose 10 words, that can be replaced with a synonym, then generate 2^10 = 1024 unique messages to identify the leaker.
That's what we did in electronics too. There were kits with electronic circuits (this was back in the day with transistors.. and a bit more), where the teacher would introduce a fault somewhere. What we were taught was to start measuring signals in the middle, and then continue just like a binary search. This became so natural that when I ended up in programming in my first job I automatically did the same when searching for bugs in software. What surprised me was that other people didn't.
In the old days of DHTML when I was in my early teens this was the way I debugged very messy JS scripts (like multi-level menus). Remove some code - see if it still breaks, remove more, and so on.
Is there a name for the fallacy where you attribute your success in life to your own intelligence, and thus assume that you are smarter than everyone else, and that you therefor must be right about everything?
The "fundamental attribution error" is a bias where people attribute their own success to their inner abilities and other people success to external circumstances. (It's the reverse when thinking about failure)
For the second part of "I'm superior and know-it-all", I'd say it's good ol' jerk-ery?
The temptation that when you are smart you should become the guardian of the world, a world based on your learnings, your ultimate truths, truths you find easier and more quickly found than by the lay-person. Or so the temptation goes. It allows you to license your morality; the ends justify the means. What you are doing evilly now will be paid off twice-fold by the good it will lead to later. Right?
There's the Fundamental Attribution Error and Dunning-Kruger effects too. And on behavior... Illusory Superiority combines with Moral Licensing (allowing yourself to be equally good and evil because you "match the two") and the dis-inhibition effect which people with greater success take more risks (including affecting other people negatively).
I think these effects all sort of combine. It's not necessary intelligence but power, at least as perceived by the individual that seems to be a bit of an issue (e.g. the individual who thinks they are smarter at doing X innately feels more powerful and then has less inhibition about expressing their superiority and trying to dominate over others).
We've all seen the person who ought to have moved on who hangs on to their former glory fail to understand they are not in prime condition and who tries to exert power they nolonger hold too.. to me that is the real opposite of imposter syndrome. it's when peoples perception of themself and social dynamics don't move with the times.
Just because it's one of my pet peeves, this is not what Dunning Kruger says. What it says is that people who are poorly skilled in a task will overestimate their skill and those highly skilled will underestimate, but not that the poorly skilled estimate themselves to be better than the highly skilled.
From the wikipedia article you link:
> Among laypeople, the Dunning–Kruger effect is often misunderstood as the claim that people with low intelligence are more confident in their knowledge and skills than people with high intelligence.
My response was directed specifically at the OP's second question, about the "opposite" of impostor syndrome, and not the first one.
The dunning krugger effect is widely regarded as the polar opposite of it:
- "If the Dunning-Kruger effect is being overconfident in one's knowledge or performance, its polar opposite is imposter syndrome or the feeling that one is undeserving of success. People who have imposter syndrome are plagued by self-doubts and constantly feel like frauds who will be unmasked any second." [1]
- "This is the opposite to the Dunning-Kruger effect. The Imposter Syndrome is a cognitive bias where someone is unable to acknowledge their own competence. Even when they may have
multiple successes they struggle to attribute their success to internal factors." [2]
- "The opposite of the Peter Principle and Dunning-Kruger effect is the imposter syndrome. This is when smart, capable people underestimate their (...)" [3]
I once had a colleague who had a favourite interview question, it had something to do with graph data structure and would always ask the question, many times candidate would reply and get rejected, strangely enough after a while all those candidates who replied got rejected. So we all gathered around him to work through what questions he was asking. We got to this question, and working with him on this question for a while we realised his solution to his own problem was wrong.
Turns out he was using this one question to reject people his whole career.
It was a humbling experience to all of us, to recheck everything before we asked questions. Most of the candidates you interview are perfect hires. Some times its you who is wrong.
>strangely enough after a while all those candidates who replied got rejected.
That also seems like a bad interview strategy. Make a mistake and you're out? Did you have so many perfect people to hire that you could just sort out almost everybody?
“As a SWE, I seek to understand important context first, before jumping to build or code. First, I’d like to ask if you’ll guess randomly and fairly, or adversarially?”
“Secondly, when significant money is involved, I make sure to verify any inputs. I’m considering the situation, not you personally, untrusted. How can I verify it, or do you want me to proceed assuming that’s verified?”
Those are great questions, but it’s also about how you ask it. SWE is not pure engineering. Communications is vitally important.
"as you, my interviewer, are a capable SWE I assume you gave me all the context needed to solve the problem".
The interviewing game of asking clarification questions is silly and should stop. In the system design portion I can understand it, but not when asked a direct technical question.
It's perfectly fine to ask followup questions with added constraints or just directly say that the specification is fuzzy and needs to be clarified first, but having that dance around the basic specs in nonsense (as if you wouldn't know if you're dealing with a 10PB array or 1kb at work).
This is anything but a direct technical question though.
> It's perfectly fine to ask followup questions with added constraints, but having the guessing game to figure out those constraints is nonsense.
You say that. I say people being able to ask the right question is one of the most important skills to be a productive developer. So of course as an interviewer I want to know if they can do it.
I don't know how it works where you are, but we don't have a big book of perfectly defined specifications for our work. I guess if we could get one of those that would improve our productivity. But until we obtain one we will keep testing candidates on their ability to ask questions.
Sure, and there's a way to test the ability to ask questions that isn't some "gotcha" type question.
As in interviewer you can just say "the specifications aren't clear, what questions would you might want to ask to clarify them?".
It's not like in the day to day work you go around defining specifications for every tiny function - the default specification are clear from the work environment.
Let's say you had to implement a "find dups in this array" at work, you probably won't go around collecting requirements for that, so asking that in an interview and having the silly dance of "Oh, the interviewee didn't ask if the array fits in memory or not" is silly imo - and doesn't show anything other than whether the candidate memorized the need to ask that or not.
and like I said before, fuzzy specification are more suitable for the system/product design part, and can also be part of the coding part, but they shouldn't appear as some "gotcha".
> As in interviewer you can just say "the specifications aren't clear, what questions would you might want to ask to clarify them?".
I do say that, yes. Not necessarily with those words, but I tell interviewees that they are free to ask questions and in fact recommend that they do rather than they go start coding immediately and accidentaly solve the wrong problem. (Heck! Some people solve a harder problem than we intended to ask from them!)
> I say people being able to ask the right question is one of the most important skills to be a productive developer.
But you never know if by asking the "right" question you'll jeopardize the entire interview problem. Some interviewers may have only prepared 75% of the problem and haven't went through all the posibilities. If you ask a question that may pose itself as a "treat" (e.g., making half the problem non-sense and therefore there's no need to implement it) your interviewer may simply consider you a no-go.
And it's not about malice, but simply that you may be better prepared than the interviewer and some times that leads to a no offer. I wouldn't mind working in a place like that, so I don't usually ask "too clever" questions.
> But you never know if by asking the "right" question you'll jeopardize the entire interview problem.
Yeah. That can happen. As an interviewer i would tell the interviewee that they are right and it is because the example is a bit contrived and would ask them to pretend it still makes sense. If they are polite about the thing it would actually count in their favour.
The candidate who can ask that is already better than the candidate that jumps straight into a solution. If I'm the interviewer, I'd be impressed with such a candidate.
And frankly this is a needed skill. Candidates who automatically think about adversarial scenarios tend to write more defensive code, not to mention fewer vulnerabilities.
As with most interview questions, I’d expect this to be about how you think through it and show your work. If an interviewer asked this question and you found a mistake, that probably helps you get the job.
Some other interesting points here: Ballmer works hard to de-emphasize and diplomatically move away from discussing this exact question once it becomes clear that Chang's not approaching it by thinking explicitly about binary search and expected value.
Which is not surprising, because she's a professional journalist! It's amazing that Ballmer (like so many technical interviewers) is so pleased with this question that he couldn't help bringing it up, even though it's not really that relevant to Chang's question.
I really am curious about the Nash equilibrium solution. I assume that as a commenter has mentioned, for the guesser it involves returning random numbers near the binary search. But I’m curious if for the picker it involves a uniform or non uniform initial distribution?? I’m sure someone on HN knows/can explain?
There's obviously a huge gap between the 5-number game and the 100-number game; it's possible that the best mixed strategies settle down as the number of choices gets larger, or (for all I know) it's possible that the best mixed strategies get crazier and crazier. I'd love a ping if anyone does any real exploration of the 6-, 7-, etc.-number games.
"Candidate always starts by guessing 50" can't possibly be part of a Nash equilibrium, since then Ballmer would never ever choose 50 as his secret number. (And indeed, he says as much in the linked video.) And if he'll never choose 50, then it's obviously silly for the candidate to waste a guess on it... and so on. Nash equilibria are usually (always?) located at mixed strategies.
It is also unclear if one has to keep playing. The expected value is very different if after the fifth guess one can thank Balmer for the opportunity and walk away.
Reminds me of the viral video, goes something like "I'll pay you $20 if I can pour 2 cups of water on your head" and then only pour 1 cup and walk away.
IANAL, but they either have to honor the verbal contract (pour an additional cup AND pay $20) OR the contract is void and therefore they can be sued for assault. Of course, the "can" in "if I can" may be construed as "being able to", but that's up to the jury I guess.
This can be a big problem in teams that are homogeneous: All were hired using the same process, so all are maths/physics majors with good analytical skills but insufficient software engineering skills.
What often happens maths/physics majors excel at programming the small, but cannot architect things in the large. As a friend once put it about one such person: "He can only do it as long as he can fit the whole problem in his head at once."
It's great to have mathematicians and physicists in the team. But you for sure want a sufficient number of trained and experienced software engineers as well.
This is a game with imperfect information, and the optimal strategy for each player is probably different from "pick any number at random" and "run off-the shelf binary search".
Now I am not so sure, but the first player makes the move first (selects the number), then the second player makes their move without seeing the results of the first player's move. Meaning after the first move there is information hidden from the second player.
Didn't Steve Ballmer start off at MSFT essentially in a biz ops role, supporting execs when the company was super small? Interesting how he became technical as the company grew. Pretty rare.
He graduated with a mathematics degree from Harvard so the concept of binary search would have likely been familiar to him. But you’re right, as far as I can tell, he never did any technical work like programming in his career.
Not only that, he was also better than Gates at math. From the acquired podcast episode on Microsoft:
> Ben: He's gregarious. Anyone who's ever met Steve or seen a video of Steve, you are well aware that this man has a presence. But the thing that people don't know about him is he is so unbelievably analytical. Steve is the guy that outscored Bill Gates on the Putnam exam.
It's surprising the extent to which the tech community overfits towards classifying intelligent individuals as either exclusively technical or nontechnical. Recruiters are especially weak in this regard, e.g., if you've ever been effective at sales or people leadership, you are likely ineffective at swe or data science or vice versa. The most intelligent folks I've worked with are very diverse in their interests and abilities. You can see this in an elementary school GT classroom. Why does the tech community believe this is always an either/or proposition?
Yep. Pigeonholing by narrow thinking individuals who aren't accustomed to ambiguity or lateral thinking, especially when exhibit talents in more than just technical areas, a person becomes "nontechnical" to a nonzero proportion of technical people while remaining "too technical" for a large fraction of business people.
PS: Recruiters generally come from the same cloth as car sales and sports, so they're not usually going to be the sharpest pencils in the drawer.
>PS: Recruiters generally come from the same cloth as car sales and sports, so they're not usually going to be the sharpest pencils in the drawer.
Aren't you committing a similar mistake here, saying that recruiters can only recruit? At my former job one of the recruiters was an engineer for 20 years. He said he just wanted to do something different after all these years.
> The most intelligent folks I've worked with are very diverse in their interests and abilities.
> as either exclusively technical or nontechnical
This applies outside of tech or generally in any role e.g. if you're a backend engineer they assume you don't know frontend or if you're a marketing specialist you're not good at sales.
I never get it either. We're people not machines but most people have this assumption like we're a game character - you get a job / trait and that's it.
Because we live in an era of specialization. Look at a companies job page - even startups have silos. I don't think this is strange or unusual. Its hard to be good at everything. If I'm spending 8+ hours per day doing sales, where am I going to find the time to be good at other things? Most people are working for the weekend or to spend time with their families. Diving into far off subjects related to work isn't always exciting.
Ballmer was good at riding coat-tails of others as a supporting figure but eventually started running MSFT into the ground. He demanded to personally evaluate every M&A activity >$10M. No bueno.
I know no one asked for it, but here's a slightly simplified version of this script in Python:
from itertools import count
from statistics import mean
INITIAL_PAYMENT = 5 # dollars paid if first guess is correct, minus 1 for each incorrect guess
RANGE = range(1, 101) # the range of numbers to guess from
def simulate_guess(target):
low = min(RANGE)
high = max(RANGE)
for bad_guesses in count():
match guess := (low + high) // 2:
case _ if guess < target:
low = guess + 1
case _ if guess > target:
high = guess - 1
case target:
return INITIAL_PAYMENT - bad_guesses
average_payout = mean(map(simulate_guess, RANGE))
print(f"\nExpected value per game: ${average_payout:0.2f}")
I believe you’d have to do a game theory analysis to actually get the answer (compute the mixed strategy that produces a Nash equilibrium). My intuition is that this yields <0 EV (because it’s already so small against a uniformly random strategy, which can’t be optimal) but I didn’t do the calculation.
I'm so glad I've managed my career such that I've never had to answer bullshit interview questions like this. The only purpose of these is to stroke the asker's ego, they tell you nothing at all about the candidate. What a waste of time.
Same. I don't know if I have been lucky. I have worked with 6 companies in the past (startups, multinationals, consultancy companies, etc.) and I have never had to answer brain teasers.
I believe what Ballmer wanted to hear are clarifying questions.
Eg. Can I stop at any time? If yes, I will stop before I go into the negative. If I cannot stop but must continue the game until I guessed the right number, I will most likely lose money. Then go into a simple computation of XY terms where X is the probability and Y is the payout or loss for maybe a dozen terms.
Ballmer states the question as "I'm thinking of a number between 1 and 100." He does not state that the number is an integer. If he's thinking of anything other than an integer, you're unlikely to be successful in finding it via binary search.
Is it really wrong though?
In this case maybe you could game the system.
What if you combine binary search and game theory?(knowing that he’s trying to beat me and that binary search is my best strategy I have information reducing the randomness of the selection. I know Ballmer is going to choose worst case numbers like 58)
begin with binary search to narrow down the range, then within the range guess the binary search worst case answers. It’s still not guaranteed to win, but might be fun for the chance of taking his money.
Although leading by telling him that the worst case for the optimal algorithm of binary search on n=100 also gets you the job at Microsoft which is worth more than the two bucks you might earn playing the game.
Title is wrong in implying Balmer is incorrect and the article shows that the title is wrong. If clickbait is misleading, then this is worse than clickbait, no?
> Ballmer states that the answer is "No" for two reasons: firstly, because he can pick numbers that'll be the most difficult for you...
The article goes on to show that there are numbers where a binary search always has the guesser paying $1
Although Ballmer could still be incorrect, because a 'sufficiently logical' player would also presumably know that he could pick numbers that'll be the most difficult to find via binary-search, so by the same logic you could also meta-game it, and assume any number that can be found in 5 steps with a binary search is immediately out. This would narrow the search space to only 37 numbers, which can then easily be found within 5 guesses.
But he also knows that you know that he could pick numbers that will be the most difficult... So could then pick one of the numbers that actually are guessable within 5 guesses to trick you.
But then you also know that he knows that you know that he could pick difficult numbers too.
I'm not entirely sure if this invalidates Ballmer's advantage, but I would be interested to know what the 'perfect' strategy would be for this game considering the meta-game.
There isn't much of metagame if number is only in Ballmers mind. No matter what guesses you choose he can force you to make at least log_2(100) guesses. Doing anything except splitting in half will only increase amount of guesses. There are two things that can change the game, requiring Baller to write the number on a piece of paper before the start. Other thing you could do is writing a number on piece of paper yourself halfway during the game. If opponent is changing the number adversarially with goal of maximizing guesses you can force them to "pick" a specific number. Afterwards you can open the piece of paper and claim that you actually guessed the number with the first attempt.
This assumes Ballmer is cheating, rather than just behaving adversarially but within the rules.
If we accept cheating is allowed, we can also potentially accept other 'cheating' scenarios where the player repeatedly punches Ballmer in the face until he discloses the number, thus winning. Or where the player just refuses to pay at the end.
IMO the problem is only interesting if we assume any form of cheating isn't allowed.
and given that the first rule still holds where he chooses hard numbers, then the expected value of the game is negative (aside from meta-gaming this, which is out of scope for a technical problem)
but you don't know. Only he knows that he's going to pick numbers the binary search will fail on and he states as much as his reason that you shouldn't play the game.
The best interview that I ever had was one in which the dude wore a fedora and we white-boarded. And yes we talked about algorithms. But I got the sense that he didn’t care if I knew any particular thing per se so much as was trying to figure out if I would be a fun person to jam with on hard problems.
Deep down we all know what programming, was supposed to be about, which is when you drop the ego and channel your childlike creative curiosity to create something you love, or to find like truth. People who know how to reliably find this state and enjoy it will become an “intelligent” person regardless of their original IQ. When I hire a full time role now, I look for people who are best able to channel this in the context of our team and our mission.
The problems come in when the creative output of the programming is being managed in a capitalistic system. If you take capital with the plans to build something and make money, then you have an entirely new set of constraints. You committed to a deadline so now you lose the ability to say “no”. Most programmers hate scrum/agile with a passion because these processes are basically the manifestation between the conflicts and misunderstandings that happen between shareholders and creators.
Nice, you played the game and you earned $0.20 [twenty cents]. Definitely a bad choice. But then you got viral on HN, and made good on your investment.
about the probabilities: say Steve's number is 59. I say 50, Steve says higher, so there are just 50 numbers left and the new probability is 1/50, I say 75, Steve says lower, so the probability is 1/24 (otherwise it would be 1/25), and so on
I wrote similar code to show that as long as you choose your starting number randomly you will have positive EV. Not sure how they kept using this interview question without realizing this.
For reasons given in the comments, both players probably choose a mixed strategy at equilibrium. If someone actually managed to find / prove a mixed strategy equilibrium for this game right there in the interview you probably couldn't go wrong hiring them on the spot.
The (TV) interview is kind of funny because the journalist asks him, after he goes through the idiotic song and dance of this brain teaser: "so what did you learn about me?" This is actually a very insightful question. What did you learn, Mr Ballmer?
To which he literally has no answer: "I learned you need to step back and really ask if you're going to make money on this thing".. uh, okay Steve. Cool. Thanks for your contribution to possibly the worst technical hiring practices in just about any professional field. The technicals are less interesting than seeing even he himself has no real justification for this kind of intellectual hazing.
He was being polite and understanding. They aren’t in a job interview setting, it’s not a correct frame to judge. But if it were, she would totally fail with that attitude of not thinking through the problem. That’s what his comment about stepping back meant.
Well even if they were in an interview setting, how often in your day-to-day are you put on blast, being critically watched and scrutinized while having to give birth to an elegant solution to some stupid random question? How often does this solve-immediately-or-GTFO happens at work? They made so much money in the 90's that they could smash every other Ivy league candidate with a bat or make them do 500 jumping jacks, and still have people line up to interview.
> how often in your day-to-day are you put on blast, being critically watched and scrutinized while having to give birth to an elegant solution to some stupid random question? How often does this solve-immediately-or-GTFO happens at work?
Well, honestly, once to several times a week during backlog grooming, meetings with stakeholders and other similar stuff. I find it much easier to work with people with fast recall who can be confident and correct even under time pressure. Nobody wants to sit through many time-consuming meetings that end with “we will check that and get back to you later”.
There is a succinct chess joke that summarises what being an expert means: a journalist asked Casablanca how many moves he thinks ahead. He replied, only one, but it is always the best one.
If Ballmer asks me this question then I would obviously take it - expecting and hoping to lose. I would then guess the worst possible answers just so I’ll have a story to tell my grandkids how a billionaire fleeced me out of a few bucks while deciding the trajectory of my career. /s
I have opened the article in question. I have NOT worked through the technical problem, the complications and interpretation surrounding Ballmer, nor have I digested the contention being presented. I am capable of these things but I am on the clock and I do not value performing the work required for this particular article.
With that said, I wanted to share the following. Perhaps it will spur discussion.
Our leadership -- whether in our professional circumstances, in our sovereign and communal circumstances, or in our choice to lead ourselves; perhaps it is in these leaders that a view, or a decision, or a proclamation -- perhaps it is in these impulses that the world is changed.
Can you assign truth to an impulse? Is it a communication for consideration? Is it a demand for compliance?
I assert that you can do so. The words were spoken. Thus, the impulse was true.
However, if the interviewee assumes that Ballmer is being adversarial, then you can pick a different value as your initial guess, which causes the probabilities to shift. Even the OP assumes that the interviewee will start guessing with 50, but, because of the way binary search works, you can select an initial guess that is offset from 50 (with a randomized offset each time) to defeat trivial adversarial attacks that attempt to game the heuristic, while still mostly reaping the benefits of binary search.
I'd be interested to see someone do the analysis of what the optimal random-offset-selection algorithm would be to counter trivial adversarial choices.