
Secretary Problem - tosh
https://en.wikipedia.org/wiki/Secretary_problem
======
extr
I was asked a version of this as an interview question a few months ago.
Despite recognizing the form of it immediately and giving some intuition for
why it converges to 1/e, the interviewer had nothing else prepared and had
expected it to be new to me, so he took me through a brutal one hour of his
personal "guided proof" of the answer, including drawing a massive grid on a
whiteboard and stepping through iterations. At one point I may have hinted
that his method was overcomplicating things and tried to steer the
conversation towards other aspects of the problem (ex: the time based
formulation), no, let's get back on track with this grid. I didn't get the
job, and I was specifically called out for doing poorly on that portion of the
interview. I used to think this problem was really cool, now it just pisses me
off remembering that situation.

~~~
dolson
Maybe you were the first person he interviewed.

~~~
extr
It was one of the team members, not a manager or lead, so I would have been
inclined to overlook it had it not been alongside 3 other "case interviews"
that were tricky whiteboard coding/math problems in disguise. I think I spent
a grand total of 20 minutes of my 8 hour day discussing my history and
relevant experience.

~~~
schoen
I think dolson was making a joke that the interviewer was actually following
this strategy in the hiring process.

~~~
extr
I think I replied to the wrong person.

------
gumby
I've always liked this problem mathematically but I'd never used it for
_actual_ hiring (though I have happily used it for certain statistical
sampling).

When actually hiring, I care about the time value of opportunity cost. That
is, we interview and get hire the first candidate that we feel can do the job
well and we all otherwise like, then the benefit of having her on the job ASAP
exceeds any possibly value from another candidate that might show up some
uncertain time later.

~~~
kyleschiller
When actually hiring, you also have benchmarks that aren't purely relative,
and past experience to fall back on.

~~~
gumby
> When actually hiring, you also have benchmarks that aren't purely relative,
> and past experience to fall back on.

(This problem is framed as hiring the _best_ which I assume means closest to
optimal weights on those non-relative benchmarks -- whatever "optimal" means.)

But your comment is interesting, as this problem was originally as "the wife
problem": how do "you" \-- presuming incorrectly even at the time all
mathematicians to be male -- find the optimal wife.

The thing that made your comment funny to me is that in majority of marriage
cases (by anyone to anyone) neither party has much, or any past experience to
fall back on!

~~~
Retra
In those cases, even if you select poorly, you can change your weight and
preferences afterward.

------
dreamcompiler
I ran into this problem many years ago when writing a memory allocator. Say
you keep a list of free blocks. You don't keep them sorted, but you do keep
track of how long the list is. You now need to find a block of at least size n
but not much larger, and the free list is so long you'd rather not look
through the whole thing for each allocation.

There are a bunch of ways to write memory allocators, but with the above
constraints, it's the same problem.

~~~
gnulinux
You can use this idea in the clock algorithm to approximate LRU.

------
rhallie11
I dated a guy once who, after a couple of weeks, asked me to marry him--citing
this problem as his reasoning.

I said no.

Butt he's all I ever think about whenever I hear it mentioned, so.

~~~
khedoros1
Did you ask his value for "n", and how he chose that value?

~~~
rhallie11
It was 2, and I honestly have no idea why

~~~
Bromskloss
What? That makes it even weirder. That means you weren't even being chosen
because you were considered to outperform anyone else. Either you were the
first candidate, and no comparison could be made, or your were the last
candidate, and had to be chosen anyway!

~~~
rhallie11
I was his second girlfriend, and he'd just gotten out of a multi-year
relationship. We met while interning in SV. We broke up soon after I went back
to school, and I think he just ended up getting back with his ex, so...

~~~
twic
Ah, so once again, we have a situation where the documentation describes the
use of an elegant algorithm, but the actual implementation is a random pile of
hacks and heuristics slapped together by someone who seems not to have really
understood what they were doing.

------
capocannoniere
I like this explanation of the Secretary problem (aka optimal stopping
problem) by @lpolovets [1]

> The basic problem is this: there are N applicants for a secretarial
> position. The applicants are interviewed in random order, and you must
> accept or reject a candidate immediately after interviewing them. After you
> reject someone, there's no way to bring them back. There is only one
> position available, so as soon as you accept a candidate you are done. What
> strategy should you pursue in order to maximize the likelihood of hiring the
> best candidate out of the N applicants?

> It turns out that the best strategy is to look at the first N/e applicants
> (i.e. the first ~37%), reject all of them, then accept the first applicant
> that is better than all of the initial rejects. It turns out this strategy
> has a ~37% (1/e) chance of picking the best candidate, regardless of how
> large N is. This is counterintuitive. It means that even if you have one
> billion applicants in some random order, you will find the very best one
> ~37% of the time with this strategy, even though you will (on average) make
> your choice without meeting the last few hundred million candidates.

> I think this has some interesting practical implications. For example, as a
> VC, one possible strategy that I can pursue is to look for interesting
> sectors (3D printing, drones, bitcoin, etc.) and try to invest in the very
> best startup in each of those sectors. The Secretary Problem is a decent
> approximation of this. I can go on AngelList and see that there are thirty
> 3D printing startups in LA/NYC/SF, which are the markets I'm interested in.
> One way to pick the best one of these 30 would be to meet 11 of these
> startups, pass on all of them, then invest in the next 3D printing startup I
> meet that's better than all of those. Of course, this would be a jerk move.
> I'd be wasting the time of a lot of founders if I knew I was planning to
> pass on their startups no matter what. However, if I happened to pass on a
> dozen 3D printing startups by chance, and I knew that there were ~30 in
> total, then if the 14th one was better than the first dozen, I would know
> there was a pretty good chance it was the best one out of all 30 even though
> I had yet to see the last 16.

[1] [https://www.quora.com/Mathematics/What-are-the-most-
interest...](https://www.quora.com/Mathematics/What-are-the-most-interesting-
or-popular-probability-puzzles-in-which-the-intuition-is-contrary-to-the-
solution/answer/Leo-Polovets?share=1)

~~~
djrogers
The secretary problem and investing have one important difference that throws
this all out the window - you can meet with multiple startups without
rejecting them, and make a decision when you've met them all (or all the ones
you want to meet). There's no requirement in the real world to accept/reject
immediately post-interview without any possibility of recall.

> Of course, this would be a jerk move. I'd be wasting the time of a lot of
> founders if I knew I was planning to pass on their startups no matter what.

On a macro-level, this isn't a jerk move at all - the other way would have you
'waste' the time of 29 founders instead of 11.

~~~
dsr_
No, the biggest problem is that nobody is perfect at ranking startups, period.
And as proven by that "antiportfolio" page of a day or two ago, even generally
successful VCs can't do that reliably.

------
scardine
I started listening to the audio book version of "Algorithms to Live By: The
Computer Science of Human Decisions"[1] by Brian Christian and Tom Griffiths.
It starts by talking about the Secretary Problem and the 37% rule and it is
very entertaining how much of our daily problems are some variation of a small
set of well known problems that have interesting (and often counter-intuitive)
mathematically proofed solutions.

Before I get downvoted to hell, let me say THIS IS NOT AN AFFILIATE LINK, I
will earn nothing if you click it (I hate spam as much as the next guy but
people here tend to be a little trigger happy).

[1] [https://www.audible.com/pd/Business/Algorithms-to-Live-By-
Au...](https://www.audible.com/pd/Business/Algorithms-to-Live-By-
Audiobook/B01D24NLWO/)

------
dang
A previous thread:
[https://news.ycombinator.com/item?id=7337759](https://news.ycombinator.com/item?id=7337759).

------
kornork
If the chance you'll pick the best option is 37%, what is the chance you'll
pick someone in the top 2, or the top 5?

Another way of asking is, does it raise the odds you'll get someone really
good, even if they aren't the best?

~~~
Bromskloss
I haven't calculated it exactly, but we were playing around with simulation
code elsewhere among the comments, and my version [0] calculates the hiring
frequency for each candidate, not just the best one, so you could look at the
tail of the list called "hires".

[0]
[https://news.ycombinator.com/item?id=15573921](https://news.ycombinator.com/item?id=15573921)

------
f_allwein
Did not know about this, but I read a simplified version of it a long time
ago, which said "carefully examine the first choice, reject it, then go for
the first choice that is better." Apparently, this would give me an above
average option in most cases (did not do the math).

I quite like that and sometimes use it, e.g. to decide which restaurant to
have a meal in.

~~~
zellyn
The Power of Two Random Choices: it's a surprisingly decent heuristic

[https://www.eecs.harvard.edu/~michaelm/postscripts/handbook2...](https://www.eecs.harvard.edu/~michaelm/postscripts/handbook2001.pdf)

~~~
f_allwein
Oh interesting, thanks for sharing!

------
imrankhan17
I did some analysis of this problem in Python here:
[https://imrankhan17.github.io/pages/Solving%20the%20secretar...](https://imrankhan17.github.io/pages/Solving%20the%20secretary%20problem%20with%20Python.html)

------
danesparza
I'd really like a real-life example of this.

I can't help but think this is a solution in search of a problem.

~~~
jpalomaki
Case: You are wondering around a new city with few friends. Everybody is
hungry and getting annoyed. No mobile devices are available or you do not
trust the online recommendations. How do you find a good place to eat?

First you estimate based on the level of hunger, quality of shoes and the
restaurant density of the area how many places you can check before you starve
to death. This rule tells that you should first walk pass n/2.718 restaurants
and then pick the best one you have seen so far.

~~~
Too
Almost correct but the secretary problem only allows you to accept or
_permanently_ reject an option. What you describe allows going back to
previously rejected options.

------
quickthrower2
This is the awe inspiring bit:

> One reason why the secretary problem has received so much attention is that
> the optimal policy for the problem (the stopping rule) is simple and selects
> the single best candidate about 37% of the time, irrespective of whether
> there are 100 or 100 million applicants.

Another example of how our intuitions are poor at probabilities.

100M! Might give that a Monty Carlo sim. Or even try it on a big DB table at
work.

------
jagthebeetle
There's probably no lack of toy code, but since I am a skeptic / sceptic,
quick nonsense code for pythonistas :
[https://gist.github.com/anonymous/ef975cdcb26044de9ba93e4e74...](https://gist.github.com/anonymous/ef975cdcb26044de9ba93e4e74de29b1)

Guess what? Skepticism allayed!

~~~
carapace
Nerd sniped. I couldn't help it. What do you think> :-)

    
    
        from math import e, floor
        from random import randint
    
    
        def secretary_search(candidates):
          """Rejects first n / e candidates and then selects the first better."""
          n_reject = int(floor(len(candidates) / e))
          best = max(candidates[:n_reject])
          for candidate in candidates[n_reject:]:
            if candidate > best:
              return candidate
          return candidates[-1]
    
    
        success = 0.0
        size = 100
        trials = 10000
        for _ in range(trials):
          candidates = [randint(1, 1000) for _ in range(size)]
          success += max(candidates) == secretary_search(candidates)
    
        print success / trials

~~~
Bromskloss
My version! [0]

\- More functional, with iterators and stuff. (It's half-baked, though, so
please take it further!)

\- Handles small numbers of candidates correctly.

\- Agrees with the table on Wikipedia [1], and with the 1/e limit.

\- Calculates hiring frequency for all candidates, not just the best one.

\- Faster!

[0]
[https://gist.github.com/anonymous/9d735c5c77bd2e0939c69f5dd4...](https://gist.github.com/anonymous/9d735c5c77bd2e0939c69f5dd4140cd4)

[1]
[https://en.wikipedia.org/wiki/Secretary_problem#Deriving_the...](https://en.wikipedia.org/wiki/Secretary_problem#Deriving_the_optimal_policy)

------
psergeant
This would feel much more interesting if there was a cost attached to being
forced to hire a terrible candidate.

~~~
Bromskloss
Isn't the cost that you get a terrible candidate?

~~~
psergeant
Yes. That isn’t factored into the algorithms though.

------
mrguyorama
Interestingly, it seems this has been discussed here before:
[http://www.thelowlyprogrammer.com/2010/04/introducing-
marria...](http://www.thelowlyprogrammer.com/2010/04/introducing-marriage-
sort.html)

