
What happens when you add a new teller? (2008) - isp
http://www.johndcook.com/blog/2008/10/21/what-happens-when-you-add-a-new-teller/
======
zw123456
Anyone with experience in telecom will immediately recognize this as a traffic
engineering problem easily solved with the Erlang formula:
[http://en.wikipedia.org/wiki/Erlang_(unit)](http://en.wikipedia.org/wiki/Erlang_\(unit\))
Not to be confused with the Erlang Language. ErlangC assumes queuing, which is
probably applicable to this case.

~~~
OldSchool
Wow, out of 28 comments at the moment, yours is the only one to point this
out.

------
a_c_s
I think this would be far more informative if they started by explaining the
single teller case: it isn't obvious how customers that arrive at a rate of
slightly less than 1 every 10 minutes and take 10 minutes to serve would have
an average waiting time of 5 hours.

~~~
basseq
I think it's non-obvious because it quickly gets into queuing theory. To
simplify (and expose a flaw in the case[1]), assume a bank is open 9am to 5pm
(8 hours). They'd expect to see 46(.4, but let's round) people in a day.

So there's two factors: flow rate and service time.

FLOW RATE:

Those people aren't coming in the door every 10.3 minutes: they're all coming
in at 9:00 before work, 12:00 on their lunch breaks, or 4:45 right before
closing. So you'd see 15-20 people all working through the doors within 10-20
minutes of each other. Ouch! This is modeled in queue theory as a Poisson
distribution of arrival times.

SERVICE TIME:

Again, we're dealing with averages and distributions, but this time it's
exponential. For every person who comes in with a check to deposit (let's say
that takes 3m), there's another extreme case (let's say someone who wants to
deposit $2,500 in Canadian pennies). As someone in the back of the line, you
have to wait for all customers before you to be served before it's your turn.

All of a sudden you're in a 29-person line and waiting 5 hours. THAT math
makes sense: 29 people × 10 minutes per person = 4.8 hours.

THE FLAW [1]

Of course, this queue model is continuous: the bank doesn't open or close (as
most banks do). Moreover, arrival times are deterministic: you can model based
on a distribution, but you could quickly measure expected arrival times.

Process efficiency and queue theory are interesting topics. My favorite case
is Toyota's six sigma production line engineers helping a NYC food kitchen cut
wait times from 90 minutes to 18 minutes with simple adjustments:

[http://www.nytimes.com/2013/07/27/nyregion/in-lieu-of-
money-...](http://www.nytimes.com/2013/07/27/nyregion/in-lieu-of-money-toyota-
donates-efficiency-to-new-york-charity.html?_r=0)

~~~
thaumasiotes
That was depressing. The article details the adjustments Toyota made to the
food kitchen's system:

> The kitchen, which can seat 50 people, typically opened for dinner at 4
> p.m., and when all the chairs were filled, a line would form outside. Mr.
> Foriest would wait for enough space to open up to allow 10 people in. The
> average wait time could be up to an hour and a half.

> [Toyota] eliminated the 10-at-a-time system, allowing diners to flow in one
> by one as soon as a chair was free.

Talk about low-hanging fruit. This is quite literally a case of cutting
waiting times by saying "hey, why don't we just stop telling people they have
to wait?"

~~~
derefr
It's a non-obvious solution if you stop thinking of it in technical terms:
they were effectively telling people (who arrived in groups) that they would
not be allowed in when their group could fit, but instead would be forced to
split up.

~~~
thaumasiotes
They're not arriving in groups. If they _were_ arriving in groups, breaking up
those groups so you could form them into platoons of 10 would still make no
sense.

------
isp
The related post, "Server utilization: Joel on queuing" (2009), includes a
useful visualisation: [http://www.johndcook.com/blog/2009/01/30/server-
utilization-...](http://www.johndcook.com/blog/2009/01/30/server-utilization-
joel-on-queuing/)

------
SocksCanClose
The inverse corollary being the essay (I can't find it) about the sociology of
lines at Tokyo Disneyland.

Question: what happens when there's no line for a specific ride at Tokyo
Disneyland? Answer: nobody wants to ride the ride! Second question: what
happens when there's a line for a specific ride at Tokyo Disneyland? Answer:
everybody starts queuing up for that ride, even if they don't know which ride
it is!

~~~
derefr
So... have (plainclothes) park staff queue for the empty rides?

~~~
SocksCanClose
yeah, they actually do!

------
marcus_holmes
Interesting to note the MBA view:

If the one teller is 100% busy serving customers, then that's efficient. The
queue is clearly temporary and due to demand spikes

If two tellers are idle 70% of the time, that's inefficient and one of them
should be relocated.

I've had similar discussions with non-tech management about server utilisation
(why should we buy another server when we're only using 80% capacity of the
ones we've got?)

~~~
vacri
Appeal to their marketing side: excess peak demand represents angry clients
who will take their business to the bank across the street, which does have
two tellers and no wait time.

------
rcarrigan87
2 personal observations at Disney:

Wait time clocks are normally inflated by around 15-20% on really popular
rides.

Generally, if the line splits, the left side is shorter bc more people are
right handed and tend to go right.

No data to support either claim, just what I've experienced. I'd be really
curious to see some of Disney's work in line theory.

~~~
tgb
Is it right-handedness or driving on the right side?

------
ScottWhigham
It helped me to visualize the problem. I don't know if HN's formatting will
kill this but here goes. The idea is that six customers arrive at 9:00, 01,
02, 03, 04, 05, and 06 - then map them out to "What happens when you add a new
teller?" I tried to figure out how the author's explanation that adding one
teller can go from 5 hours to 3 minutes.

Edit - the text version was a disaster here. Here is an image of what I used
to try to understand:

[http://i.imgur.com/HOKgfmd.png](http://i.imgur.com/HOKgfmd.png)

One teller: average wait time is 22.5 minutes for "six users who arrive one
minute apart from 9:00 to 9:05"

Two tellers: average wait time is 8 minutes.

I think to go from "5 hours with one teller" to "Three minutes with two
tellers" requires you to space people out more than "one every one minute".

------
cabacon
See also a very nice video from ClojureWest about queues in system
architectures:
[https://www.youtube.com/watch?v=1bNOO3xxMc0](https://www.youtube.com/watch?v=1bNOO3xxMc0)

The queue are everywhere - your messaging queue, the threadpool, the hardware
threads, and other layers of the stack and APIs you use. The video adds the
interesting detail that as you add more tellers (workers) you learn of
impending disaster only in the outlier p99 (or higher) latencies; by the time
your p85 latency rises, you're already about to stall out.

------
instakill
A great guide on this can be found:
[http://ss15-teropa.divshot.io/](http://ss15-teropa.divshot.io/)

------
theVirginian
Interesting conclusion. No explanation.

------
thrownaway2424
This analogy will not resonate with the under-30 crowd. I realized the other
day that my kids don't have firsthand recollection of what "going to the bank"
might be.

~~~
wldcordeiro
Under-30 here, the analogy makes sense, I have to go to my bank somewhat
regularly for things like check cashing, transferring funds from accounts to
others and getting financial advice. I think you're mistaking people under 30
with people who haven't lived by themselves and handled their own finances
which is a subgroup of under-30's.

~~~
ominous_prime
I'm financially independent, mid 30s, married with kids, and I haven't been in
my bank branch in years.

I agree that under 30s should be able to relate, but technology is changing
how often I physically queue up somewhere.

~~~
kibibu
Ditto, and my bank doesn't even have branches. Last time I stepped into a
physical bank to do actual banking was somewhere around dickity 06.

