
Queueing theory: The science of waiting in line - chmaynard
https://www.johndcook.com/blog/2019/01/23/queueing/
======
scott_s
Understanding the basics of queueing theory means that you will be eternally
frustrated in most US grocery store checkout lines. And you may be kicked out
for screaming "Basically zero-cost queue synchronization and heavily variable
server time means a global queue is better!"

~~~
defertoreptar
It's true, but there are two things to factor in:

1) People tend to sort themselves by picking the shortest line.

2) Grocery stores try to staff only just enough registers so that no register
ever has a gap.

If you combine those two things, the efficiency closes in on what a global
queue would. My main gripe is the "unlucky" factor where the shopper gets
stuck behind a very slow person, which the global queue would fix.

The third, dark-patterny part of this is how grocers benefit from people and
their children being stuck next to impulse merchandise while they wait.

~~~
tunesmith
With multiple lines, there's also the excellent visual cue to managers - if
the average line length appears too long, you know you need additional
cashiers. And my local grocery store is excellent about bringing in additional
cashiers if any of the lines are two deep or more.

With a global line, the length of the line isn't a good visual indicator of
needing more cashiers. I suppose you could just periodically count the line
length, and count the cashiers, and divide, but it's not a quick eye check
anymore.

~~~
scott_s
Global line length is no different than average line length of multiple lines.

~~~
paulie_a
On a global average that is probably true. On an individual/personal level it
is not. I take a strategic approach, there might be twice as many people in
one line but it will be faster. No one in that line is breaking out the check
book. They won't be arguing about a 25 cent coupon that expired. On the plus
side I've noticed less places accepting checks. Personally I think they should
be banned for basic consumer shopping entirely. Unless it is 4 digits, use a
card or cash.

As for self checkout lines there should be a huge sign that asks "are you
prepared for this technology?, Really? Are you sure?"

------
velosol
So should I leave space on the highway so my lowest speed through the
occasional slowdown is higher than average or am I hurting the traffic flow?

I recall reading about (highway) traffic being a queuing problem where the
arrival rate exceeded the clearance rate due to a transient action (someone
tapping their brakes because they were distracted or a bright glint from a
glass truck going the other way). If you decrease the arrival rate slightly
such that the clearance rate can remove the stopped/slowed cars then the
entire traffic flow will return to the norm for the volume of traffic on the
road.

Sound? Valid?

~~~
kokokokoko
Braking or a quick lane change causing traffic jams is a well known and
refuted myth. Similar to the idea that driving slower on the highway is more
dangerous. We have massive amounts of data on both of those. I wouldn't put
too much value in that source.

~~~
velosol
And that would be my fault - my parenthetical was just imagined reasons why
there might be a saturation-related slow down.

In trying to look for the source I did come across [1] from a few years ago
and may have been remembered incorrectly.

Edit: More details at [2], [3], and [4] some noting transient braking may be a
factor.

[1]: [https://www.npr.org/2013/11/29/247825768/phantom-traffic-
jam...](https://www.npr.org/2013/11/29/247825768/phantom-traffic-jams-what-
causes-mysterious-highway-backups)

[2]: [https://www.vox.com/2014/11/24/7276027/traffic-
jam](https://www.vox.com/2014/11/24/7276027/traffic-jam) [3]:
[http://math.mit.edu/projects/traffic/](http://math.mit.edu/projects/traffic/)
[4]:
[https://www.sciencedaily.com/releases/2007/12/071219103102.h...](https://www.sciencedaily.com/releases/2007/12/071219103102.htm)

~~~
kokokokoko
Right, those are brief disruptions that cause no significant variation in
total transit times.

Much like the idea that slower drivers cause more accidents, this is based on
other drivers experiencing discomfort and having to be more careful to avoid
the slower car. It feels like it is more dangerous, but the data we have shows
that it does not result in more accidents.

Traffic jams that cause meaningful changes to total transit time are almost
always based upon significant disruption events(accidents) or uneven
capacity(merging traffic at a higher rate than the freeway can absorb).

We know from psychology that a person will perceive something that requires
more attention and effort as taking longer and perceive something that
requires less attention and effort as taking less time, even if the times are
equal.

~~~
friendzis
> Much like the idea that slower drivers cause more accidents

Slower vehicles are more likely to end up in an accident. Speed - accident
rate curve is IIRC called Solomon curve. I haven't heard this theory
disproved.

> this is based on other drivers experiencing discomfort and having to be more
> careful to avoid the slower car

I have heard experts saying that observations indicate that slower drivers
cause accidents indirectly. They are usually slow enough to irritate other
drivers, but fast enough to make it hard for an average car to overtake them
safely, which pushes some faster drivers into dangerous situations, sometimes
resulting in accidents from which the slower drivers escape unscratched.

~~~
tatersolid
> Much like the idea that slower drivers cause more accidents

I think this might be statistically valid, due to correlation with the root
cause. Drivers who are inexperienced or uncomfortable due to poor eyesight,
poor reflexes, unfamiliarity with the vehicle or area naturally slow down.

These are the same sorts of “driver disadvantages” that would cause more
accidents.

No references, just intuition based on decades driving in Chicago traffic.

------
theoh
Some goodies via Jon Bentley:

[http://www.eecs.harvard.edu/~margo/cs261/background/p176-ben...](http://www.eecs.harvard.edu/~margo/cs261/background/p176-bentley.pdf)

And a different presentation of some of the same material, on consequences of
Little's Law, from his book Programming Pearls, quoted here:
[http://www.cs.fsu.edu/~baker/opsys/notes/queueing.html](http://www.cs.fsu.edu/~baker/opsys/notes/queueing.html)

(I'll paste this here as it's already being shared online)

"For instance, if you're in line waiting to get into a popular nightspot, you
might figure out how long you'll have to wait by standing there for a while
and trying to estimate the rate at which people are entering. With Little's
Law, though, you could reason, `This place holds about 60 people, and the
average Joe will be in there about 3 hours, so we're entering at the rate of
about 20 people an hour. The line has 20 people in it, so that means we'll
wait about an hour."

Bentley also cites the following, attributing them to Peter Denning:

"The average number of objects in a queue is the product of the entry rate and
the average holding time."

"I have 150 cases of wine in my basement and I consume (and purchase) 25 cases
per year. How long do I hold each case? Little's Law tells me to divide 150
cases by 25 cases/year, which gives 6 years per case."

"The response-time formula for a multi-user system can be proved using
Little's Law and flow balance. Assume n users of average think time z are
connected to an arbitrary system with response time r. Each user cycles
between thinking and waiting-for-response, so the total number of jobs in the
meta-system (consisting of users and the computer system) is fixed at n. If
you cut the path from the system's output to the users, you see a meta-system
with average load n, average response time z+r, and throughput x (measured in
jobs per time unit). Little's Law says n = x*(z+r), and solving for r gives r
= n/x - z."

------
vanderZwan
Tangential: in the article submitted the other day[0] that interviewed
Japanese demoscener 0x4015, he brought up a time-lapse video of half a million
people queueing for an event in Tokyo:

> _I should also mention Comic Market (Comiket). It’s the world’s largest
> self-published manga exhibition & sale fair, and about half a million people
> (back then, quarter-million) visit there. This is a time-lapse video of the
> queue for the Comiket entrance. It gets demo-ish after 40 seconds and I find
> it interesting._

It is oddly mesmerizing to watch, indeed[1].

[0]
[https://news.ycombinator.com/item?id=18965767](https://news.ycombinator.com/item?id=18965767)

[1]
[https://www.youtube.com/watch?v=ermNqkUUiJw](https://www.youtube.com/watch?v=ermNqkUUiJw)

------
nullandvoid
Vaguely on the topic of queing I noticed this year on black Friday several
sites had queue systems ( e.g 'game' store was one I remember and even had a
limited time you could spend on the site after the queue ).

Part of me wonders whether this was really a server capacity issue or there
was some sort of psychological gaming scheme for customers to feel like they
need to make a commitment to purchase.

Does anyone have some information on this sort of tactic if so?

~~~
theNJR
If this wasn't a gaming scheme, it soon will be. I bet it worked.

------
ggm
Two large Australian department stores implemented single global queueing for
non-self-serve when they introduced self-serve. Its a mixed bag because they
also reduced staff on the line to match maximal delay for minimum fuss: ie,
they don't aim to serve this line any faster than the worst-case single line
instance.

It doesn't invalidate the single global line theory: it points out that the
_qui bono_ moment in a queue is not you the customer: its the profit centre.
They do anything to maximise profit absent a strong signal they went too far.

its morally akin to basic time-and-motion/taylorism.

~~~
Tor3
The signal I usually give is that when I arrive at a line of (understaffed)
counters, with all of them having long queues, I turn around with my basket or
cart, put everything back on the shelves, and leave. I do that if I it's clear
that I have to wait more than a few minutes.

~~~
mmikeff
The signal is clearer if you abandon the trolley and walk out.

------
ojbyrne
It’s interesting that all the grocery stores with automated checkouts I’ve
been to have single lines for multiple automated checkouts, but multiple
lines/single servers for human checkouts.

~~~
foolfoolz
that happens a lot at grocery stores because of carts. it’s even more obvious
at costco.

stores where people are more likely to hold items in their hand or a small
basket can do global queue like home goods or old navy. but once you have
10-40 carts lined up it’s a shit show. imagine costco on the weekend with
everyone trying to push their cart through a maze of switchbacks.

self checkout likely has less carts through it than staffed chashier lines.
also their close grouping makes this a little less of an issue

------
Jtsummers
Queuing theory is something I'm familiar with from my educational background
and general interest in math and systems. But I don't actually know of a good
introduction to the topic, it's just a set of information I've accumulated and
internalized over the years. Any suggestions from the HN crowd for something I
can pass around to some of my team leads and managers that will help them to
understand these topics?

~~~
sacheendra
The book Queuing Theory in Action:
[http://www.cs.cmu.edu/~harchol/PerformanceModeling/book.html](http://www.cs.cmu.edu/~harchol/PerformanceModeling/book.html)

~~~
r4um
+1 For the above mentioned book, especially if you are looking at computer
systems/networks.

For more pragmatic treatment found "Fundamentals of Queueing Theory"[1] really
good.

[1]
[http://mason.gmu.edu/~jshortle/fqt5th.html](http://mason.gmu.edu/~jshortle/fqt5th.html)

------
adanto6840
We've gone to great lengths to get as close to accurate 'queueing theory' for
our game, SimAirport. The systems allow for a global queue for many desks, or
one queue per desk, or any hybrid of one queue per N desks, etc...

It's tough because scaling time & 'movement distance' in a simulation game
isn't easy -- but sounds like that's actually an issue in real-life, too. :)

------
benj111
I was expecting them to explain why the queue I join is always the slowest.

~~~
air7
You're probably joking, but you have spent more time overall in slow queues
than you did in fast queues... This is similar to the fact that "your friends
have more friends than you do" [0]

[0]
[https://en.wikipedia.org/wiki/Friendship_paradox](https://en.wikipedia.org/wiki/Friendship_paradox)

~~~
mmikeff
My rubric for picking the quickest queue is the one with the lowest total age.
A queue of pensioners will take longer to pack bags, find the right payment
card, redeem vouchers and chat with the cashier.

~~~
kqr
Also tourists and people with many products in their vicinity. Suddenly I
really want to collect numbers of this. What's the proportion of age to
product count, for example? How many tourists are one pensioner?

------
elviejo
Is there a nice and visual queueing theory simulator for python or other
programming language?

~~~
nileshtrivedi
I had written a quick visual demo using the Python mode of Processing
language. Perhaps someone here can build upon it.

[https://gitlab.com/nileshtrivedi/queue_simulation](https://gitlab.com/nileshtrivedi/queue_simulation)

------
crdrost
There is also a rough application of some queueing theory ideas (one might
also say, the more general lessons one sees in the physics of transport
phenomena, before one specializes in continuums etc) to business, called
"theory of constraints." The basic idea is that there is probably right now
only a handful of things that are slowing down your ability to make more
money, and if you can figure out what they are and add extra capacity to those
things, you can presumably get "more bang for your buck" than attempts to
improve every part of the system individually. A lot of authors also discuss
similar concepts under "systems thinking."

More specifically, the theory of constraints is kind of half-applied to the
"projects" contexts that we work in in software in the "critical chain"
methodology. It basically says in our context that for each project you have,
at any given point during a sprint, there should be one person who is "holding
the baton" for that project, they need to be aware that any delays in their
work are getting communicated to the project launch date, while others working
in parallel are usually exempt. Essentially exactly one person on the project
should be panicking, and everyone else should have spare capacity to handle
the routine "hey Alice from HR wants us to change all of the articles in the
table which Bob has authored so that they no longer say that they were
authored by Bob since his last day is next Friday, can someone write SQL to do
that?" \-- even if Carol has the most domain knowledge about that, if she's
holding some other baton, that comes first. In fact, if Carol can split her
work -- one common one in our field would be between functionality and
tests/documentation -- so that she can pass the baton a little sooner, often
that is a net win, too: she finishes the core functionality up to testing-by-
hand, she delivers it to Darryl who is the next to hold the baton, then she
documents and adds unit tests in a separate PR before she loses her context
about it.

Similarly the management of these projects is different because you don't want
to estimate "hey Carol we think this is a two-day PR and we will hassle you
come day three," since if she judges it to be a half-day PR then it's in her
best interests to spend a week on other urgent-seeming requests as those
minimize who is hassling her. But instead you want to communicate "hey, this
is going to take however long it takes, and I know I cannot make it happen
faster by hassling you. I want you to know that even though we guessed that
this was a within-one-day task we put an extra safety cushion in the project
in case it goes longer, but you have the entire project's safety cushion if
you ultimately need it: conversely, any time you save right now can be passed
on to your other developers. What I need from you is a daily guess about when
you realistically think you can pass that baton so that I can tell Darryl,
'hey, heads up, you're about to have to drop everything and pick up this
baton.' You can change that estimate at any time, I just want to minimize that
friction and encourage you to sprint as hard as you can on this since you're
on the critical path, if anyone tries to tell you that they have something
more important for you to be doing, you tell them to talk to me and I will do
it personally."

Also the queueing theory makes me a lot more patient with myself. Like I used
to get really hard on myself about how "you spent all of this time on the job
doing this other thing that didn't need to get done, you lazy procrastinator
you!" ... with some queueing theory you start to ask, "what would happen if we
defined load as the maximum usage of CPU, RAM, network -- what would happen if
we ran a server at 100% load? Machine failure, yes... but also high latency,
and things would just get dropped forever. So what would happen if I run
_myself_ at 100% load, is that really what's best for the company?" and in a
couple of cases (like on these critical paths, with a crucial deadline) that
_is_ what's best for the company, but in most cases actually the company
_should_ want you to have some idle bandwidth so that you can promptly serve
their ad-hoc needs.

~~~
crdrost
s/spend a week/spend a day/

Apparently I am trying to edit this after some threshold time...

------
sixstringtheory
Interesting Freakonomics episode [0] about this, where among other things they
suggest that the most efficient queue puts newcomers at the front, essentially
LIFO... thus transforming the queue to a stack!

[0]: [http://freakonomics.com/podcast/what-are-you-waiting-
for/](http://freakonomics.com/podcast/what-are-you-waiting-for/)

~~~
jedberg
But only if all the players are rational. :)

------
jimpick
And then there’s how they wait in line in Cuba...

[https://www.washingtonpost.com/world/the_americas/in-an-
onli...](https://www.washingtonpost.com/world/the_americas/in-an-online-world-
cuba-remains-a-stand-in-line-
society/2015/05/31/07459d7e-fd90-11e4-8c77-bf274685e1df_story.html?utm_term=.7047f0afbd48)

------
amelius
> With only one teller, customers will have to wait nearly five hours on
> average before they are served. But if you add a second teller, the average
> waiting time is not just cut in half; it goes down to about 3 minutes. The
> waiting time is reduced by a factor of 93x.

This is exactly why asynchronous programming (where every task competes for
the same CPU) is such a bad idea.

~~~
icebraining
I don't see what this has to do with asynchronous or synchronous programming.
If you have more work to do than what the core can handle, you're going to
wait a while no matter what style of programming you use.

~~~
amelius
The point is that the async style of programming will introduce unpredictable
and potentially wildly varying service times. Contrast this to multithreaded
programming, where the program is cut into thin time-slices which all have a
similar, predictable duration.

~~~
kqr
I'm not sure I follow... If I understand the async style correctly, the idea
is that a task will yield the CPU as soon as it is about to wait for I/O an
insignificant time. This allows CPU bound tasks to race to idle, and prevents
context switches unless they are nearly certain to be needed.

I guess I understand your concern in the case of multiples tasks that are very
CPU heavy, but that hardly seems like a universal issue.

I also think it's a bit of a mathematical sleight of hand to use measures very
sensitive to outliers with a distribution that generates large outliers in
order to make a point. Perhaps the median or the 95th percentile is more
useful. But less dramatic.

------
smarri
I wonder if a noble goal for queuing systems would be to eradicate the need
for the queue all together. E.g. more frequent public transport, designated
security screening times at airports and gigs, supermarket checkout without
tills or home delivery as an alternative. Theoretically it would be nice, but
could it be ever be practical...

------
Markoff
besides self service cashiers i saw here in Europe global queues implemented
only in Kaufland hypermarket, they actually don't use one big global queue but
have queues assigned to group maybe of 4 cashiers max., in the end anything
it's better than waiting at one cashier queue

------
bdz
Are there any good reads about matchmaking in video games? Which is basically
the same queueing theory

