
The German Tank Problem - eadan
https://www.eadan.net/blog/german-tank-problem/
======
dooglius
This is only the toy version of the actual problems solved by the Allies,
which were more nuanced, and involved reasoning about the tank manufacturing
pipeline. The write-up [0] doesn't go into the math but makes an interesting
read.

[0] [https://sci-hub.tw/10.2307/2280189](https://sci-hub.tw/10.2307/2280189)

~~~
Causality1
How accurate did the allies' model turn out to be when compared to the real
number?

~~~
dooglius
Quite well, see pg. 86 for a plot of all predictions

------
tzury
More about Frequentist and Bayesian analysis can be found here:

[https://en.wikipedia.org/wiki/German_tank_problem](https://en.wikipedia.org/wiki/German_tank_problem)

Matter of fact...

    
    
        According to conventional Allied intelligence estimates, the Germans 
        were producing around 1,400 tanks a month between June 1940 and September 1942. 
    
        Applying the formula below to the serial numbers of captured tanks, the number 
        was calculated to be 246 a month. After the war, captured German production 
        figures from the ministry of Albert Speer showed the actual number to be 245.

~~~
debbiedowner
I was actually surprised that MVUEs and the fellow point estimators are called
frequentist (though it makes sense). In school we always referred to them as
non-Bayesian, at the same time frequentist always seemed like a dirty word to
us students so maybe that's why

------
jackfoxy
How ironic that the nation that led the world in the frontiers of maths in the
19th century completely missed the boat in the applied math of signals
intelligence in WWII. I'm referring to the tank serial numbers and the lack of
care in Enigma codes, except by the Kriegsmarine, but even they eventually
lost a code book to the allies, which they apparently considered an
impossibility.

~~~
PhasmaFelis
They had a lot of opsec problems. There's a great story about how using "cool"
codenames instead of random ones bit them in the ass.

It's nearly impossible for a bomber to navigate long distances in the dark
over a blacked-out country, so the Germans came up with a radio navigation
system involving beams transmitted from the mainland to intersect over the
target, which the British figured out how to jam; the Germans came up with
another nav system, and the Brits eventually jammed that one too.

The British knew the Germans would be trying to find yet another way. They'd
learned from Enigma decrypts about a new device called _Wotan_. One researcher
looked up the word, learned that it was the name of a one-eyed god, and
concluded that the new system would use a single transmitter with a
rangefinding transponder aboard the bomber, instead of multiple beams like the
previous ones. Starting from there, they had a countermeasure online and ready
to go before the Germans even _deployed_ Wotan. When the Nazis realized they'd
been outmaneuvered from the start, they gave up on radio-guided bombing
completely, at least against Britain.

~~~
noir_lord
We also caught every single German spy and turned them all (iirc it was all)
but didn’t know we’d got them all till after the wars conclusion.

British intelligence was pretty impressive during WWII.

~~~
neaden
To be fair, some of the German spies were pretty bad at their jobs. Josef
Jakobs stands out as a man who was just not a good spy:
[https://en.wikipedia.org/wiki/Josef_Jakobs](https://en.wikipedia.org/wiki/Josef_Jakobs)

------
laGrenouille
Interesting article, though I think it incorrectly leaves the reader thinking
that there is some interesting informating hidden in the average spacing of
the numbers. In fact, all you need to know is that maximum observation and the
number of observations. Once you simplify the average spacing goes away.

If M is the maximum serial number of N is the total number of observations,
using the formula in the post:

    
    
        M + (avg. spacing) = M + M / N - 1 = (N + 1) / N * M
    

To me that gives a more clear picture of what the unbiased estimator is doing:
inflate the maximum value by a factor that limits towards one as the sample
size grows.

~~~
comicjk
If you just assume that the sample mean = the population mean, then you get
the right answer, at least for this example. I don't see why the article fools
around with the maximum at all - isn't the maximum a much more noisy statistic
than the mean?

~~~
skosch
The range matters – had they found 10 serial numbers between 100000 and
101000, would the mean still be a meaningful estimate of the production rate?
In this case, the author just tacitly assumes the minimum to be zero.

------
mhh__
For anyone else interested in WW2 reverse engineering and design etc.,
[https://www.youtube.com/watch?v=GJCF-
Ufapu8](https://www.youtube.com/watch?v=GJCF-Ufapu8) "The secret war" is a
huge documentary covering british efforts to counter german electronic warfare
and V-weapons.

------
spectramax
Why didn't they use randomized and scrambled serial numbers? Sort of like what
Amazon does to their order numbers. I know it can still be cracked but
serially numbering military equipment is not very smart. I was setting up a
Shopify store the other day and it doesn't allow for a lookup table to be used
for order numbers. I don't want competitors to know that I've sold so many X
items. Same thing with Squarespace and Square e-commerce stores. It blows my
mind that a multi-billion dollar ecom giant has not implemented despite of
forum posts and requests from users.

~~~
Nitramp
World War II was (at least one of) the first industrialized war. So the whole
situation was genuinely novel to most participants.

Additionally, the German army command didn't think that way. Where the US
relied on overpowering by materiel dominance, and the Soviets fought and won
through unimaginable human sacrifice, the considerable initial success of the
German army was based on better, smarter tactics, individual leadership,
bravery, ruthlessness, etc. The leadership assumed they'd be able to win the
war that way, even when the war had turned into a much more industrial
operation.

You can see that in operations such as the Battle of the Bulge, the war in
Normandy, and most importantly in the the Russian campaign.

This is of course over-generalizing, but I believe the general mode of
thinking was there, and that'd explain the lack of attention on such details.

~~~
greedo
Don't underestimate Soviet industrial capabilities during the war. The Soviets
produced over 58K T-34 type tanks compared to Germany producing 37K (PzIII
through Pz6).

~~~
ptaipale
Though a substantial part of the steel used to make those tanks came from USA.
As did much of the trucks and other equipment used by Soviets.

I recall seeing actual numbers (proportion of American steel in Soviet
production), but couldn't find them, does someone have a source?

Anyway, e.g. this article talks about it:

[https://www.rbth.com/defence/2016/03/14/lend-lease-how-
ameri...](https://www.rbth.com/defence/2016/03/14/lend-lease-how-american-
supplies-aided-the-ussr-in-its-darkest-hour_575559)

~~~
adventured
At the point of German-Soviet conflict, the US had about six times the steel
production of the USSR, six times the iron production, eight times the oil
production, and three to four times the coal production. It definitely
wouldn't be surprising if US steel was a large share of Soviet figures.

The scale of resources delivered to prop up the Soviets was extraordinary,
including what the British sent them.

In just 3 1/2 years the British sent them[1]:

3,000+ Hurricanes aircraft, 4,000+ other aircraft, 27 naval vessels, 5,218
tanks, 5,000+ anti-tank guns, 4,020 ambulances and trucks, 323 machinery
trucks, 1,212 Universal Carriers and Loyd Carriers, 1,721 motorcycles, £1.15bn
worth of aircraft engines, 1,474 radar sets, 4,338 radio sets, 600 naval radar
and sonar sets

And the US sent them:

427,284 trucks, 13,303 combat vehicles, 35,170 motorcycles, 2,328 ordnance
service vehicles, 2,670,371 tons of petroleum products (gasoline and oil) or
57.8 percent of the High-octane aviation fuel,[32] 4,478,116 tons of
foodstuffs (canned meats, sugar, flour, salt, etc.), 1,911 steam locomotives,
66 Diesel locomotives, 9,920 flat cars, 1,000 dump cars, 120 tank cars, and 35
heavy machinery cars. Provided ordnance goods (ammunition, artillery shells,
mines, assorted explosives) amounted to 53 percent of total domestic
production

Beyond Russia also notes:

"The USSR received a total of 44,000 American jeeps, 375,883 cargo trucks,
8,071 tractors and 12,700 tanks. Additionally, 1,541,590 blankets, 331,066
liters of alcohol, 15,417,000 pairs of army boots, 106,893 tons of cotton,
2,670,000 tons of petroleum products and 4,478,000 tons of food supplies"

The notion of sending a country 375,000 trucks and 1,900 locomotives in just
three years, is incredible to think of today.

[1] [https://en.wikipedia.org/wiki/Lend-
Lease](https://en.wikipedia.org/wiki/Lend-Lease)

~~~
olegious
The Soviets repaid the debt with unimaginable human losses: 25-30 MILLION war
dead.

80 percent of all German military casualties occurred on the Eastern Front.

Just think about those numbers.

~~~
neaden
While the Soviet Union lost a huge number of people, the 25-30 million dead is
an inflated estimate because it counts people living in areas that were
conquered by the Soviet Union. Polish people by and large do not appreciate
being lumped in with the people who had invaded them the year before.

~~~
ptaipale
And quite many of the dead of course died with a bullet from behind, or in
slave labor camps run by Soviets themselves.

------
srean
The job interview version: If you are being interviewed for a position by
engineers who have their employee ids (serially allocated) on their badge find
the number of employees from those ids assuming all engineers are equally
likely to be on the panel of 8.

~~~
chiph
I have looked at my payroll check numbers from contracting firms to see how
they're doing as a business. If the interval between check numbers drops in a
month, I have a good idea that there aren't as many people working there
anymore.

~~~
feintruled
I worked for a big company and we used to put bug numbers in our change
releases. We were told to stop doing this, as some customers would see that
their bug would appear to have been given a lower priority when they saw lower
numbers coming in first.

------
wcoenen
See also Doomsday Argument.

[https://en.wikipedia.org/wiki/Doomsday_argument](https://en.wikipedia.org/wiki/Doomsday_argument)

~~~
sulam
Take a silly premise, get a silly argument and a really silly conclusion.

Confounding question: 1000 years ago, would this argument look any different?
Answer: mathematically speaking, it would not. In fact, far more humans have
been born than you could have predicted using this method. Conclusion: the
argument is flawed.

~~~
wcoenen
The people living a 1000 years ago indeed could have used the same argument to
show that they should be 95% certain to be in the last 95% of all humans to be
ever born, and history indeed showed that this was not the case; the dice fell
on the other 5% possibility for them and the population increased more than
20x.

However, the argument will still give the correct prediction for most humans
that try to use it. Just not for the few that were in the special position to
be born early in the sequence of all humans. The argument essentially tells
you that you have no reason to believe that you are also in that special
position.

~~~
jameshart
You are trying to _predict_ the length of the sequence of all humans, so you
have no way to know whether or not you are ‘early in the sequence’. You can’t
use the number you are trying to predict as an input to your prediction.

The tank problem doesn’t tell you how many tanks Germany will go on to build -
it just tells you how many they have already built.

‘Good news! The war is almost over! The chances are these tank serial numbers
all fall among the last 95% of the tanks Germany will ever produce!’

~~~
wcoenen
The difference is that the german tank sample can only be drawn from the pool
of existing tanks so far. But when you consider your own birth, that's a
sample out of the pool of all the humans, including future ones. Because
future humans can also ponder their own place in the sequence, and where they
find themselves is distributed uniformly over the sequence of all humans.

To apply the doomsday argument to tanks, we need to fix the sampling. It won't
work for allies grabbing german tank samples. We have to think from the point
of view of the tank; say a copy of your consciousness is being uploaded to an
army of tanks. If you wake up as tank #734, _could_ there eventually be
millions of tanks? Maybe so, but there's only a 5% chance that you are one of
the first 5% tanks, so there's a very good chance that there will be less than
734*20 tanks in total.

------
Nomentatus
This all seems to assume the tank serial numbers would be captured at one
moment in time ("captured 15 of these tanks uniformly at random.") But in fact
the tank shells dribble in over time which biases the gap, the gaps at the
highest numbers are going to be greater. Earlier tanks have had many more
chances to be destroyed or captured. So using average gap is clearly not going
to give the best estimate. If you restrict yourself to tanks from the latest
large battle, that will cancel out the dribble effect though.

------
nevir
FWIW, this is part of why Amazon's product identifiers (ASINs) are obfuscated
the way they are

~~~
eadan
People have used a similar strategy to estimate iPhone production [0].

[0]
[https://www.theguardian.com/technology/blog/2008/oct/08/ipho...](https://www.theguardian.com/technology/blog/2008/oct/08/iphone.apple)

~~~
shereadsthenews
There's a zillion things you can estimate this way. A lot of sites use
sequential cookies, user IDs, etc. Until about a decade ago UPS tracking
numbers were sequential for each shipper which made it trivial to estimate
output for online shops. Apple invoice numbers used to be dense and sequential
and you only needed the number to retrieve the invoice. The IMEI is actually
just about the worst way to have estimated iPhone sales in 2008; at that time
you could literally have crawled Apple's website for every invoice whether
sold online or in stores.

------
kevingrahl
Since no one else commented on that yet; I just wanted to say that I like the
simple layout OP is using.

Not much clutter & straight to the point. Loads fast and it’s under 630KB.

Could certainly be improved but it’s nice not having to load >25MB just to
read an article.

------
d--
This is also a good (applied, with simple code) example of the use of
probabilistic programming. I can't get myself to read full books, but somehow
this simple example gave me some intuition and additional pointers to follow.

------
dang
Related from 2016:
[https://news.ycombinator.com/item?id=13095178](https://news.ycombinator.com/item?id=13095178)

2015:
[https://news.ycombinator.com/item?id=10517882](https://news.ycombinator.com/item?id=10517882)

2009:
[https://news.ycombinator.com/item?id=670065](https://news.ycombinator.com/item?id=670065)

------
squeakynick
It depends on how you intend to 'score' the estimate.

Are you looking for the answer that is the 'most likely', or one that has the
'lowest least squared error', or maybe one that is 'unbiased' (mean error)?

[http://datagenetics.com/blog/march22014/index.html](http://datagenetics.com/blog/march22014/index.html)

------
slyu
I recommend Think Bayes by Allen Downey if you want to study more. It's a free
book available online.
[http://www.greenteapress.com/thinkbayes/thinkbayes.pdf](http://www.greenteapress.com/thinkbayes/thinkbayes.pdf)

------
RickJWagner
"the Germans, being Germans, had numbered their parts in the order they rolled
off the production line"

Probably in today's world this is racist or nationalist or something. But (as
someone of German descent) I have to admit it's funny.

------
ngneer
I remember studying this problem in the context of anonymity a few years back,
defining immeasurability as the property whereby an adversary cannot
distinguish between different node counts, for example. The tank problem is
related to mark recapture techniques for animal population size estimation.
Shameless plug,

[http://www.cs.technion.ac.il/users/wwwb/cgi-bin/tr-
get.cgi/2...](http://www.cs.technion.ac.il/users/wwwb/cgi-bin/tr-
get.cgi/2011/MSC/MSC-2011-06.pdf)

------
debbiedowner
Nice write up.

Little bit of a funny though: Note how num_tanks ~ Unif(max(captured),2000)
was defined, so you already have p[ parameter | data ]. Isn't this already a
posterior?

I get however how if you had the r.v.s num_tanks ~ Unif(M,2000), observed |
num_tanks ~ Unif(1,num_tanks), M some constant, that you could find a
posterior distribution num_tanks | vector<observed> by first finding the joint
via E[ 1[num_tanks < t]P[observed | num_tanks] ]

------
joker3
Given the praise for Bayesian methods here, I'm surprised the author didn't
discuss the Bayesian solution. See [http://isaacslavitt.com/2015/12/19/german-
tank-problem-with-...](http://isaacslavitt.com/2015/12/19/german-tank-problem-
with-pymc-and-pystan/) for a similar exposition.

~~~
eadan
I implement the Bayesian solution in pymc3 at the end [0].

[0] [https://www.eadan.net/blog/german-tank-
problem/#probabilisti...](https://www.eadan.net/blog/german-tank-
problem/#probabilistic-programming)

------
coldcode
More impressive than using modern tools is that people in WW2 figured this out
and modeled it on paper using slide rules.

~~~
eadan
It's actually fairly straightforward to derive the estimators by hand [0].
It's just a bit tedious to do in a blog post.

[0]
[https://en.wikipedia.org/wiki/Discrete_uniform_distribution#...](https://en.wikipedia.org/wiki/Discrete_uniform_distribution#Estimation_of_maximum)

------
ngcc_hk
Very interesting. Especially the three links and in particular

[https://github.com/CamDavidsonPilon/Probabilistic-
Programmin...](https://github.com/CamDavidsonPilon/Probabilistic-Programming-
and-Bayesian-Methods-for-Hackers)

------
salty_biscuits
Why call it probabilistic programming? It is bayesian inference with mcmc (or
am I missing something).

------
matchagaucho
Moral of the story: Don't auto-increment serial numbers :-)

------
jaimex2
Was there only one tank factory?

~~~
jandrese
No but each tank also had a manufacturer code, so this technique still worked.

------
mruts
Seeing that it's a uniform distribution, let's start out with assuming our
sample mean (the average serial number we find) has the same distribution as
the true mean (the actual number of tanks in existence). If this is true,
then:

2 x mean

should be an unbiased estimator of the true mean. But because we are probably
under sampling the extremes, we could use the Bessel correction:

1/(n-1) x summation_{i=1}^n(sample_i)

I would guess this comes out to a better estimation than what the article
says.

Bessel's correction might be a bit of overkill, since it's intended to work
with normal distributions. But I still suspect it comes out to a better
estimation that what the blog post says.

~~~
comicjk
I thought the same, and tried it. The mean sample mean after 10 million runs
is 500.47, which is very close to the true mean, 500.5. Bessel's correction is
not correct here - it's effectively multiplying by n/(n-1), so your estimate
would be 536. Bessel's correction is made for estimating true _variance_ using
a sample, not for estimating true mean.

~~~
mruts
Yes of course, the mean is the first moment and a Bessel correction would be
inappropriate. Now I feel stupid. The mean calculated by sum(p_t x_i) or 1/n
sum(x_i) is already the best linear unbiased estimate. Maybe we can't get
better than twice the mean?

