Another fun bit of biochemical history: Chaim Weizmann (1) was a biochemist and staunch Zionist who gained the attention of First Lord of the British Admiralty, Winston Churchill, for cultivating a bacterium, Clostridium acetobutylicum, that could produce acetone, which was in short supply and required for the production of cordite, the key propellent in naval artillery during World War I. In gratitude for Weizmann's contribution to the war effort, George Lloyd asked him what Britain could do for him, to which he replied "not for me, but for my people", which begat the Balfour Declaration (2) establishing Britain's commitment to provide a Home for the Jewish People (3).
(3) It is essentially lost to history that the Balfour Declaration also provided that the Palestinian people should not be displaced. It is also mostly lost to history that there was a great deal of politicking between Weizmann arriving in England and his audience with George Lloyd, including a world tour Weizmann orchestrated to promote one Albert Einstein.
In practice, this is clearly true. But it is not for lack of trying, at least on some people's parts. I recently read 'A Peace to End All Peace', David Fromkin, which goes in depth into the background here, including Weizmann's role and the making of the Balfour declaration. A passage from the book's conclusion has stuck with me:
"It took Europe a millennium and a half to resolve its post-Roman crisis of social and political identity: nearly a thousand years to settle on the nation-state form of political organization, and nearly five hundred years more to determine which nations were entitled to be states. Whether civilization would survive the raids and conflicts of rival warrior bands; whether church or state, pope or emperor, would rule; whether Catholic or Protestant would prevail in Christendom; whether dynastic empire, national state, or city-state would command fealty; and whether, for example, a townsman of Dijon belonged to the Burgundian or to the French nation, were issues painfully worked out through ages of searching and strife, during which the losers—the Albigensians of southern France, for example—were often annihilated. It was only at the end of the nineteenth century, with the creation of Germany and Italy, that an accepted map of western Europe finally emerged, some 1,500 years after the old Roman map started to become obsolete. The continuing crisis in the Middle East in our time may prove to be nowhere near so profound or so long-lasting. But its issue is the same: how diverse peoples are to regroup to create new political identities for themselves after the collapse of an ages-old imperial order to which they had grown accustomed."
One big difference with the Middle East is that the present day borders were largely drawn up by foreign powers (England, France) rather than evolving organically as they did in post-Roman Europe. This is also a source of much of the ongoing conflicts in the region including Israel/Palestine.
That's one of the big points made in 'A Peace to End all Peace', I just felt my comment was too long as is. Here's how Fromkin puts it, edited down for length:
Some of the disputes, like those elsewhere in the world, are about rulers or frontiers, but what is typical of the Middle East is that more fundamental claims are also advanced, drawing into question not merely the dimensions and boundaries, but the right to exist, of countries that immediately or eventually emerged from the British and French decisions of the early 1920s: Iraq, Israel, Jordan, and Lebanon.
...
The disputes go deeper still: ... whether the transplanted modern system of politics invented in Europe—characterized, among other things, by the division of the earth into independent secular states based on national citizenship—will survive in the foreign soil of the Middle East. In the rest of the world European political assumptions are so taken for granted that nobody thinks about them anymore; but at least one of these assumptions, the modern belief in secular civil government, is an alien creed in a region most of whose inhabitants, for more than a thousand years, have avowed faith in a Holy Law that governs all of life, including government and politics.
> rather than evolving organically as they did in post-Roman Europe
You think the borders left after Rome fell were organic? They just used the old Roman province borders. Rome replaced the rule of every country it conquered and the conquest lasted so long that there was no trace of the old rule left, they all had to rediscover themselves afterwards, including figuring out how to redraw the borders.
Yes but they were active participants in that process (through wars, treaties, etc). It’s not like Russia came in, took a map, drew some lines, and gave France to the Bourbons and England to the Plantagenets.
Doesn't prove much. The Jews needed the fertile lands that were very much settled, there were not all flocking there for some desert.
Quoting Ben-Gurion:
A people which fights against the usurpation of its land will not tire so easily. ... When we say that the Arabs are the aggressors and we defend ourselves — this is only half the truth. ... [P]olitically we are the aggressors and they defend themselves. The country is theirs, because they inhabit it, whereas we want to come here and settle down, and in their view we want to take away from them their country.
> It is essentially lost to history that the Balfour Declaration also provided that the Palestinian people should not be displaced.
It is equally essentially lost on most people today, that the Balfour Declaration did not create the state of Israel, nor did the United Nations create it, nor the UK, nor the US, nor was it post-war "resettlement plan" for displaced Jews by the Allies. To the contrary, Jews at that time were barred from entering the British Mandate.
Israelis declared themselves a state, similar to Americans declaring themselves independent of Britain, against the wishes of the UK. So it should also be noted that no "plan" for that region called for a civil war: but a civil war broke out. So it's revisionist shoehorning of the events that played out to say that they didn't match such and such of various plans that may have previously been made but never came to fruition. The cis-Jordan Arabs were not treated worse by the Israelis than they themselves were attempting to treat the Israelis; the Arabs did lose the military conflict, however, after which they displaced their own longstanding Jewish populations.
I continue to be amazed the things you learn on HN. When I was in Poland last summer I learned that a number of Jewish leaders there between the two world wars advocated for a Jewish homeland. But they were never able to convince the government to publicly declare that support. Despite the fact at the time Jews made up to 25% of Poland's population.
It also can't be overstated that Balfour himself was a staunch racist and antisemite. His motivation for passing the Declaration was at least in part the idea that if the UK gives the Jews their own country, the UK will have a powerful ally in the global Jewish conspiracy - and also encourage Jews to leave the UK and not manipulate in its politics, culture and economy.
From Balfour's own writing in 1919 cited in his Wikipedia article:
> [Zionism would] mitigate the age-long miseries created for Western civilization by the presence in its midst of a Body [the Jews] which it too long regarded as alien and even hostile, but which it was equally unable to expel or to absorb.
He's the best example how support of Zionism and antisemitism aren't mutually contradictory and can actually go hand in hand. It's likewise often lost on people that there was a Zionist project helping German Jews emigrate to Palestine with support of the German government even after Hitler came to power, although of course (like all migration) it ended with the beginning of World War 2. This isn't to say the Nazis were fond of this project but they didn't actively oppose it. They did however pass laws requiring emigrating Jews to liquidate their assets (i.e. sell off any businesses or property) and significantly limiting the amount of wealth they could transfer out of the country just like they later dispossessed (and subsequently re-privatized) Jewish business owners and confiscated their property during the Holocaust.
I'm not saying that this is a bad comment, or that it should not have been made; but at the same time, I am also a bit wearied to see that the HN comment section has achieved a Time To Palestine of 1 on this post.
depends on who you ask, for german jews under the nazis before world war 2, it was almost the only place to go to, and definitely saved them from certain death in places like the netherlands.
not to mention after world war 2 for those who came back from the camps and found their homes taken.
moreover zionism did not start with the balfour declaration, but decades before with mass killings of jews in eastern europe
the western allies certainly could and should have done much pre-war, and arguably in the war (mosquito strikes on the camps, for example). my point was that few can dispute that, as we see today, the whole imposition of israel on its neighbours has been a disaster.
that's only when taking only palestinians into account. When the balfour declaration was signed already 10% of then population was jewish.
Mass killings of jews had already started in eastern europe decades before the holocaust. which was what prompted zionism. This is not a singular event, but a movement that would happen with the british or without them.
And any result of the conflict would cause serious "disaster" for one people or another
Really the ultimate cause for all of that pain and suffering is the failure of the Ottoman Empire which was quite cosmopolitan while it lasted.
There is plenty to criticize about the actions taken by many parties post-collapse, but I strongly doubt it's at all possible for external powers to take a failed empire and keep all the pieces happy in the aftermath. Those people have to do most of that themselves, and in this case they didn't for many reasons. Europe was ready for the transition away from monarchy and empire, the middle east wasn't. Russia/USSR tried a path that worked but not particularly well. Luck and circumstance also played a part.
A century of war and strife post empire collapse isn't exactly unusual.
Well it was cosmopolitan because it conquered many people in an area that was cosmopolitan from the start (in particular Constantinople) and as long as they didn't fight back for their independence they were tolerated.
That's not much different from the current state of Israel which allows Arabs to live in it (and even have political representation!).
I don't know how to square the elevation of the tolerance in the ottoman empire while at the same time decrying the apartheid within Israel's borders.
The situation is fucked up. It's hard to have justice and peace.
> I don't know how to square the elevation of the tolerance in the ottoman empire while at the same time decrying the apartheid within Israel's borders.
Well for starters I don’t test the Ottoman Empire in the 19th century and Israel in the 21st against the same standard. We’re not saying the Ottomans were great by a modern western standard, but compared to most of the west in their day they were quite good.
yeah, fair enough. I don't think there one "right way" of looking at all this. I think it's important to keep all those things in mind at the same time, including what you're saying about standards and how they change over time.
My point was that when we look back at times of peaceful coexistence within empires, it was often the results of suppression of dissent. Empires are fundamentally at odds with national self determination. Empires are by definition violent subjugation of people and self-determination movements often require violence to counteract that.
It's easy to take past empires for granted and assume they are "how things are"; but when they fail and crumble they do so for a reason, they contain the seed of their own destruction, because they impose stability and order over internal and extern forces that push against them.
Are you confused about why purchasing a large amount of nearly empty land with minimal disruption to the small number of people living there produced less conflict than a large number of people moving into an already-occupied area? The Alaska Purchase and the Israel/Palestine situation are two of the least similar things you could possibly pick to compare.
But your example is a bit like 'why is it so difficult to build a hundred-storey building? My uncle John built a shed in his back yard once, perhaps we should ask him to share how he did it.'
Alaska was sparsely populated and had no (as of then discovered) natural resources. It was also not strongly tied to ethnic identity and religion as Palestine is both to Jews and Muslim Arabs. And it did not have a huge influx of people needing land which immediately set the scene for conflict with established inhabitants.
What’s missing in your world view is that “high functioning” countries largely ride on the backs of lower functioning ones and depend on low cost labor to sustain their quality of living. Eg the US is just coming off of 20+ years of riding on China’s back.
> largely ride on the backs of lower functioning ones
Where has this lie come from? "Higher functioning" countries generally stem from countries that were doing well long before they met many other countries. The British invented naval clocks, required to circumnavigate the globe, before they went round the globe on a regular basis. How did they do that if everyone was equal before they met?
Really? You’re going to claim that the prosperity of Britain did not come from systematically plundering the rest of the world? Is this what you’re going to go with as your counterpoint?
It didn't. It came from the Industrial Revolution and a step change in the productivity of british society. Many economic studies have shown that the "Empire" and especially parts of it like India, the African colonies etc were a net negative to Britain economically speaking. This should not be surprising - just as Afghanistan or Iraq were a net negative to the US more recently, but still allowed certain segments of US society (the military industrial complex) to enrich themselves at the expense of other parts.
Probably in 50 or 100 years there will be some Afghan nationalist movement which will be telling everyone that the entirety of US wealth is based on plunder of Afghanistan. That's roughly how it is with India nationalists and Britain today, but of course, it is nonsense.
>Really? You’re going to claim that the prosperity of Britain did not come from systematically plundering the rest of the world?
I think the OP's question is how did Britain get it a position where it had the capability to "systematically plunder the rest of the world"?
Do you think that was luck? Were the British simply "evil", and everyone else not so? One has to assume they were already high-functioning, prior to the plundering.
I never claimed it was luck. I merely pointed out the mechanism of acquiring the riches. Eg most people in the US have no idea much of the East Coast was built with money from the Chinese opium trade, and how many of our “elites” got their start as basically drug cartel members. There’s lots of this in the history of any empire. You just have to dig a little - the empires don’t like to talk about how they made their first trillion
In the case of Britain, it was the following (and more):
- be colonised by the Romans 2000 years ago
- spend the next 1000 years rediscovering and (eventually) surpassing the Romans' achievements
- invest heavily in education (when the Incas were building their monuments, most of Oxford University had already been constructed) and subsequently have 1000 years of cumulative effect of those investments
- be highly connected to Western Europe, with all the continual advancements shared between countries all trying to outdo each other
- be an island nation, with a seafaring tradition that was accelerated by all of the above to the point of being the best in the world
That happened after they got powerful. It is like the Mongol invasions of Europe and China, they didn't become powerful from those invasions those invasions happened since they were powerful.
Do you believe that just no one thought to "plunder" other places? Think about what enabled the British to go afar and make an empire. The ability came before the empire.
Yes. Same as you can be exploited and grow your earnings over time as you move up the value chain. Note however how unbelievably strenuously the US is trying to keep them from progressing technologically. It’s to the point of imposing its will on _other_ countries by now. That is not accidental, and it has nothing to do with “national security” or whatever. It’s just to be able to milk that teat for a little longer. Military gear by design uses older chips that are easy to produce should a war break out.
America's biggest sources of imports are Mexico, Canada, China, Germany, and Japan.
I'm wishing we could do more to improve the lives of people living in countries like Sudan, Venezuela, Burundi, Haiti, DR Congo, etc. They're humans too even if they don't make the news. Right now those countries are too dysfunctional to be competitive for most exports.
Problem is that they've been made so dysfunctional by colonialism, and then often even after formally gaining independence, they continued to be purposefully destroyed by colonial forces through colonial taxes, financing corruptions, political influences, etc. Haiti was one of the riches countries in that part of the world, until France killed their economy by sanctions in revenge for their successful slave uprising. It lasted until 20th century, Haiti was paying France huge money. Some African countries are still to this day paying France taxes and France keeps all their money in the French central bank. Other countries like Belguim, Spain and even US were not much better. US dealers were buying off the colonial debts and traded with them on US stock-exchanges until 1940s or so.
And today this continues in the form of corruption by multi-national corporations. Almost all diamond mines are owned by a few western companies. Do you really think they want progressive democratic government in any of those countries? Nope, corrupted dictators are way better for business...
> Every country was a utopia until the British and French got there?
Nope, but you can bet that in every single case it got way worse when British, French, Spanish, Belgians, Russians, Italians, Venetians, Turks, etc. occupied them by often unspeakably brutal use of force, and then exploited their natural and human resources while giving close to nothing in return.
1) Alaska was (and still is) a huge, and very sparsely populated land
2) For the native population it was not a big difference whether Russians or Americans ruled them, none of them treated them well - it was a colony and all they cared of was getting the natural resources.
3) Russians were not kicked out of their homes, as there was very few of them there in the first place (mostly working for the government). And then there was also proportionally very little American settlers coming in compared to the vastness of the land available.
If you want a better comparison with US history, look at the Trail of Tears, it's much closer to what happened in Israel...
Interesting read. I don't think this came up in my stats classes:
> Gosset solved many problems at the brewery with his new technique. The self-taught statistician published his t-test under the pseudonym “Student” because Guinness didn’t want to tip off competitors to its research. Although Gosset pioneered industrial quality control and contributed loads of other ideas to quantitative research, most textbooks still call his great achievement the “Student’s t-test.”
This is such a great story, it should be included in every intro stats class (I did, back when I taught intro stats).
Gosset didn’t have the mathematical background to derive the correct distribution theoretically, so he figured out what it was by simulating drawing samples of different sizes thousands of times and fitting curves. Simulating, in those days, meant writing numbers on thousands of cards, then shuffling and drawing a sample. Calculate the mean and standard deviation. Repeat. Thousands of times. He published the result with an apologetic shrug for not being able to prove it properly.
It's interesting how mathematically shallow most stats presentations are. In most other areas I've studied, you start from some basics like axioms and gradually build up machinery by proving theorems etc. But presentations I've seen of the t-test focus on when and how to use it, without going very deep into the derivation at all.
This leaves me skeptical of the movement to replace calculus with stats in high school. It's true that an ordinary citizen will find stats more useful. But for students who will go on to become scientists and engineers, I think they should study calculus. Calculus is a better on-ramp to the sort of rigor you need in upper-level math. And I'm concerned that a bad "cargo cult" stats class may be worse than no stats education at all. Calculus education seems harder to screw up.
Calculus is also mathematically shallow in that sense: the subject where you start with axioms and gradually build up the machinery of calculus is (Real) Analysis, which is not part of the standard calculus curriculum and which the vast majority of people taking calculus will never study [1]. A typical Calculus class expects students to memorize and use things like trig function integrals which are presented without proof; not so different from memorizing and using statistical tests presented without proof, in my opinion.
In an intro statistics class I think conceptual depth is more important than mathematical depth. It's more important that students really understand the concept of probabilistic inference, both hypothesis tests and confidence intervals, than that they understand the mathematical derivation of the t distribution [2].
Unfortunately intro stats classes often fail on this count as well. One of the (many) straws that eventually broke my desire to teach was a committee decision--a committee composed entirely of people not teaching intro stats--to disallow students from bringing formula cheatsheets to exams, effectively forcing us to make the students memorize formulas rather than focusing on conceptual understanding.
[1] When I took Real Analysis there was a calculus class that met right before us in the same room, which often ran over so that the calculus students would be packing up as we entered the room. One day as we're sitting down one of them asks us what class we're there for, and then asks what Real Analysis is all about, since he's never heard of it. One of my classmates responded with the absolutely perfect "Well, our homework last night was integrating x^2 from 0 to 1."
[2] I'd say the same goes for Calculus, for what it's worth; actually understanding what an integral means is more important than being able to set up the Reimann sum and take the limit.
> A typical Calculus class expects students to memorize and use things like trig function integrals which are presented without proof; not so different from memorizing and using statistical tests presented without proof, in my opinion.
You can see the derivatives based on their shapes, it is very intuitive that way unlike statistics. Calculus tries to show the intuitive explanations for the different parts, and it works very well it is how we get engineers and those engineers has built almost everything we see in modern society.
Intuition is much more useful than formalism unless you need to prove things, and calculus is one of the most intuitive subjects in math once it clicks. That intuition then stays with students for their entire lives, often they don't understand they got it from calculus class. Humans has very poor intuition for things like velocity and acceleration until after they take calculus, but people forget that since those things feels so extremely obvious after you have taken the class. That just shows how great the class is, that it helps solidify such important concepts that people don't even realize they used to have trouble with them.
> Unfortunately intro stats classes often fail on this count as well
Because unlike calculus statistics is very unintuitive. The only ways to teach it is either via extreme formalism or by showing people what numbers to put where, there is no intuitive way to teach statistics as far as we know. That makes statistics classes and knowledge you learn there churn much faster than the intuition built in calculus, making the class inherently less useful.
All the useful things I know about statistics I learned in middle school, stuff like confidence intervals, sampling bias etc. Everyone learns that in middle school, later stats classes just focus on churning numbers, I learned nothing in later stats classes I was forced to take in college.
Not sure why you are downvoted for this. Your points have merit. I agree that stats classes can have a "cookbook" flavor and do not generally lead to a deep understanding of probability. But I would rather fix the stats classes than abandon the topic.
Does anyone really argue to replace calculus with stats? I thought the idea was to offer both and let students choose based on their interests.
Propbabilities, combinatorics, logics and sets are the most valuable things from high school maths that benefitted me all the way through from teenager to professor.
Calculus is intellectually stimulating, but for my line of work (dealing with uncertaintly, risk, decision making, AI), other parts of mathematics are more useful. However, I would not argue calculus should be replaced. I would argue for more "proper" maths to replace "recipe-like" maths. It's more important to go deeper on a topic than what the topic is.
> (dealing with uncertaintly, risk, decision making, AI)
Rate of change, slopes, finding maximal points etc is extremely important in all of that, and that concept you learned and got intuition for in calculus. Calculus isn't worthless for you at all, you just undervalue it.
You might not have needed to study more calculus than the original courses, but that just makes that course even more valuable, it is so useful in so many domains even though it is a low level math class.
Um. Wow. That's quite a story. But, it's not real. "owever, Guinness had a policy of not publishing company data, and allowed Gosset to publish his observations on the strict understanding that he did so anonymously."
I'm 1906, Gosset was the guest of Person at UCL, and since Gosset had a First in Math, and Professor Pearson was the leading mathematician and publisher of the Bell curve..
Gosset spent a year at UCL. University College London. A year with an expert looking over his shoulder? I would think that he would publish with an extreme amount of confidence, forgoing the need for an apolocetic shrug, which I have never ever heard of. Never, and I have a degree in math with a minor in Statistics. They had playing cards. You are arguing for large sample sizes, which is not economical - precisely against the design of the test - which looks surprisingly suspicious.
> I would think that he would publish with an extreme amount of confidence, forgoing the need for an apolocetic shrug, which I have never ever heard of. Never, and I have a degree in math with a minor in Statistics.
Good for you. As you might have guessed from reading that I used to teach statistics, I have a bit more than a minor in the subject. Your attempt to appeal to authority, not to put too fine a point on it, falls flat.
Just because you haven’t heard of a thing don’t mean it isn’t true. We can, after all, just read the original paper:
> Before I had succeeded in solving my problem analytically, I had endeavoured to do so empirically. The material used was a correlation table containing the height and left middle finger measurements of 3000 criminals, from a paper by W. R. Macdonnell (Biometrika, i, p. 219). The measurements were written out on 3000 pieces of cardboard, which were then very thoroughly shuffled and drawn at random. As each card was drawn its numbers were written down in a book, which thus contains the measurements of 3000 criminals in a random order. Finally, each consecutive set of 4 was taken as a sample—750 in all—and the mean, standard deviation, and correlation5 of each sample determined. The difference between the mean of each sample and the mean of the population was then divided by the standard deviation of the sample, giving us the z of Section III.
As for the apologetic shrug, in the course of the “analytic solution” we have:
> The law of formation of these moment coefficients appears to be a simple one, but I have not seen my way to a general proof.
and then after a bit more math guessing the correct distribution based on the moments
> Consequently a curve of Prof. Pearson’s Type III may he expected to fit the distribution of s2.
My story is slightly off; Gosset only used one sample size rather than several different sample sizes. But he did use simulation with thousands of hand written cards as his approach to the problem, he did fail to prove the correct distribution (moments are not sufficient to determine the distribution), and he did publish with an apologetic shrug.
> But, it's not real. "owever, Guinness had a policy of not publishing company data, and allowed Gosset to publish his observations on the strict understanding that he did so anonymously."
Except that this part is true. Obviously, he is well-known in academic circles, but Guinness did have a policy against its employees to publish their research using a pseudonym[1].
[1] Specifically, they can publish with three conditions:
1) To not mention Guinness or its competitors,
2) To not mention anything about beer (so topics specifically about beer is forbidden), and
3) To not publish using their surname (which in practical effect is to publish using a pseudonym).
I always thought that name was strange but I never thought to look it up. Stats books are so dry, they don’t have the inclination to share these kinds of stories.
Doubly unfortunate because the philosophical aspects of statistics are more important to students than most of maths. There are something like 4 different schools of thought [0] and people will have a natural propensity to one of them.
Although they all agree on the formulas and rigorous aspects, it is actually a challenging proposition to comprehend what someone is doing if you strongly see the world from one perspective and don't realise that academics are potentially approaching the interpretation in one of 3 other ways.
It adds a lot of dryness to the textbook because the author can really only talk about the objective parts in an introductory classroom setting. But if you're getting taught by a frequentist and have a subjectivity bent it is easy to spend a year or two confused before someone clues you in that there are unresolved questions of interpretation.
Interesting, my university book of the subject was pretty tiny but it did talk of the different interpretations, but it only mentioned frequentist and bayesian. I did not suspect the story was much more complicated.
> Stats books are so dry, they don’t have the inclination to share these kinds of stories.
It doesn't have to be that way. My pandemic lockdown read was a 10 dollar Stats textbook[1], that comes with tons of classic examples: the Salk polio vaccine, a prosecutor misusing the multiplication rule using purely circumstantial evidence ("what are the odds that police pulled over the wrong couple matching 10 different pieces of description by the victim?"), the classic Gallup poll showing FDR would defeat Landon (versus incumbent _Literary Digest_ showing a Landon win), Gosset's history with Guiness, the early history of probability as gambling strategy, a controversy over Mendel's data on pea plant heredity being _too_ clean, and so on.
Sadly, while this book left me well prepared to apply statistical reasoning in my day job, it's departure from typical pedagogy left me feeling unprepared for further reading based on perusal of Stats Wikipedia -- what's a kernel? what's a moment? etc.
In a former life I taught some intro stat courses from this book. It’s a good book for an intro course for people who aren’t going on to further stat classes, although I think for a current class I’d want something that acknowledges how statistics and computers have gotten all tied up with each other. (I don’t have any recommendations - it’s not my job to know this any more.)
You probably missed it. This is something stat book authors love to mention. I don’t remember a stat intro book that doesn’t have a footnote for “Student t”.
> In this illuminating volume, Robert P. Abelson delves into the too-often dismissed problems of interpreting quantitative data and then presenting them in the context of a coherent story about one's research. Unlike too many books on statistics, this is a remarkably engaging read, filled with fascinating real-life (and real-research) examples rather than with recipes for analysis. It will be of true interest and lasting value to beginning graduate students and seasoned researchers alike. The focus of the book is that the purpose of statistics is to organize a useful argument from quantitative evidence, using a form of principled rhetoric. Five criteria, described by the acronym MAGIC (magnitude, articulation, generality, interestingness, and credibility) are proposed as crucial features of a persuasive, principled argument. Particular statistical methods are discussed, with minimum use of formulas and heavy data sets. The ideas throughout the book revolve around elementary probability theory, t tests, and simple issues of research design. It is therefore assumed that the reader has already had some access to elementary statistics. Many examples are included to explain the connection of statistics to substantive claims about real phenomena.
I wonder if dry reading means written without the influence of drink. I couldn't find an answer online. But, if so, it would be ironic to describe a stat book that ignored a brewer as dry.
I agree with the sentiment, but I always have wondered what t-test a real engineer uses and why they only teach the "Student" version. Given the context, a bit of a clarifier would have been appreciated.
At least they let him publish, albeit under a pseudonym. It makes me wonder how many potentially useful discoveries were made in industrial settings, and wound up being buried due to management not wanting to risk leaking competitive information. The good news, I suppose, would be if you believe that it's rarely the case that only one person could ever discover something. Then you can conclude that all (most?) such discoveries were eventually (or will eventually be) rediscovered independently.
On a related note... I wonder how much valuable research disappears (more or less) when companies fold, get acquired, etc. Take MCC[1] for example. I've been doing a lot of reading lately that involves old papers from the 1990's on "agents" and "multi-agent systems". And time and time again, in the references, you'll see something like "MCC Technical Report TR86-32791" or some-such. Occasionally said report can be found online, but quite a few of them seem to be either hard - or impossible - to find. Maybe there's an archive of physical papers stored away somewhere, but FSM knows where the heck such a thing would be, or how hard it would be to get access.
A similar situation came up a while back when we started discussing "sharding" here on HN[2]. There was a lot of effort spent trying to identify when the term first arose, and a lot of evidence pointed to a particular paper that was internal to CCA, who were acquired by Xerox. And now that original paper seems to be unobtanium. The paper probably still exists somewhere in the bowels of Xerox, but good luck ever getting your hands on it.
> It makes me wonder how many potentially useful discoveries were made in industrial settings, and wound up being buried due to management not wanting to risk
Probably a lot. I’ve come to find out that some dinosaur companies won’t even let their programmers open up issues on open source repos (forget sending patches or releasing their own software).
The logic goes like this: if someone found the log4j zero day before it was reported they could comb through all issues and see the companies that the users worked for then try to target them. In this case any comment would indicate possible involvement.
The least bit of security, through the tiniest extra bit of obscurity. Thankfully many of these companies are starting to come around and realizing that a lack of involvement with open source is more risky than accidental 3rd hand information leaking (like what dependencies doesn’t certain company use).
The easiest counter to this is that, to my knowledge at least, it’s easier to build a vulnerability scanner than to scrape repos for more targeted attacks.
The "No lieutenant, your men are already dead" defense. I like it.
I think that if your threat model includes nation states (and the companies I was referencing above was largely S&P500 financial institutions) then you have to think the attacker also doesn’t want to trip off any alarms with a ham fisted port scan blasting the precious zeroday exploit all over the internet. Your point is still extremely valid though.
Which is why the counter I provided is that the best defense is to get as many engineers’ eyes on the problem and in the codebase as possible to prevent or find it before it becomes an issue. Things like lib XZ are scary, but it’s even scarier if not caught before it’s in the wild.
The dirty secret is that nation states can get your software dependency list pretty easily in a number of ways (e.g. sending agents to meetups to nerd out & make friends would be an expensive way but there’s other social engineering attacks I’ve observed).
The other secret is that monitoring software can’t detect anomalies ahead of time & the vulnerability scan will not show up meaningfully any different than all the other random traffic already happening. Your nation state can hide it’s vulnerability scan amongst all the other vulnerability scanners already running (both legit as a service when you request it against your server & illegitimate actors trying to find a way in). So at best a ham fisted search is unlikely to really tip your hand in a meaningful way unless it requires having penetrated a few layers of your security to begin with.
As for libxz, the scary part is that as an industry we recognize the security challenge of not compensating maintainers and yet we have lackluster responses to fixing it (e.g. Google trying to pay OSS maintainers to harden their security while completely ignoring that a huge problem is that the maintainers can’t devote full time which opens an avenue for malicious actors to overwhelm maintainers & take control socially as happened with libxz).
Many international conferences are regularly held in Dublin, and attendees often visit the Guinness brewery as part of conferences' social events, where a memorial plaque reminds them of Gosset and his important contributions to statistics.
My first year of working at AWS was in the "DUB1" site, which was part of the Digital Hub (a tech and incubator space). As it happened ... Amazon's small office was in William Sealy Gosset's old laboratory, right beside St. Patrick's Tower where the Guinness cooperage was. As a former statistics lecturer, I excitedly told everyone I worked with how lucky we were, to almost no reaction! Can you imagine?
A long time ago I submitted William Sealy Gosset as a suggestion for commemoration with an Irish Postage Stamp; but nothing has ever come of it. I hope some day he gets more recogonition.
Guinness was way ahead of their time and in many ways the Google of their day (building accomodation, high pay, great perks). My granddad worked there so all my dad's brother and sisters learned to swim in the Guinness swimming pool!
Funnest tidbit is that the widget to get a good head from cans won the best invention of the year, the year the internet wax invented.
I went to school beside the Guinness brewery. The smell of the hops brewing will remain with me forever.
However the school was dirt poor in many ways and they wouldn't sponsor our school football team to buy some kit. This was back in the 80's Ireland with massive unemployment and huge emigration.
Ironically the school had a computer lab way beyond its time when one ex-pupil donated a couple of Apple Macs and a dozen Apple IIe's. That's were I cut my teeth on some - probably BASIC - programming, learning myself.
On Topic - Guinness were always canny and ahead of their time. Getting a job there was like winning the lotto, you were pretty much made for life.
I love these sorts of of things. I was fortunate enough to have this fact mentioned in my stats book in undergrad, and later when I was in Dublin and touring the Guinness brewery, they had a small exhibit on Gosset (although Gossett was actually based out of their brewery in England). Another fun fact that I learned in organic chemistry was that Alexander Borodin (the Russian composer who composed "In the Steppes of Central Asia" and "Prince Igor") was only a composer in his spare time and was actually professionally an organic chemist who was the co-discoverer of the aldol reaction
I introduced the voiceover artist at the beginning, she’s a Japanese speaker that I found on fiver. I chose her because the talk was being given at a Japanese speaking conference (Ruby Kaigi).
I love the foil of having a second character on screen but having an accented cartoon was not the right effect I was going for.
I’m still experimenting with multiple characters in my talks but my most recent one I did doesn’t use any actors and I read their lines like a narrator. I think the effect works much better https://m.youtube.com/watch?v=-8UQMH6p-Mw&list=PL9oQ7yETvN12...
ah, appreciate the reply, and the spirit of creativity.
Kinda feel like I got caught being overly critical and here is the actual creator! Thanks for receiving it well and also for putting your stuff out there.
I’m kinda glad you said something to be honest. I wanted to mention the voiceover being cringe but it’s also hard to pre-apologize for something without raising a lot of alarm bells.
I’m still proud of the overall talk. I’m glad I pushed a limit, found it, and learned from it.
Your comment stated what you saw and how you felt about it. I think you did a great job speaking up without lashing out or talking down. I appreciate that.
I appreciate you sticking up for me. I can see this comment being taken poorly by others. I also think we generally need to learn to empathize that the creator might experience our words different than how we mean them (and therefore be kinder when we post).
In this case, I was aware that the rabbit is tough to watch so I have the empathy for my viewer and the comment came across as honest rather than harsh. I don’t think you should be downvoted. If it was a different time or place then I 100% could have taken it the wrong way.
This doesn't surprise me; any industry involved with basically any manufacturing seems like a perfect testing ground for statistical methods. There's enough scale in these things where subtle differences can save tons of money, so it can be beneficial pretty quickly.
I always thought the t-test was clever just because of how simple it was compared to more advanced stuff.
We covered this in my Research Methods class as part of my Systems Engineering Masters. Growing and agriculture in the early 1900s benefitted from things like ANOVA, blocking factors, nuisance factors, factors, levels, ranges and experiment design such as full factorial design. It is taken for granted these days. F statistic, F crit, etc
I feel like OP unfortunately mostly misses the point of Gosset's work and the t-test, in the usual way of people taught the contemporary bastardization of Gosset/Fisher/Pearson/Neyman as NHST. The important thing isn't that it lets you calculate some _p_-value; the important thing is that it is a framework for decision-making, where you can trade off your false-positive and your false-negative rates to make the economically rational decision: https://www.tandfonline.com/doi/full/10.1080/00031305.2018.1...
In this case, because it is a well-understood problem with the rates of bad batches easily established from the brewery's records of testing & drinking, it lets the brewery decide on how many 'off' batches it wants to risk in exchange for saving the cost of a certain number of test-samples. You decide you want to risk 1 bad batch in 100 for a false negative while rejecting 1 good batch in 20, then you need _n_ samples etc. And this directly translates better measurements (by lowering variance, eg. by blocking) into money: the lower the variance, the fewer samples you need to achieve any given tradeoff, thereby saving the brewery money on scrapped material or testing. The smaller the better, hence Student's inability to use asymptotics or approximations: they might be off by orders of magnitude. (He would even try to do _n_ = 2 tests!)
Or they might be trying to tightly optimize alcohol content, to avoid taxation for passing high-alcohol content thresholds, but also avoid going too low to disappoint their customers, so Student would explicitly calculate out scenarios, for example:
> Thus, Gosset concluded, “In order to get the accuracy we require [that is, 10 to 1 odds with 0.5 accuracy], we must, therefore, take the mean of [at least] four determinations.” The Guinness Board cheered. The Apprentice Brewer found an economical way to assess the behavior of population parameters, using very small samples.
(If you're thinking this sounds like a very subjective-Bayesian decision-theory thing to write, you are right, although Student would have rejected that, like most statisticians, and emphasized that he was dealing with populations with known base rates, and so nothing Bayesian was necessary; it was just a frequentist decision-theory approach.)
Sometimes I wonder about the actual utility of the T-test compared to just looking at a pair of boxplots, with jittered points (or some other indication of the number of data points).
If it isn't plainly evident from the boxplots (assuming you've got "enough points") do T-tests alone ever make a truly compelling argument?
But you still need to choose a significance threshold, which is just an opinion. There's nothing "scientific" about p=0.05; surely God loves p=0.06 almost as much.
I read this story and several other very interesting ones in this great book detailing the history of evolution of modern statistics --
The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century by David Salsburg.
The author himself personally met with several leading figures that he describes. Highly recommended!
> Gosset recognized that this approach only worked with large sample sizes, whereas small samples of hops wouldn’t guarantee that normal distribution. So he meticulously tabulated new distributions for smaller sample sizes.
Does it mean Gosset stop before the distribution converging to normal distribution?
The nitrogen is just a propellant for forcing the beer through tiny holes that make the dissolved CO2 come out of solution in tiny bubbles, which form a more stable foam. Very little nitrogen actually dissolves into the beer.
The students' t distribution has a symmetric PDF (with no skew), and thus you assume that the sample and/or population also have such a PDF (Probability Distribution Function).
>> the Kolmogorov-Smirnov two-sample test, which essentially compares the empirical distribution functions of the two samples without implicitly assuming normality. http://en.wikipedia.org/wiki/Kolmogorov–Smirnov_test
> The generalized matrix t-distribution is the compound distribution that results from an infinite mixture of a matrix normal distribution with an inverse multivariate gamma distribution placed over either of its covariance matrices.
But does a matrix t-distribution describe nonlinear variance in complex wave functions?
> In quantum mechanics a statistical ensemble (probability distribution over possible quantum states) is described by a density operator S, which is a non-negative, self-adjoint, trace-class operator of trace 1 on the Hilbert space H describing the quantum system.
A Q12 question:
How frequently are quantum density operators described by a parametric t distribution?
(1) https://en.wikipedia.org/wiki/Chaim_Weizmann
(2) https://en.wikipedia.org/wiki/Balfour_Declaration
(3) It is essentially lost to history that the Balfour Declaration also provided that the Palestinian people should not be displaced. It is also mostly lost to history that there was a great deal of politicking between Weizmann arriving in England and his audience with George Lloyd, including a world tour Weizmann orchestrated to promote one Albert Einstein.