Hacker News new | past | comments | ask | show | jobs | submit login
Probability and Markets [pdf] (janestreet.com)
289 points by max_ on June 16, 2023 | hide | past | favorite | 72 comments

For a much better introduction to financial markets I highly recommend


> The price is a review (even if only a few words) on your preferred social media, either tagged #PricingMoney or with the link jdawiseman.com/PricingMoney.html. There is no paywall, nor registration, nor even cookies, so this price cannot be enforced. Nonetheless, please be fair: please post a review or comment or acknowledgement, tagged or linked or both. Thank you.

This is racking up some upvotes so I've submitted it separately if discussion is desired and so on


During my university studies, I found it surprising that although some topics from advanced courses might be applicable in certain jobs like at Jane Street, interviews often emphasised the fundamentals and understanding of basic concepts such as basic probability theory, rather than some specialised knowledge in Lie processes.

If you look through this PDF it does not even cover a 1.semester course in probability theory from a bachelors degree in CS/math.

I always go back to this lecture series to brush up my fundaments. https://projects.iq.harvard.edu/stat110/youtube

Not Lie processes (are you referring to Lévy?), but stochastic calculus in general, which requires a strong intuition for differential equations, calculus, and statistics. Quantitative finance heavily favor those with a background in the physical sciences rather than discrete math like CS.

That is curious. Could you elaborate or point me to some sources, I would like to investigate that relationship further. I remember Jim Simmons saying that the Renaissance fund employs many physicist PhDs and other kinds of scientists. I thought they were there because of their specialisation but perhaps the reason is what you explained above or both

Physicists are popular because they deal with uncertainty and complex models across the time domain, which is another description of markets. When you zoom out far enough, a fluid dynamicist and a market making quant are not that different.

If you want books about trading mathematics and interviews in general, the green book (practical guide to quant finance interviews by Zhou) is the usual go-to. Falcon's Heard On the Street is another classic. Note both of these focus on generic interviews (the quant equivalent of leetcoding) usually for recruiting fresh grads directly out of school. You can probably skip some steps if you have the work experience or know the right people at certain funds.

If you want actual books on trading and modeling, that's another topic entirely.

Figure out which area specifically you are interested in: https://teddit.net/r/wallstreetbets/comments/kcs4xy/what_qua...

If you want an autobiography, Emanuel Derman's My Life as a Quant is well-regarded. I have never read it myself but Amazon has often recommended me The Physics of Wall Street.

yes I meant Lévy I don't really know what went through my head.

I also have to say that you don't need to have a strong intuition for differential equations even through they are very related. Stochastic calculus can be studied entirely in its own right.

And with respect to quantitative finance my experience is that while a candidate knowing about or having coursework in stochastic calculus is certainly relevant its not favourable.

This also depends on the exact job right, if you have to be pricing XVa products then you better have know Girsanov, Ito and what have you =)

I second this. The fundamentals take you far, and Joe Blitzstein has fantastic material.

Slightly off topic, but how do you feel about his “story proofs”? They were the one part of his lectures that never really clicked with me, but I never talked to anyone else about it.

Readers, if you want a great into to probability co-authored by Joe Blitzstein, see https://stat110.net/.

Why can't I download the PDF?

I enjoyed them - I think it shows how you can make a (relatively) rigorous, yet intuitive, argument that does not necessarily have all of the trappings/wall decoration of traditional proofs. The art is noticing when the story is complete vs. when it is papering over an important mathematical aspect.

I took the course in person, so it might have been more understandable in that context.

Me too - particularly when the story proof was more concise than the algebraic proof, or hinted at the deeper reason as to why something must be true. I remember them better too.

I heard they try to hire for aptitude/talent. The sophisticated stuff they will teach you if you’re hired.

This is similar to how math Olympiad students are picked, most questions involve combinatorics and Euclidean geometry instead of advanced calculus, as anyone can understand a basic counting problem, but it’d take a few semesters to define the terms in a typical calculus problem.

Although Olympiad is only counting, there is considerable prep and practice that goes into it.

That’s true. I meant (almost) everyone can understand the ‘statement’ of Olympiad-level counting problems, but of course solving them is another issue.

I heard convex optimiztion techniques are widely used as well, is that true?

Yes, especially in, but limited to, portfolio construction. Check out the mosek portfolio construction cookbook.

Yes. I worked on convex optimization in trading (deeply involved with fine-tuning, constraint selection, utility function construction, etc.).

For the experts in the thread complaining about the simplicity of the excellent introductory text in the OP, here's an exciting reading for you:

    Statistical Consequences of Fat Tails: Real World Preasymptotics, Epistemology, and Applications by Nassim Nicholas Taleb

Beware that it may be way more advanced than what you'd expect though.

The takeaways from NNT’s books are summarized in the top Amazon review. The summary is actually pretty useful and relevant and highlights some of the pitfalls of applying common statistical concepts to fat tailed distributions and the change of mindset that is needed.

We tend to gloss over assumptions like finite variance or iid but they matter a lot for reasoning correctly on fat tailed distributions.

Also the law of large numbers only works if you don’t have a game over scenario or fat tails.


Personally -- I couldn't get past the first 2 chapters of the book. The notations it introduces are pretty unfamiliar for a newcomer and it quickly becomes really hard to follow. I genuinely would like to hear from somebody who managed to go through the entire book above, and what their main takeaways were (from the chapters that follow the two introductory ones).

The non-technical introduction chapter is pretty easy to follow, and I would recommend reading it.

The rule of thumb with mathematics books is that either you skip the first 2 chapters and read the whole thing, or you only read those 2.

There is a Technical Incerto reading group (this book) that you may find useful.


would you recommend any books/resources before jumping straight to this? Thanks!

Anything that would help build a strong probability theory background.

Assuming you have some freshman-college math literacy, here are a few great introductory books to Probability theory:

    Introduction To Probability by D. Bertsekas & J. Tsitsiklis

    Elementary Probability Theory: With Stochastic Processes and an Introduction to Mathematical Finance  by K. L. Chung & F. AitSahlia 

    An Introduction to Probability Theory and Its Applications Vol. 1 by W. Feller

I'm sure there are some great video lectures on the subject as well, but unfortunately I can't point you to any relevant material since textbooks is what I used when I was in college. But if I had to guess, the latest relevant MIT OCW course that has video lectures available would be sufficient.

first sentence in the doc: The goal of this document is to present an introduction to probability and markets in what we hope is a fairly intuitive and accessible way

HN thread: wHeRe is compleXitY?!?

I remember going through that Pdf when I was interviewing with them. Very helpful stuff. I feel like I learned more about statistics while going through hundreds of Jane Street interview questions than I did during my university studies.

The publication ends with "Interview Myths", so I think this is geared to potential job applicants at Jane Street...

Just like everything they publish.


The first section on randomness seems to imply that all you need to know is information. That if you knew all the data about the weather that you could predict the weather. I think it's more complicated than that. Even if you knew the position, type, and velocity of every molecule of the atmosphere at a given moment you would still need a model that explains how they interact over time in order to predict the future. So, I'm not sure that actually is a knowable unknown because that assumes there is a model that can be made in addition to the information. Maybe the weather is chaotic.

But is that chaos predictable? Current weather predictions are based on statistical models. Storm trackers, for example, typically have bands that expand as time gets further out. Without knowing every molecule that affects a hurricane, for example, they can predict within some confidence interval where the storm will head. The data meteorologists have is relatively coarse, and yet they can make predictions within some level of probability. That level of probability allows municipalities to issue evacuation orders early as opposed to throwing hands up and saying there is no certainty in anything, so let's just see what happens.

Their discussion of die rolling doesn't say they can predict the outcome, rather that they can predict the probability of the outcome. In trading, life, and gaming that probability is one component of many that go into decision making. Knowing how much that randomness effects the outcome at what probability can be a successful strategy (but of course not guaranteed) in all of those domains.

> Even if you knew the position, type, and velocity of every molecule of the atmosphere at a given moment you would still need a model that explains how they interact over time in order to predict the future. So, I'm not sure that actually is a knowable unknown because that assumes there is a model that can be made in addition to the information. Maybe the weather is chaotic.

It is, but that doesn't mean you couldn't predict it with perfect information. A chaotic system is roughly speaking one that is

- sensitive to initial conditions, in the sense that the distance between nearby trajectories in phase space grows as e^{l*t} for time t and some positive number l

- mixing, meaning that given any two open sets in phase space X and Y there's at least one trajectory from a point in X to a point in Y.

and so if you have any error bounds at all on your measurement of the initial state (which of course you always do) then you can't predict where it ends up in the long term. But there are plenty of chaotic systems for which exact numerical computations are quite simple. The logistic map x_{n+1} = rx_{n}*(1-x_{n}) for instance, is chaotic for many values of r.

> – You “fail” the interview if you get the wrong answer

> We care about how you think about and approach tough problems, which includes how you take hints and respond to being wrong.

First and second sentences need to swapped.

What was the probability that one of JaneStreet’s alumni, Sam Bankman Fried, would use JaneStreet’s brand name to build up credibility and then pull off the biggest ponzi scheme in human history?

Slightly lower than one of MIT's alumni doing the same.

It's telling that Jane Street stayed away from the whole mess despite the connection.

They were happy to trade crypto and provide market making facilities throughout the ordeal.

They only decided to “step back” now that regulators are bringing the hammer down

This wasn’t an innocent bystander firm. They were complicit, either directly or indirectly

Buying prospectors' pyrite for the little it’s worth is different from driving aspiring miners to mountains without food or water.

> use JaneStreet’s brand name to build up credibility

How is that different from bragging that you finished MIT, Harvard, Oxford, you worked at Google or that your startup is in YC.

Well for one, JaneStreet is an active market maker, trader, and investor in crypto, including in many projects created and/or backed by SBF

The fact that JaneStreet hired both SBF and Caroline shows their interview process has a few flaws.

Why? Maybe they were good employees? Just because they did something bad after really isn't thst relevant.

It shows JS doesn’t scan for morality or ethics or judgement.

Unless you happen to have some evidence presenting unethical and/or immoral actions on the part of SFB and Caroline during their tenures at JS, it only shows that JS don't try be an oracle predicting future morality or ethics or judgement of their candidates for the rest of their lives.

JS aren't future tellers when it comes to their employees' lives after they leave JS, and JS has no control over those employees once they depart either.

Were you screened for morality or ethics at your current job?

I was. You were too. It's just called "culture fit" in polite society.

These are hard to scan for. Also people can change?

Plenty of tech nerds have turned into complete psychopaths when they pocketed their first billion.

That seems easy, then why is getting a job there so hard? the pdf is undergrad level. it is covered in any elementary stat or probability book. I guess you have to do it on the fly or something. or have a good intuition for estimating probability when the formula is too cumbersome.

When was this published? There is no publication or revision dates in the pdf. Maybe 1999 when Jane Street started?

WaybackMachine’s oldest snapshot for this url is Sept 2019.


This seems like a sub-optimal way of finding top talent, even though the results still speak for themselves. Obviously Jane Street has had a lot of success using brain teasers to screen for talent.

If i were a hedge fund recruiter, here is what I would do:

Go on /r/ wallstreetbets and find guys who consistently are pulling huge $ like this guy:


Send a DM job offer to all of them, 7 figures plus performance comp , or at least an interview that will be paid anyway, so as to not waste the recipient's time.

Why ask brain teasers to try to find skills that correlate with trading when you can just pick out the best traders who already have demonstrable skill? that is how sports recruiting works. they find the people who can play the sport at best level.

Jane Street is a market maker.

WSB is a gambling forum where people are addicted to chasing huge rewards by taking huge risks.

This would be about as interesting as going to a casino and recruting people who have a big win streak at the slot machine.

but there is evidence that some traders there are more skilled than others, like the guy I linked to (or someone like Warren Buffett or Jim Simons ). it's not all random like a casino. Plus, to make so much money at options trading so consistently, requires knowledge of position sizing, market intuition, pattern recognition, and other skills relevant for all aspects of trading.

Nice and practical !

Honestly I expected it to go over far more advanced topics than these, most of these are covered in the last year of high school over here.

> Honestly I expected it to go over far more advanced topics than these, most of these are covered in the last year of high school over here.

Opening sentence: The goal of this document is to present an introduction to probability and markets in what we hope is a fairly intuitive and accessible way.

Basic reading and comprehension is not covered in his last year of high school it seems.

right no I guess thats not what I meant to say, rather that I expected any learning material from Jane Street(which has some really rigorous interviews) to give a better heads-up to the type of thinking thats required to do well in those interviews.

> type of thinking thats required to do well in those interviews

They have a whole page dedicated to that: https://www.janestreet.com/join-jane-street/interviewing/

Attention to detail and clear communication, including good grammar and spelling, are probably important.

I teach post-grad stats classes in the US, with no real prerequisites besides a Bachelor's degree. Most but not all of my students have passed a college level stats class that covered these concepts, and maybe half of those did not retain a true understanding of these concepts by the time they got to my classroom. In my experience, math is the least like/most hated of the core subjects, and stats is one of the least liked/most feared math subjects.

What makes statistics seems so daunting to the average student over, say, discrete mathematics? I feel like statistics has so many immediately obvious real world use cases and useful analogies to help frame it (die rolls, etc).

As a person with a bachelor's in math who dislikes statistics, it's the one field I know where "gotcha" problems are more common in reality than in class. And they are nasty problems. It takes a lot of education until you can confidently say what simplifications are justified for your situation. And a lot of professional statisticians like to flex on newcomers for simple mistakes.

You're right that some education would help people with everyday situations. It'd be nice if schools had intro to probability and statistics as the only required math course for all students. Even just covering fractions for probabilities, mean/median/mode, and then memorizing basic tricks or terms to look up on Wikipedia/Wolfram throughout life.

The very idea underlying statistics (that we can learn something useful from a class of events by ignoring specifically that which makes them different from each other) is a very recent invention -- only a few hundred years old -- and it's still counter-intuitive to a lot of people.

Ask people to predict whether the Jones' have a dog and they'll start asking questions about the specifics of the Jones', like do they have children, do they live in the suburbs, have they always wanted a dog, etc. Then they try to construct a narrative based on "logical" conclusions.

When the right approach might be to ask "what's the background rate of dog ownership in the Jones' country of residence?" But that takes ignoring the specifics, and that doesn't come easily to people. It took several millennia for anyone to realise it could be useful.

> Ask people to predict whether the Jones' have a dog and they'll start asking questions about the specifics of the Jones', like do they have children, do they live in the suburbs, have they always wanted a dog, etc. Then they try to construct a narrative based on "logical" conclusions.

This is the core idea of Bayesian statistics.

You fall back to the background rate only if you have no ability to access more pertinent facts.

When people ask about whether the Jones' have children, they do not (most of the time) have an accurate conditional probability of dog ownership given a certain number of children. They are trying to construct a story that seems plausible.

The difference between Bayesian reasoning and narrative fallacy is a coherent evaluation of joint probabilities -- the bit most people skip over.

It's doubtful people have access to population-level base rates either, in which case they would also arrive at the wrong answer, no narrative needed.

But why do you point the finger at narrative instead of a simple lack of data? In the dog example, those questions are absolutely good ones to ask, so long as the data can be found.

One reason to reject this mental opposition between 'Bayesian reasoning' and 'narrative fallacy' is because narratives are essential to coming up with good conditions on which to split the population in the first place.

'I would ask whether they have children, because children like dogs and therefore it's plausible that families with children have a higher rate of dog ownership' is a reasonable narrative that justifies collecting an extra column in your dataset.

Granted, one shouldn't accept that story uncritically without checking against the actual statistics. But narratives themselves are unavoidable even when you're doing statistics correctly!

This is one of those things where it's impossible to prove one approach better or worse than the other. But history has shown (starting with merchant ship insurance in the 1700s going up to Tetlock's superforecaster research more recently) that starting from general base rates gives, on average, more realistic odds than starting from specifics.

You see this all the time in sports betting, too. People overcorrect from the base rate when receiving news. People love a narrative that makes sense to them, but statistically it rarely holds up against statistical reasoning with a very small set of variables. (For more examples, see Meehl's research around clinical vs actuarial judgment.)

Yes, you're correct that a hypothesis is one of those narratives, but if you're operating at that level of scientific abstraction you can ignore my comment. Most people don't, and just roll with the story and forget about the probabilities.

Easy to get things wrong from the wrong intuition or understanding of the problem. Cross mathematic discipline etc. But I feel that most applications just need high school math.

Turns out trading doesn't require that much advanced math

Despite it being "covered" in many places, few otherwise intelligent people actually grok this stuff or have any real intuition for it.

simple things will go very far

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact