Get this book, read it and understand it.
Quants tend to be in 1 of 3 categories:
1) pricing quants,
you work for a bank or investment house like Goldamn. You know stochastic calculus very well, you know finite differencing like the back of your hand. You'll know every way to look at a derivative product. You can program in matlab. You create the next big thing like CDO's or CDS's
If you love math, this is where you want to be.
2) Quants who trade
You work at a hedge fund. You can program in python with scikit or use R. You don't know calculus maybe as well as a pricing quant but you know some area of the market much much better. You know stats like its your mother tongue.
3) Risk quant/programmer. You do modelling all day for a trader or risk manager. You can take a portfolio and model any feature that someone ask for. Var, beta, greeks, you can spit them out quickly. You know C++, excel and R. You might have been an engineer in a former life.
This is often considered the low position in the quant hierarchy. This is how some programmers break into the industry.
4) I lied there is actually special category 4). This is the professors of quants. You sit around all day and think about the next big equation, or how to model derivatives better than black scholes. If you are one of these people chances are your name is known in the industry:)
EDIT someone asked if you can do these without a math degree. The short answer is probably not. YOu'll need a math, engineering, or physics degree, unless you really are driven to learn hte math yourself.
A comp sci degree might work but you would be the exception and you'd be fighting up hill. Ask your self honestly, how many branches of mathematics have you self taught yourself and you'll get your answer. For most the answer is none, for a select few the answer is yes.
Someone asked about brain teasers. They do get asked, you have to deal with it, whining in an interview that this type of question gives no useful hiring indicator won't get you hired. I don't ask them but they are common:(
We throw around brain teasers at work during the day trying to stump each other. I guess some of it is trying to look smart. Some of it is that we just really like to dissect any problem and figure it out.
I think the biggest reason for asking brain teasers is that at a hedge fund there are no rules for how to make money, excluding legal. They want people who can think outside the box. it turns out that its really tough to test for "can the candidate think outside the box"?
Trading is a higher stress job than most other math or programming jobs, what we are trying to see is not only are you smart, but can you think on your feet and not get stressed out, because if an interview stresses you out, whats going to happens when a $50,000,000 position that should be going up starts to go down.
Are you going to complain that the model says it should go up and the market isn't being fair, or are you going to accept what's happening and get back to work? You'd be surprised at the number of people who chose option 1.
I ask a bunch of questions that some people feel are brain teasers. In between questions I'l throw out what's the square root of 225 to see how the candidate reacts. Good traders seem to be exceptional at mental math, with very, very few exceptions.
I'll also ask alot of probability questions. Get ready to know what your expected payoff is if you gamble on dice games, its basically what you do every minute of the day when trading:)
Now if you are a programmer, how do you get into the industry?
You need to know stats, machine learning, and programming, really well.
And I don't mean know machine learning, like "I tool an nlp library and stiched it together to do sentiment analysis on a corpus of text". I will ask what algorithms the underlying library used. You used SVM, great talk to me about your kernel selection methods. I want to know that you understand the math, and more importantly the assumptions and limitations of the library you are using.
The reason is that when you trade on a model that is based on your machine learning, I want to know that you know when it breaks. Finance is an industry that loves to model but has crashes that are predicted to happen 1 in 1000 year events happen every 10 years.
I love helping programmers who want to become quants get into the industry. Please feel free to ask if you have questions!
My other two options are working at an ambitious new robotics startup, and a PhD in deep learning.
I have a few questions:
1) What would you say is a reasonable salary range for someone with a master's degree in computer engineering and a year of experience in back office, as well as an assortment of ML side projects? How high could you expect it to be in 2, 5, 10 years?
2) Is it very difficult to break into the industry? This opportunity just landed in my lap (recruiter), and I'd like to know how likely it is that I'll find something like it again.
3) Will a PhD in machine learning (and the resulting five year gap in the industry) make me more or less employable? How will it affect my salary/job opportunities?
4) Just how much of the job is reading and implementing machine learning papers, and how much of it is general software engineering?
5) Where can you derive meaning and satisfaction from a job in quantitative finance? How do you reconcile the opportunity cost to society from not working on directly socially beneficial applications in fields like medicine and artificial intelligence?
There are two fine institutions that may help with this: taxation and charity. If you earn a lot of money, then you pay a lot of taxes and can afford to give a lot away.
Everyone likes to complain about taxation, but it's what turns a free market full of individuals and corporations all (to a good first approximation) trying to maximize their own wealth into something that benefits everyone.
And the most effective charities seem to be able to save a life (or provide a kinda-equivalent amount of other benefits) for something in the vicinity of $2000. (Important cautionary note: all such figures are very rough and you shouldn't trust them too much.)
So, suppose you have a choice between a quant-finance job of, let's suppose, exactly zero social value, and a job in medicine that pays $50k/year less. And suppose you'd be equally happy in either aside from ethical concerns. Then by taking the quant-finance job and giving away all your extra earnings, you give your government (let's say) an extra $20k/year to spend on schools and hospitals and police and roads -- and, unfortunately, various other things you might approve of less, so let's say it's the equivalent of $10k/year going to something obviously valuable like teaching, so you're paying for about 20% of an elementary-school teacher. And you give an extra $30k to (I hope) very effective charities, so maybe you are saving 10 lives a year.
So it comes down to the question: is the medical job more beneficial to the world than 20% of an elementary-school teacher and 10 poor Africans' lives per year?
You might answer that either way, but at any rate I don't think it's obvious that the answer is that the medical job is better.
(Some notes: I am not claiming that anyone who goes into finance in preference to another worse-paid job is obliged to give away all the extra money they earn. Only that doing so is one option, and that it might work out pretty well ethically speaking. It might be psychologically difficult to give away so much of one's earnings. It might be harder to "derive meaning and satisfaction" from things as indirect as tax and charity, compared with deriving them from one's actual work. It can be argued that quant-finance has positive social value, but I'd be skeptical of claims that it has much. I do not work in finance and never have, though being a mathematician it's always possible that one day I might.)
If this is a strong concern of yours you should probably go with one of the other two options. There's opportunities to make money in many industries. You might not end up with as high as a net worth as you would've if you went into quantitative finance, but that's not an absolute rule and you'll more likely burn out if you don't find satisfaction in your job. This isn't a dig towards anyone that is in the finance industry and enjoys their job, but it's common for people to end up quitting from dissatisfaction/work conditions.
Besides that, a robotics startup sounds pretty exciting.
Off the top of my head...
> 1) What would you say is a reasonable salary range for someone with a master's degree in computer engineering and a year of experience in back office, as well as an assortment of ML side projects? How high could you expect it to be in 2, 5, 10 years?
First off your education counts for nothing when negotiating salary. If you can do the job, you get the salary for the job. Some people really have a tough time of letting go of this. I don't care if you have Phd or are a high school drop out, you get paid based on performance and role.
In Toronto, starting $100,000 with raise to 200,000 at the high end in 10 years.
Bonus is 0 - 2x that, expect about 0.75 . Alot of that would be based on the firms record and not your own. It doesn't sound like you'd be actually making money so you have a chance to be higher if you develop trading strategies.
That's great money but not the kind of money some people think. You don't get huge bonuses until you, yourself, produce even larger profits.
> 2) Is it very difficult to break into the industry? This opportunity just landed in my lap (recruiter), and I'd like to know how likely it is that I'll find something like it again
Connections really help. A Phd really helps. Writing a piece of open source software that a firm uses really helps, Writing a paper that the firm uses really helps. Jobs can be hard to come by as the industry is pretty incestuous. People move around alot and that means someone trying to break in has to answer the question of why hire you instead of the guy whose done this for 10 years and I know everyone he's worked for.
> 3) Will a PhD in machine learning (and the resulting five year gap in the industry) make me more or less employable? How will it affect my salary/job opportunities?
If its for pricing, it probably won't help at all. If its a HFT then it helps.
Machine learning is used alot less than people think at most funds. Alot of people are under the assumption that you can just apply some machine learning to the market and make money, it just isn't possible for most people. There are just too many factors that can affect the price of a stock.
how do you model panic? Consumer confidence? Russia invading the Ukraine? OPEC selling oil below $80 a barrel? The US invading Iraq?
All of these affect the price of stocks, and its often modeled as an additional fudge parameter, that is positive or negative according to the whims of the modeler on that particular day, in other words, its a hack in the greatest sense:)
A lot less than you'd like. a ratio of 10:1 plumbing vs paper implementation is about what I do and I get to chose what i do:( If you think about it, the algorithm is small compared to the surrounding code you need just to get an order out the door.
Back testing can be 3 weeks out of 4 sometimes because for every idea you have that succeeds, you'll have 10 that fail at some level.
> 5) Where can you derive meaning and satisfaction from a job in quantitative finance? How do you reconcile the opportunity cost to society from not working on directly socially beneficial applications in fields like medicine and artificial intelligence?
Not touching this question with a 10 foot pole. I know I'm really excited for Monday mornings, others might not be.
The upside is not the same as being a very successful trader, but it can still be head and shoulders above working at the Amazon/Google/Microsoft's of the world. Writing high performance, high availability trading infrastructure is hard, and scaling it globally is even harder. A successful HFT firm will be iterating on it's infrastructure fast enough that there is always interesting work to do, especially if you like distributed systems.
1) High guaranteed, little upside
2) Low guaranteed, high upside
3) Moderate guaranteed, little upside, better QoL?
4) Low, none..., excellent QoL
I'm not sure how I'd feel discussing kernels of an SVM with someone -- to me, that feels pedantic. I generally feel that the fundamental driver to 'models no longer working' is evolving covariance dynamics (changing linkages) -- this has very little to the mechanisms of most ML tools.
Perhaps the metaphor I'm looking for is that I wouldn't expect a carpenter to be able to build and explain the components of a power drill to use it to help my house.
QuantFi is a huge discipline -- you build teams with hopefully a mixed bag of skills. On the trading side, I'd rather have someone with experience w.r.t to the market + products than math-jutsu any day.
There are trader/quants who basically work for free for just upside at lots of prop shops + HFs. There's actually deals WORSE than that where people sacrifice a lot for upside participation in their strategy.
Trading at a bank is a mixed objective problem generally -- much of what you do is NOT about making $ so understanding how and why people make what they make is not trivial.
I can't imagine a world where any of these wouldn't be at least low 6 figure roles -- the banks/HFs/prop-shops have to pay to compete with the Googles/Facebooks of the world.
One-hops (1-2 year stint then transition out)
1. Programming gig at finance facing software shop
2. Programming gig at exchange
1. Work on popular open source framework in heavy use (ZeroMQ, QuickFix, pandas, sympy, sklearn, etc)
2. Find headhunter/recruiting agency
3. Attend local finance oriented meetups and schmooze
I wouldn't trivialize the hiring process -- all of these roles are pretty competitive.
I've worked with some brilliant mathematical people, and I've worked with many who were not so brilliant. There are a ton of people who do the 'math' and 'stat' work at major financial institutions who are nowhere near as good at math as they should be. I haven't found it to be much different that any other industry from a programmatic perspective.
So far, it's been a fantastic read. Detailed, good information, and very well written. I saw the recommendation for the "Heard on the Street" book. I was wondering if people can recommend some other books for moving into finance?
I'm thinking of kicking the tires and refresh my stats/probability knowledge, but most online courses I could find are pretty introductory in nature. What would be a good path for me to be seriously considered for #2?
Or to put it another way: if your hiring culture enshrines shallow, game-show style intellectualism along with such a condescending, "just suck it up" attitude towards candidates... then don't whine that you can't find decent programmers.
In my school you had a class for 6 months: introduction to programming, that's it! The rest of the 5 years where basically only math/algorithms (machine learning, statistics, probability, compiler algorithms and such).
I've heard that this is changing as the industry values practical programming experience, but boy would I've been pissed if I'd spent 5 years to learn ancient tech.
I'm quite pissed about people getting a comp. sci. degree from knowing some C++, HTML5 and bubble-sort nowadays from shabby universities. Get of my lawn!
I would have thought no exceptions. What are the characteristics of the few who make good traders despite not being exceptional at mental math?
Writing correct trading algorithms has more to do with knowing what your blind spots are than being good at math party tricks.
I am already a good programmer and my mathematics ability is not lacking. My weakest point is probably finance, as I only have a rudimentary understanding of economics.
This is interesting, but I have a slightly different interpretation of the question: Assuming you _can_ learn the math yourself, will anybody believe you? Is the degree necessary, if only for pedigree purposes?
In a job interview for a position where the math is a requirement, it's going to be easy to convince people. There's very likely going to be someone who knows the math much better than you in the interview.
My experience being a quant and being around quants is that, sadly, they don't get to use much machine learning. Some do, but it seems like 97-98% of the work is much more mundane.
Am I correct in perceiving that many hiring managers expect more in the way of interesting experience than they really need (or have to offer in the work that needs done)?
Yes I think you are Michael. Especially for quant jobs 2 and 3 that I enumerated.
The only reason I listed machine learning is that it is the trogan horse that gets programmers to learning stats. I can't over emphasize enough just how much you need to know stats. half of trading is knowing stats as all good traders trade only when they have the advantage. 100% upside for 10% downside is a good rule of thumb.
I mentioned the deep dive into the machine learning techniques as unfortunately most of the programmers I meet who call themselves data scientists, just aren't very good at stats. I'll ask them what methods they use and they'll tell me the library they used.
Unfortunately most interviews go like this:
me: I see you did some K-means clustering, tell me about it.
them: Well, we used R to cluster related books into groups.
me: how did you choose the inital partition
them: blank stare, um the library does that.
me: yes, well I mean did you use Forgy or Random Partition
them: I don't know what those are. That's just random trivia that I would look up when working. Do you find asking random trivia helps you hire people:
me:...So do you have any questions for me?
Hedge funds have lots of mundane programming jobs. Most are back office stitching together WPF and accounting systems. Even at the HFT firms I've seen only a small portion of the programmers actually do anything on the main trading system.
I'd love to hear about your Jane Street experience here:)
Like most things the closer you are to the money the more you'll make and the more exciting your job will be:)
Most quant jobs that I've seen were very mundane, category 3 is most of the quant jobs. Though most will tell you that they are in category 1 :)
Instead of trying to "expose" these people for supposedly over-representing their competence in some skill area X mentioned in an ancillary way on their resume (relating to some job they had N years in the past), you might want to try just asking them a simple normal, human question first: "So was this something you go to delve into the internal workings of? Or did you just use an API for some small task?"
If it's the former answer, then fine -- drill away into the internals. If it's the latter (which is by far the more typical case -- the way things go in most dev environments), then you can just say "That's fine", and look for some other bullet point to zoom in on. Or if in fact X is a major requirement for you -- you can say "That's fine, but we're really looking for people who have done at least a fair amount of X here. But we thank you very much for taking the to talk with us."
Either way -- there's no reason to go into snark mode and assume these people are frauds or idiots just because they don't have a lot of detail to offer you. The bottom line is that unless they mention X as some major skill area, you pretty much have to assume that it was just another random thing they were forced to learn that day (you know, to keep their jobs). And unless it's mentioned in a way that would suggest they spent a significant amount of time on it, of course they aren't going to remember very much beyond the most superficial aspects of how it works.
If you call yourself a data scientist but don't know a lick of statistics then you are the one being misleading. You can't just pick this stuff up overnight. It is a bit like claiming you are a systems developer but then in the interview you don't know a thing about pointers or what the heap is. If your actual knowledge of systems development is "I wrote a python wrapper around a c library once" then representing yourself as a systems dev is intentionally misleading your interviewer.
Which (unless the candidate advertises themselves as having significant experience in algorithm X) is about as useful as asking them if they know the airspeed velocity of an unladen swallow.
In the example, the candidate wasn't grilled on the default partition scheme; he or she was grilled on the partition scheme that was used. If someone thinks that's an unimportant implementation detail, why would you ever want to hire them?
In some of these jobs, maybe. But the person I was initially responding to wasn't looking for someone with "deep statistical knowledge"; rather, he was looking for that species which goes by the trendy, loosey-goosey catch-all moniker, "data scientist".
And if you're going to put "data scientist" in a job title -- with no further qualifications -- then you had better understand that it's nearly useless as a description of anything (beyond a few colloquial definitions floating about, which there's still no real consensus on).
If on the other hand, you want an "MS/PhD background in Machine Learning and/or Statistics, or equivalent work experience" then that's fine too, of course -- just put it in the job description. It's really quite easy to do, and it will save everyone tons and tons of time (and grief).
Again, it matters to the extent that they, themselves, emphasize it as a core skill area. When someone simply says "did $foo using X" in some project description, I personally don't read anything more into it than that.
But the question's not as esoteric or impossible as you're making it out to be. "What's a confidence interval" is something any good undergraduate applying for finance jobs should be able to answer, regardless of major. I'd expect an enthusiastic undergrad stats or compsci major to care about the implementation details of an estimator they'd used for something nontrivial. (Which seemed to be the case in the original comment.)
i just dont think these environments r made for you or me...... i used to win many math comps and was at the top of my class for years & years. in college i went into media programming & psychology because i didnt really want to study stuff i thought was boring all day just to be a market kiss-ass
as a result, i have a pretty boring job but i let my mind wander creatively & still work on a ton of music & creative projects. I've designed some pretty sophisticated machine learning algorighms/research in audio signal processing but i don't get to study it all day because there is not much market demand.
i know i have more raw talent than most people i interact with (in fact im confident i can self-teach myself various technical disciplines to near-expertise) but i've also come to understand that employers don't care. beyond entry level, they are just looking for someone who says the right things and has already done work that resembles the job they are hiring for.
sometimes i consider self-teaching toward a goal like being a quant, but then i remember that that world is not for me. im not insanely greedy so its not a big deal. and though my enterprise work now is boring, there are several things on the horizon that seem much more promising & appealing than finance. i am accumulating skills & enjoying having room in my head for other thoughts. i don't work in the startup world because i haven't encountered anyone with compelling ideas but i suppose thats where people like me are destined to end up.... i stay sharp & am confident that i could code a mini-enterprise in a year, once i find the right idea.
Ah, that makes perfect sense.
To me, the appeal of machine learning is that it's challenging and respected enough that programmers doing it get the autonomy to work in any subfield of computer science, from the high level to the low. If you're a data scientist and you say you want to use Clojure or Haskell (a high-level concern) or that you want to do GPU programming or dive deep on assembly (low-level work) you can. Machine learning, 10 years ago, was extremely appealing because software managers were figuring out that they needed it, but most admitted they knew little about it, so they gave a lot of autonomy to individual contributors. (That may change, and "data science" may become thoroughly commoditized.)
It's the Fundamental Theorem of Employment: you're usually hired either to do (a) something your boss can't do for himself or (b) something he doesn't want to do. With (a) you get respect and autonomy and high pay; with (b) you get treated like a commodity. "Data scientist" (or software "architect" vs. "engineer") is, often, a programmer who's managed to learn enough of "the hard stuff" to move himself over to (a). It's the (b) category of engineers who get stuck on "Scrum teams".
I feel like some of the crowding of "data science" (and, as you noted, not all of the "data scientists" know what they're talking about) comes from the way that "Agile"-style micromanagement has made the rest of programming so braindead. There are people like me who enjoy the hard mathematical aspect, but others who've just learned that if they call themselves "data scientists" they get more interesting projects and don't have to munch on Scrum tickets. For them, the math is an impediment rather than a challenge and an attraction.
I mentioned the deep dive into the machine learning techniques as unfortunately most of the programmers I meet who call themselves data scientists, just aren't very good at stats.
There's a depth vs. breadth problem, because machine learning is a much, much bigger field than many people think. I've gone pretty deep on penalized regressions (e.g. ridge and Lasso with large numbers of features) but know only the basics about tree-based models. I can read the papers on neural network architectures (e.g. convolutional nets) and implement them, and I understand the theory that led to them, but I still lack some of the intuition (like why rectified linear units are more useful in image processing than regular logistics).
I feel like there are some people who pick up a lot of vocabulary and interview well (but not with you, because you actually know the field) but are really just playing around with parameters. I like the math, but "data science" is mostly bullshit and I hope the term will die; I want to see more machine learning and less business pomposity.