Hacker News new | past | comments | ask | show | jobs | submit login
Crunching 200 years of stock, bond, currency and commodity data (bloomberg.com)
308 points by chollida1 45 days ago | hide | past | web | favorite | 132 comments

Coming from a CS/Econ/Finance background....

Efficiency as “the market fully processes all information” is decidedly untrue.

Efficiency as “Its very hard to arb markets without risk better than a passive strategy” is very true.

Most professional money managers don’t beat the market once you factor in fees. Many private equity funds produce outsize returns, but it’s based on higher risk and taking advantage of tax laws.

Over time, positive performance for mutual funds don’t persist. (If you’re in the top decile performance one year, you’re no more likely to be there next year)

Despite all of this, there are ways people make money. But it’s a small subset of professionals. It’s high frequency traders who find ways to cut the line on mounds of pennies. It inside information from hedge funds. Or earlier access to non public information. But it’s generally not in areas that normal people like you and me can access.

It inside information from hedge funds. Or earlier access to non public information. But it’s generally not in areas that normal people like you and me can access.

Asymmetric information is pretty far from what used to be said about the perfect market and rational actors. It's "there's a sucker born every minute" and "if it seems too good to be true it probably is" economics.

I might be misunderstanding what you're saying here, but are you sure you're right? Fama originally predicated the model of the efficient market (the efficient market hypothesis) on the idea of informational efficiency. Information asymmetry is a fundamental measure involved in the idealized model of an efficient market.

What you're mentioning about rational actors is actually a different topic altogether in economics.

Or have I misunderstood what you're getting at?

No, I'm not sure. I have ill-informed opinions which go to bad economics and I'd probably be written out of the reddit economics conversation which has a special place in hell for amateurs like me.

So.. I quoted the comment I commented on. What I feel here, is that small traders, at home traders, and people whose investments are managed arms-length by funds, are in a different place to people who operate "in" the market. And to some extent, these things which are driving significantly above longterm trend are things which the market regulator says it tries to limit: that whisper of future intent that three guys get in the mens room, and two women don't get in their other toilet, leads to a distorted outcome: the clients who benefit did not benefit from worked knowledge, they benefited from a point in time creating a distortion in who knew what to expect.

The LIBOR people setting the prices for money who decided to listen to their chinese-walled banker friends and set company friendly price, did everyone a huge disservice, in the act of making some people fantastic outcomes predicated on different knowledge of the process. LIBOR was not setting information, it was a closed-room price fixing game.

I was interested, so I did some research here.

Rational Choice Theory https://en.wikipedia.org/wiki/Rational_choice_theory

Rational Behavior https://www.investopedia.com/terms/r/rational-behavior.asp

> Most mainstream academic economics theories are based on rational choice theory.

> While most conventional economic theories assume rational behavior on the part of consumers and investors, behavioral finance is a field of study that substitutes the idea of “normal” people for perfectly rational ones. It allows for issues of psychology and emotion to enter the equation, understanding that these factors alter the actions of investors, and can lead to decisions that may not appear to be entirely rational or logical in nature. This can include making decisions based primarily on emotion, such as investing in a company for which the investor has positive feelings, even if financial models suggest the investment is not wise.

Behavioral finance https://www.investopedia.com/terms/b/behavioralfinance.asp

Bounded rationality > Relationship to behavioral economics https://en.wikipedia.org/wiki/Bounded_rationality

Perfectly rational decisions can be and are made without perfect information; bounded by the information available at the time. If we all had perfect information, there would be no entropy and no advantage; just lag and delay between credible reports and order entry.

Information asymmetry https://en.wikipedia.org/wiki/Information_asymmetry

Heed these words wisely: What foolish games! Always breaking my heart.


> Asymmetric games also naturally model certain real-world scenarios such as automated auctions where buyers and sellers operate with different motivations. Our results give us new insights into these situations and reveal a surprisingly simple way to analyse them. While our interest is in how this theory applies to the interaction of multiple AI systems, we believe the results could also be of use in economics, evolutionary biology and empirical game theory among others.


> A Pareto improvement is a change to a different allocation that makes at least one individual or preference criterion better off without making any other individual or preference criterion worse off, given a certain initial allocation of goods among a set of individuals. An allocation is defined as "Pareto efficient" or "Pareto optimal" when no further Pareto improvements can be made, in which case we are assumed to have reached Pareto optimality.

Which, I think, brings me to equitable availability of maximum superalgo efficiency and limits of real value creation in capital and commodities markets; which'll have to be a topic for a different day.

I'm a finance professional as well, and I generally agree with you. But every consumer has access to one piece of very vital non-public information, Namely, his or her's personal preference.

For example, say that I own a Tesla car, and think that it's pretty great. That's a very valuable piece of non-public information: I think Tesla cars are good. If one was to act upon this private information 3 or 4 years ago, they would have done very well indeed. Same with Google, or Amazon, or a multitude of other public companies.

I think people overestimate how difficult it is to make money in the public markets as individuals. Hedge funds, and especially mutual funds, are quite constrained in their ability to up size positions and put on profitable trades. They have redemptions, they have to hold assets in cash to liquidate shares (mostly this in a problem for mutual funds, not hedge funds), they might have to hold a long only portfolio (again, a problem with mutual funds, not hedge funds).

Individuals don't really have this problem, since they are managing their own money. If they want to put 50% of their capital into one name, no one is going to stop them. Moreover, they have full transparency into redemptions and inflows.

I have personally have done quite well with my own investments, achieving a better return than the firm that I work at. This has only been possible because I have complete insight into my risk tolerance, and don't have management and performance fees eating into my returns.

Another example. My wife's grandfather's wife in the early 2000s loved books and loved reading. Consequentially, she loved Amazon and what the company could provide for her. She convinced her husband to invest a sizeable chunk of their net worth (while not huge, was substantial, maybe 2 million) in one name, Amazon. They both dead recently, but the estate is very very valuable at this point, all because of that single Amazon investment shortly after they IPO'd.

Just because there are huge players in the market, doesn't mean an individual can't compete. In fact, individuals are probably in the best position to reap the highest returns from the public markets.

>I have personally have done quite well with my own investments, achieving a better return than the firm that I work at. This has only been possible because I have complete insight into my risk tolerance, and don't have management and performance fees eating into my returns.

How do you know it isn't just random luck? Same goes for your suggestions to invest in one "good" stock. Some will pick Amazon and profit greatly and some will pick AOL or Yahoo and be destitute. This is the whole point of diversification and the rise of index funds and I'm a little amazed to see a finance professional recommending things like this when studies repeatedly have shown that picking stocks does not work when you get a large enough sample size.

Historically, many funds made their bread on charging management fees for beta, and now that index funds can do the same thing but at a much lower price, they're going to go extinct.

I'm not suggesting that you put all of your money into one name, on the contrary, I think an index tracking ETF should make up the majority of one's portfolio. Maybe 20% should go into bonds or pure alpha and maybe 20% should go into 3 or 4 single name bets.

Institutional investors nowadays are doing something very similar. A core of their portfolio is now cost passive maybe with some smart beta tilt. The rest is allocated into bonds or pure alpha, whill a small portion is put into high conviction equity bets or hedge funds with beta exposure. This allows them to significantly reduce their management feels.

If you cite a few of the specific studies you're talking about, I'll be happy to explain why they're either 1) not actually concluding what you think they're concluding, or 2) simply incorrect. I can tell you very confidently that there is no academic consensus claiming alpha does not exist. It's rather the contrary, actually, and it's far more nuanced than your comment implies.

Outside of academic finance you can also find pretty obvious indications that alpna exists. Berkshire Hathaway has averaged about a 20% return between 1964 and 2018. Over the same timespan the S&P 500 averaged about a 7% annual return. Smaller funds have done even better than that.

When you actually do the math on some of these firms' performance, you see that the likelihood of their returns emerging by chance is too small to be explained by the number of hedge funds that has ever existed. There are funds which haven't lost money for 20 - 30 years, and whose returns are multiples of the market over the same time.

The question is not only whether alpha exists, though, but whether you can a) identify it, and b) access it.

Berkshire Hathaway’s stellar outperformance seems to have declined in recent years, right, and you and I can’t invest in Renaissance’s Medallion fund.

I’d concur that the individual investor has much more freedom in terms of basic asset allocation than most professional fund managers (except maybe hedge funds), but otherwise the general advice to go for a broad based index fund still stands.

Exactly. In 1999 I had enough money with AOL that I could have completely paid for my 4 years of college tuition. I knew nothing about financial markets or investing but I thought and was told that AOL was a good company.....obviously that didn't last.

> I'm a little amazed to see a finance professional recommending things like this when studies repeatedly have shown that picking stocks does not work when you get a large enough sample size.

If every financial professional believed that, there would no be finance professionals. With me maybe the quote is true "It's hard to make a person understand something when their job depends on them not." I work at a firm that tries to make money for their investors and if I believed it was impossible, I probably wouldn't be working for them in the first place.

In no way am I an unbiased commentator, my livelihood depends on me believing that I can be smarter, than I can be better than everyone else. And while most people can't, many people can. There are a handful of funds that have demonstrated significant outperformance over a long period of time. Funds like RenTech or Bridgewater.

A similar story: I bought AMZN because I thought the Kindle was amazing. I correctly foresaw that it would get cheaper and better (despite people sneering that $400 for the gen 1 was obscene and that they preferred paper), and that AMZN could become one of the biggest sellers of books just like AAPL became one of the leading sellers of music with the iTunes store.

At the end of the day though, I think I made money on the stock for other reasons (AWS; Prime; AMZN eating all of retail).

Similarly, your family member seems to have made a good bet for all of the wrong reasons.

So I'm not sure your example is a great one.

Maybe, but delivering exceptional value in one area (books), is probably correlated to providing value in other areas (AWS, retail, etc).

You could imagine a similar situation with Google. You buy google stock because they created a really good search engine. But being able to deliver a great search engine also, it seems, is correlated to providing great advertising services, cloud computing, and email.

Being good at one thing is probably correlated to being good at other things (especially if they are related). While being bad at one thing is probably correlated with being bad at other things.

Having a good product is no indication that the stocks are good.

- Stock price can be overvalued, even for that good product

- A costly proccess to make that fancy product could eat up the margins

- Company strategy, investment in the future etc need to be considered.

Basically you need to know how much a company is really worth to buy stocks, not just "I like their product"

Exactly. It’s “Is your view of the product quality and it’s impact on the company, industry and stock market price better than the aggregate wisdom of everyone betting on it?”

Historically the answer is “Less than 49% of the time for most people, including professionals”

And yet...his wife's granny ended up investing a couple of mil in a stock priced in the single digits that is now worth over $1500/share. Don't overthink things.

Btw: if you actually knew what happened to the share price at that time (it was probably down 90% in a year) then you wouldn't call it anything other than what it is...a good decision.

This is pure anecdata and survivorship bias. I like bitcoin, and yet...

Holding amazon through a 90% decline is having nerves of steel.

Liking bitcoin, that's just stupid.

Hindsight is 20/20. Investing 2 mil because "I like books" is never a good decision. If it works, you were lucky, not smart.

The parent comment was talking about risk-adjusted returns. You're talking about inherently highly risky bets "50% of their capital into one name".

I don't think it will surprise anyone that risky bets do sometimes pay off.

This was Peter Lynch’s theory.

But that one piece generally doesn’t beat the market.

It can’t. In aggregate we are average.

I think your comments are true though. Risk tolerance plus a long timeline is good for long term returns.

>Efficiency as “Its very hard to arb markets without risk better than a passive strategy” is very true

Isn't that the best we can hope for though?

The inefficiency is basically not knowing how to price something, which on some level is always true. At the moment a giant meteorite strike is priced at a level that is probably wrong? Tomorrow the meteorite will or won't strike and tell us which way we were wrong.

Essentially outperformance is down to being quicker on the trigger.

Edit: Or as you say, guessing whether the meteorite will or won't strike.

Most professional money managers aren't trying to beat the market. Distribution is way more important than performance. Trying to outperform the market is risky. Even if you have a good fund manager, they will eventually go elsewhere (if you look at most of big managers, but not all, their business model is turning a 0x employee into a 1x employee...10x employees get paid their marginal product).

The key to running an asset management business is marketing, and building assets with unsophisticated clients that won't sell (mainly retail investors/advisers). This doesn't work as well anymore but it is very weird to argue that mutual funds are meant to beat the market. That was never the pitch.

It is also fairly straightforward to outperform the market if you are a retail investor, you have a huge edge: size. You have just to look at the stuff that no-one else is looking at. I am not sure why but people have extreme amounts of difficulty with this concept.

People become investors because they want people to think they are big time, they want respect...they don't really care about performance. I remember a few years ago, when I worked in the industry, I came across a one-month 15% arb. My job at the time was to pitch ideas, and no-one was interested..."it isn't what we/I do". I cleaned out my account, sold everything, borrowed a substantial amount...99% of people won't do that. It is crazy to them...but investing is simple: look at what everyone else is doing, and don't do that because it is fucking dumb.

Also, most HFs don't have non-public information...sorry, they just don't. Some people are just genuinely better at making decisions than other people - 95% of people make the same mistake over and over, regardless of intelligence...ppl who are too smart usually do worse imo - and the most successful get around the problem of scale (a good example is Baupost: they invest in complex stuff that no-one else is trading).

Pursuit of efficiency is a common mistake. I believe, though I could be wrong, that Bridgewater concluded the only goal in investing is to make the correct choice more than 60% of the time though greater than 80% is extremely stellar. So long as you make the correct decision more than the wrong decision and account for the degree of risk you can afford to be inefficient and still outperform the competition by an order of magnitude.

Sometimes it is hard to know what the right decision is when current market forces suggest you should make the wrong decision or when every decision is apparently the right decision. In those cases you need historical data to compare against. The way they describe it it isn't any different than writing a basic algorithm, except there are many more variables to consider.

That’s actually a huge opportunity though. If you have a small edge in information, you can make millions.

Undoubtedly, there can be no such thing as perfect market efficiency because information takes time (even in our interconnected age) to travel, be grok’d and then acted upon.

Companies that focus on building an edge on either analytics or an edge on information gathering are where the real money is.

While a lot of people can acquire a lot of money by being a slightly better predictor of what the short term market thinks is the the future cash flow of public companies will be, it is a bit sad that this skill is valued so highly. So many really smart and hardworking people spend their whole lives doing it. They can end up with a lot of money, but I don't think they produce much wealth. Similar to how getting people to click on ads is really lucrative but also is more of a zero sum game than productive.

Not sure how to get more people to work on more interesting things like fusion power, rocket ships, electric powered transportation, etc. instead of skimming off of zero sum transactions. Just taxing the rich at high rates also kills the few golden geese.

> If you have a small edge in information, you can make millions.

This is one of the misnomer. Economics has enough random stuff and complexity that small edge doesn’t make huge difference. For example, you can get satellite photos of 100 sample Walmart stores and observe it’s parking lot to conclude that perhaps their quarter would end in negative side. But you might not be aware that the slack is being picked up more than enough by their online operations or that they have new deals for partnerships or some law about to be passed that taxes competition more.

Uh, no.

The key word in the parent poster’s sentence was “can”. You only need one example to prove it - tons of failing examples do not disprove it.

For example:

If you lived next door to the head server guy at Wells Fargo, and the other night his wife called you to cancel their dinner with you because “there is some big problem at work that he’s dealing with and he has to work thru the night”, and then you decided to short WFC that next morning, then you would have done quite well, but importantly, the information edge would be not only small, but also non-specific.

Well, the guy could be some manager of some small project with some release deadline on his head. This may not have any impact on quarterly financials of $10B company. Big companies like this are very complex creatures and trying to bet because you had glimpse on one data point is trying to predict weather for next month because you felt breeze on your walk.

You're correct - I should have been more clear on the titling - when I think head server guy, I'm thinking a VP or above level.

To be fair, if you are making huge trading decisions like this on consumer retail, and ignoring the online component of the third largest online retailer in the US, you're going to lose all your money eventually anyway.

> That’s actually a huge opportunity though. If you have a small edge in information, you can make millions.

Also, if you can act on public information faster than others then you effectively have that edge in information.

If you don't mind sharing, how did you turn your CS/Econ/Finance background into a career? Feel free to be vague, but I'm curious as to how you make a living. Thanks!

Started in CS, became a database programmer, went to grad school in Econ and Finance. Spent some time in Capital Markets after school, but realized I liked tech more. (I liked studying Econ and Finance more than working in that environment. The opposite of CS. Maybe I identify more with programmers than financiers?) I combined the two in Fintech as a way to get back into technology. Now I work in the business side of tech.

Thanks for sharing! Hoping to follow in your footsteps, except with ds/ml instead of database. Nice to hear it’s working out for you.

If you don't know about him already, you might find Martin Armstrong interesting.


How's the sovereign debt crisis coming along? We're what, 3 years late at this point?

People can still make money because it’s still a rolodex game.

Most hedge funds are making worse returns than average investors. They only funds that are consistently making money are highly sophisticated quantitative top-down systematic traders. And their returns having nothing to do with their rolodex. Names like RenTech, Bridgewater, AQR, 2 Sigma.

I think people in our line of work can easily beat the market because we know how a lot of money is being spent and is going to be spent, well before finance guys do.

I made huge returns buying amazon and nvidia 5 or 6 years ago because I worked for a big tech company just beginning big moves into aws and machine learning and everyone I knew at other companies was telling me the same story. I bought amazon at $300 and nvidia at $30.

When people say you can’t beat the market, I think they’re wrong in that you can be well positioned to see large amounts of money being moved around well before the market becomes aware of it, if you’re in the right position.

I think you just have to think hard about how well distributed the knowledge you have is.

For example— I think we all know kubernetes is going to be a big big thing and it’s only getting started— I don’t think the market as a whole knows anything about it — my only question is which companies are set to profit from it? I suspect that commodity providers are going to slowly eat away at aws’s profit margins as people move to k8s hosting. But I don’t know who is going to be the big winner yet.

What else did you own in those 5-6 years? What was your overall performance vs. the market?

Up about 350+% over all over the past 5 years even with some losers in there. I’m just picking stocks for fun and never put any more money than my initial stake. I recognize that there’s luck and risk involved. I’m just saying that if you think carefully and have access to information that’s not widely distributed, you can make money.

I think one thing you have to get over is assuming that because everyone in your professional circle knows something, that everyone knows it. It can take years before disruptive change gets noticed, even while you’re working on it every day.

Former fund manager here.

Behavioural economics seems like a very fashionable explanation for market behaviour. However, it doesn't seem falsifiable. Just because some factor seems to work, it doesn't have to be because of herd effects. In fact it's never explained exactly which of the many BE effects is at play, or how to know which ones are not active at a given time. I've never seen anyone predict that now, loss aversion is excessive, therefore the market is going to go up or down or whatever. It seems to always be "we found this violation of EMH" -> behavioural econ is true.

Regarding the things mentioned in the article, none of them are new. That doesn't mean that anyone can make money doing them. I like to do restaurant analogies when it comes to hedge funds, maybe because I grew up in one and went on to manage a couple of funds, but also there's certain similarities.

Everyone thinks they can cook. It's just taking the same ingredients that are publicly available and putting them through some process, right? Well unsurprisingly it is indeed possible to make a meal that is better than what you can buy at a restaurant. Can you do it cheaper and with less effort? Not likely. On the investment side though, it really matters a lot whether you can execute cheaply. And you can get lucky with the right combination on either side of this analogy.

Another analogy is recipes. How many restaurants owe their success to having better recipes than everyone else? I struggle to think of any. A restaurant is not merely a collection of recipes. You can stick exactly the same instructions in front of two different teams and the experience will turn out differently. The same goes for these strategies. Plenty of funds do trend following, yet somehow they do not end up with the same returns.

And finally, a recipe is merely a guide. Look in any cookbook, there are loads of questions left over after you read a recipe. Do I peal the potatoes? How long do I cook things? At what temperature? And so on. These strategies are just like that. So, 12-1 momentum. How much of each future? Is there an execution strategy? What you end up doing depends a lot on what exactly the environment is, who your staff are, etc.

>I like to do restaurant analogies when it comes to hedge funds, maybe because I grew up in one and went on to manage a couple of funds, but also there's certain similarities.

This may be one of the best explanations I have seen. Very simple and easy to understand. Bravo!

I'm genuinely curious, where do you invest your money for the next 10+ years?

You looking for my advice or what I do? Funnily enough they are different, since I have access to vehicles that others don't.

Plus I'm a particular person with a particular situation, advisees are a general sort of group with an average situation.

What he/she is stating here is that they are an accredited investor. They can invest in things that the average joe cannot.

> (5) Any natural person whose individual net worth, or joint net worth with that person's spouse, exceeds $1,000,000.

[5]"Electronic Code of Federal Regulations" (https://www.ecfr.gov/cgi-bin/retrieveECFR?gp=&SID=8edfd12967...)

Thanks! I'll make my research on what kind of investments accredited investors have access .

If you have any interesting leads on this, I'm interested!

Yes, trying to get my long term investment plan done. I'm allocating a small % (few thousands a month) to riskier things. I'm betting on few cryptocurrencies now, but wouldn't mind to explore your advices :)

Also debating with myself I should do my investment via my personal account or via an offshore company.

Either / both

You looking for my advice or what I do? Funnily enough they are different, since I have access to vehicles that others don't.

Plus I'm a particular person with a particular situation, advisers are a general sort of group with an average situation.

> the market is not as efficient as it is supposed to be

I have a recent business degree, but not finance or econ. In school we were strictly taught the market is efficient and I fully believed that. But I've seen online and discussed in person a (seemingly) growing idea that markets are not in fact efficient at all.

Are there any economists/finance people that can shed light on this? Are these new ideas gaining traction, or old ideas just sticking around?

Check out the book "How Markets Fail" for a discussion on how the efficient market hypothesis gained popularity (part 1) and a discussion of the many ways that real markets do not maintain the preconditions for markets to be efficient.

In particular, the precondition that Adam Smith stated was that increasing production required linearly increased costs. However, this is frequently not the case (utilities, railroads, software, social media networks, etc.). Also, markets require a certain minimum number of producers (and consumers) in order to remain efficient. Consolidation can reduce the number of producers below the level that meets the precondition of efficiency, and then barriers to entry to prevent newcomers from joining the market.

As far as efficient market theory applied to the stock market, I've always thought that efficient market theory was trivially disprovable: did all companies in the US lose 20% of their value all at the same time this past Nov - Dec, in the same season that they are selling all their inventory during the holiday season? And then regain 10% of it all at the same time the next month? Did Apple lose 50% of its value from between July and December, despite the fact that its revenues, costs, dividends, assets, people-capital, know-how, brand, consumer sentiment, etc. remained roughly the same? Compare to, say, PG&E during the same period.

John Cassidy's "How Markets Fail" is a must-read IMHO. I went through 4 years of an economics degree at a top tier university (top 5 globally in Econ), and Cassidy's book just happened to be the weekly assigned reading for a class my last semester.

Everything then made sense. All the inconsistencies you hear about economics, with one camp saying one idea and another talking about something totally different, both passing it off as consensus, fit into place.

TLDR: Cassidy's book is in 2 parts. The first part, he constructs the classical/rational model of economics through intellectual history. The second part, he deconstructs, drawing from all the critiques of inefficiency, both behavioral and mathematical. (Some of the most fascinating aspects are the beautiful mathematical models that can't be solved). It's all well written prose, no need to have a pen and calculator to follow along.

The title is misleading IMHO, because it seems to be a populist polemical. It's not. It should be called something like "A Brief History of Economics", echoing Hawkin's Brief History of Time.

FWIW, the efficient market hypothesis has very little to do with the efficiency (or absence thereof) of market based economies, efficient production, etc.

I think maybe a better question is not if markets are fully efficient, but if markets are more efficient than the alternatives. I read the Cassidy book years ago, so I might be misremembering, but I don't think that was something he spent much time discussing. I don’t think many would argue that markets can't fail and aren't fully efficient - he spent the entire book demonstrating that.

The idea that regulators have the ability to design, implement, and maintain laws that correct or protect from market failures without producing unintended consequences, perverse incentives, significant costs, or market distortions more severe than the original market failure was not sufficiently addressed, from what I remember. Despite their problems, markets tend to be dynamic, adaptive, and innovative, while regulations tend to be rigid, slow, and reactionary. I don't know what the answer to market failure is, but I found the book unsatisfactory in that regard.

Cassidy spends plenty of time stating that regulations are often bad for markets, and he spells out clearly which types of markets need to be regulated: specifically markets with negative spillovers, mono/oligopolies, and the financial industry because of its unique ability to impact money supply.

He goes to GREAT lengths to talk about all of the potential pitfalls and problems with placing regulations on these industries. And he even gives many examples of times in the past where things did and did not work.

Sure, you can get a positive outcome from the wrong approach. But examples are probably the best thing we can look to...

I think there are a few good responses to the question of "are markets efficient?".

1) Markets are efficient enough in that you generally don't see any real big arbitrage opportunities. Small ones will always be present, and as is often pointed out its not worth the big players time to search them out and capitalize.

A trader also sees this as evidenced by the fact that we are using more and more data and ever increasingly complicated math to eek out smaller and smaller chunks of alpha.

2) different people have different ideas as to what the correct price actually is.

Consider a company that is continually losing money.

- Fundamental analysts will value this as a poor company due to its future cash flows dwindling. They may consider this a short opportunity

- Merger Arb folks may bid up this up as a potential merger target thus raising the price.

- ETF's tracking an index just don't care, they'll buy the company in proportion to its index weight.

- Stat arb folks will be agnostic to price and only compare it to a peer that they feel it will trade in relation ship with, these people will be both long and short this company at different times, often unrelated to what its price currently is.

3) Physics often uses the simplifying assumption of no friction to make things easier. The efficient market hypothesis works under the assumption that everyone knows about news immediately and has time to react. Some investors react in micro seconds, some in weeks. Both affect the price

Well in the UK I have as a amateur investor taken advantage of arbitrage in the last few years. One on a big trust Witan that had gone out to a much bigger discount than normal due to a stock overhang caused by a forced sale. The other was Electra private equity which was on a huge discount.

It's only an arb if your net exposure is zero. If you want an arb, look at Hansa. You can buy their trust and sell an underlying porfolio which, if you do it right, should come out to very close to zero exposure. But most CEFs aren't selling at a discount because of liquidity, they are just shit (if the manager is decent, they should buy back shares to close the discount).

The market can stay irrational longer than you can stay solvent.

Efficient market theory is mostly used to justify deregulation, to protect the profits of the big players who can throw the most weight around.

Some economists support this (perhaps because their paychecks depend on it, or they were brainwashed at the University of Chicago where the movement was strongest). But I doubt most do, especially these days, so soon after the reminder of the Great Recesssion, 80 years after the Great Depression taught the initial lessons. But Keynes got boring for some I guess, and the allure of the "totally rational human" was clearly too great for some, and attractive to lobbyists.


Isn't the modern interpretation of Keynes a complete bastardization of what he was actually teaching?

AFAIK, Keynes claimed that the market is more efficient than not. But he was decidedly not a believer of the efficient market hypothesis! Further, he was mostly concerned with money supply and risk-free government bonds.

Sometime between the 60s and the 80s, Keynes got associated with Supply-Side Economics and Laissez Faires (which from my understanding Keynes was completely against).

Also, when Keynes was writing -- FIRE (Finance, Insurance, and Real Estate) amounted to about 5% of the U.S. economy. Now it amounts to more than 30%. So, ideas that weren't concerned with those markets probably need to be concerned with them now.

I'm honestly baffled why anyone would take "markets are efficient" seriously.

It seems such an obviously rhetorical (i.e. politically expedient, not fact-based) narrative intended to sell the idea that markets are the very best of all possible ways to organise everything - in spite of all the centuries of historical evidence that markets consistently run in an irrational bubble-driven boom-bust cycle with almost insanely destructive opportunity and recovery costs.

What “markets are efficient” means is really that you can’t make more money on the stock market than average unless you put the same time and brain power to it that hedge fund people do, and even then you can only make about as much money as hedge fund traders do. (Assuming a few things, like that you’re only trading based on public knowledge)

The logic is this: if this were not the case, then some of those very smart hedge fund people were take advantage of this strategy, making that strategy become the average that everyone wants to beat.

A simple thought experiment:

The Efficient Market Hypothesis states that every asset is fairly priced, because anything that CAN be known about it has been ALREADY factored into the calculation of price.

That's like walking down the street, seeing $20 and thinking, "Nah, if there was money on the ground someone would have picked it up ALREADY".

Markets trend towards Efficiency, they are not in themselves efficient

That's actually not what the efficient market hypothesis (EMH) postulates. The EMH does not state that it's impossible to find opportunities to make a better-than-market return. It states that it's impossible to predict those opportunities.

In other words, the EMH does not preclude $20 bills from lying on the street. It does imply that, in an idealized market which is informationally efficient, you cannot predict where people are dropping $20 bills around the city.

Your example is one of the most common mis-analogies of what the EMH states. But in point of fact, if the EMH precluded any better-than-market returns, it would not be possible to have a market since you'd have no risk or volatility.

You could make a similar comparison with evolution -- if the jungle is so competitive, how does the monkey find any food? The answer is that there is fruit available, but the monkey has to invest energy and expertise (both now and via billions of years of evolution) to extract it. The bigger the fruit, the more difficult to get it.

The problem with your analogy is that, in the real world, you don't know if the $20 is real or not. It could even be a -$20 bill.

If you believe the market is 100% efficient, then it follows it always has been this way. Do you really think the market is equally efficient as it was a 100 years ago? If not, then it follows that market efficiency is a gradient, and gets more efficient over time. I do not believe the market will ever hit 100% perfect efficiency. There is room for improvement and room for longer term alpha.

Another argument against EMH is to assume it is true. Market wages for quants and hedge fund employees would reflect their true value. But wages for quants are a lot higher than the cost of using a random number generator. If all trading success can be attributed to luck or chance, it would make no economic sense to hire expensive quants, hence EMH being true leads to a contradiction.

I do believe in a no free lunch theorem for market pricing. Averaged over all traders and all stocks and all strategies, there is no perfect approach that beats the market consistently. But that is of little practical value (in inference and search the no free lunch holds, but we can still use prior information to limit search ranges, focus on the profitable markets, and use approaches that worked on similar problems).

I am tempted to ask which school and truly in the post crisis era? In my econ podcasts they keep lamenting how things like this are still thaught, but I was somewhat incredulous. I studied econ in the early aughts and there was more behavioral, industrial and institutional economics than macro in my program. That was long before the crisis and still looks like the proper mix.

And then on the question. Perfect information is false and there are degrees in the definition of how efficient markets are, with the most common one approximating 'somewhat' on the not too short timescale. Mileage varies between commodities, stocks, your staple teaching example of the market for used cars (lemons) going towards the more micro models with network effects etc.

Institutional economics will teach you that no market functions without surrounding institutional arrangements and that there is some relationship between the nature of the product and the optimal 'institutional burden' (my words). Mechanism design is the subject where they try to design optimally structured markets.

Economics is awesome and anyone teaching only perfect information even to undergrads or business students should be fired. Ceterum censeo

Economics is not a science. The end.

It's mostly ideology with mathematical excuses.

At best, it's non predictive modeling.

Such and utter BS. There is so much science in econ. Just look at some of the work of the nobel laureates. Gary Becker treating social relationships as outcomes of economic relations. Great and provacative work. Ostrom on tragedy of the commons and solutions? That has properly changed the way we treat things like fishery, with more sustainable outcomes when applied correctly. Had we learned more of Douglas North, we wouldn't have so much kleptocracy in eastern Europe in the post-USSR period. Fama in finance, etc. etc.

Too easy to dismiss that much work. And don't blame economists for the crisis. That's politics. There are a lot more lawyers in politics than economists...

We don't discredit the whole of psychological science for the failures of a load of social pop-psychologists. Perhaps, perhaps we can discredit the whole of nutritional science. And even there I'd say we have learned some but that the financial incentives are so biased against proper learning that it's hard to blame the field as a whole.

Sane people do discredit most of psychological science. BTW, they're a real science where people could find an answer, and economics almost entirely isn't.

Probably the best piece of economics I've seen is Coase's theorem. While we base our laws on this idea, it's still a shaggy dog story, and basing it on dialectical materialism, the divine right of kings or Schmitt's friend/enemy principle would work just as well. Citation: places which never heard of Coase's theorem work/worked just as well.

> Economics is not a science. The end.

That’s too facile. I’d say, insofar as it is a science, it’s not about the real world, and insofar as it’s about the real world, it's not a science.

But even that doesn’t give it enough credit.

To me that's hiding another question of whether or not it's worthwhile. Like when we ask if something is or isn't art, or if writing software is or isn't engineering. There shouldn't be any link between saying economics is or isn't a science and giving it credit.

Efficiency doesn’t have to imply perfect information. (There are many forms) Just that probabilities are baked into the price so that there are no easy arbs.

If markets were efficient we wouldn't see such enormous and consistent outsized returns from value investors of the Graham-Buffett school, as mentioned in another HN thread today: https://www8.gsb.columbia.edu/articles/columbia-business/sup...

The absoluteness of the efficient market hypothesis also leads to some interesting paradoxes that don't make much sense when you look at the real world: https://en.m.wikipedia.org/wiki/Grossman-Stiglitz_Paradox

Are you certain you remember all of your schooling? Any basic finance class will cover the efficient market hypothesis, but also cover exceptions and temporary violations. This is standard course content and hasn't changed significantly in decades.

No industry practitioner actually believe in strong EMH. Otherwise they would be doing something else. If the market was efficient, then no one, and I mean no one, would be able to achieve a higher Sharpe ratio than anyone else. A higher return would always imply higher risk. This is clearly not true, and the Sharpe ratio of different funds and investments has a high variance.

Maybe you can say that weak EMH is true, that is, that the market is mostly efficient (and I believe that, as a professional market practitioner), but there will always be opportunities to make superior risk adjusted returns giving enough skill and wit.

Perfect markets are efficient. There are no perfect markets because the participants are not all perfectly rational or in possession of all relevant, complete and correct information.

Imperfect markets (real ones) are not efficient.

Hypothesis is the key word. It's not a law, or a tested theory.

Also, it's predicated on all participants having perfect info, which doesn't happen in the real world.

Is it useful? Mostly no, in my opinion, other than realizing that when my own idea of valuation differs from the market's, it's usually because I'm missing something.

All models are flawed. The question is not whether the market is efficient, but rather if it is useful to model the market as efficient for the purposes of economic research. Such a model does seem to work well in a lot of cases.

The EMH has been losing some steam because it doesn't match the data. In the last couple decades there has been a lot of data crunching like this, and academics have found these same factors. This study confirms those results with more and older data.

People have come up with two classes of explanations, touched on in the article. One is human behavioral factors and cognitive biases, the other is structural biases in the economy, like investment managers being more incentivized to avoid relative losses than to make absolute gains.

The market model is a gross simplification of reality. The basic model doesn't incorporate irrational decision-makers, data asymmetries, externalities, etc.

Markets are probably only efficient if P = NP, and it probably doesn't:


There is a lot of 'probably' going on here. He structures his entire P=NP argument on the basis that most computer scientists believe it to be false. That's fairly weak.

He has shown that markets are only efficient if P=NP, but that doesn't say much about whether they are therefore inefficient.

"In short, if P≠NP, then the market cannot be efficient for very long because the ability of investorsto check all strategies will be quickly overwhelmed by the exponential number of possible strategies there are to check."

He also states this, but even an overwhelming amount of strategies can be condensed into algorithms that makes 99% of them useless. See the game of chess for similar ideas. 99% of the possible moves are useless, so it seems as if there are an exponential amount of good moves when only a fraction are viable. Chess is NP, but still partially solvable under P conditions. I believe the market is like this too.

The LIBOR scam and similar schemes blow out the fiction that markets are efficient. One counter-example suffices to defeat a general claim.

> In school we were strictly taught the market is efficient and I fully believed that. But I've seen online and discussed in person a (seemingly) growing idea that markets are not in fact efficient at all.

I can tell pretty much everything I need to know about a person based on which one of these ideas they would say or defend.

Always lots of new ideas and theories that hope to reduce the participants in a market down to an explainable behavior.

But there are lots of ways to extract value from a market, which is more important than the debate around "already 'efficient' or not".

I think many people may be surprised how much of quantitative investing is similar to what was done here: finding strategies that are proven to work well over time and in different market conditions. It's not all black boxes and HFT.

Here is a great video of Cliff Assess of AQR, one of the largest quant hedge funds basically saying the same thing [0]. They focus on Value, Momentum, Carry, and Defensive.

[0] https://www.youtube.com/watch?v=FqaP3VTKccE

Commodity futures trader here... this article reminds me why I got into futures: because I don't have patience to hold positions for days, weeks, months, or years. If that's what you are doing, and it's working for you - keep on doing it.

All the bloomberg articles in the world won't help you trade profitably nor consistently. At the end of the day(for me), success is a very repetitive set of tasks that rely on math and probabilities to succeed. Inside information is great(no really, it is great when you have it), but discipline was the most important skill for me to master. Before I began, I was sure I had it - and was entirely wrong.

You know what's very good at a very repetitive set of tasks that rely on math and probabilities - and is 100% disciplined, right?

Holding a position for years is the best bet for me because I don't have the patience to investigate the same reports (with small changes to the numbers) constantly to figure out what is going to change in the near future. I just buy a good index and let it grow. I'm up 3x my money over the last 10 years and I think about that account once a year when I have to put it on my taxes - except for one time when I wanted to make a big purchase: I had plenty of money to spare in that account.

That's fantastic! I was never able to hold over long periods myself, but that's an excellent success story you shared.

At times like now(bullish), do you continue to grow your position or wait for dips? I think there is great long term profit to be had in people allowing fortunes to grow in long term index funds, I just wish I was one of the people who could take advantage of it. My niche revolves around VIX futures contracts and their effects on equity index futures - so very short term(intraday).

The account I was talking about was created on a one time windfall (see my other responses), and had no other contribution.

Most of my investment is in my 401k, which I max out. As such I'm dollar cost averaging and not otherwise thinking about it (except my last paycheck in December is a mini bonus). My day job is writing computer programs. When I get home every night I have dinner with the family, play a bit and put the kids to bed. This makes it easy to ignore the stock market day to day: I have better things to do than think about it. Once a year I check balances, decide I can't retire yet (won't happen for years) and forget about it.

You picked the best 10 years to start investing :)

That was pure luck, I had some stock in a previous company that got bought out. I decided I didn't want to think (if I had been thinking about it I would have sold years before) so I dumped it in the index.

What index did you invest in?

Vanguard S&P500.

The chances of this data being accurate is close to zero. I have seen data looking at factors for major markets outside the US, and it is wildly inaccurate even as recent as the 1970s. Mainly trading costs and liquidity but delistings too.

In addition, I know someone who looked through the paid data offerings for a similar purpose. He ended up employing ten to twenty people year round to build out a proprietary dataset because it was just not accurate.

Furthermore, the farther back you go the of an issue you have with survivorship bias. Finding all the companies that went bankrupt or bought out at a discount and correctly accounting for the losses you would have made investing in them is difficult.

To people saying the market is hopelessly efficient, It's not as bad as it sounds. Working on an S&P500 option strategy that can capture the total return of the S&P500 since 1990 while risking only 4-5% of capital per year (only 2% when one includes a nominal 2% interest rate). There are tons of strategies like this. There are many ways to get excess risk adjusted returns. Index funds though are still the best bet for most people though.

You can’t get 20x leverage on the upside without 20x leverage on the downside (unless you pay for it).

There are not tons of way to get excess risk adjusted returns.

Really interested to read specifics. Any links?

There is a risk to this sort of long-term analysis - they identify 'following the trend' as a good strategy; but it was a good strategy under different circumstances. The advent of computers is a huge deal that may have changed the game.

Even as recently as 1970, how easy was it to execute a trend following strategy? Gathering and processing the data would have been a mammoth activity requiring several clerks and lots of record keeping. Demonstrating the it was a good strategy would also have been very difficult (more clerks, lots of by-hand calculation). Only a serious trading house could have pulled it off. The ability of actuaries to employ statistical models has also changed completely in the last 10-20 years. The amount of money following a trend-following strategy is also an extremely interesting but unknowable figure.

If the advent of computers makes it trivial even for a roaming programmer to implement a trend following strategy, is it still going to be robust? Now that it can be demonstrated to be a strong strategy, will the market correct for it?

The core of the article, for me:

Now that quants have discovered them and attempted to profit from them, there’s a risk that those effects will be arbitraged away. But the sheer durability of the factors does imply there is something to factor investing.

Can someone who has made money in finance or knows how it works, explain how/why short term gain/loss doesn't translate into long term gain/loss?

It seems like if there was any signal left above the noise, that trading bots would exploit it. But that's obviously not the case, because I could hop on Robinhood today and make a quick 2% riding the daily heartbeat of the stock. My last experience with this was 18 years ago with my dad where we day traded Apple and got up about 40% or $40,000 over 4 months between the dot bomb and 9/11 (which lead to subsequently wiped out all our gains in a week). If things have changed so much that this is no longer possible, then I apologize. But the article suggests that finding correlations still works.

I fundamentally don't understand how people on Wall Street can make hundreds of thousands of dollars per year investing (with other people's money, I guess?), but meanwhile the average return of the stock market is only 10% per year.

What I'm getting at is that either:

1) there is predictability in investing (meaning that we could all profit if we just had the money to trade, which is inherently discriminatory against the poor)


2) there is no predictability in investing (meaning that it is gambling and should perhaps be regulated or banned as such)

I'm getting the feeling that all of this has fingers in issues like wealth inequality and generational wealth. Perhaps this is simply a "because that's the way it is" situation, but that was said about stuff like slavery and child marriage.

> (which lead to subsequently wiped out all our gains in a week)

This is the key point. Many strategies that yield above-market returns often do so only because they're taking on large amounts of risk, even if that risk isn't immediately apparent. Think about trading cryptocurrencies, buying penny stocks, selling naked options, etc. You _can_ make a lot of money doing any of these things, but you'll be exposed to quite a bit of downside along the way, and most people who try will end up in the red.

Aka “picking up pennies in front of a steamroller” or “selling insurance” or “writing naked options” etc.

...given sufficient hours/time in the game.

Just to bring the thought home.

Im a blackjack player, and i calculate my returns at 100hours of play because of this manifestation of the law of large numbers

> there is predictability in investing

Even if you could predict the future of your investment the market will eventually catch on.

Let's say you can accurately predict the market's up or down direction a week in advance. You know it will go up so you start buying. The act of you buying drives prices up until even you don't think it's a good investment. Essentially, you've accelerated your future prediction to become today's reality.

Markets suck up information. Maybe you could make predictions in secret but as soon as you start publicizing those predictions or acting on the market, you've already changed the conditions.

All it takes is a bug in a financial model for someone to start dumping shares and affecting the market.

While markets are self correcting, they are not predictable or guaranteed to be rational.

Many people who are predicting markets are getting lucky or hoping to do it so publicly that it becomes a set fulfilling profacy.

I disagree - if "The act of you buying drives prices up", then you are making the market, not speculating about the future.

Trying to predict markets has a high risk of ruin.

>"The act of you buying drives prices up", then you are making the market, not speculating about the future.

You seem a tad anal here, what is the difference here?

There is a vast difference between short term and long term investing.

Short term investing is cutthroat competition to take advantages of the slight gap between the market and pure unpredictable behaviors.

Long term investing is buying a portion of future profits. There is still uncertainty but much less especially for properly researched and diversified portfolios.

Of course you shouldn't make major financial decisions based on what you read from internet strangers, do your own research.

A book I recommend is A Random Walk Down Wall Street https://www.amazon.com/gp/product/0393246116

> we could all profit if we just had the money to trade, which is inherently discriminatory against the poor

Well, this is capitalism 101: if you have capital you can get returns on capital (with a risk/reward tradeoff, but there is such a thing as a "riskfree" return), and people have been complaining about this since before Marx.

Predictability: you should see it more as a competitive sport. Nobody can predict who's going to win reliably, but being informed and doing the work will give you better results than not.

(Also, not everybody is a "trader"; quite a lot of people are effectively commissioned salesmen for either the buy side or the sell side)

I'm not surprised by this, these strategies are based on pretty timeless human foibles.

People are always going to prefer to buy something everyone else is buying, (no one was ever fired for buying IBM). Some people will want to buy 'cheap', some investors will just want to receive a dividend cheque every year.

Jack Bogle would warn against "fighting the last war".

This means past results have no bearing on future results.

What is trend following?

Trend following is exactly what it sounds like. Pick a trend(over a time frame you choose, using a set of indicators you choose), then join the market on the side of the trend. To be successful, you must join the trend early, and be vigilant and disciplined enough to cut your losses when your trend fails to gain momentum and/or reverses.

From https://investresolve.com/blog/over-two-centuries-of-global-... (linked to in the bloomberg article) :

Trend and momentum are defined as the 12-month-minus-1-month excess return. While this is consistent with traditional measures of cross-sectional momentum, “trend” as proposed by Moskowitz, Ooi and Pedersen; and Hurst, Ooi and Pedersen does not impose a skip month. The current authors make the case that a skip month is more conservative because it allows for a delay in data diffusion in very early periods.

It's pretty similar to momentum investing, except using real returns rather than relative performance: https://www.aqr.com/Insights/Research/Journal-Article/Fact-F...

Mind if I ask a poll question- Do you believe 'markets grow at 7-13%/yr, over 10 years'

I've been taught this like its a fact.


John Bogle [1] claims that the long term annualized return from the stock market has averaged about 6.8%. The lowest 50 year period was 5.8%, and highest was 7.9%. As far as 10 year periods, the lowest was 2.3%, and highest was 11.5%.

But, these percentages are after adjusting for inflation, maybe before inflation would be closer to your "7-13%".

[1] Common sense on mutual funds, p16

It is a fact in the sense that the average stock market return over the course of the 20th century was ~10%. Of course, as every prospectus tells you, past performance is not a guarantee of future results, but what else are you going to use as a baseline?

Disclaimer: I have limited experience with anything finance, my profession is programming.

My understanding is that 10 years ago was a stock market crash(2008), and since then we've been in a historic bull run. My understanding is that it WILL crash again (obviously), we just don't know when. But the stock market has always been like this, cycling between bull/bear runs.

Wouldn't it make more sense to use a larger dataset that isn't just a bull run? Maybe the last 25 years or whatnot?

No. Over 30 or 50 years maybe but 10 years is too short especially considering we just had an incredible bull run.

I think many leaders in the space have said they expect lower returns in the coming years as well. Obviously this list is biased towards those who I personally believe in:

https://www.cnbc.com/2017/03/22/jack-bogle-believes-the-stoc... https://www.cnbc.com/2018/09/14/nobel-prize-winner-shiller-s... https://www.cnbc.com/2018/07/06/vanguard-youll-make-less-mon...

I've heard recently that we can expect lower returns in the 21st century, on the order of 5%, simply due to more efficient markets. It makes sense given that information spreads more rapidly today which in turn reduces informational advantages. I like the following poker analogy, even if it's not a perfect one.

The markets in the 60s, 70s and 80s were like poker tables with plenty of fish and few sharks. The sharks came out with bigger returns due to asymmetrical informational advantages which resulted in exploitation of mistakes. Today, the tables are full of sharks which means less mistakes and smaller returns.

Intuitively, this makes sense, since market gains are really just market valuation errors.

This doesnt make any sense. Short term trading gains/losses fit the poker table metaphor. The market as a whole is pretty insulated from those dynamics over longer than ~3month timescales.

10 years is too short. Take a look at https://blog.nawaz.org/posts/2015/Dec/pay-down-mortgage-or-i...

You'll find charts for 5, 10, 20 and 30 year windows for the S&P 500. If you look at the 10 year one, you can see it has gone as high as almost 20%/yr for some 10 year period, and has given negative returns at least twice. That's without inflation.

If you adjust for inflation, then it's more like a peak of 17%, and multiple 10 year intervals that were negative.

Even a 30 year window (which I think is where the traditional 7% after inflation comes from), there were periods below 5%. The highest was a bit over 10% (inflation adjusted).

This is assuming a lumped sum in the initial period. If you're looking at putting money in every month/year and calculating the effective returns for those, you'll find those plots as well.

No, sort of. 13% is way off, but 7% is about the inflation-adjusted average yearly real return of the US stock market over the last 140 years. But any particular 10-year period can be up or down. You only get some amount of safety if you're investing over 40+ years.

My understanding is that the consensus number used for eg retirement planning was 6% real (after inflation) annual returns for decades, but today (“secular stagnation” etc) I’d probably conservatively calculate with 5% or even 4%.

Applications are open for YC Summer 2019

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact