Hacker News new | past | comments | ask | show | jobs | submit login
Investing Returns on the S&P500 (github.com/zonination)
618 points by minimaxir on June 29, 2016 | hide | past | favorite | 340 comments

I don't think it is fair to say that next 100 years will be same as last 100 years:

1. GDP growth is not as high as it used to be anywhere in developed world: http://www.oecd.org/std/productivity-stats/oecd-compendium-o...

2. USA is superpower at the peak. Plenty of other stock market economies hasn't been so successful. E.g. Argentina used to be one of the richest country in the world. Investing in history is easy, e.g. you can also say about Apple stock that it will always recover based on past data, but plenty of other companies hasn't.

3. Most of the 100 years we have inflation: http://inflationdata.com/Inflation/Inflation_Rate/Long_Term_...

Right now interests rate are close to zero, which is rare. Long term trends may change if we continue to operate in those climate.

Edit: Inflation was factored in the graphs. Sorry I was wrong about that.

>"anywhere in developed world"

I hate this terminology. If the US has, in fact, "finished" developing, then the future of the S&P will be bleak. I don't think that's the case at all, though. I think development has just started and we'll see fantastic advances in bio-informatics, solar power, 3D printing and a number of other fields in the upcoming decades.

When I was a teenager, I pirated music. I seriously hope my children pirate gourmet cheeses and burritos as teenagers. I download apps that organize my notes. May they download the bio-app that lets them dunk from the free throw line.

>> If the US has, in fact, "finished" developing, then the future of the S&P will be bleak.

Most of S&P are global corporations. They're considered American simply by virtue of having headquarters in the US.

This only shifts the question as to whether or not the rest of the world can emulate the growth of the United States, and if so, where specifically and to what extent.

I'd suggest taking a look at Robert Gordon's The Rise and Fall of American Growth. Gordon's coverage of past technology and both the mode and extent of impacts, and of recent developments, including information technology medicine, and other areas, is sobering.

And that's despite disagreeing with Gordon fairly strongly in a number of other areas.

Vaclav Smil and Manfred Weissenbacher both have excellent books describing the impacts of technolgy, and especially energy, on history. I strongly suspect that's the most significant component.

Gordon's book is part of a larger series much of which addresses the question of why concerning the Industrial Revolution and the absolutely unprecedented growth of the past 250 years or so, much of that occurring only in the 20th century. It's hardly the only treatment of that phenomenon, but I do agree with Gordon, and others, that it's perhaps the most pressing question, still unanswered, of economics.

Who will be able to buy that stuff?

I'm not seeing the end is nigh unemployment rate. But it will definitely get worse...

With the dropping off potential buyers you reduce the possibility for increased profit.

Sure you can just increase the margin ( and I think that is what we will see more ave more I future) but that growth is also limited.

New markets will emerge and make profits. But economics as a whole will shrink.

US is far from finishing deveping, fortunately or unfortunately. Take trains, for example, area where US is far behind China, Japan, or European countries, despite the size of its territory. I'm sure there are other examples.

Our infrastructure needs a lot of work ($4T dollars worth). Improving our infrastructure has a good ROI (GDP multiplier).




The problem with infrastructure spending isn't that it doesn't potentially have a good ROI, it is that the process channels the money to wasteful projects that don't have a good ROI.

This is what happens in every ecosystem as the available energy sources become fully exploited. Featherbedding and cronyism are the parasitic vines which strangle enough trees so that when the next lightning strike comes the conflagration is inevitable. They cannot exist without the forest. When the forest grows to maturity they become inevitable.

Tangent, but the US freight rail networks are much better than Europe. US is good at freight rail and bad at passenger service, Europe is vice versa.

Actually, I find Europe's passenger service uninspiring compared to Japan or especially Taiwan.

Taiwan's trains are subsidized

So are Japan's. The idea isn't to get a profit off of tickets vs operating costs. The idea is to improve the efficiency of the country and economy. It's an investment.

If the US had a similar political climate the east coast would already have high speed rail from New York to Miami.

Rail needs to be profitable, but roads certainly don't. For some reason.

Japan has a very low subsidy and most of its railways are private and receive none.

I think that Japan's population density and relatively small size makes it much easier to make money on railways.

But they don't need to be, really, when they can just put it on a boat for the whole journey, or most of the journey and then just truck it a few hours inland.

Transportation in general is weak in the US compared to other countries.

He probably means "developed world" as compared to third-world countries. I agree, though, that the U.S. have not finished developing but have instead just begun developing except on a new frontier now.

"Developed" is a relative term (more fully, it would be "More Developed"), not a statement of completion (this is somewhat confused by the currently-accepted alternative, "developing", but it was more clear with the more accurate, previously common, alternative "Less Developed".)

But we live in the best of all possible worlds!

Don't be so Candid

I thought it was Chumscrubber.

Have you visited other possible worlds?

all of them, that's still nil though.

nil has very interesting qualities.

Nothing has no values. Nil is nothing thoug, or what properties should it have?

It noths. With vigor.

The question being, is that what I believe, or fear?

> USA is superpower at the peak.

That remains to be seen

> Argentina used to be richest country in the world.

That is not true. In the early 20th century they were top 10, but never surpassed Britain or the US in GDP per capita.

> That remains to be seen

The fact that it remains to be seen is kind of the point. It is possible that it is at peak, and if it is true, then it is likely that historical data is not very useful in understanding market returns.

Couldn't you have made this same point at most times in modern American history? Wasn't it possible that the US was at its peak in 1970? Or 1980? Or 1990? I suspect it is only due to hindsight that we know it wasn't the peak then.

Yes, that's true. I'm not saying that the US is necessarily at its peak, just pointing out the fundamental problem with extrapolating based on historical data.

It assumes that the future will look like the past, even though there are some plausible reasons to think that it won't.

Some of those reasons would push you harder in the direction of investing in stocks (e.g. increasing automation, globalization) and some would push you in the opposite (e.g. declining gdp growth, rising inequality, political instability).

Actually, that isn't quite right. All the negatives you mention should push you away from investing in stocks only if you think that other investment options under those scenarios would do better. But why would they ?

> That remains to be seen

What would a more powerful USA look like? It doesn't appear that the country is after an empire the same way the British had one 100 years ago, so raw aggression is out. I can't imagine a realistic scenario that doesn't require the implosion of other nations to embiggen America.

An even more wealthy USA? Imagine one of those horror scenarios where robots / automation start taking over many, many jobs and the multinationals that own them are based out of the US.

Not saying it would happen but I think that's one scenario that would be even "more powerful".

With off shore holding accounts, do google et al count?

> What would a more powerful USA look like?

Passenger rail; durable houses; less unemployment and more labor-force participation; a population not balkanized along political/cultural lines.

With an increase in immigration and a cheapening of robotic workers and kiosks... where is this decrease in unemployment and increase in labor-force participation going to come from?

I mean, there is "always" going to be jobs for plumbers and spots for artisan made stuff... but those jobs aren't going to replace the amount of jobs that will be lost when $15 becomes more expensive than iRobot...

> With an increase in immigration and a cheapening of robotic workers and kiosks... where is this decrease in unemployment and increase in labor-force participation going to come from?

I don't know where it would come from (perhaps the same magical unicorn tooth fairy who would remove balkanization, inspire durable construction, and give us more passenger rail), but the US would look a lot stronger if it had it.

This includes politically stronger; poverty, disengagement from the labor force, and high inequality are not good for a country's political stability.

200 years most people farmed, 15 years ago SEO, apps, and YouTubers didn't exist either, yes YouTubers... News jobs are already being created.

> News jobs are already being created.

Not fast enough to replace the abolished ones, and most of those new jobs are knowledge work, which can't scale as there's a IQ floor that will stop most people from being able to work them. The more we shift to knowledge work, the higher unemployment will climb. You're making the mistake of thinking the past predicts the future, but today's automation is unprecedented, it will not follow the same pattern because it has different effects.

Humans have a knack for adaption. We'll figure out the employment problems that result from automation and robotics. We'd make great pets.

Humans have a knack for selfishness and war, it's more likely there will be much needless suffering and death along any path to an eventual solution to those problems, which will likely be a solution that most suffer under.

A UBI, supplied taxafion, and significant deregulation would enable citizens to participate in the economy on much more liberal terms and you would almost always see participation rates rise while total hours worked fall.

simplified taxation*. Yay autocorrect on mobile from hours ago.

The USA is at 'peak' , we can roughly say this, not because of anything America is doing, but because the 'rest of the world' is coming online.

50 years ago there was 'America', the 'Soviets' and 'Europe'. Everything else was a footnote.

What we've seen with China 'coming out of nowhere' in the last 25 years is about to happen 4x with the rest of the world.

America and Europe's share of global economy is falling fast, and it simply has to do with electricity, plumbing, democracy, knowledge and productivity gains in the rest of the world.

India is coming on a little slowly, but surely.

Pakistan is 180 Million people. Nigeria almost that - and they have oil.

Africa is a billion people.

There are a lot of Asians outside of China, India and Japan.

As they start to become connected to the global economy, relative power shift will happen.

I don't see the emergence of any new 'superpower' and America will likely remain the only 'superpower' in the classical sense, but it won't be as powerful as it is today - relatively speaking.

If you look at the growth of Japan after WW2 - a period of fast growth, then slowing down as it approaches parity with the west - this is what the underdeveloped world will look like, but a little slower.

Also - a very important point - a lot of real growth is not measured in the GDP.

For example, the value that is created by governments is measured simply in 'government spending'. But $1 of gov. spending hopefully creates more value than $1!

For example, if the US Gov spent $50B and created this amazing national public transport system that everyone used, and cost next to nothing - the 'economic surplus' would be massive, but the GDP would directly be down. Why? Because the only thing that would count towards the GDP would be the $50B spent by the gov, not the $Trillion in value derived from it. Obviously, there would be indirect business benefits ... point being - the GDP is not necessarily the best way to measure things for many obvious reasons.

> I don't see the emergence of any new 'superpower' and America will likely remain the only 'superpower' in the classical sense, but it won't be as powerful as it is today - relatively speaking.

Maybe. But the trend towards privatising military force could end up accelerating USA's slide down in terms of military power. If we are indeed moving back to the "normal" (historically speaking) "neo medieval" world were military force is generally mercenary and not national standing armies - coupled with the commodization of advanced military technology - the world could radically change quite quickly.

Maybe the USA won't be able to pay its contractors - and China and/or Russia can? Maybe China will restructure the PLA as the world's biggest military contractor?

I don't really think you are wrong - but it can be tricky to predict the future. Perhaps especially the sudden stumbles we seem to make every so often as a species back into new dark ages and decent into chaos.

"But the trend towards privatising military force could end up accelerating USA's slide down in terms of military power. "

+ This is a very marginal thing.

" If we are indeed moving back to the "normal" (historically speaking) "neo medieval" world were military force is generally mercenary and not national standing armies"

+ This is not happening. Don't worry about it.

Listen - war is too expensive. It is only economically advantageous when the parties are not interconnected.

Some American businesses can economically from a war in Iraq by getting big contracts ... but a war with any real nation would cause massive upheaval in markets.

The moment Brexit happened - it was a big shock to markets - and that was a small thing.

90% of people and businesses 'lose' in a big war, so it's very, very unlikely that China, Russia, USA get involved directly in a war - it's just too costly for everyone. USA is China's #1 customer.

The only 'rising military power' that will make a difference is China.

Everyone else - won't matter.

It's also very, very expensive to challenge the Americans in any way, even regionally, so the only actors that will do it are non-state entities, i.e. insurgents like Al Qeda etc..

The 'big shift' over the next 50 years is that 4 billion people who are not even on the radar today are going from $2 a day, to $50 a day in terms of GDP.

Iraq was "a real nation" before the first gulf war. Indeed, as you point out - the US lost that war by any sensible measure - but that doesn't mean the same mistakes won't be repeated. And the gulf operations were something of a 50/50 split between contractors and military personnel, at least towards the end of the protracted conflict.

Don't forget that "the end of war" as proclaimed after world war one - because a new world war would be too costly, and too horrible for anyone to contemplate. Of course no one will start a war they think they will lose. You start wars you are "certain" you will win. And then they escalate. And escalate.

> The 'big shift' over the next 50 years is that 4 billion people who are not even on the radar today are going from $2 a day, to $50 a day in terms of GDP.

I absolutely agree that this is going to be the biggest change. With accompanying logistics, new (political and economical) battles over water rights and so on. If there's a new "private" war, I'm guessing it will start with a large contractor being hired by some regime to "calm down" "unreasonable unrest" over stealing natural resources and handing them over to corporations, like for example privatizing water (or as one can see in Niger, privatizing oil).

I certainly hope we won't see a third world war, but I'm also afraid of people that seem to think that large scale war is "unlikely". Nothing in history suggests that it is.

[ed: And "insurgents" are likely to always be proxy wars. The US created the modern resistance in Afghanistan to fight Sovjet power, and then end up fighting that same insurgency, now supported by Saudis, which are allies on paper.]

It doesn't appear that the country is after an empire the same way the British had one 100 years ago

US empire is defined more by squashing incipient threats to its global hegemony than by UK-style colony management.

UK-style colony management was also largely about access to colonial resources, and colonial markets. Colonial management was a means to that end.

The US empire is doing precisely that, through a mixture of soft and hard power.

To put it another way, the British Empire had a "merchantalist" foundation - maintain a positive balance of trade and surplus material assets by forcing its colonies to engage in trade on preferential terms - while the U.S. has been "free trade" oriented in its imperial years, with balance of trade being less important than retaining GDP.

Both have imperial dynamics, but for the U.S. the important part is being the "world's policeman" as it comes with the authority to destabilize regions and install puppets where their sovereign governments may act against U.S. interests. With this more limited administrative footprint, they're free to focus on use of violence and propaganda, while taking up domestic market policies that benefit net importer businesses and thus justify maintaining the empire.

Where people say that the Pax Americana is ending it is in part because the dynamic has grown more multipolar since the end of the Cold War, with a diverse group of nations asserting their interests without being overthrown.

It's a perfectly cromulent word.


The USA has a huge empire, probably the largest ever seen, but it's not an empire of government expansion it's an empire built by companies and culture.

Empire is the wrong term. It's misleading, it's fraught, it leads to lazy thinking, facile notions and comparisons, and can facilitate unjustified anti-Americanism on one side, cavalier adventurism on another (think Cheney, neocons, AEI), and perhaps isolationism in a current sense ("we're an empire but we shouldn't be").

"Superpower" is better.

What constitutes an empire though, the US is more than twice the size of the Roman Empire at it's peak. Surely that is not an empire?

It also influences other countries to an extent which is similar to other empires like the British did. It invades countries to keep trade going, and it's companies dominate the world much like the companies of the British empire did.

The only difference is that the US has finished expanding.

So what really constitutes an empire? Does it need an emperor? Does it need to constantly be questing to expand in lands?

To quickly take on comparisons with the British Empire --

War: True, the US engages in war, and the British Empire engaged in war. But so have many countries that were no empires. When the US engages in war, it leaves and tries to set up a gov't as the new authority. This has worked well (Japan, Germany, etc.) and not so well (Iraq, South Korea initially).

Companies: Imperial British companies literally captured their markets -- they had their own armies! The British East India Company for example came to rule much of the subcontinent, with their own soldiers! Subjects of British imperial rule were told what to trade. They might have done well for themselves sometimes (or been devastated at other times), but in any case the relationship was clear.

Global leadership: The US took the mantle as global leader from the British. This wasn't desired. Indeed after WWI the US refused to play leader. Some historians believe that WWII could have been avoided if the US would have fulfilled its duty as most powerful to provide leadership.

After WWII the US became serious about being a leader, by which I mean taking the initiative on issues of global coordination, both security-wise and financial. This is another reason the US is so easily confused with an empire. The UN, IMF, World Bank, NATO, Bretton Woods, these were largely American initiatives, but they were not imperial dictats.

Again, the importance in my way of thinking is that countries with relationships with the US are not subjects of the US. That is the key distinction that must never be forgotten.

A question: In your way of thinking, is the EU an empire?

The definition of empire is a group of states or countries ruled by a single supreme authority.

All other uses of the term are metaphorical. We can talk about the McDonald's hamburger empire.

When we talk about the American Empire, we're also using the term metaphorically, because think what you will of the US, but it doesn't rule as a supreme authority over anything but a few islands. It may have friends, allies, interests, influence, bargaining power, but it has no subjects, no subjugated peoples. It may have largely founded what is now the world's economic and security order after WWII, but those are all voluntary.

The problem is that the overlap between superpower and empire makes the term empire confusing in the American case. "Powerful like an empire" and "an empire" are very different.

When we talk or think about the "American Empire", we seek similarities and ignore differences with the Roman Empire, European colonial empires, even the Athenian Empire or Soviet Empire. It makes it sound like the US can tell other countries what to do as it pleases, that the US rules over others. That's a very important mistake, even a dangerous mistake. Allies are not subjects. When a country asks the US military to leave, it leaves. See the Philippines as a good example (and further back as an example of the bad old days, when the US was briefly interested in having a real empire).

Today's US is an empire in the sense that Microsoft is an empire -- it is an empire of the willing who can leave whenever they like.

And if you can leave whenever you'd like, it's not a real empire.

> I can't imagine a realistic scenario that doesn't require the implosion of other nations to embiggen America.

This has happened! In the past 30 years, no less.

To what are you referring? https://en.wikipedia.org/wiki/Territorial_evolution_of_the_U... ends in 1970.

Yeah, but that was a surprise to quite a few people too.

Imperialism is alive and well today, it's just that it's less noticeable since total war has been infeasible since the atomic age.

Go tell that to people who live in Syria, Yemen, Nigeria, Palestine or Libya.

Puppet wars, not Total War. There's a difference: https://en.wikipedia.org/wiki/Total_war

We haven't experienced Total War in our lifetime (thankfully), but when referencing the imperial dynasties in history as being more noticeable, it's usually because the wars they held were total.

Yugoslavia, Serbia, Ukraine, Turkey...

Wealth is not a zero sum game.

That depends on how you generate it. When apple builds wealth, not zero sum; when a hedge fund builds wealth; zero sum. 40% of our economy is financial services now, there's a lot of zero sum wealth creation going on.

Even hedge funds aren't zero sum. They receive dividends on the investments they hold which is a return that is not zero sum.

Of course, the more active a fund is, the farther it gets from zero sum, but it would never be fully zero sum.

Additionally, financial services overall are far from zero sum. There is a great deal of efficiency and market lubrication created by financial services and if you exclude the high frequency traders, you leave a huge majority that is still quite beneficial to people on both sides of the transaction.

> Even hedge funds aren't zero sum. They receive dividends on the investments they hold which is a return that is not zero sum.

Once again, that depends on the trade, short term trades don't make their money from dividends, but from timing the market.

> Additionally, financial services overall are far from zero sum.

Beside the point, plenty of them are zero-sum-never said they were all zero-sum; that exception exist doesn't change the point that zero-sum vs non-zero-sum is a false dichotomy, the real world is both zero-sum and non-zero-sum, depending on the way the wealth is being generated. It's tiresome watching people continually repeat the incorrect mantra that wealth isn't a zero-sum game when quite often, it is.

No, I didn't say there are exceptions. I said MOST financial services are NOT zero sum. The zero sum ones are the exception.

Utterly missing the point, goodbye.


Embiggen is a perfectly cromulent word.

> So at Year 1, we take every point on the S&P500 curve, look at every point on the S&P500 that's one year ahead, add in dividends and subtract inflation, and record all points as a relative gain or loss for Year 1.

Point #3 is wrong.

While inflation was adjusted for it was not accurately adjusted for as it ignored taxes. If your returns are 10% and inflation is 10% you get taxed on that 10% and lose money.

PS: Now if this is for 401k accounts or something that's another story.

You dont get taxed until you sell. The performance shown here is for a buy and hold strategy.

Dividends are also taxed, so even reinvesting has a cost.

Granted, IRA/401k etc exist and in years with 0-3% it's not that big a deal, but over 20 years it's often a significant cost.

Dividends are not taxed in tax-deferred accounts like IRA/401(k).

Yes, that's why I mentioned them. Sorry, if I was not clear, though they also generally have associated fees. So again, nominal breakeven is not the same as actual breakeven.

> Dividends are not taxed in tax-deferred accounts like IRA/401(k)

Only if you never withdraw them.

No, you do get taxed on dividends.

You do, but that's why dividends are becoming increasingly rare in the stock market. Many firms are preferring buybacks instead, which cause a pop in the stock price for remaining stockholders and so get taxed as capital gains.

Qualified dividends (which is most of them) are also taxed at long-term capital gains rates.


I don't believe "dividends are becoming increasingly rare in the stock market". One measure of that is to compare the S&P 500 yield with treasury yield, the latter being near a historic low, but the S&P yield near a consistent 2% for over a decade, and quite attractive now relative to treasuries.


Exactly correct. From a tax management perspective, low inflation and low yields is preferable to high inflation and high yields.

I'm not sure why you are being downvoted, as the dividend component of the S&P 500 is a substantial component of the total return.

This is misleading. The question is, what is a better strategy? Like, you lose real value after this scenario, but you would have lost more if you'd have kept the money under your mattress. The only relevant question is: compared to other strategies, do you come out ahead?

I think the default option is usually to spend the money now.

If your choices are go on a 'cruse' today, or invest your money wait 15 years and then pay for a cruse waiting seems pointless. If you wait 15 years and can't pay for a cruse your clearly worse off. If you wait 15 years and can pay for a cruse and have money left over then that's an advantage to investing.

Sure, you can consider several investment strategy's. But, you can't see the future when your deciding what to do today. So, you can't say ahead of time what the best strategy is.

> While inflation was adjusted for it was not accurately adjusted for as it ignored taxes. If your returns are 10% and inflation is 10% you get taxed on that 10% and lose money.

And if you'd just put the money in a mattress you're down 10%, far more than the loss of paying taxes on 10%.

It's not wrong. We don't know how markets behave at near-zero-to-negative interest rates over long periods of time. The types of central bank policies that are in play right now are unprecedented.

This seems like a good place to insert a reference to Fooled by Randomness[1], which focuses on the point that you make in your second item.

[1]: https://www.amazon.com/Fooled-Randomness-Hidden-Markets-Ince...

The author did explicitly factor in inflation.

Space/Mars could start to be in play before the century is over. That's one example of a (literal) new frontier for growth.

Hehe, depending on the setting in sci-fi stories, that doesn't play out well, politically, and a lot of spending goes to the military.

So what alternative method do you propose for predicting growth (or decline) over the next century?

100 years ago you would have probably mostly invested in the UK, French, German and Russian stock market as well as the US. So if you want to predict the next 100 years I would try to combine the performance of the UK and the US, on the assumption that US is going to behave economically like an existing power rather than an up and coming power.

Also, just as some risky "foreign" markets did awesome a hundred years ago-- that's what the US was to the investing world in 1900, and it did great-- some will do great over the next 100 years. Some will have wipe outs as Russia, Germany and Argentina did. The future is hard to know.

Really though, owning some vanguard funds is a good idea. I'd just hold on to a little real estate as well.

You would've bought Russian stock in 1916?

If one took the long-term "buy and hold" strategy, there would be no reason not to include Russia in your portfolio, in proportion to its relative market cap.

Of course, "buy and hold" would have been very wrong in that era (two of your superpowers would permanently tank), as it might be in this one.

Or the UK, France, and Germany during WW1. Seems suboptimal.

Not in 1916 but why not in 1890 ?

Because the country just got over serfdom, corruption was rampant, and its imperial ambitions were too big for its boots?

A crystal ball.

Seriously now, this is like asking: What methods do you propose for building a perpetuum mobile? This Universe does not allow for the future to be predicted, like it doesn't allow for the laws of TD to be broken. All the predictions of growth/decline you read/hear from people are just educated guesses, nothing else. Some of them might get lucky with their predictions, some of them might not, but let's not foul ourselves into thinking that we can predict the future only because we can make nice-looking charts about what happened in the past

This is a strange reply.

You're saying we can't understand the world through science?

Physical laws like the laws of thermodynamics are methods for predicting the future that we obtain by inferring them from what we've observed in the past.

You can argue that particular domains (like the global economy) are nearly impossible to predict with much accuracy because we are unable to observe many of the driving variables and the underlying causal structure is too complicated for us to comprehend the laws of the system's behavior. But to say that, in general, the universe does not allow forecasting the future seems really weird.

> You're saying we can't understand the world through science?

We're having a hard enough time as it is understanding the physical, inanimate world, when it comes to applying "science" to human endeavors the results are much less certain, approaching crystal-ball-reasoning. People did try in the past to do just that, i.e. to make scientific "sense" about what happened in the past with us, humans, with the hidden aim of trying to predict what will happen in the future. Karl Popper had a really good book explaining why that was a terrible idea (https://en.wikipedia.org/wiki/The_Poverty_of_Historicism):

> The book is a treatise on scientific method in the social sciences.[2] Popper defines historicism as: “an approach to the social sciences which assumes that historical prediction is their principal aim…”.[3] “The belief… that it is the task of the social sciences to lay bare the law of evolution of society in order to foretell its future… might be described as the central Historicist doctrine.”.[4]

> Popper’s criticisms of the poverty of the idea of historical prediction can broadly be split into three areas: fundamental problems with the idea itself, common inconsistencies in the arguments of historicists, and the negative practical effects of implementing Historicist ideas.

> iii) Individual human action or reaction can never be predicted with certainty, therefore neither can the future: “the human factor is the ultimately uncertain and wayward element in social life and in all social institutions. Indeed this is the element which ultimately cannot be completely controlled by institutions (as Spinoza first saw); for every attempt at controlling it completely must lead to tyranny; which means, to the omnipotence of the human factor – the whims of a few men, or even one.”.

And, to put it more generally, we do know that the Sun will probably die in the next 4-5 billion years, that is science based on induction, i.e. we saw that other stars similar to the Sun have lived that long in the past and so we expect the Sun to have the same lifespan. But this is not prediction nor forecasting the future, is really a very good guess. Science itself is a really nice and educated guess. The problem is that we're much better at guessing what will happen to the Sun, let's say, compared to what will happen to the stock exchange in the next 10 years or to who the president of Rwanda will be in 15 to 20 years.

David Hume has put down in writing the issue with induction better than a guy like me could ever do. From here (https://en.wikipedia.org/wiki/Problem_of_induction#David_Hum...):

> Only through previous observation can it be predicted, inductively, what will actually happen with the balls. In general, it is not necessary that causal relation in the future resemble causal relations in the past, as it is always conceivable otherwise

I don't have a crystal ball. However, it would be more useful if more stock indexes would be considered than just one S&P 500.

E.g. Dow Jones: http://www.macrotrends.net/1319/dow-jones-100-year-historica...

Also from other nations.

Nobody uses the dow. It's price-weighted. It's most famous for being reported at the 6PM local news. But seeing the Russell 2000 would be great.

Dow Jones is probably one of the least interesting ones due to the small number of stocks in it.

it's not even the small number that makes it useless, it's the fact that it is not weighted by market cap.

Demographics. Africa, Nigeria in particular, looks like China and India of decades past.

Perhaps, but Nigeria is a disaster waiting to happen. Their economy is dangerously dependent on crude petroleum exports[0] (second only to Saudi Arabia, I guess). This, along with them having destroyed 95% of their forests[1] in the last few decades, and having an expected population growth from 180→400+ million by 2040, is absolutely unsustainable and will be depressing to watch unfold.

[0] http://atlas.media.mit.edu/en/profile/country/nga/

[1] http://www.focusonforests.org/fcontent/nigeria.htm

Late response. I have to reply.

Yes. China and India (and Pakistan and Brazil and...) are disasters in progress. Yet there are encouraging signs.

I choose to hope that Nigeria and others will improve their lots as they climb the economic ladder. Their people certainly deserve it.

I think another thing to factor in is the increase in computerized trading. With more and more volume comprising of strategies that focus on the short term, it seems like the returns may not be as shown since the data the research is based on covers traditional trading periods.

not just HFT, buy algo shops like PDT and Renaissance Technologies being more and more frequent players

A tuyere is a weird, tough, heat proof, somewhat hard to make (depending on your local technological level) compressed air nozzle. If there's a steel mill blast furnace nearby, you can make a fat stack of cash making and selling tuyeres. Its kind of a specialists metalworking job in that an idiot can make one that doesn't last and falls apart at the worst possible time causing thousands of dollars of missed production, but a specialist can make a good one. If you operate a steel mill blast furnace you won't make much money without a supplier of cheap, quickly replaced, reliable tuyeres (and you also require a zillion other things...) If there's no steel mill nearby then a tuyere maker won't be in business very long.

Anyway the point is much like some agriculture analogies you plop down a steel mill next to a metal shop specializing in tuyere construction and repair, and they both thrive. Take away either, and the other eventually disappears. Going back in history there is no spontaneous generation of either blast furnaces or tuyere metalworking shops separately. They always and only develop, grow, and survive together, simultaneously.

Now lets talk about another pair of technologies that can only thrive in the presence of each other... the capitalist stock market, and early industrialization. Unlike the tuyere example, for both of these, without the other, they can exist, barely, but only a thousandth the size in the long run if required to operate alone. You can kinda industrialize without a stock market, communist style, but it doesn't work nearly as well.

The USA is in a post industrial era. Therefore soon we'll be in a post stock market era. Stock trading will still exist, just maybe a thousandth the size, volume, and importance.

Even today the decline is evident. I'd estimate a century ago "everyone" in my family and maybe in the country worked for a stock market financed industrial company, or indirectly in that all their revenue came from stock market financed industrial companies. The value of the DJIA directly influenced if the railroad expanded and therefore promoted my g-grandfather. Today? Half the population does not work (young, old, sick, unemployable, etc), my wife works for the .gov so the market means nothing to her, my sister and I work at (different) megacorporations that are so large the market cannot handle them anymore (like our employers influence the market, not the market influences our employers LOL). Many of my friends work for non-profits or privately owned/financed firms. My dad's last day at his career position in a stock market financed company was in the early 80s, and it was mostly contracting at privately owned companies for the next 20 years. I've only worked at two companies financed by the stock market and I've never had a career and probably never will. I could do pretty well without the market.

Many information-age business models (SaaS, marketplaces, advertising & other grow-first-then-monetize consumer businesses) are even more dependent upon capital than their industrial-age predecessors, since they run at a loss until they completely saturate a market and then turn on the money spigot. They've just been going to the private markets rather than the public markets because the Information Age started with a massive speculative bubble that scared everyone off. I think that as people realize that the Internet is real and actually will change everything about society, they'll be an increasing call for the general public to get in on this wealth creation.

thats some of problem.

once they are a money spigot they don't need the capital market.

when they are money burning they are too risky for what we consider acceptable for a stock listing (though biotech seems to be an exception for some odd reason)

there are no 'safe' way to deploy capital that returns 6%

But that's always been the nature of capital - there are no returns without risk. Capital's always been a bet on the uncertain future - the whole reason we have the concept of a modern corporation is so that risk can be spread across lots of people who can afford to lose the money they put up.

The percentage of investors who lost their shirt in the Industrial Revolution dwarfs those who lost their shirt in the dot-com boom. It's just that everybody who went broke investing in a refinery or mining scheme that was just snake oil dropped out of the history books, and we only hear about the people who got fabulous wealthy.

> I'd estimate a century ago "everyone" in my family and maybe in the country worked for a stock market financed industrial company, or indirectly in that all their revenue came from stock market financed industrial companies.

That is not true at all of my family - they were all farmers until my parent's generation. To pull out some real statistics: in 1870, over 50% of Americans were farmers. A century ago it looks like about 30%.


Or, the finance market has priced itself above the average income. Without a doubt, the US economy would produce more if there were higher demand - financed by higher wages. That would increase financial market participation.

How are you efficiently saving for retirement? There are markets for real estate and gov&corp bonds too!

Cool, this is very similar to what I had people do at one point as part of the interview process.

Give them a bunch of historical data

- find me the longest period that we would be flat/negative

- find the best time to invest

- find the optimal portfolio off stocks to hold over a given period.

I think I've said this before but I see too many people who think that they need to have a huge public repository of code to show off in an interview.

If you want to be a programmer at a hedge fund just having some very shallow analysis like the above would get a programmer to the top of the resume pile.

Now if you want to be a quant the bar is obviously much higher:)

I'm interested in walking the quant path. Just how high is the bar? And would there be an equivalent project that would put a prospective quant to the top of the resume pile?

I've attached two pdf's on what you should expect.

The latter was the take home an old company used to give quants as a test. If you can answer the questions on teh practicum then you probably have enough math skills to start as a quant.


Just in case anyone else doesn't know what a "quant" is: quantitative analyst.


if you enjoyed that too bad you'll never do any of it on the job

Question 4 was fun :)

Not sure what technique they wanted you to use, but it was easy to work back from the desired outcome, keeping track of the current IOU balance and the bet that would leave you in the desired position. I assume the early questions illuminate the general way of doing this.

==== SPOILER ====

Create a 4 * 4 matrix BET_m,n and a 5 * 5 matrix IOU_m,n, where m and n are the number of wins for each team respectively, and each matrix encodes how much is bet and owed after that many wins.

Our initial constraints give IOU_0,0 = 0, as nothing is owed until a team wins a game. IOU_4,* = 100 and IOU_* ,4 = -100.

It's hard to see what BET_0,0 should be starting from the start, so instead lets work backwards.

If a team has 3 wins, and wins again, then the bet plus current IOU must equal 100 if it's my team winning, -100 if the other.

IOU_3,3 + BET_3,3 = IOU_4,3 = 100

IOU_3,3 - BET_3,3 = IOU_3,4 = -100

Thus BET_3,3 = 100 and IOU_3,3 = 0

Our matrices now look like

    BET = [. . . .
           . . . .
           . . . .
           . . . 100]
    IOU = [0   .   .   .   -100
           .   .   .   .   -100
           .   .   .   .   -100
           .   .   .   0   -100
           100 100 100 100 .   ]
IOU_3,2 + BET_3,2 = 100

IOU_3,2 - BET_3,2 = 0

So BET_3,2 = 50 and IOU_3,2 = 50

By symmetry BET_2,3 = 50 and IOU_2,3 = -50

Working along the m=3 and n=3 edges we get

    BET = [.    .  .  12.5
           .    .  .  25
           .    .  .  50
           12.5 25 50 100 ]
    IOU = [0    .   .   -87.5 -100
           .    .   .   -75   -100
           .    .   .   -50   -100
           87.5 75  50  0     -100
           100  100 100 100   .   ]
In general, our constraints are:

IOU_m,n + BET_m,n = IOU_m+1,n

IOU_m,n - BET_m,n = IOU_m,n+1

Filling in through the middle

    BET = [31.25 31.25 25   12.5
           31.25 37.5  37.5 25
           25    37.5  50   50
           12.5  25    50   100 ]
    IOU = [0     -31.25  -62.5 -87.5 -100
           31.25 0       -37.5 -75   -100
           62.5  37.5    0     -50   -100
           87.5  75      50    0     -100
           100   100     100   100   .   ]

Thanks, that is very valuable.

From "Max Dama on Automated Trading": > Work shall set you free

The English translation of Arbeit macht frei? Really?

Remind me

The NYTimes has a great visualization of S&P 500 returns for money invested any year between 1920 and 2009 and withdrawn between 1921 and 2010:


I brought up this chart as a counterpoint to the OP's graph in a separate discussion.

The difference is that this one factors in "taxes and fees".

In this chart, it says 1979-1999 is +8.2% per year.

Using the OP's data source, 1979-1999 would be 12.464% (using https://dqydj.com/sp-500-return-calculator/ which uses the same Schiller data set).

That's 4.2% per year obliterated by "taxes and fees". That seems excessive for "taxes and fees", given that it's buy-and-hold.

I don't know what to make of this discrepancy but I find it aggravating. Either Schiller's data has some strange assumptions built into it, or the OP's analysis draws improperly bullish conclusions from it, or the NYT chart is unreasonably bearish.

Either way, I still strongly believe people should not be generally expecting their investments to quadruple in 20 years. There's a whole investment industry that is based off of bad assumptions, and it's keeping people from investing in actually producing value for themselves and others.

> and it's keeping people from investing in actually producing value for themselves and others.

What do you mean?

Well, starting a business is a good example - doing the work to find an actual market opportunity and investing in it. Or, I could scrimp to put that money in the S&P, or I could invest in myself through further education.

It's unclear if the NYT accounts for dividend reinvestment.

Leaving out compounding interest could massively affect the returns shown in the chart.

Excellent visual, I would love to see this updated for 2016.

You can see that there are quite a few vertical slices where you will always result in a positive return. It's just a matter of how long you are willing to hold your money, and for what kind of return. That's why I'm attracted to index funds for investing.

How do people feel about investing in the SP500 right now? I'm not too bullish considering the current peak and the US political future.


I saw that visualization back in 2011 and I think it's actually a pretty poor one. It uses shades of red for what are objectively not bad outcomes (return greater than inflation is red, real return between 3 and 7% is pink). IMO that's misleading.

It would be helpful if it compared against the same visualization for other straightforward market investments, like bonds, or savings accounts / CDs. Those asset classes would be red (or pink for long-term bonds, perhaps) across the board. Stocks look great in comparison.

But picking a slightly more reasonable color scale would help.

> It uses shades of red for what are objectively not bad outcomes (return greater than inflation is red, real return between 3 and 7% is pink). IMO that's misleading.

Not sure if I'm reading it wrong, but the key shows <=0% as red, 0-3% as pink, 3-7% as biege.

3-7% looks more pink than beige to me, but I'm a little red-green colorblind, so what do I know.

IMO it would make more sense to show 0-3% real return — strictly above inflation — as beige and 3-7% real return as a shade of green.

I did a similar analysis in 2008 to determine whether an n% drop in the index could be help inform an investment strategy: https://saffell.wordpress.com/2008/10/26/does-timing-the-mar...

Yay, another viz perfect for the colorblind :D

That's from Ed Easterling's Crestmont Research. Easterling's updated the matrix since:


Looking at the discussions here I find it interesting that even in something as number driven as the stock market everybody argues about the meaning and the validity of the numbers. There really is no clear picture.

But somehow the regular guy is supposed to navigate his way through this jungle of conflicting, confusing or meaningless numbers. And considering the long time frames most people don't have much opportunity for leaning from mistakes and doing better. Once you realize that your strategy doesn't work you have already wasted many, many years and lots of money.

1. If you're right 51% of the time when you invest you're going to get rich. Unfortunately the odds of you being right 51% of the time are incredibly low. 2. Heed Rule #1, and put most (if not all) of your money in broad market ETFs. 3. If you DO decide to actively invest, think about your strengths both in terms of character and industry knowledge and play to those. I don't know jack about healthcare and have had my head handed to me in Biotech because I am willing to hold through a lot of pain. What I AM good at is seeing strong secular trends in tech stocks and value discrepancies there. So most of my cash is in broad market ETFs, and I do a small amount of active investing in tech and telecom. And I work in Equity Research..

It astounds me that so many people think they can be great investors. It would be like me trying to build an ark with no experience in carpentry or ship-building. Really it just strikes me as a legal and socially acceptable way to gamble. Please recognize that this is a zero-sum game, and the other person taking your trade is likely a professional who spends their entire work week doing deep analysis.

> If you're right 51% of the time when you invest you're going to get rich.

Sorry, but this is completely incorrect.

The approximate formula for success in investing or trading is "percentage right" * "average win profit" - "percentage wrong" * "average loss" = overall profit.

Many traders are correct only 20% of the time (I'm looking at you, stock options traders) but make fortunes because they understand this formula.

You can also lose a lot of money even if you are right 90% of the time because you fail to manage a big loss properly.

Can confirm. Been trading options for 3 years, lose money on 90% of my trades, but still up 30% for the year.

It's basic expected value, magnify your wins and minimize your loss to what you know you can lose.

So your counterparty makes money on 90% of his trades and is down 30% for the year? He must feel like a schmuck.

That is not at all how the market works.

Why would there only be one, perfectly symmetrical, counterparty?

And take in mind, most returns are relative to your initial investment, if you reinvest your profits so you have a static % 'at risk' the bad results will hit you harder than the good ones. Lose 50% and you need to make 100% RoI to get back to the start. Win 100% and you only need to lose 50% to get knocked back.

And there are also catastrophic losses, if you play it too risky, where you won't have enough to continue investing after a particular hard hit.

A simplification to explain the main point, which is that it is very hard to beat the market. Even for professionals.

Yes, and you can also be be right most of the time, and have below market returns because of transaction costs or management fees.

This is the big point.

Investment houses don't start from the same place as an index fund. Their returns are penalized from the start due to the fee structure. So not only do they have to beat the market, they have to do it by a sizeable margin before the end user sees a profit advantage.

For a retail investor it is hard to justify not choosing an index fund.

+1 for using industry knowledge. Most of my money is invested in ETFs, however, I've been able to beat the market by significant amounts when I invested in tech stocks because I understood what drives the price. There are plenty of small cap tech stocks that have doubled / tripled the last few years.

Understand the market, understand the product and technology well, and you can do significantly better than any wall street analyst. Don't rush into it, but do your research. Look at the numbers and the growth potential.

> Understand the market, understand the product and technology well, and you can do significantly better than any wall street analyst. Don't rush into it, but do your research. Look at the numbers and the growth potential.

This is very dangerous advice because it just ain't true. Under efficient markets, you can do equally well as "any wall street analyst"

1. Markets aren't efficient. They're full of emotion and greed. They can be irrational. Just look at Brexit - even stocks that had zero exposure to the UK (directly or indirectly) sold off significantly.

2. Why would you assume you can't do better than an analyst, in your area of expertise? Someone who understands tech well will be able to make better tech investments, than, let's say, investments in mining. That seems pretty obvious to me. That doesn't means you'll always get it right, but it does increase chances significantly.

1. Did they actually have zero exposure to the UK? There are knock-on effects. If many airlines had exposure to the UK and this oil company only sold to american airline companies, they still effectively have exposure to the UK. If this spring manufacturer had the majority of their deals with that oil company, then they're going to suffer with them. I'm not saying you're wrong. I used the same hypothesis to buy on Brexit panic, but I don't know if it was correct.

2. Honestly, the analysts got it wrong in their own area of expertise often enough that I don't think it makes sense to pretend they're oracles in comparison to anyone else.

You don't just have to beat the analysts. You have to beat all the other people in tech who think they can beat the analysts.

Markets are efficient because you cannot effectively predict how long emotion, greed, and irrationality will drive the bus before rationality once again prevails.

That makes no sense at all to me. How does that make them efficient?

Efficient essentially means random.

The efficient markets hypothesis alone isn't falsifiable. You need a model of information and how it interacts with prices: this gives you a so-called joint hypothesis. The joint hypothesis is merely extremely difficult to falsify.

I'm not saying it's not very hard to make money by trading. But be aware that you can't just assume efficient markets and have done with it.

But how do you understand the market?

Is looking at the company numbers enough, or do you have to look at companies in detail?

Is domain knowledge enough, or do you need to look at corporate culture, people and business plans?

How fast do you need to be, in term of reacting to events?

To answer your first question: be a part of it, either as a consumer or a producer.

My professional life now revolves around mining. I'm confident to know who to invest in and who not to.

My personal life centers around various hobbies - I'm confident I could choose a few companies in those spaces to invest in. Unfortunately, most of those are well performing private companies.

Just curious - are you worried about getting too coupled to the industry you work in? I try not to invest in tech since I work in tech. So a big downturn in tech would not only be bad for my job but also investments. Or would you possibly bet against mining?

I wouldn't put my whole portfolio in it but I'm not really worried about being too tied to the industry I work in. That might also be because it's mining. Mining is a large portion of the world's GDP (estimated to be in the range of 45-60% either directly or as a supplier to others). If mining as an industry suffers, we're all probably in a bit of trouble.

Tech is disrupting many existing industries, meaning ultimately more money will flow from those sectors into tech. I'm extremely bullish on tech for the next 5-10 years.

Supply and demand... if you know a stock is heavily shorted and Martin Shkreli owns 70% of all the stock and tells his broker to turn off the borrowability, then there's A) a lot less stock to short B) current shorts have to deliver C) less shares to buy when they cover

what happen?

"If you're right 51% of the time when you invest you're going to get rich."

I completely disagree with this statement and I've seen it (or a variation of it) many times over the years. You can have a win rate of 99%, but still lose money if the 1% losing trades/investments wipe out all your wins. The win rate isn't what people should be looking, but the profit expectancy. Other than that, I agree with what your are saying, particularly #2. I don't think the majority of people should be actively trading/investing, they don't have the time to build the expertise and will probably not invest the time in learning about their bias.

Yes it was a simplification to explain the main point, which is that it's very very hard to beat the market.

Ignorant question: Why ETFs rather than the equivalent mutual funds (which I think are usually available)? I have some sense of the differences, but I've never taken the time to figure out the pros and cons. (I've been investing mostly in index funds for the past 15 years or so; ETFs weren't really on my radar when I started.)

ETFs are generally cheaper to own in terms of their expense ratios, something like <=1% vs 1% – 3% for mutual finds. It‘s easier to get into and out of ETFs; They trade all day just as a stock does. Mutual funds you enter into at the market‘s close at a price set at that time. Also, ETFs are bit more transparent as to their capital gains taxes costs. Less surprises when you liquidate.

Yikes! Are there really index funds with 1% expense ratios? I think my Vanguard funds are all under 0.2%, some way under. (I've got the impression that Vanguard's ETF expense ratios are similar or identical to the corresponding fund expense ratios.)

Thanks for the overview. I'd heard about the constant trading difference before (I've tended not to think about it, since I'm a buy-and-hold-for-years type). Do you have a sense of how much cost winds up being associated with broker commissions and/or the buy-ask spread? I hadn't been aware of the simpler capital gains situation (thus far, I've just blindly copied down whatever's summarized on the year-end tax statement they provide onto my IRS forms).

Vanguards ETF expense ratios seem to be slightly lower than/on par with the admiral type funds from my experience.

Vanguard ETF expense ratios are almost always^* exactly equal to the corresponding admiral type funds. The reason is because Vanguard holds a patent on the dual-share class[1] idea, where the ETF and admiral share mutual fund are precisely the same fund.

*: The exceptions tend to be new-ish funds which are less liquid and have purchase costs on the mutual fund side. I haven't seen more than one or two of these recently.

[1]: http://www.ft.com/cms/s/0/3e0cc962-ec0c-11e4-b428-00144feab7...

Even 1% is a very high total expense ratio.

For most practical purposes, there's not much difference between ETFs and Mutual Funds.

Some point out some greater risk in ETFs because they can lend their securities : http://www.etf.com/etf-education-center/21031-understanding-...

As other person pointed out, mutual funds are much more expensive.

It depends on the mutual fund and the ETF. For Vanguard 500, the VOO ETF and the VFIAX index fund are effectively the same, they even have the same expense ratio. The latter you need $10k upfront though. (VFINX, if you only have between $3k and $10k, has a higher expense ratio.)

VFIAX is not a mutual fund, it's just the "Admiral" class of the VOO ETF. Admirals shares offer lower expense ratios in many instances, but I guess the VOO expense was already at its floor.

VFIAX is a mutual fund. Vanguard holds a patent which allows a dual share class, where the ETFs and mutual funds share the same pool of securities. This allows share conversions from mutual fund->ETF, interestingly.


Not exactly a good argument for mutual funds being the same cost as ETFs though. It's only technically a mutual fund from what I can tell. I see your point though.

Vanguard's Admiral share mutual funds almost never differ in cost from their ETFs.

As for it only "technically" being a mutual fund: consider that VFIAX predates VOO by 10 years. How does that square with VFIAX only "technically" being a mutual fund, when presumably it was a real, honest mutual fund in the years 2000-2010 (before VOO was introduced)

"this is a zero-sum game"

Minor quibble. That is not accurate.

You see teenagers buying hotdogs with a credit card. Visa and Mastercard go public, you buy shares, it goes up, up, up.

There's no zero sum here and people can spot special values without deep analysis.

If you bought the shares after IPO, then someone sold them to you, and they missed out on all of those gains. Even in the IPO, it's very common that you're buying shares from insiders and private investors rather than from the company itself.

(But even if trading is zero-sum in a strict monetary sense, different risk preferences, desire to liquidate positions in order to buy something you value more, etc. mean that it's not zero-sum in a utility sense.)

You had to buy from someone. Even Visa loses out when going public because they could have offered at a higher price. So in that sense it is a zero sum game. You profit instead of Visa (or whomever the seller is).

There are, however, externalities that make it less costly for Visa to sell lower than the theoretical max. Happy investors are easier to manage, market momentum is strong, etc.

But those things will almost certainly be priced into the market already, so you will capture that value with purchases of VOO.

(Brexit nonwithstanding)

...actually that is a hole in my argument.

Okay: if you can find a cultural trend that you earnestly believe thatFinance people are out of touch with, then it makes sense to do arbitrage there.

+1 on ETFs. And I guess, think about at what point in the cycle you want to invest in equities. They are quite highly priced at the moment.

Market timing is a fool's game. Find out the right asset allocation for your personal risk tolerance and retirement goals, stick your money in, rebalance annually (contribute monthly if fees are low), and then sit and wait.

As a general rule yes. But if you are thinking of sticking a large amount of cash in the stock market right now and plan to stay in long term, I would at least give it a shot in terms of trying to time it right in the cycle.

Right now we are quite high in the cycle - 9 years from the last crisis, end of QE, slowing growth - so at least in my view, it's an asymmetric bet to wait and see. But what do I know.

> As a general rule yes. But if you are thinking of sticking a large amount of cash in the stock market right now and plan to stay in long term, I would at least give it a shot in terms of trying to time it right in the cycle.

Vanguard says you're wrong.



Most people have a financial adviser, but for the industrious I recommend: http://jlcollinsnh.com/stock-series/

And for the lazy, this is a good resource: https://www.bogleheads.org/wiki/Lazy_portfolios

What? Most* people (at least in the USA) don't even own a single stock, let alone have a financial adviser.

*(Actually, looks like it depends on your source: 48%[1] or 52%[2], so let's say "about half of people")

1: http://money.cnn.com/2015/04/10/investing/investing-52-perce...

2: http://www.gallup.com/poll/190883/half-americans-own-stocks-...

I doubt most people have a financial adviser. What percent of people even have savings more than a single paycheck?

If you're just as likely to be right as wrong, then is the only risk that your money is taken by fees (if you don't count your time as wasted)?

Nope. Because if you are wrong once and lose 50% you need to be right by 100% to get back to even. So after a number of losses you will run out of money ever if your subsequent picks will be right. Unless you have a rich dad who gives you another chance...

That is basically the argument for managed pensions, fiduciary managed retirement accounts, etc

Does the stock data used suffer from Surviver bias?

That is a free download of historical data that lacks failing, delisted companies of the past.

This is a good question. The S&P 500 kind of dodges this question by its very nature.

The S&P 500 constantly changes the stocks it holds. So if a stock falls out of favor, say Sears, its dropped from the index.

It's also important to Note that comparing the S&P 500 from 100 years ago to today is very naive as the methodology for what it holds has changed significantly over the years.

Comparing the returns of the S&P 500 from 1900 to now is kind of like comparing the statistics of a baseball player from 1900 and now in isolation and using that as the sole determination of which player was better.


It's also only one statistic (as used). You would find differences for what years were loses 10/15/25 years down in different markets. For instance, Japan.

Yes sometimes companies are delisted from the S&P500, but if you invest in an S&P500 index fund, then your investment is also automatically adjusted to remove that company.

Which makes me think: with index funds becoming more and more popular, should we see bigger and bigger crashes of stocks when they are removed from an index?

Potentially yes, but some hedge funds play the index dynamic by trading into or out of stocks before/after they are added.

And then that index fund has to take the loss that comes with having bought a stock that failed and sell at a loss, while not capturing the gains of stocks that got big enough to make it on the index.

Shouldn't the index "return" thus diverge from the return of an actual index fund in practice given enough years?

That's not how it works, the return of the index is the actual return of the included stocks while they are in the index. There will be winners and losers, but the return is actually what you get.

I'd appreciate if you could expand on that, because I share the same concern as GP. It seems to me that the index fund has to sell a lousy company at a low price (since it's being delisted) and buy a strong one that's being included in the index. Whereas the index, being just a number, can magically perform the swap without taking a hit.

Sometimes it's OK to sell a lousy company at a low price--it's a lousy company! The beauty of a wide index fund like the S&P is that any one company diving won't hurt you. Companies typically last in the S&P for a long time, so the potential gains of adding companies earlier or selling them at a different time average out. Even Facebook has more than doubled since being added to the S&P and that's the definition of a strong growing company. If you really like a company that is being booted from the S&P you can just go buy it directly after the announcement and you'll most likely be able to pick it up on quite a discount.

You can use different indexes if you want to have smaller companies (the Russell 2000 is a good choice), but the whole point of the S&P is that it is made up of established firms. Perhaps lower reward, but also lower risk.

Is there any data out there on variations? It would be interesting to see how other strategies compare such as buying the next 100 largest market caps outside the S&P 500, also not selling stocks that are removed.

The index "magically performs the swap" doing a calculation equivalent to selling at the rebalancing date the stocks that get out of the index (and buying those that replace them) at their price that day.

Oh, that makes sense. Do you have any reference for it? I found this explanation http://www.investopedia.com/ask/answers/040215/what-does-sp-..., but it doesn't seem all that trustworthy, and it seems to suggest that S&P doesn't account for survivor bias.

The return of the ETF is only as good as what the portfolio managers can accomplish in reality. They are very good at this, but the cost is there. It's referred to as tracking error.

The tracking error is not really "a cost", it's about variability. An ETF can have low tracking error but high cost if the returns are highly correlated with the index but systematically lower. http://www.etf.com/etf-education-center/21030-understanding-...

I don't see how that addresses the dynamic I described, beyond re-asserting the conclusion I was questioning.

> Shouldn't the index "return" thus diverge from the return of an actual index fund in practice given enough years?

That's what I am taking issue with.

That much I get. I was asking for your basis for disputing the logic; all I see so far is a reassertion of the original conclusion. If you don't understand what dynamic I was describing, please say so. As it stands, I don't see what the reason is that you're trying to give for why it is wrong.

Likewise, are there bigger potential crashes for Mainstreet Investors in store as more and more of the populace shift to index funds? If there's a critical mass of people on index funds, what sorts of changes are there to overall risk and liquidity in the market?

The survivorship bias applies to the U.S. data as a whole. I've found this paper [1] that adds data from 15 more countries, some of which experienced interruptions (germany) but still excludes those that suffered "permanent interruptions" (eastern bloc) to the sample. Needless to say, the results are not as exciting.

Memorable quote form the paper (quoted in turn from Samuelson):

> We only have one history of capitalism. Inferences based on a sample of one must never be accorded sure-thing interpretations. When a thirty-five-year-old lost 82% of his pension portfolio between 1929 and 1932, do you think it was fore-ordained in heaven that later it would come back and fructify to +400% by his retirement at sixty-five? How did 1932 Tsarist executives fare in their retirement years on the Left Bank of Paris?

[1]: http://merage.uci.edu/~jorion/papers/risk.pdf

Warren Buffett bet $1mm that S&P500 will outperform a hedge fund over a 10 year period.[1]

That's good enough for me, I'll follow the oracle.

[1] http://longbets.org/362/

This bet ends in less than 2 years. Does anybody know what would be the result if it would end now?

edit: I guess it depends on the detailed terms that they haven't disclosed, but educated guesses are fine

65.67% vs. 21.87% returns so far (net after fees)


Almost impossible for Warren to lose at this point.

Hedge funds are downstream of money with performance-driven interest.

Markets (the major players that skew the averages) are downstream of banks who use money the Fed's pay the interest on.

His gamble is that the elite would rather give the market unlimited welfare a la Japan to keep the signals of valid global demand alive.

The Fed will not allow a correction... mostly for political reasons.

I don't think that was the gamble when Buffet made the bet. The gamble then was, in ten years of completely unknown (at the time) market conditions, the S&P 500 will outperform hedge funds.

If you picked a different 10 years (say, 199X-200X or 200X-201X), I suspect that Buffet wins for most values of X, but I have not checked.

Whoa, thanks. I would have definitely taken the losing side of the bet.

His bet is on returns net of fees. There are some theories that any alpha generated by a fund is eventually just captured in the fee structure.

I seem to remember that as of Berkshire Hathaway's annual shareholder meeting in April, Buffett was winning by a long way.

> bet $1mm that S&P500 will outperform a hedge fund over a 10 year period.

He's actually betting against a portfolio of hedge funds going as far as to include fund of funds (typically a terrible bet). I assume he is only using US Equity hedge funds too?

I'd be interested to see the distribution of individual funds in his portfolio that beat the S&P vs those that didn't. In general though these numbers have huge survivor bias since numbers are self reporting and ignore dead funds.

It is also worth noting that the index went from 1400 to 900 over the course of 2008. So when he started the bet is clearly going to be relevant. You could of course repeat the experiment using every possible start date from, say, 2000 - though your portfolio of hedge funds will be hard to construct.

He is clearly winning, but it seems like an odd question to ask.

Planet Money did a recent episode on this with the guy who bet against Buffet. Really interesting.


The delicious irony of a person that has outperformed the market betting that it's impossible to outperform the market... and (probably) winning. :-)

Lots of talk in here concerning superpowers falling, and not much about how it's way more favorable for companies to use debt financing in a low interest rate environment... Lol developers should stick to developing

Who do you think created HFT (High Frequency Trading) systems? Surely bankers stick to banking but hire developers to give them that competitive edge. The odds are that software engineers would have come up with the idea of and implementation of, a working high frequency trading system before bankers would have despite banking probably not being a software engineer's specialty.

I think market makers used technology to do exactly what they've always done at increasingly higher speeds. History also shows you're wrong, for hundreds of years market makers have sought an edge by trying to get information faster and faster. Hell, it used to be carrier pigeons that gave them the edge. There's really not much new here...

> for hundreds of years market makers have sought an edge by trying to get information faster and faster.

And then it spurred the telegraph, and then the telephone, and then... :)

I'd be curious to see results for other countries, the graph of "Chance of Losing in the Stock Market" in particular. In the case of Japan, it appears that if you'd still have significant losses if you invested 25-30 years ago: https://finance.yahoo.com/echarts?s=%5En225+Interactive#{"ra...

this is the biggest fallacy in the investing thesis. buy and hold didn't work in russia, or argetina, or many others. because we live in a country that's prospered (for a plethora of reasons), we assume the prosperity must continue unabated forever.

Would investment in a Russian market managed fund panned out any better?

I think they're referring to the fact that the Russian stock market collapsed completely (i.e. total loss) in the beginning of the 20th century.

Clearly. But they were considering investment strategies (buy&hold vs alternatives). My point was that few if any investment policies would hedge you against market extinction events.

I would probably have preferred to own real-estate in Russia... Although If we look back to before ww2, probably not...

If a fund manager is relatively close to Kremlin, than yes.

If you're in Japan you can still invest in the US stock market... and should to diversify. I invest in international markets even though I am in the U.S.

Just to point out to people who may not be aware, this leaves you open to currency risk. The FTSE 100 dropped 3% in GBP terms last week, but maybe 10% in USD terms.

This cuts both ways of course: you can make money on favourable currency moves. But it's an important risk to be aware of before buying assets in a foreign currency.

You want this though. For example if you was English and invested all in uk investments you would have dropped massively USD terms.

If you spread your investments in many different currencies in pound terms you would have actually gained from this crisis.

Diversification also applies to currency

Eventually most people have to redeem their investments, though, and they're going to do so in their home currency.

A US investor who bought foreign assets in 2014 and sold them today would have been hurt by the 25% USD/EUR rally in 2014-2015. The dollar rose against almost all other currencies too (besides, for example the yen).

I moved my all of my UK pension to non-UK bonds (mainly US) one day before the referendum result. It was a mission and a half using my pension fund's online system, truly awful UX, I almost gave up.

I don't normally change the holdings or attempt active investing, but it just seemed like a very asymmetric bet - nearly 50/50 polls, very large potential downside if not diversifying outside the UK/sterling.

On the other hand you often already have exposure to the fate of your own country, because you live and work there.

"Buy and Hold" applied to the S&P500 actually means "sell when an individual company loses enough market cap, buy one that has gained enough".

The other reason none of these returns are realistic for an average person:

1) People don't get a lump sum at the beginning of their investment history

2) Ah, but you say, dollar-cost-averaging. The problem there is that people get more money to invest when times are good, and less when times are bad.

3) As a result, even when buying in responsibly, people are buying more when the market is high, and less when the market is low.

It distorts performance. Plus, betting on having average-or-better performance is a risky bet. Don't count on more than 2.5% / year lifetime.

It gets worse: people buy when times are good and sell when times are bad, if they've not held enough liquidity. This tends to amplify buying prices and depress selling prices.

It's also a strong reason to maintain liquidity and keep your nut (recurring expenses) low.

The dynamic turns up in a number of other areas. Natural resources markets: extractors are often pressed to provide more supply when prices fall because they've a large fixed cost structure, often loans, and the resource is all they have to sell. Monopolisation or cartelisation, or government-regulated stockpiles, are a common response (Standard Oil, the As-Is agreement, the TRO/Dept. of Interior quota program, Naval / Strategic oil reserve -- see also Tea Pot Dome scandal, Harbord list, OPEC). Farmers will double down in drought, see Ken Burns and Tim Egan on the Dust Bowl. Migrant or refugee labour (Dust Bowl and others).

Economic agents -- persons, groups, firms -- face both short-term essential and long-term cost structures. They can operate, for a time, below long-term costs, but only at the cost of degrading their long-term capabilities. They'll do so because if they fail to meet short-term needs, they die.

It's a tremendous market distortion. Perhaps one that drives the entire modern economy.

A lot of companies offer retirement plans where a % of income is invested monthly, a la dollar cost averaging (e.g. 401k plans). It's really easy to set up and let ride on autopilot.

But that still skews the dollars invested. At age 22 (college grad) you might make $50K. You put in 6% and get a 3% match, for $4500 into your retirement fund.

Fast forward to age 50, when you're making 200K (this is 28 years from now, so inflation and pay raises bump you very high), you're putting in the same %, but that equates to $18K/year.

So even with dollar cost averaging, the majority of your money is invested in a small, 10 year time window when you are at peak earnings.

Ahem, good comment, but:

At age 22, you make 25k and get 0% matching, and can afford 0 to put into your still-hypothetical retirement fund.

At age 50, if you're the median person, you make 50k. And you need that money to put your kid through college.

is your planet accepting applications?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact