
Nate Silver confuses cause and effect, ends up defending corruption - anon1385
http://mathbabe.org/2012/12/20/nate-silver-confuses-cause-and-effect-ends-up-defending-corruption/
======
DigitalJack
I'm not discounting her complaints, but the following is not a confusion of
cause and effect:

"Silver confuses cause and effect. We didn’t have a financial crisis because
of a bad model or a few bad models. We had bad models because of a corrupt and
criminally fraudulent financial system."

A= Financial Crisis B= Bad Models C= Fraudulent System.

Nate said "A<-B" Author says "A<-B<-C"

That is not a mix up of cause and effect.

Author's main complaint seems to be that Nate assumes bad models are an
accident, and Author claims they were intentional.

Again, not a mixup of cause and effect. At worst it's a naive interpretation
of the correct cause.

~~~
sillysaurus
The bad models were a consequence of a fraudulent system. So by definition,
the models weren't the cause of the financial crisis. Therefore the bad models
were an effect, in the same way the financial crisis was an effect.

If this is true, then the author's claim (mix-up of cause and effect) must be
correct.

~~~
cube13
But it's not.

She states:

>Rather, the entire industry crucially depended on the false models. Indeed
they changed the data to conform with the models, which is to say it was an
intentional combination of using flawed models and using irrelevant historical
data (see points 64-69 here for more).

So C(corruption) directly leads to B(bad model). She does not argue that
A(everything falling apart)<-C anywhere in the piece. Silver is claiming that
A<-B. He is not commenting about C. So while Silver's claim may not identify
the absolute root cause, the author does not actually prove that his cause and
effect analysis is flawed. If anything, she has shown that it may be
incomplete.

~~~
sillysaurus
_But it's not [true]._

What, specifically, isn't true?

 _So C(corruption) directly leads to B(bad model). She does not argue that
A(everything falling apart) <-C anywhere in the piece._

The point of the piece is that the financial meltdown stemmed from corruption,
not from models. If this wasn't her point, then why else would she have
written this piece?

~~~
cube13
She does not prove that claim, though. She argues that corruption causes bad
models. For Silver's cause and effect analysis to be incorrect, she needed to
prove that the inaccurate models had absolutely nothing to do with the
financial meltdown.

~~~
btilly
The models can be part of the meltdown without really being a cause.

If I choose to shoot you with a gun, and have a variety of appropriate guns,
it is not the fault of the specific gun chosen that it was used to kill you. I
had motive, opportunity, and alternate means available.

------
flatline
Having just read this book, I believe that this commentary is wrong on nearly
all points. Specifically:

\- The ratings agencies...did not accidentally have bad underlying models.

He first talks about how the models were defective, and then goes on at length
to describe the perverse incentives for developing and keeping these models,
and holds the responsible parties to the fire for willful ignorance and
unabated greed. He could have made some of these points more strongly, but I
don't think that he skirted over the real issues.

\- the only goal of a modeler is to produce an accurate model.

Actually he speaks quite a bit about the reasons that various forecasters
generate models in the first place, and their motivations and
responsibilities. For example, the Weather Channel's skewed "wet" forecasts.
They will, for example, predict a 20% chance of rain if their models show only
a 5% chance. On the one hand it is a bit dishonest, but on the other hand
there is a net utility in doing so. Part of the problem is that the general
public doesn't understand what the percentages really signify, so it may be of
some benefit to emphasize that it really could rain, it's just not very
likely, so it behooves people to at least have some backup plan for rain. The
other side of this is that people will not remember the good forecasts as much
as the bad, and the Weather Channel has business considerations in mind.

\- He spends very little time on the question of how people act inside larger
systems, where a given modeler might be more interested in keeping their job
or getting a big bonus than in making their model as accurate as possible.

How much of the book do you suggest he devote to this? He addressed the issue
directly, as the next couple paragraphs state. He also talks to quite a few
institutional players at large organizations, both public and private.

\- Having said all that, I have major problems with this book and what it
claims to explain. In fact, I’m angry.

Really, stop with the faux outrage, because you don't seem all that angry
throughout the post, you more seem to be disappointed he did not devote the
book to your own pet topics. I find this kind of criticism to be mostly
without merit. For me, there were three overriding themes in "The Signal and
the Noise": Bayesian thinking (this being the primary one), the goals and
motivations of forecasters, and examples of what forecasters have done right
and what they have done wrong. I think it covered the bases pretty well, and
it was an entertaining and lively read on the whole.

~~~
cube13
The author is a woman, just FYI.

~~~
flatline
The author of the book is a man; the author of the commentary is a woman.
Hmm...did I mix up a pronoun somewhere?

~~~
cube13
Whoops, I thought you did early on. Sorry about that!

~~~
proksoup
I also read "this" as "his" in that first sentence, and had to do a double
take.

------
cbs
This is a really long and angry way of complaining that Silver assumes good
faith on the part of those tasked with creating models and policy.

The underlying issue is my biggest peeve with both the buisness and political
world. There is a popular viewpoint spread by many defacto authority figures
that one should presume good faith from all groups involved, even though
everything I can see tells me the opposite.

Did Silver decide to perpetuate that by being afraid to address the topic of
malice in his book or did he fall victim to cultural attitudes himself? Either
way this rant is all over the place, it complains about Silver mixing cause
and effect while itself attacking a symptom, not the problem.

Edit: This post is written temporarily presupposing that the author is correct
in her take on Silver's book, the comment by flatline in this thread hits some
good points on why she may be a bit off. It was so long ago that I read the
book I don't particularly remember how many inches Silver dedicated to
incentives let alone care to debate if that was enough given the goals of his
book.

~~~
pnathan
> The underlying issue is my biggest peeve with both the buisness and
> political world. There is a popular viewpoint spread by many defacto
> authority figures that one should presume good faith from all groups
> involved, even though everything I can see tells me the opposite.

This is a pretty big deal.

Assuming good faith by authority figures is a popular and common fallacy from
people who sit in the "Lawful" quadrant. N.b., I see this a lot in academic
circles. Arguing from authority is the M.O. there, and assuming from authority
is the consequence.

There's also assuming bad faith, a equally fallacious and (at least equally)
popular idea from people who sit in the "Chaotic" quadrant.

We have to remember to have nuance in our opinions and dealings with others.

------
unreal37
About half-way through the rant she says:

[[Call me “asinine,” but I have less faith in the experts than Nate Silver: I
don’t want to trust the very people who got us into this mess, while
benefitting from it, to also be in charge of cleaning it up. And, being part
of the Occupy movement, I obviously think that this is the time for mass
movements.]]

Ahhh, so she was part of the Occupy movement and comes from the world-view
that the financial system and government is corrupt. She should have said that
up front. Makes what she is saying make more sense.

Nate Silver doesn't believe those things, and so that largely explains why
they come to different conclusions.

~~~
fatbird
She has a more reasonable, and more specific, point than that.

The naivete of which she accuses Silver is that Silver assumes that the
modelers are striving for accuracy in their models (and that they failed to
achieve accuracy). With some linked evidence, she asserts that the modelers in
the financial industry knew that their models were inaccurate, but that those
models supported the corrupt narrative that enriched them, and so perpetuated
them. In other words, in the finance world (she says, with some apparent
understanding and evidence) the wilfully inaccurate models were a means to a
corrupt end.

~~~
jacoblyles
I disagree that she had reasonable points. I don't see how malice played a
significant role at all in the meltdown. Carelessness, bad incentives, and
etc. - sure. Maybe a bit of malice. But most of the misallocation of capital
was done by firms doing things like trusting AAA rated securities when
everybody else was doing the same and it was in their financial interest to do
so. Actions like this were not _wise_ , but it's hard to see malice if you're
not a card-carrying Marxist.

~~~
fatbird
She doesn't attribute malice to anyone. She describes the belief that the
issue with models in the meltdown was inaccuracy as "maliciously" wrong, which
I take to be hyperbole on her part to emphasize the category flaw she sees in
this (namely, that if it's just inaccurate models, then better mathematicians
are what's needed; in her view, the financial system at the heart of the
meltdown was "corrupt and criminally fraudulent", so it's not mathematicians
that are needed, and concentrating on them allows criminals to go free).

The fundamental problem she sees is that incentives in the financial industry
are not aligned with accurate models, that inaccurate models were deliberately
used to further short-term performance at the expense of further inflating the
bubble.

"most of the misallocation of capital was done by firms doing things like
trusting AAA rated securities"

On this, she links to specific evidence that the "trust" they were exercising
was knowingly misplaced because the incentive was always to get the next big
bonus.

------
tomkarlo
Having spent part of my career in finance, the part she saying about modeling
in that industry is true for some situations but not others. (Unsurprisingly.)
An insurance company is generally going to try to get its own premium pricing
models right, for example. But if the point of the model is to _sell_ someone
on something, than look out. I remember a senior banker saying to me "model
this merger, and make sure it comes out to be accretive by X cents per share."
It was irrelevant to him if the model was accurate - he just wanted ammunition
to convince his client to do the acquisition.

------
tlb
The villains in this story (ratings agencies) had lots of models, bad and
good. They chose to publicly report the ones that suited their interest,
rather than the most accurate ones.

The way to improve the situation is to educate the consumers of those models
(securities purchasers) about what models are best, and which are misleading.

It's more productive to say, "Buyer beware, sellers are misleading you in the
following ways..." than, "Shame on corrupt sellers!". Silver's book is doing
the first, which does not constitute defense of corruption.

~~~
tomrod
I'm not sure I follow. Ex ante the models were unknown as to their
performance?

------
daguar
The author is being polemical to get readership.

Her criticism boils down to, "but there are agent-principal problems!"

But I think that's a bit of a sideswipe at Nate, who is dealing with a
different domain of problem: modeling inaccuracy when the incentives ARE
aligned in favor of optimizing predictive accuracy.

~~~
pdonis
But then the financial crisis is not within the domain of problem Nate is
dealing with, because, as mathbabe says, the incentives were _not_ aligned in
favor of optimizing predictive accuracy. So Nate should not have talked about
the financial crisis at all, yet he still did.

------
PaulHoule
Bayesian methods are the best for people who want to turn de-biasing into re-
biasing, particularly when you're dealing with lots of output variables.
(Generally when the variables are few, as they are in the things this document
talks about, a screwy prior sticks out like a sore thumb.)

Sometimes the distribution that you ~can~ sample isn't really the distribution
that you wish you could sample, and sometimes changing the prior in such a
model is a way to make it behave as if it was sampled correctly to begin with.

------
quizotic
Silver attempts a dispassionate analysis. Her post simply asserts what she
believes to be the problem with the housing bubble and ensuing meltdown,
without much evidence or analysis.

She may be right. But Silver's larger point is evidence-based analysis.
Where's the evidence to support her position? Why is her assertion any
different than the political pundits' assertions? Is an email exchange between
a couple of traders enough to prove global complicit awareness? There may well
HAVE been global complicit awareness. But without enough data to statistically
support her position, this response seems to prove Silver's point that we're
better off looking at the data than we are working off what we 'know' is
right.

~~~
pixl97
This is anecdotal, but I'll posit it anyway. I did network administration for
a number of title companies, mortgage lenders, real-estate agents, and other
assorted firms involved in the selling and buying of houses between 2004 and
2011. Most of these people were what you would consider hard working honest
employees, but due to perverse incentives helped in their part of making all
of this worse. I saw plenty of lenders have the client flat out lie about
their income. It didn't seem to matter, the banks rarely rejected the loans if
the paper looked good. The better you lied, the more sales you got. If you
were totally honest, people heard that it was hard to get a loan or closure
from your firm and would head to others. At the time there was seemly no risk,
it wasn't till after the financial crash that I heard of any arrests over
paper manipulation. The problem is the people that have the data now are going
to be very shy about releasing it. It will show fraud by the buyers, possible
fraud by the lenders, poor research by the big banks. By the time the crash
came it was a game. Interest only loans? You've got to be kidding me.

At a murder scene, evidence is how one determines the cause and the killer,
but if your killer has means he can manipulate that evidence. The issue we
have now is that the murder (banks) are the group holding all the evidence.
They don't want it looked in to, it would show they were an accomplice.

------
jasonwatkinspdx
Amazing how willing the poster is to criticize Nate Silvers based on assumed
conflicts of interest and naivety about his own incentives mislead him, while
remaining silent on her own.

> "He gets well-paid for his political consulting work and speaker appearances
> at hedge funds like D.E. Shaw and Jane Street, and, in order to maintain
> this income, it’s critical that he perfects a patina of modeling genius
> combined with an easily digested message for his financial and political
> clients."

> "Silver is selling a story we all want to hear, and a story we all want to
> be true. Unfortunately for us and for the world, it’s not."

And of course, she derives also derives her income from speaking, consulting,
and is writing her own book. She certainly benefits from positioning herself
as a more expert Nate Silvers. "This best seller is wrong, buy my book to find
out the details why" is pretty effective marketing.

By her own logic we should criticize her just as strongly.

------
fretless
Hmm, i should read this book, I probably have a different perspective. I
worked in the banking group at one of the 2 major rating agencies, then
structured some large ABS transactions at Merrill Lynch and at one of the
largest issuers, but haven't worked in finance for a while

------
malachismith
Analysis and modeling should always be free of dogma. Otherwise you will (as
the other of this post demonstrates) start using your analysis and modeling to
justify and prove your dogma (rather than to understand). The fact that
Silver's analysis and modeling did not prove her point does not invalidate it.
In fact, you could argue that the root of her "anger" is that his book proves
her _beliefs_ to be at least unproven if not entirely false.

------
chris123
From someone involved in real estate, real estate finance, stocks, venture
capital, financial modeling, and behavioral finance (since before that phrase
was even coined), who sold his home and three other properties from 2004 to
summer 2008 and rented so as to exit before the crash, all these bubbles and
crashes are, IHMO, not about modeling, they are about greed, fear, conflict
between self interest on collective interest, and perverse incentive and
compensation systems (in many areas, from politics to lobbying to regulatory
to finance to sales to brokerage to legal to money management to many (all?)
of the other important things for finance-related models). None of those
things have changed materially. Nate can build the fanciest financial
asset/credit/price model from his wettest dream and it will fail, probably at
the worst possible time, such as once you go all in on it. Many reasons for
that. But I bet he could get a job on Wall Street pumping out models that back
up what will make the firm the most amount of money in the short term, damed
the best interests of pretty much anyone, the firm included, in the long term.
See Goldman Sachs :)

------
woodchuck64
> We didn’t have a financial crisis because of a bad model or a few bad
> models. We had bad models because of a corrupt and criminally fraudulent
> financial system.

Those who are attracted to financial careers often have a deep and abiding
love for money, much like wolves have a deep and abiding love for sheep. Sheep
farmers are smart enough to not to hire wolves as shepherds; but in the
financial industry, the wolves are already in charge.

------
ryguytilidie
The whole "Nate Silver makes pundits look bad so pundits will all grasp at
straws and twist his words to try to make him look wrong so they can feel
relevant again" act is getting a tad old.

------
drcode
I agree that Nate Silver's book is short on many subjects (such as politics)
but the goal of his book is not to explain "Why did X happen" but simply to
discuss the challenges of prediction within different domains. It's a book
that helps people learn the art/science/challenges of prediction.

Saying that his book lacks detailed discussions of incentives (while true)
misses the point of his book.

------
Symmetry
It's not surprising that the banks that "won" the financial crisis should have
had an incentive to use bad models. But there were more losers than winners,
and for the losers this was a straightforward matter of their models being
wrong when they didn't want them to.

------
juddlyon
"Let me give you some concrete examples from his book."

I wish every blog post included a line like this.

