
Secret Data: On replication in economics - nkurz
http://johnhcochrane.blogspot.com/2015/12/secret-data.html
======
cs702
From the original post: "Much research in economics and finance uses secret
data. The journals publish results and conclusions, but the data and sometimes
even the programs are not available for review or inspection. Replication,
even just checking what the author(s) did given their data, is getting
harder."

If the data and programs remain secret, it's not really science; it's just
another example of Richard Feynman's _cargo cult science_ :
[http://calteches.library.caltech.edu/51/2/CargoCult.htm](http://calteches.library.caltech.edu/51/2/CargoCult.htm)

Sadly, much if not most "economics" research is cargo cult science... often
motivated by political, religious, and moralistic ideologies.

\--

EDIT: In response to davidw's comment below, here are a few other economists
weighting on the topic:

* [http://paulromer.net/wp-content/uploads/2015/05/Mathiness.pd...](http://paulromer.net/wp-content/uploads/2015/05/Mathiness.pdf)

* [http://paulromer.net/mathiness/](http://paulromer.net/mathiness/) (shorter version)

* [http://noahpinionblog.blogspot.com/2015/09/theory-vs-data-in...](http://noahpinionblog.blogspot.com/2015/09/theory-vs-data-in-economics.html)

* [http://noahpinionblog.blogspot.com/2013/12/freshwater-vs-sal...](http://noahpinionblog.blogspot.com/2013/12/freshwater-vs-saltwater-divides-macro.html)

* [http://news.mit.edu/2010/krugman-event](http://news.mit.edu/2010/krugman-event)

~~~
davidw
> Sadly, much if not most "economics" research is cargo cult science... often
> motivated by political, religious, and moralistic ideologies.

Got any data for that, or do you just happen to dislike some of the
conclusions?

I agree entirely about science without data not being much good, though.

~~~
cs702
The author of the original post, John Cochrane, is a leading economist[1], and
_he_ is complaining that "much research in economics and finance uses secret
data."

How's that for evidence? Did you read his post?

In case it wasn't obvious, I _agree_ with him!

\--

[1]
[http://faculty.chicagobooth.edu/john.cochrane/](http://faculty.chicagobooth.edu/john.cochrane/)

~~~
davidw
You jumped to conclusions from that, though.

~~~
cs702
No, I din't!

I come across this issue repeatedly in the economics literature, but didn't
have the inclination to google up more evidence when writing the above
comment, as I assumed "everyone knows this." Evidently I was wrong about
_that_. Here are a few other economists (other than Cochrane) weighting on the
topic:

* [http://paulromer.net/wp-content/uploads/2015/05/Mathiness.pd...](http://paulromer.net/wp-content/uploads/2015/05/Mathiness.pdf)

* [http://paulromer.net/mathiness/](http://paulromer.net/mathiness/) (shorter version)

* [http://noahpinionblog.blogspot.com/2015/09/theory-vs-data-in...](http://noahpinionblog.blogspot.com/2015/09/theory-vs-data-in-economics.html)

* [http://noahpinionblog.blogspot.com/2013/12/freshwater-vs-sal...](http://noahpinionblog.blogspot.com/2013/12/freshwater-vs-saltwater-divides-macro.html)

* [http://news.mit.edu/2010/krugman-event](http://news.mit.edu/2010/krugman-event)

Finally, if this is of interest, I highly recommend you read this opinion
piece by Richard Posner, in which he concluded that the dominant conception of
economics until the global financial crisis was the abstract-theoretical study
of "rational choice" as opposed to the study of how economies actually work:

[https://newrepublic.com/article/69601/how-i-became-
keynesian](https://newrepublic.com/article/69601/how-i-became-keynesian)

If I get the time, I'll google up more blog posts and supporting evidence.

~~~
davidw
Some of what you point out is that macroeconomics is difficult. Even micro is
difficult, because you can't just go running all the experiments on people
that you'd like to, but running experiments with entire countries is "kind of"
difficult.

That doesn't mean it's worthless or mostly partisan or 'religious' (that seems
particularly nonsensical).

It's fair to point out that economics - like most human fields of study and
endeavor - has its problems, but I do object to the broad-brush one-liner
style of discussion.

~~~
cs702
LOL. Yes, running experiments on entire countries is "kind of" difficult, to
put it mildly.

But the links I shared suggest the divide in macroeconomics goes well beyond
that. Quoting from Romer's blog post
([http://paulromer.net/mathiness](http://paulromer.net/mathiness)): "The goal
in starting this discussion is to ensure that economics is a science that
makes progress toward truth. A necessary condition for making this kind of
progress is a capacity for reaching consensus that is grounded in logic and
evidence. Given how deeply entrenched positions seem to have become in
macroeconomics, this discussion could be unpleasant. If animosity surfaces, it
will be tempting to postpone this discussion. We should resist this
temptation." In the most nice and polite way possible, he's accusing his
colleagues of engaging in cargo cult science.

