Hacker News new | comments | show | ask | jobs | submit login
Hill for the data scientist: an xkcd story (livefreeordichotomize.com)
122 points by dsr12 57 days ago | hide | past | web | 11 comments | favorite

There's another way of looking for causality directly from data. If A and B are uncorrelated but both correlate with C then you know that C isn't causally upstream of either.

So if you have an A and a C and want to figure out the causality you need to look for a B that correlates with C but not A?

How can two things not correlate with each other, but both correlate with a third thing? This seems like saying x=5, y=3, and that there's some variable z which x and y are both equal to?

The number of people who like the taste of vanilla (I suppose) doesn't correlate with hours of sunlight in some area. Both correlate with amount of ice cream sold.

The simplest way to explain is when when both cause C some % of the time. It's useful to run a simulation if you doubt it.

Interesting, is there a law or algorithm/equation for that?

Edit: sorry, instrumental variables do not describe what the original poster was talking about, but could be useful for understanding the connections between correlation and causation in other ways.

The concept of an instrumental variable


covers some of it. Judea Pearl's book Causality summarizes lots of recent (past few decades) work in the area, with a focus on his theoretical framework. There are lots of algorithms for discovering the structural of causality for sets of variables larger than three. Having interventional data always makes these things easier, but it's not the only way to get at causality.

I always thought correlation was transitive.Isn't it? It is surely reflexive but I might have to go back and study more about those properties.

Anyone else frustrated hovering over the strips looking for the title-text?

As an aside, i feel there has been a lack of such strips from XKCD lately.

Strips in article:

* https://xkcd.com/552/

* https://xkcd.com/1592/

* https://xkcd.com/242/

* https://xkcd.com/1217/

* https://xkcd.com/925/

* https://xkcd.com/323/

* https://xkcd.com/605/

* https://xkcd.com/1170/

* https://xkcd.com/1462/

* https://xkcd.com/882/

Latest strip at time of writing is https://xkcd.com/1772/

Current strip is https://xkcd.com/1868/

Eyeballing it I think its still likely uniform even if there hasn't been one that would have made it into the article in the last 100.

Applications are open for YC Winter 2018

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact