Hacker News new | past | comments | ask | show | jobs | submit login

How would I determine if the pattern was linear or logarithmic, whether an observation was an outlier, or whether two series were related?

A tiny bit ad hominem, but I take it you are not a statistician. The first thing we do is to graphically illustrate the data, because human brains are so good at recognising patterns.

It is important to experiment with several variations of display, also including and excluding outliers and different series, to see if there are biases that mislead you.

Only then do we come up with a mathematical model to fit.

Actually reading the data line by line can be extremely misleading - because you can only compare a few numbers at a time, which fails to give you an insight into the overall variation.

I am in fact a statistician. I work in statistics, mathematics, and computer science. I am often severely frustrated by the oversimplifying and misleading natures of conventional charts.

I have seen good charts. Yes, they exist, and I actually see them quite often. It just so happens that the vast majority of charts and graphical illustrations I see are omitting details at best and dangerously misleading at worst.

This is very true.

Reading Tufte carefully, one realizes that (a) he's read more Derrida than he's willing to admit (there are entire unattributed quotations, probably by accident) and (b) great charting is an art form, perilously predicated on someone being both quantitatively educated and visually gifted.

There is no silver bullet but the table, and even then, Simpson's paradox is always ready to bite you.

Interesting. Me too, but I have been more into actuarial work for the last five years. One thing I have noticed is that they love tables of numbers. Maybe I need to reconsider.

How do you feel about waterfall charts? (presumably with the table attached)

How do you

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact