

Is There Such a Thing As Predictive Analytics? - skempe
http://www.dataversity.net/is-there-such-a-thing-as-predictive-analytics/

======
crusso
The author is right in one sense: there's a lot of black magic around
predictive analytics, machine learning, modeling, etc.

However, some of these techniques yield a great deal of paydirt. Look at the
spam, or rather the lack thereof, in your inbox. Thank the enormous success of
predictive analytics for eliminating a great deal of it.

As he mentioned, weather forecasting is actually pretty good, all things
considered. Kvetching about unexpected rainy days demonstrates a
misunderstanding of how we should evaluate the success of weather forecasting
techniques.

As useful as these techniques are, there is a great deal of ignorance in where
they are useful and where they aren't. People selling things and those with
various political agendas use that ignorance to foist "modeling" on an
unsuspecting public.

No complex modeling used for predictive analytics is useful without a great
deal of validation. You come up with a model and set of techniques, you feed
in historical data, you compare the outputs of your predictions with
subsequent historical data, you adjust your techniques to produce more
accurate predictions, wash, rinse, repeat. There's no real getting around the
need to perform this cycle ad nauseum if you want results that aren't just
noise.

------
endersshadow
I've done a lot of this thing called "predictive analytics," and I've found
that what works best is machine-aided prognostication. It highlights some
things that folks aren't aware of, and points to some seasonality that may not
be particularly intuitive.

I've found it particularly useful in predicting inventory and packaging
distributions. For example, if you have a product like soda that you can
package a myriad of different ways, predictive analytics is a great way to
help determine the ratio and amount you need to put into each type of package.
I've seen it used with great success, reducing the amount of excess inventory
and increasing sales (because stock-outs don't happen).

It's a tool, like any other. It's not a silver bullet that will tell you
exactly what the future will hold, but it can give you a good guideline on
what to expect, and from there, you can act upon that information.

------
rywa
Some systems are more complex than others. Others exhibit more apparent
regularity. This also comes into play depending on what scale you're working
at. Although it's difficult to predict the weather that doesn't mean it's
equally to difficult to predict something like the load on an electric grid,
which depends on the weather. Electricity companies have been doing this for
decades.

~~~
crusso
It's also important to understand the perceived success of predictive
analytics in the context of the problem being solved.

For example in predictive analytics used to determine product distribution at
major retailers, the goal isn't to be perfect. The goal is to be a little bit
better than what humans (or their previous methods) could do. If you go to
Target management and prove that your predictive analytics is 3.3% better at
predicting how many blenders they'll need in each of their stores, but that
3.3% netted them 5 million dollars, they'll sit up and listen.

------
dgreensp
Philosophically speaking, he is defining the term "prediction" right out of
existence! It's not that "real" predictions are "right" in some sense, even
probabilistically. There is only ever such thing as one take on the future
based on certain data from a certain perspective.

~~~
msellout
Right. He's critiquing the term "predictive model" by saying we should use the
term "hypothetical model" instead. Is he suggesting it's impossible to test a
hypothesis? I thought Kant and Hume had this argument long ago.

------
msellout
Taleb wrote better. Yes, prediction is hard.

If you're interested in these topics, do some reading on over-fitting,
stationarity, and nonlinear dynamics.

