
Humans-in-the-loop forecasting: integrating data science and business planning - Anon84
http://www.unofficialgoogledatascience.com/2019/12/humans-in-loop-forecasting-integrating.html
======
kfk
The black box forecasting “platforms” baffle me. AWS has been pushing a
forecasting module lately. They push the exact same thing to big corp and
small start ups. Then they just push “forecasting” without putting it into a
context. As this article describes there are different contexts and different
outputs for forecasting. Platforms are great but at least until we have AI we
still need data scientists to take the right steps.

~~~
mr_toad
> Platforms are great but at least until we have AI we still need data
> scientists to take the right steps.

Yes, but with some caveats.

You’re going to want to hire someone with technical chops to run the system,
and interpret the results, but the days of second guessing the machine are
over.

Rob Hyndman has long argued that automated time-series forecasting beats human
judgement (before ML was even a thing). And even where humans could beat
automated systems, they don’t scale.

~~~
vharuck
My fear is these systems promise great results for execs who build models to
answer new questions by themselves. And that's just irresponsible.

I do forecasting in my job, and use Hyndman's `forecast` R package. I totally
rely on it's automated model creation. That said, I only have the moderate
confidence in its results because I took the time to research forecasting and
started with an understanding of how to use data to answer questions. (To be
clear, any lack of confidence in the results is from a fear I'm doing it
wrong).

Non-data people are _terrible_ at choosing the right data to answer the
question. Sure, automating the model terms or even the choice of model is
probably done better by machine, with human tweaking only needed for rare
cases. But domain experts will often choose totally wrong numbers for models
to predict.

If the target's obvious (e.g., quarterly revenue they already have a method
for calculating), then letting execs do it themselves is probably fine. But
I'd never trust them to take an abstract question, choose an appropriate
measure to answer it, and then find good data without consulting a skilled
analyst.

------
vavooom
"Strategic forecasts drive high stakes decisions at longer horizons, so they
should not be approached simply as a black box forecasting service, divorced
from decision-making."

In summary, the author seems to argue that there is should be an inverse
relationship between the time to prediction and how complicated the desired
model should be.

------
pygy_
I'm waiting for Google to productize Alpha Zero (or whatever its descendant is
called nowadays) as a "C-suite as a service" thing for directors to play with.

