Hacker News new | past | comments | ask | show | jobs | submit login

> Instead of comparing yourself to arbitrary thresholds, use the raw metric as a KPI instead. Use "minutes until airborne" as the KPI. That you can optimise indefinitely, and it is an honest representation of your current process, not how it compares to a number someone threw out at some time.

This is an example of a more general phenomenon - when you have outcome buckets, it's generally better to do your modeling on raw outcomes ["it took us 32 minutes"] than to try to do it on the bucketed outcomes ["we didn't make it in 30 minutes"]. Bucketing throws most of your information away.

Andrew Gelman talks about this all the time in the context of election modeling, where it's better to predict how many votes something will get, which is a continuous variable (and then, based on the predicted vote, predict whether the thing will win or not), rather than trying to predict whether the thing will win, which is a discrete variable.




> Use "minutes until airborne" as the KPI.

Average minutes to airborne? Median? Mode? 90th percentile? 99th percentile?

Optimizing for each of those would result in a different process, different outcomes, and quite possibly worse outcomes for the stakeholders than "proportion under 30 minutes" which is just a sixth way to slice the same data.


> Average minutes to airborne? Median? Mode? 90th percentile? 99th percentile?

Facetious answer: yes.

More useful answer: you want to record the full timeseries of individual data points. From this you can derive mean, median, mode, 90th percentile, 99th percentile and any other statistic that is shaped like hole in your cost–benefit calculations. Including trends and changing variance.

For reporting purposes, the upper process behaviour limit is probably a good one as "the" numeric value of the KPI, but the true value is in the timeseries.


Did you mean to respond to my parent comment?




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: