
Data Science for Startups: Business Intelligence - bweber
https://towardsdatascience.com/data-science-for-startups-business-intelligence-f4a2ba728e75
======
mlthoughts2018
> the next step is being able to answer all sorts of questions about product
> health within your organization. A novice data scientist might think that
> this type of work is outside the role of a data scientist, but identifying
> key metrics for product health is one of the core facets of the role.

This strikes me as disingenuous word play. Whether or not the identification
of key metrics is important, it's fair to ask whether it ought to be part of
the job duties for a data scientist, and your answer to that question probably
reveals more about your personal inclinations than about what is optimal for a
given business.

Personally, my experience has been that most of the metrics and ad hoc reports
that product- or business-minded managers ask for are junk wastes of time
which, at best, create new surface area for fueling political debates about
what's important, allowing different managers and cliques in the company to
duke it out over whose priorities should be favored. Dressing them up as sexy
data-driven "insights" is just one more tool for politics.

When the words "data scientist" come to mean "ad hoc business report jockey"
then it seems like you might as well dissolve the question, because in that
company, they've already chosen what they want out of data science-- arbitrary
and whimsical political fodder.

In other companies, data scientists are actual statisticians. It can be really
hard for business people to let go of the keys to the car, but a well-trained
statistician knows that you have to devise you experiments or your proposed
models based on whatever inference goals you have, which would be business
goals in many cases. By taking a generic statistical perspective, you will
home in on the metrics and model diagnostics that carry _actual_ meaning for
decision making, and many times you'll just pragmatically ignore all the
business junk (not always, because some of it rests on useful principles, but
a lot of it really is junk).

Anyway, it's such a loaded topic. It all depends on the private mental model
someone has in their brain when they hear "data science."

Some people start thinking about rigorous statistical models, trade-offs
between a frequentist test or a Bayesian setup, whether or not a machine
learning model is appropriate, what kind of diagnostic would make sense for a
special regression model... these people need to be given freedom to make
choices that are statistically rigorous, and trusted that they understand the
inference goals in terms of business priorities for the company (probably even
better than the business people do).

Other people hear "data science" and they think of a sharp-featured
20-something wearing a sweater vest parachuting in the business meeting with a
sexy slide deck on their Macbook, ready to spin whatever story is needed.
These people ... should not be in charge of anything.

