
The Dangers of Intermediate Targets: IQ, Cholesterol, and 99%-ile Latency - kryptiskt
http://danluu.com/percentile-latency/
======
emmfr
Useful related concept in the social sciences: Goodhart's law [1]

“When a measure becomes a target, it ceases to be a good measure.”

[1]
[https://en.wikipedia.org/wiki/Goodhart's_law](https://en.wikipedia.org/wiki/Goodhart's_law)

~~~
paol
With the corollary that every measure always becomes a target...

------
majke
I hate blog posts without dates. Is that a new blog post or old?

Anyway, to the merit:

> But the only trick is that there is no trick

Yes a thousand times. Very well put. This is my understanding of the
"Premature Optimization" mantra: it's either you got everything right, and
there is little need for miro-optimizing, or everything wrong and micro-
optimizing won't get you much.

~~~
venomsnake
Well no ... premature optimization is something else. When you start a project
- you make some assumptions about load, shape whatever. Optimizing based on
these assumptions before being validated by the real world testing is sub
optimal. You don't know where the bottlenecks will be, or how users will
interact with your software.

~~~
tjradcliffe
Premature optimization includes both what you describe and what the GP talks
about. Adopting the general rule, "No optimization without quantification"
helps in all cases: validate those assumptions early in the design process
(with prototypes, user observations, whatever) and with regard to optimization
of code later on, profile it intelligently to identify bottlenecks, don't just
assume the issue arise from what seem like the obvious culprits.

------
carbocation
The treatment of cholesterol could use some clarification.

As the author notes, statins reduce MI risk and they reduce cholesterol (LDL
cholesterol specifically).

He then addresses torcetrapib, which was designed to raise HDL-cholesterol.
This was testing quite a different hypothesis from that of statins. I don't
think it makes sense to lump these two together and say "sometimes cholesterol
works and sometimes it doesn't". Rather, it's more fruitful to see the
biomarkers separately.

------
atemerev
This is how market crashes now happen. People at hedge funds optimize to be
right 99.99% of times, ignoring those huge black swans at 0.01%.

~~~
grandalf
The idea of a _black swan event_ is that it's beyond normal expectations. This
is a bit misleading because we all know such events will occur. Because we
know this, how should we act?

Suppose you are on a small island with 50 other people. You know that there
will eventually be a tsunami. What percentage of your time do you spend on
tsunami preparedness vs other productive pursuits?

When it comes to regulation, we know catastrophic events will occur
eventually, but the time horizon is distant enough (where the probability of
such events approaches 1) that we collectively decide to roll the dice and
accept the risk.

Then when such an event occurs we pretend that nobody knew it would happen. We
did know it would happen, we just blindly hoped it would not happen during our
lifetime|tenure|time-in-office, etc.

The further we are from a previous event, the easier it becomes (politically)
to erode the protections that are in place. So much lobbying successfully
attempts to do this.

This is predictable, and as our society evolves in finance more and more risk
can be arbitraged, and so transfer of risk from the individual or firm to the
state is easier than ever before. Not only will a firm fail but its pension
fund will go away and the insurance company that insured its pension fund's
underwriting capital is only sufficient to handle two such events per year
because firms lobbied to have the requirements reduced. The capital freed due
to lessened underwriting requirements now funds some other industry which is
propped on this risk transfer.

So over time we move in lock step to ignore systemic risk and to double down
and double down again on existing assumptions about how much underwriting is
necessary, etc.

Quantitative Easing is the intentional erosion of risk capital standards,
intended to put more capital into the economy to do exactly this.

While it's a relief that the bailout largely worked, it's discomfiting to
think about the many perverse incentives that have now been added to the mix.

