
A tale of latency and broken windows - yenkel
https://yenkel.dev/posts/a-tale-of-latency-and-broken-windows
======
hyperman1
A simple way to include performance in the development activities without
ending ib premature optimization territory is to introduce timing budgets.

Decide for each operation how long it us allowed to take. E.g. pushing this
button reacts in 100ms. Make your tests record these times. Optimize when it
becomes clear you're not going to make it.

There was an intervieuw with the Intel engineer that got the linux laptops
booting in 10s. They budgetted, x seconds for kernel, y seconds for X11, etc.
.

~~~
kevsim
This sounds totally reasonable for a company like Auth0 that has a) a big
engineering team and b) tons of people who are depending on their APIs not
being slow.

Do you think it's easy enough to get this sort of machinery set up even for
small startups?

~~~
toast0
If you're collecting stats anyway, it's not usually too hard to collect a
latency metric and process it.

There's room to argue about methodology (ie client vs server latency, where
and how timing probes happen), but just pick something and try to measure no
more than 10 phases of the requests (timing probes have costs too), and see
what happens. After you have a baseline, you can set targets and/or see how
releases change your data.

------
teddyuk
This reminds me of a system I worked on, the devs all developed locally with
all the services but in prod the services were on different hosts and the
performance was terrible.

This reminds me of the system I worked on, the devs had promised a 3-tier
system but delivered a 2-tier system so management had them add a middle layer
and the performance was terrible.

This reminds me of the system I worked on, the devs decided a lockless
architecture was the way for better performance but with the limited threads
we had at the time, apart from the few people who kept hold of a thread until
they were done, the performance was terrible.

This reminds me of the system I worked on, the database design was a serious
bottleneck but no one wanted to fix that so the app tier tried to change their
queries, the performance was terrible.

I don't know why people spend so much time optimizing the wrong thing - it
happens over and over again!

~~~
Chris2048
Management not wanting to admit a large architectural problem, slapping a
small bugfix/task on the board in the hope it will take less time before to
(appear) fix before performance review.

------
nabilhat
I'm lost on the broken windows analogy. Projects like software are constructed
from these discrete elements; criminal activity is not. Bank robbery doesn't
have a dependency on littering and jaywalking.

It makes more sense from the perspective of the real life outcomes of the law
enforcement policy. By making an outsized response to trivial infractions a
goal divorced from measurement against the desired result, finding reasons to
harrass citizens became a performance target, and the policy is widely
considered to be a failure in its intended outcomes and unintended
consequences. Similarly, "Always Eschew Latency" as a task level target
independent of desired project outcomes is likely to result in a product that
performs ahead of schedule in every way except delivery date.

~~~
hyperman1
AFAIK the theory says littering does have an effect on bank robbery: If people
see litter, you instill a mentality where caring for the neighbourhood is
worthless. From that 'each for his' , 'dog eat dog' world view, the step to
big crimes is smaller.

Now I have no idea if the theory is correct.

~~~
webmaven
> AFAIK the theory says littering does have an effect on bank robbery: If
> people see litter, you instill a mentality where caring for the
> neighbourhood is worthless. From that 'each for his' , 'dog eat dog' world
> view, the step to big crimes is smaller.

> Now I have no idea if the theory is correct.

The theory is correct, but the effect is smaller than proponents supposed (and
seems to affect minor property crime more than violence). The poster child for
the theory is the reduction in crime in NYC, but more recent analysis has
shown that a larger driver was the elimination of lead from gasoline and
paint.

