
How Basic Performance Analysis Saved Us Millions - kiyanwang
https://blog.heapanalytics.com/basic-performance-analysis-saved-us-millions/
======
mafro
Previously:
[https://news.ycombinator.com/item?id=14384370](https://news.ycombinator.com/item?id=14384370)

------
bartread
This is an interesting write-up although I'd question the take home at the
end: _" It really doesn’t take much to find 10x wins."_

Sometimes there's a really obvious problem that you can solve that will get
you that kind of 10x performance boost. Often though you have to work your ass
off for it or, even having found a problem where fixing it might even double
your performance, the fix might be complex, touching multiple parts of the
system.

In my experience, whilst you can often find quick wins, and sometimes these
will be enough, there will always be cases where the analysis is much harder,
or where more complex fixes are required.

(Disclaimer: I am a performance consultant, focussed mainly on SQL Server and
.NET. My background is that _years and years_ ago I lead the team that built
Red Gate's ANTS Performance Profiler and ANTS Memory Profiler, and also worked
on a number of their SQL Server tools.)

~~~
jng
This. I would call the optimization described on the article as "low-hanging
fruit": if you don't have the right DB indices or you are not batching your
operations, you can find that kind of improvement easily, mostly just because
your code was not following the "fast path".

Still very useful article even if just to show how to use profiling tools,
which you will need to even for much-higher-hanging fruit.

------
snaily
Fom the flame graphs I don't get how there is a 10x speedup - the optimized
piece (ExecOpenIndices) is less than 90% of the total?

------
mankash666
Nice analysis and write up. What's the reason for choosing PGSQL? Your
workload appears ideal for a NOSQL DB like mongo

~~~
golergka
> Your workload appears ideal for a NOSQL DB like mongo

Can you expand on that a little bit, please?

~~~
ddorian43
That is a good workload for column-stores. Mongodb is row-store (an
inefficient one at that).

~~~
mankash666
The aim of the product is to display usage trends to it's customer, in real
time.

My understanding as a non CS, self taught programmer is that such workloads
are better served by NoSql DB s. Yes, this comes with the lack of acid
compliance, but spotting trends in real-time (apparently) don't require acid
compliance.

Please correct me if wrong as I'm getting down voted, which is rather amusing

~~~
ddorian43
Nosql is very big. Compare redis + cassandra + elasticsearch and you'll see.
You have to look at: data-structures the db uses, query patterns, data-
velocity, data-shape, etc etc. Basically how the db really works in a _low_
level, to understand what is it good for.

Current case they: have a lot of data, need a lot of compression (or low
overhead as possible), need as much query-selectivity as possible (reading as
few rows as possible for each query) that's why they have a unique case of
selective indexes.

Bonus: mongodb agg sucks. Seach online for comparison to pg or es agg.

