

Software Optimization: A Systematic Approach - jknupp
http://www.jeffknupp.com/blog/2012/07/10/software-optimization-a-systematic-approach/

======
reirob
Why I appreciate articles about Software Optimization and I also think that it
is often underrated I have experienced in last projects something slightly
different: The systems on which I worked were composed by several separate
processes that are spawned in chains and communicate mostly over files. While
this might seem an arcane architecture by today's standards it has the
advantage of high flexibility, because you can replace/optimize every single
process at the OS level and you can very efficiently introduce new logic and
even make the system scale over many machines.

However in this case case the bottleneck is IO and not CPU. And hence the
optimization in this case would be to minimize file access, rereading
information that was already present at prior steps of the process chains. I
would love to learn efficient methods to measure in such architectures what
are the spots to optimize.

~~~
jknupp
An upcoming part of the series discusses determining what work is _necessary_
, at a very fine level, and working to remove all else. This may seem like a
needlesly simplistic exercise, but consider the example you gave: the
processes need to communicate with one another. The method by which they do so
is an implementation detail. Even if they eventually use the files for another
purpose, those are two separate requirements (the first to communicate with
other processes and the second to journal that communication). Replacing file
based communication with something like a simple TCP based message protocol
would likely yield great dividends. For processes on the same machine, an IPC
mechanism like domain sockets or shared memory would surely be faster.

Of course, the first thing to do is determine if increased performance is
actually required. No sense in making something fast if no one benefits.

~~~
nkurz
_For processes on the same machine, an IPC mechanism like domain sockets or
shared memory would surely be faster._

Why are you sure that these would be faster than reading and writing a file?

 _There is a key truth to be mindful of while doing optimization work:
programmers are, as a rule, terrible at anticipating at the cause of slowness
in their application. This is counterintuitive but almost always true._

