"Another problem is I think that the open source tools that are available don't work at the right level. Like when you are running profilers, they are helping you profile assembly code generated by a C compiler. The performance characteristics he's taking advantage of are at a lower level. I think you need CPU-specific performance counters and so forth, and there's a different way of getting those for each make/model. Are there even open source tools that allow you to get this information? I'd appreciate a pointer."
The way forward is a sampling profiler.
By sampling the stack every (eg) millisecond, you can build a picture of what is taking the longest.
A sampling profiler still doesn't tell you what specific instructions are taking longer, how much memory contention is happening between CPUs, how many cache and TLB misses there are, etc.
The way forward is a sampling profiler.
By sampling the stack every (eg) millisecond, you can build a picture of what is taking the longest.