We are pretty much done with CPU speed advances and compiler optimizations. If you use an Intel i7 with the Intel C compiler, there might be a few percent left to optimize, but not much. Free lunch is over for a decade now.
You might get more cores at the same speed, but even that seems to be limited due to heat issues.
You can get more efficient CPU (e.g. SIMD instructions) but the compiler cannot optimize for them very well. Some people say implicit SIMD is a bad idea anyways.