> Unless you use a language with cheap and fast green threads that multiplex on OS threads (like Haskell), you should use a number of threads that is roughly in the same order of magnitude as the number of CPU cores + hyper threads.
The optimal number of threads per core is workload and machine dependent, but rarely is that number 1 per hardware thread[1]. For maximum CPU utilization and computational efficiency, you need multiple software threads per hardware thread so that the OS always has a thread ready to go when another thread blocks or the CPU has an unused component. How much the threads block and what parts of the CPU the exercise is the workload-dependent part. Ideally, the number of threads would be a tunable performance parameter.
As an example of this, I recently wrote some math-heavy code to transform the cells of a matrix by a particular transfer function. Each cell calculation was independent, so I parallelized them. The most efficient parallelization for my 6-core AMD processor turned out to be 6 threads per core.
[1] The main exception I can think of is when running multiple applications in parallel that in turn parallelize their work.
I'm sorry, but if your "math-heavy" code performs better when oversubscribed, then it is a massively suboptimal implementation. As a performance-oriented mathematical library developer, with colleagues at many of the major supercomputing centers and vendors, oversubscription is a hazard that we guard against on CPU architectures because it is always slower. Many supercomputing environments use a kernel that does not even support over-subscription.
Note that GPUs are different in the sense that over-decomposition is necessary to cover latency, but even there, you generally get better performance at lower occupancy due to better register reuse. SMT CPU systems, most notably, Blue Gene/Q, often require use of multiple hardware threads per core to cover instruction latency and keep the prefetcher busy (due to allocation of outstanding requests). This is not over-subscription because those are bonified hardware threads.
> The optimal number of threads per core is workload and machine dependent, but rarely is that number 1 per hardware thread.
This is exactly why I said it should be in the same order of magnitude, not exactly the same. Definitely not 250+ threads on a dual core. It might make sense to add a few more threads to keep the CPU busy if one or more of the threads will have to wait.
The optimal number of threads per core is workload and machine dependent, but rarely is that number 1 per hardware thread[1]. For maximum CPU utilization and computational efficiency, you need multiple software threads per hardware thread so that the OS always has a thread ready to go when another thread blocks or the CPU has an unused component. How much the threads block and what parts of the CPU the exercise is the workload-dependent part. Ideally, the number of threads would be a tunable performance parameter.
As an example of this, I recently wrote some math-heavy code to transform the cells of a matrix by a particular transfer function. Each cell calculation was independent, so I parallelized them. The most efficient parallelization for my 6-core AMD processor turned out to be 6 threads per core.
[1] The main exception I can think of is when running multiple applications in parallel that in turn parallelize their work.