On: How concurrency works: A visual guide

[CyberDildonics]

Optimizing isn't either SIMD or large scale multi-threading. You need both, which is why CPUs and GPUs both use both techniques.

[dragontamer → CyberDildonics]

Fair enough.

GPUs in particular have a very hyperthread/SMT like model where multiple true threads (aka instruction pointers) are juggled while waiting for RAM to respond.

Still, the intermediate organizational step where SIMD gives you a simpler form of parallelism is underrated and understudied IMO.