On: Prefix sum: 20 GB/s (2.6x baseline)

This is partially due to the compromises of mappingvector intrinsics into C (with C++ only being marginally better). In a more vector-oriented language, such as shader languages, this:

  s1 = vaddq_u32(s1, vextq_u32(z, s1, 2));
  s1 = vaddq_u32(s1, vdupq_laneq_u32(s0, 3));

would be more like this:

  s1.xy += s1.zw;
  s1 += s0.w;

To be fair, even in standard C11 you can do a bit better than the CPU manufacturer’s syntax

  #define vaddv(A, B) _Generic((A),
      int8x8_t:    vaddv_s8((A), (B)),
      uint8x8_t:   vaddv_u8((A), (B)),
      int8x16_t:   vaddvq_s8((A), (B)),
      uint8x16_t:  vaddvq_u8((A), (B)),
      int16x4_t:   vaddv_s16((A), (B)),
      uint16x4_t:  vaddv_u16((A), (B)),
      int16x8_t:   vaddvq_s16((A), (B)),
      uint16x8_t:  vaddvq_u16((A), (B)),
      int32x2_t:   vaddv_s32((A), (B)),
      uint32x2_t:  vaddv_u32((A), (B)),
      float32x2_t: vaddv_f32((A), (B)),
      int32x4_t:   vaddvq_s32((A), (B)),
      uint32x4_t:  vaddvq_u32((A), (B)),
      float32x4_t: vaddvq_f32((A), (B)),
      int64x2_t:   vaddvq_s64((A), (B)),
      uint64x2_t:  vaddvq_u64((A), (B)),
      float64x2_t: vaddvq_f64((A), (B)))

while in GNU C you can in fact use normal arithmetic and indexing (but not swizzles) on vector types.