On: Fearless SIMD v1.0

[modulovalue]

I’m not sure I agree on the impossible part, I feel like a sufficiently smart interprocedural analysis that also implements range analysis interprocedurally could prove a lot to where it becomes useful.

I guess what I would like to see is SIMD libraries being able to confidently say nobody needs to use intrinsics (or differentiate between relaxed/normal SIMD on the user API level) because the language + high level SIMD APIs are smart enough to choose the right implementation.

IIRC IEEE min/max with proper NaN handling needs 8 instructions on x86 vs 1 on arm64 I find it very sad that we apparently haven’t really solved that yet without forcing the user to use different APIs.

[orlp → modulovalue]

You can do it with just 3 instructions for IEEE 754-2019 minimumNumber (ignores NaN):

        vminpd          ymm2, ymm1, ymm0
        vcmpunordpd     ymm0, ymm0, ymm0
        vblendvpd       ymm0, ymm2, ymm1, ymm0

If you want proper IEEE 754-2019 minimum (propagate NaN, -0.0 < +0.0, NaN bitpattern picked in the usual way) you can do it in 6:

        vminpd          ymm1, ymm0, ymm1
        vbroadcastsd    ymm2, qword ptr [rip + .LCPI0_0]
        vandpd          ymm2, ymm0, ymm2
        vorpd           ymm1, ymm2, ymm1
        vcmpunordpd     ymm2, ymm0, ymm0
        vblendvpd       ymm0, ymm1, ymm0, ymm2

I personally find this a load of nonsense I don't care about.

If you want propagating NaNs but don't care about signed zero or NaN payload/sign, you can use

        vminpd  ymm2, ymm0, ymm1
        vminpd  ymm1, ymm1, ymm0
        vorpd   ymm0, ymm1, ymm2

What I do in Polars is a bit different, there for propagating NaNs I do

    if (self < other) | self.is_nan() { self } else { other}

this isn't fully optimal on x86-64 but it's fairly simple and autovectorizes decently on various platforms, here's AVX2:

        vcmpltpd        ymm2, ymm0, ymm1
        vcmpunordpd     ymm3, ymm0, ymm0
        vorpd           ymm2, ymm3, ymm2
        vblendvpd       ymm0, ymm1, ymm0, ymm2

[dzaima → orlp]

That vminpd+vminpd+vorpd actually does handle signed zero properly! Screws up NaN payloads to the max though. Can be easily extended to canonicalize the NaN with 2 instrs + constant though of course. (which ends up at the same number of instrs as your proper impl (albeit with worse port distribution and latency), but you get to have a canonical NaN!)

Hit upon https://github.com/llvm/llvm-project/issues/217376 while playing around with proper minimumnum, failing to SMT-verify whatever version of LLVM I had; did find a funky working 6-instr (+ constant) version though:

    vpandn      ymm2, ymm1, ymm0
    vpcmpeqd    ymm2, ymm2, 0x80000000 # whether ymm0 is -0 and ymm1 is +0 (or other cases that magically don't cause issues)
    vcmpltpd    ymm3, ymm0, ymm1
    vcmpunordpd ymm3, ymm3, ymm1 # regular NaN-is-larger ymm0<ymm1
    vpor        ymm2, ymm2, ymm3
    vpblendvb   ymm0, ymm1, ymm0, ymm2

[dzaima → dzaima]

Stop the presses! 5 instructions!

    vpor        xmm2, xmm0, 0x80000000
    vpcmpeqd    xmm2, xmm1, xmm2
    vcmpunordps xmm2, xmm2, xmm0
    vminps      xmm0, xmm1, xmm0
    vblendvps   xmm0, xmm0, xmm1, xmm2

yay for superoptimizers

[athrowaway3z → modulovalue]

Anything without range analysis is not worth it.

Note that:

NonZerof32 * NonZerof32 -> NonNanf32

NonZerof32::from_bits(1) multiplied with itself is zero.

Doing range analysis needs the language to support it at compile time, and the dev to specify what range it is.

The only 'stable' thing i can think of is a type for 'greater-eq-one' using only addition and multiplication. Practically every other operation breaks most of the type knowledge up to that point.

[nnevatie → modulovalue]

> nobody needs to use intrinsics

Funnily enough, intrinsics have become almost trivial to write, with agentic tooling. I used to be an ISPC advocate but now I'm finding myself gravitate to direct use of intrinsics with bespoke dynamic/runtime dispatching more than ever.

[janwas → nnevatie]

+1 on agents making library/DIY a lot more attractive than bespoke compilers.

In C++, one can also have portable intrinsics plus simpler runtime dispatching using our Highway library :)

[nnevatie → janwas]

> using our Highway library

Yes, Highway is pretty nice but also quite elaborate when it comes to dealing with multiple vectorized versions and dispatch. The macros burn my eyes still.

[janwas → nnevatie]

:) Yeah, those are best copy-pasted from existing code.

I do think we've converged on the best that's possible in C++.