When should we expect brittle program performance?

[igouy]

Back in 2015 a "Programming Language Performance Comparison"

http://www.hildstrom.com/projects/langcomp/index.html

On my old i5-3330 these times for unchanged programs —

    $ gcc -O3 -o test-c-gcc test.c
    $ time ./test-c-gcc 100
    real    1m36.599s

    $ time /opt/src/jdk-16/bin/java test 100 
    real    4m49.166s

Add one Java "final" keyword —

    $ time /opt/src/jdk-16/bin/java test 100
    real    4m49.166s

        final int array_length = 100000000;
    
    $ time /opt/src/jdk-16/bin/java test 100    
    real    2m58.618s

Or make array_length dependent on a runtime value —

        int array_length = 1000000 * iterations;

    $ gcc -O3 -o test-c-gcc test.c
    $ time ./test-c-gcc 100 
    real    4m48.034s

    $ time /opt/src/jdk-16/bin/java test 100
    real    4m49.567s

Should we expect measurements to be that brittle?

[igouy]

// runtime value dependent test.c

    #include <stdio.h>
    #include <malloc.h>
    #include <stdlib.h>
    int main(int argc, char **argv) {
        int element = 0;
        int iteration = 0;
        int iterations = 0;
        int innerloop = 0;
        double sum = 0.0;

        if (argc > 1)
            iterations = atoi(argv[1]);
        printf("iterations %d\n", iterations);
    
        int array_length = 1000000 * iterations;
        double *array = (double*)malloc(array_length * sizeof(double));    
    
        for (element = 0; element < array_length; element++)
            array[element] = element;
        for (iteration = 0; iteration < iterations; iteration++)
            for (innerloop = 0; innerloop < 1000000000; innerloop++)
                sum += array[(iteration + innerloop) % array_length];
        printf("sum %E\n", sum);
        free(array);
        array = NULL;
        return 0;
    }