On: Speeding up PyTorch inference on Apple devices with AI-generated Metal kernels

[syntaxing]

This is gonna be a silly question but what does “kernel” mean in this context. I thought it meant like a Linux kernel module but doesn’t seem to be?

[tuckerman → syntaxing]

A kernel is low level function that is going to run in parallel on your accelerator (hopefully efficiently). You will have various matmuls, convolutions, etc.

If you search CUDA kernel you’d find examples but the term was used in HPC before as well.

[drdrey → syntaxing]

https://en.wikipedia.org/wiki/Compute_kernel