On: Optimizing AI Inference at Character.ai

[janalsncm]

> we implemented customized int8 kernels for matrix multiplications and attention

I would be curious how this differs from [1] which is supported in Huggingface’s transformers library.

[1] https://arxiv.org/abs/2208.07339