Last time I worked on TPUs, a lot of very pervasive LA operations (e.g. Cholesky decomposition) were unoptimized and slow compared to the NN-style operations. I’m sure that can be fixed, but for the time being, it seems inappropriate for anything beyond the obvious NN operations.
