On: M2 Ultra can run 128 streams of Llama 2 7B in para...

[eurekin]

Interesting!

I'd have one use case for classification: user text (from a jira issue) mapped to the team responsible for the fix.

Can you share some tutorials? I only just managed to get this working on windows/cuda:

https://colab.research.google.com/drive/1vk8i01apaSp59GVV2yI...

It's been a royal pain to setup