On: Things we learned about LLMs in 2024

[refulgentis]

For context, I got M2 Max MBP, 64 GB shared RAM, bought it March 2023 for $5-6K.

  Llama 3.2 1.0B - 650 t/s
  Phi 3.5   3.8B - 60 t/s.
  Llama 3.1 8.0B - 37 t/s.
  Mixtral  14.0B - 24 t/s.

Full GPU acceleration, using llama.cpp, just like LM Studio.