On: Llama.cpp: Full CUDA GPU Acceleration

[mFixman]

I don't have a PC with a powerful GPU. What's the easiest way I can play with Llama on AWS, Google Cloud, or somebody else's computer?

[Tepix → mFixman]

You can play with Llama on your CPU. Depending on the model you use and the RAM you have available, the performance may be acceptable.

[anentropic → mFixman]

using llama.cpp

it runs on the CPU

this news story is that they are now extending GPU support to llama.cpp

[hospitalJail → mFixman]

Do you know about Oobabooga?

You can probably find a google colab link.

[mFixman → hospitalJail]

Is that like a LLM Stable Diffusion? Neat.