Google's Gemma to run locally on Nvidia GPUs

[jug]

https://blogs.nvidia.com/blog/google-gemma-llm-rtx-ai-pc

[iAkashPaul]

I've quantised 2B & 7B Instruct models here https://huggingface.co/iAkashPaul/gemma-2b-it-gguf & https://huggingface.co/iAkashPaul/gemma-7b-it-gguf

[yogorenapan]

How does this compare to Mixtral (especially the Dolphin and Orca fine tunes)?

[not_really → yogorenapan]

Not great, even when compared to Mistral 7b

[ecmascript]

I'm assuming it won't run on my GPU though since I am a white man

[shapefrog → ecmascript]

I'm ok, I self identify as a black woman.

[badrabbit → ecmascript]

No, it is because you are too smart.