On: Run Google Gemma 2 2B 100% Locally

[kozak]

I tried this code, and it cannot download the model from https://huggingface.co/google/gemma-2-2b-it-GGUF/resolve/mai... without authentication.

[esleightholm → kozak]

There are 2 things you will need to do.

1. You will need to request access to the model in HuggingFace and accept the license. Head to https://huggingface.co/google/gemma-2-2b-it-GGUF/tree/main and there should be an option to request access. This will be approved almost immediately and you will receive an email saying you've been granted access.

2. Create a User Access Token in HuggingFace to download the model. Visit https://huggingface.co/settings/tokens and create a new token. Then, set the token in your environment by running: `export HF_TOKEN=<your_huggingface_token>`

Hope that helps! Any further issues, feel free to reply to this comment and I'd be happy to help.