On: My local model setup on an M4 Pro Mac Mini

[EagnaIonat]

> You could buy a $150 refurbished 16gb i5

Or I can use the Mac I already have.

Your example though, Ouch!

~8B Q4. That's around 5-10 tokens a second. Base M1 16GB mac would do 15-20 tokens a seconds. That's a 6 year old machine.

You do get what you pay for it seems.

[whatsThisBtn4 → EagnaIonat]

Oh go get a $700 Nvidia laptop.

Sorry local models are basically useless outside chat, I didn't even consider it.

[EagnaIonat → whatsThisBtn4]

You seem to be ill informed and only talking to yourself at this point.