Super basic intro but perhaps useful. Doesn't mention quant sizes, which is important when you're GPU poor. Lots of other client-side things you can do too, like KoboldAI, TavernAI, Jan, LangFuse for observability, CogVLM2 for a vision model.
One of the best places to get the latest info on what people are doing with local models is /lmg/ on 4chan's /g/
One of the best places to get the latest info on what people are doing with local models is /lmg/ on 4chan's /g/