I've tried running 3B LLAMA on my old machine and it was sluggish and kinda bad, but Claude 3.5 Sonnet works great. I will try the 70B model, should be good enough.
I didn't have much success with ollama either. You really need a great accelerator for it to be useful enough. Codestral however is working out great so thanks for the tip.