Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm still in awe that a local 3090 gpu was able to run the qwen3 coder instruct 30b-a3b exl3 q6 and...

Was able to create a sample page, tried starting a server, recognising a leftover server was running, killing it (and forced a prompt for my permission), retrying and finding out it's ip for me to open in the browser.

This isn't a demo anymore. That's actually very useful help for interns/juniors already.





How did you do your setup ?, right now the only way i know how to run LLM is through LM studio.

Using tabbyApi and exLlamav2.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: