Hacker News new | past | comments | ask | show | jobs | submit login

Agreed that others should implement it as well but coercing llama to output results with matching grammar needs work.



What kind of work? I've only given it a short try before moving to Ollama that doesn't have it, but it seemed to have worked there. (With ollama I need to use a retry system.)

edit: I researched a bit and apparently it can reduce performance, plus the streaming mode fails to report incorrect grammars. Overall these don't seem like deal-breakers.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: