Hacker News new | past | comments | ask | show | jobs | submit login

There are many reasons, though I am not sure which they had in mind. One thing is that LLMs in general tend to do better when they can be more verbose in their output and sort of “think aloud” to reach an answer. Insisting on strict output format would rob it of the benefits (because it doesn’t just not emit but completely skips those stages, or else you’d be paying for those elided output tokens).



But then why would someone specify that the response has to be in a given JSON schema (by presence of the schema itself), but then also not care if it is actually using that schema (by specifying `strict` as `false`)? That is the use-case I can't wrap my head around.


Ah, good point. It honestly feels/looks like a hack to flip a switch on their end and they’re just bubbling it back up to the user.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: