It does make some sense given they're positioning it as alternative to the normal way you'd implement such output shape, which is to slap a prompt on a frontier LLM and maybe run it in "constrained output" mode if you like things fancy. Against that use case, the "no hallucinations" and parallelism and cost claims all sound legitimate and useful -- and similarly, "but we could do that with BERT two years ago" does not.
They do actually admit that about constrained decoding somewhere in the docs. They argue it’s useless in practice because when the constraints actually kick in it harms the output too much and that it’s better to just error and retry in those cases.
That does align with my experience, though we’re not using anything close to frontier for these sort of tasks.
I am interested if it can actually improve on that. As an engineer i like the elegance of guaranteed output, but the retry works pretty well in practice.
