On: Structured Outputs in the API

[blackcat201]

Do beware on some reasoning task, our recent work[0] actually found it may cause some performance degradation as well as possible reasoning weakening in JSON. I really hope they fix this in the latest GPT-4o version.

[0] https://arxiv.org/abs/2408.02442

[kiratp → blackcat201]

Thank you! This confirms my intuition!

Structured generation seems counter to every other signal we have that chain of thought etc improves performance.