Hacker News new | past | comments | ask | show | jobs | submit login

I guess so, at least this is what people are reporting who have a lot of experience with language models, like janus (see link in sibling).

Though I should mention that mode collapse doesn't just come from supervised instruction tuning (which let the model reply to requests instead of treating them as completion prompts), but also from things like RLHF, which bias the model to give certain replies rather than others.




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: