On: LLMs Will Always Hallucinate, and We Need to Live ...

[Too]

Training with inappropriate data will make copy-paste equally bad with hallucinating. For example, fiction or sarcasm may be taken out of context and used as a serious output.

What's missing is sanity-checking the sources and the output.