On: Scientists invented a fake disease. AI told people it was real

[simianwords]

You are pointing at something that is orthogonal to this paper. The LLM did not randomly recommend or bring this disease up to people - it merely assumed the disease was true when the preprint was pointed at it.

[simmerup → simianwords]

The LLM bought up the disease because some person put a fake journal in its training data.

If the person put their product as th definitive cure for the made up disease, the LLM probably would have mentioned that too.

> merely assumed the disease was true when the preprint was pointed at it.

What do you mean by preprint pointed at it? It being the disease?

[simianwords → simmerup]

> The LLM bought up the disease because some person put a fake journal in its training data.

This is not true - the model was not trained on this fake disease. It brought it up because it found it during real time search.

>What do you mean by preprint pointed at it? It being the disease?

On this I'm wrong - it turned out that the model brought up this disease even when not mentioning it explicitly.

[simmerup → simianwords]

Interesting, will be looking into RAG now. I assume Claude was retrained regularly but Opus for example was last trained August 2025. Way older than I thought it’d be