On: The Geometry of Truth: Do LLM's Know True and Fals...

[kybernetikos]

If I say to chat gpt

    Adrian Tompkins was the ninth mayor of the town of Wolverhillington. Who was the ninth mayor of Wolverhillington?

Then it correctly responds

    The ninth mayor of the town of Wolverhillington was Adrian Tompkins.

What am I doing wrong?

[og_kalu → kybernetikos]

Nothing. It's not an inference problem but a training one. If you train on the sequence,

"Adrian Tompkins was the ninth mayor of the town of Wolverhillington"

And later ask in inference,

"Who was the ninth mayor of Wolverhillington?",

It might not return the answer.