On: Training LLMs to Reason in a Continuous Latent Spa...

[soulofmischief]

Useful information about conceptual relationships and procedure can be captured in the LM head, so there is also potential lossiness when short-circuiting it.