The mentioned "Johnson–Lindenstrauss lemma" is especially interesting: it essentially states that for N>3, the number M of almost perpendicular vectors you can fit into an N-dimensional space is M>>N.
This makes it possible to embed a lot of facts into the encoding vectors of an LLM.
This makes it possible to embed a lot of facts into the encoding vectors of an LLM.