And the scale of everything. GPT3 embedding vectors are around 12,000, vs 768 shown here.
I was curious and the 12k figure closely approximates the median synapse dimensionality of human neurons. Maybe we don’t need much more.
And the scale of everything. GPT3 embedding vectors are around 12,000, vs 768 shown here.
I was curious and the 12k figure closely approximates the median synapse dimensionality of human neurons. Maybe we don’t need much more.