
GPT-What: Intro to the next big thing - jesalg
https://medium.com/@jesalg/gpt-what-intro-to-the-next-big-thing-ea1b92c4cbbc
======
dane-pgp
> What are those 175 billion parameters?

> According to Geoffrey Hinton — the godfather of ‘deep learning’ — parameter
> can be thought of as a synapse in a human brain ...

> They’re far, far smaller, the biggest ones we have right now have about a
> billion synapses. That’s about a million times smaller than the brain.

So his estimate is now off by a factor of about 100? How long will it take for
the next couple of 100-fold increases in parameter count?

