On: The State of Transfer Learning in NLP

[make3]

there are some much smaller BERT models available on tf-hub and elsewhere if you wish.

most people also actually distill BERT to reduce the computation cost