Well, depends how much time you're willing to invest.
A small model, i.e. a few hundred million params might take you a few days of training, while bigger ones (0.5-1B) would take you months probably.
But you can go for it until you reach the scale where the time tradeoff isn't worth it and use kaggle, colab or such free GPU resources to continue trianing it.
A small model, i.e. a few hundred million params might take you a few days of training, while bigger ones (0.5-1B) would take you months probably.
But you can go for it until you reach the scale where the time tradeoff isn't worth it and use kaggle, colab or such free GPU resources to continue trianing it.