Either way GPT-2 to GPT-3 was a much bigger step and happened much quicker. What we see now are already after much fine tuning, testing with humans, data filtering etc, more of that will result in smaller and smaller improvements. There is so much money spent on these models that they have already tried so many things, so getting to the edge of the S curve happens much quicker.
