On: How to Finetune GPT-Like Large Language Models on a Custom Dataset

[CuriouslyC]

Fine tuning + context will outperform context alone, and it's cheaper to burn cycles fine tuning then use a smaller context than to use a larger context in production.

[Guillaume86 → CuriouslyC]

Fine tuning + same context will probably outperform context alone, but if you use a smaller context that does not seem to work that well as GP stated.