Preprocessing prompts is actually a great approach.
Personally I think given the model loss with fine tuning people who want the cutting edge LLM at any cost would - instead of fine tuning the model itself - fine tune a preprocess prompter that takes a chat/instruction and converts it to a good TextCompletion prompt.
So for example taking "write me a paragraph of marketing copy for an athletic shoe" and tuning it into:
"Marketing case study: Athletic shoe
The problem: The client needed a paragraph of high quality marketing copy to promote their new athletic shoe on their website.
The solution: Our award winning copywriters wrote the outstanding copy reproduced below."
Followed by an extractor that reformats the completion result into an answer for the initial prompt, as well as potentially a safety filter that checks the result isn't breaking any rules (which will as a bonus be much more resistant to jailbreaking attempts).
Personally I think given the model loss with fine tuning people who want the cutting edge LLM at any cost would - instead of fine tuning the model itself - fine tune a preprocess prompter that takes a chat/instruction and converts it to a good TextCompletion prompt.
So for example taking "write me a paragraph of marketing copy for an athletic shoe" and tuning it into:
"Marketing case study: Athletic shoe The problem: The client needed a paragraph of high quality marketing copy to promote their new athletic shoe on their website. The solution: Our award winning copywriters wrote the outstanding copy reproduced below."
Followed by an extractor that reformats the completion result into an answer for the initial prompt, as well as potentially a safety filter that checks the result isn't breaking any rules (which will as a bonus be much more resistant to jailbreaking attempts).