On: How to Finetune GPT-Like Large Language Models on a Custom Dataset

[akiselev]

Try using InstructXL for embeddings. It’s got a more complex prompt structure for generating embeddings which might be more useful