On: Ask HN: How are you extracting the best performance out of your RAG pipeline?

[imaravind]

@naveen99 That's awesome! But I'm curious how you went about data chunking, embedding etc.? Chose adhoc or applied some trial-and-error approach or something else?

[naveen99 → imaravind]

Yes generated embeddings using an LLM… no chunking. Tried a few of models. Clip and Imagebind didn’t work great for text, so went with a text focused one.

[chiccomagnus → naveen99]

By chance, have you tried preprocess.co for text extraction + chucking?