Have you tried Groq? We did a few days testing on replacing gpt4-turbo with it and, while incredibly fast, the results were horrible, even after a lot of specific prompt engineering. So many hallucinations and such. Our products all have to do with strict generation and software quality; it basically has to fill in the blanks but it was incredibly hit or miss. Some results came in within a second so even a few iterations beat gpt4 when correct, but some needed so many (that we quit) iterations that gpt4 beat it hands down.
Sure, so when openai moves to groq it might be something. Groq with the current models is impressive but doesn’t work for us is what I am saying. As I don’t actually have access to other models on groq, this is groq as it stands.
