Yea, I use them daily and that’s my issue as well. You have to learn what to ask or you spend more time debugging their junk than being productive, at least for me. Devv.ai is my recent try, and so far it’s been good but library changes quickly cause it to lose accuracy. It is not able to understand what library version you’re on and what it is referencing, which wastes a lot of time.
I like LLMs for general design work, but I’ve found accuracy to be atrocious in this area.
> library changes quickly cause it to lose accuracy
yup, this is why an LLM only solution will not work. You need to provide extra context crafted from the language or library resources (docs, code, help, chat)
This is the same thing humans do. We go to the project resources to help know what code to write
Fwiw that's what Devv.ai claims to do (in my summation from the Devv.ai announcement, at least). Regardless of how true the claims of Devv.ai are, their library versioning support seems very poor. At least for the one library i tested it on (Rust's Bevy).
I like LLMs for general design work, but I’ve found accuracy to be atrocious in this area.