This resonates with my recent experience using Bard (not the latest version of Gemini). It would produce something that initially seemed surprisingly good, but then when I actually tried to run it it turned out to be totally broken. I'd ask it to fix the error; it would magically do so but then be broken in a different way. It felt like pair programming with a junior programmer who just didn't quite get it.
This was just me interacting through text prompts. I could imagine some kind of more integrated solution where you can provide some basic test cases, and the system would run those cases through code proposed by the LLM could go a long way towards improving this.
For now it seems mainly useful as a way of getting a quick first draft of some code which I then have to fix up and get fully working myself.
This was just me interacting through text prompts. I could imagine some kind of more integrated solution where you can provide some basic test cases, and the system would run those cases through code proposed by the LLM could go a long way towards improving this.
For now it seems mainly useful as a way of getting a quick first draft of some code which I then have to fix up and get fully working myself.