I've been experimenting with using ChatGPT for coding and building up some usefu...

mobius1 · on May 18, 2023

This is awesome. I started messing around with it yesterday. I'm building a similar tool that interacts via Github issues and PRs, but being able to interact directly on the CLI is super cool. Some ideas (I may experiment with these on my own fork, and make a PR if I have success):

1. The main prompt is pretty long (https://github.com/paul-gauthier/aider/blob/main/aider/promp...), but I suspect that it could be abbreviated. I'm think the same criteria/restrictions could be enforced with fewer characters, enabling (slightly) larger files to be supported

E.g. "I want you to act as an expert software engineer and pair programmer." (69 chars) -> "Act as a software dev + pair programmer" (39 chars) - I suspect the LLM would behave similarly with these two phrases

2. It would be cool to be able to open "file sections" rather than entire files (still trying to figure out how to do this for my own tool). But the idea I had would be to support opening files like `path/to/main.go[123:200]` or `/path/to/main.go[func:FuncName]`

^the way I've gotten around this limitation with your tool is by copying functions/structures relevant to my request into a temporary file, then opening aider on the temporary file and making my request. Then I copy/paste the code generated to the actual file I want it in. It did a really good job, I'm just trying to figure out ways to reduce the number of manual steps for that

Anyway, great job, and I'm really excited to keep using this tool.

By the way, could you by any chance update the README with instructions for running directly from source rather than via pip install?

anotherpaulg · on May 27, 2023

Sorry, just noticed this reply.

I have explored the "line ranges" concept you describe a fair amount. I took another run at it again last week. I still haven't unlocked a formulation that GPT-4 can reliably work with.

It's really hard to get GPT-4 to stop trying to edit parts of the files that we haven't shown it yet. It frequently hallucinates what's in the missing sections and then happily starts editing it.

ilaksh · on May 15, 2023

Nice solution with the search and replace blocks to edit code. I spent quite awhile trying to get regular patch files for stuff to work consistently.

Then I switched to having it use sed which was better but still not totally consistent.

Can you add a license?

anotherpaulg · on May 15, 2023

I just added Apache 2.0. Thanks for the reminder, I've been meaning to add a license.

I tried `sed` as well, every variant of `diff` output, various json structured formats, etc. The edit block syntax I chose is modeled after the conflict resolution markup you get from a failed `git merge`. I find it's very helpful to ask GPT to output in a format that is already in popular use. It has probably seen plenty of examples in its training data.

Asking GPT to output edits really only works ~reliably in GPT-4. Even then, my parser has to be permissive and go with the flow. GPT-3.5 was a lost cause for outputting any editing syntax -- it only worked reliably by outputting the entire modified source file. Which was slow and wasted the small context window.

Both 3.5 and 4 seem to be able to think more clearly about the actual coding task when you allow them to use an output format they're comfortable with. If you require them to output a bespoke format, or try too hard to ask for "minimal" edits... the quality of the actual code changes suffers. The models are used to seeing uninterrupted blocks of code in plain text, and so they seem to do best if you choose an output format that embraces that.