There's an interesting section that talks about what you want from a pair programmer - questioning your assumptions, spotting errors, debating design with you, etc, and how Copilot doesn't do this, but instead just spits out the code it thinks you want. That made me think that, when using Copilot, the human is actually the copilot. You're the one spotting bugs, picking different designs, and so on.
I also think the discussion around searching versus autocomplete is interesting. What we need is something that autogenerates a Google search based on your code context and shows you some promising results. I'm imagining a pane in VS Code getting populated with Stack Overflow answers whenever I slow down.
For what it’s worth, Copilot is not being marketed as anything else than code suggestion software - an alternative to google search, perhaps more of a “I’m feeling lucky” without having to actually type the search term. Knowing how to succinctly express a problem is half the problem after all.
It's called "Copilot" and the tagline is "Your AI pair programmer." It's obviously being marketed as more than code suggestion software. It's being marketed as an AI pair programmer.
I agree that it is basically just code suggestion software, or like a big autocomplete. But it's definitely being marketed as more.
This is very true, and much like Tesla getting away with it for years (up until recently), GitHub will get away with this for a while as well. And so will the media decrying shit like "Is this the end of developers?!?"
> And so will the media decrying shit like "Is this the end of developers?!?"
Yeah, CoPilot already replaced my job. I used to be the office copypasta, going around to my coworkers desk all day long to fix their problems by googling them, then pasting in the first snippet to pop up. It didn't matter if the language matched or not, it was more the thought that counted. Now I've had to get a job as a loan rejection stamper at my local bank. It's not nearly as fun or exciting, but at least my effort makes a difference now. A big difference.
the biggest concern and reason to pump the brakes on copilot is the unresolved elephant in the room: the growing consensus that it violates FOSS licenses.
>Tests without the toil. Tests are the backbone of any robust software engineering project. Import a unit test package, and let GitHub Copilot suggest tests that match your implementation code.
Accompanied by a picture of copilot filling out the implementation of a unit test, based on the test's name.
The implication seems very clearly that it is more than a google search. According to the ads, it understands your code well enough to write the unit tests for it!
> According to the ads, it understands your code well enough to write the unit tests for it!
Yes, Github Copilot marketing has overstatements -- but it also has caveats about its limitations and creating errors.
In other words, the landing page has a mix of statements ... some hyping up the potential -- and some disclaimers to mention the reality and pitfalls.
Excerpt from that same Copilot landing page about it being wrong more often than right: https://copilot.github.com/
>How good is GitHub Copilot?
>We recently benchmarked against a set of Python functions that have good test coverage in open source repos. We blanked out the function bodies and asked GitHub Copilot to fill them in. The model got this right 43% of the time on the first try, and 57% of the time when allowed 10 attempts.
>Does GitHub Copilot write perfect code?
>No. GitHub Copilot tries to understand your intent and to generate the best code it can, but the code it suggests may not always work, or even make sense. While we are working hard to make GitHub Copilot better, code suggested by GitHub Copilot should be carefully tested, reviewed, and vetted, like any other code.
> I'm imagining a pane in VS Code getting populated with Stack Overflow answers whenever I slow down.
Right but realize how much of a marginal gain we're fighting for here. You can already do this with a separate window and some extra clicking and typing. Saving a little clicking and typing isn't a huge win, unless you're also willing to argue the most valuable work a SWE does is the clicking and typing.
This whole thing feels ultimately like a PR spot that we're all tripping over ourselves to participate with. Its the SWE equivalent of getting all excited about how self-driving is just around the corner TM. It's a premature waste of energy until real advancements are made.
I think the utility is similar to instant search on Google. I get very little by Google autocompleting the term I'm trying to type. I get quite a bit by Google suggesting things I didn't even know to search for.
The automatic SO search pane I was describing might give the same benefits. Most of the time maybe it would be useless. Sometimes it would show what I knew I wanted and save me the trivial effort of a search. The real gain would come from when it hit something that I should have asked but didn't think to. Then, I'd have a valuable insight I never would've had otherwise sitting in some pane ready for me.
Maybe it's just my imagination - but that's how I think it could be good.
I've been using TabNine for a while and found it very helpful. It really does seem like more of a hyper-intelligent autocomplete than CoPilot which feels like it wants to write it for you.
The thing I love the most about tabnine is how it learns on your own code base. So if you have particular naming schemes or weird idioms, it eventually picks up on them.
I've installed it a few weeks ago but haven't found it very useful yet. May I ask, which language(s) do you use it for and was it worth paying (I paid).
Python, works quite well, golang works super well. C++ and JS are reasonable. Rust, you might as well turn it off. The more popular the language overall, the better it does. Swift and Kotlin might have too little to have refined the model on.
Another thing: I was using the free version in beta so it was effectively the full blown version. It worked super well. I changed jobs, changed hardware, and haven't set up my key yet (hopefully still grandfathered in) but it's felt not as good - the output is a lot shorter, for one. I would likely get the paid version if I lose my beta key. But it's weird that you have the paid version and it's not useful.
It's also possible they nerfed the models over time in the name of some other tradeoff, like cpu or memory. May still be able to configure it. But I distinctly remember it spitting out entire lines or chunks of code, exactly as I would have written them.
It works best when you are writing stuff similar to your own codebase. Perhaps try vendoring some Swift or Kotlin in your project dir to get the engine to fine-tune on it?
I also think the discussion around searching versus autocomplete is interesting. What we need is something that autogenerates a Google search based on your code context and shows you some promising results. I'm imagining a pane in VS Code getting populated with Stack Overflow answers whenever I slow down.