Hacker News new | past | comments | ask | show | jobs | submit login

Taken directly from the copilot product page:

>Tests without the toil. Tests are the backbone of any robust software engineering project. Import a unit test package, and let GitHub Copilot suggest tests that match your implementation code.

Accompanied by a picture of copilot filling out the implementation of a unit test, based on the test's name.

The implication seems very clearly that it is more than a google search. According to the ads, it understands your code well enough to write the unit tests for it!




> According to the ads, it understands your code well enough to write the unit tests for it!

Yes, Github Copilot marketing has overstatements -- but it also has caveats about its limitations and creating errors.

In other words, the landing page has a mix of statements ... some hyping up the potential -- and some disclaimers to mention the reality and pitfalls.

Excerpt from that same Copilot landing page about it being wrong more often than right: https://copilot.github.com/

>How good is GitHub Copilot?

>We recently benchmarked against a set of Python functions that have good test coverage in open source repos. We blanked out the function bodies and asked GitHub Copilot to fill them in. The model got this right 43% of the time on the first try, and 57% of the time when allowed 10 attempts.

>Does GitHub Copilot write perfect code?

>No. GitHub Copilot tries to understand your intent and to generate the best code it can, but the code it suggests may not always work, or even make sense. While we are working hard to make GitHub Copilot better, code suggested by GitHub Copilot should be carefully tested, reviewed, and vetted, like any other code.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: