Hacker News new | past | comments | ask | show | jobs | submit login
Microsoft OpenAI can write Python code based on function signature and comments (youtube.com)
6 points by pimterry 13 days ago | hide | past | web | favorite | 3 comments

    def f() -> bool:
        """return True iff the Collatz Conjecture is true"""

    def g() -> bool:
        """return False iff OpenAI correctly completes this code"""
These tricks will only work for simple things, won't work for complex things (but is likely to give obviously nonsense answers), but even worse there will be a middle swamp where it'll write something complicated, plausible, and wrong.

As usual, a video of 2 or 3 simple examples, curated beforehand so you're confident they'll work, tells a different story.

Just for the sake of curiosity, I also asked GPT-2 whether it would work. Here's what it said:

    Here's why this Python code-generating AI won't work. The other
    reason is that it will require you to make something that performs a
    large and complex program. This, after all, is what the function
    call is to do, a "simple function" that returns a function that
    accepts an integer number as its result. Let's say that I say
    "hello" in the code and I get the value by making a change. To do
    this, I divide the value by a few. Next, I increment the value by
    two. The process will fail, because I need a new function which will
    return a result, but in this case I get a value by incrementing the
    value by two. This gives me the answer that is most likely true. We
    need to know exactly which value we want for the computation and to
    implement ways to handle returning it.
You said it, GPT-2.

> These tricks will only work for simple things, won't work for complex things

Even complex systems include a lot of simple code.

A practical tool based on this doesn't need to be able to generate large working systems or solve complex problems hands-free. It just needs to be able to provide autocompleteable suggestions for the simple code parts. It can do so only when it's very confident it knows what you're looking for, and you'll share a meaningful percentage of time off of all software development work. Never write any boilerplate code for anything ever again.

Accuracy is likely to improve in that case too, since inline suggestions have much more context than just a function declaration.

That's before you start thinking about running this in reverse, and using it as AI-powered linting. If I write some code and the AI is very confident that part of it isn't what it would suggest, flag it for examination. It'll only catch simple mistakes, of course, but a lot of developers make a lot of simple mistakes that they miss every day.

More interesting to me is the capacity of Idris 2 to complete a function based on type signatures -- because (when it works) you can be sure it got it's correct.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact