
Microsoft OpenAI can write Python code based on function signature and comments - pimterry
https://www.youtube.com/watch?v=fZSFNUT6iY8
======
jepler

        def f() -> bool:
            """return True iff the Collatz Conjecture is true"""
    
        def g() -> bool:
            """return False iff OpenAI correctly completes this code"""
    

These tricks will only work for simple things, won't work for complex things
(but is likely to give obviously nonsense answers), but even worse there will
be a middle swamp where it'll write something complicated, plausible, and
wrong.

As usual, a video of 2 or 3 simple examples, curated beforehand so you're
confident they'll work, tells a different story.

Just for the sake of curiosity, I also asked GPT-2 whether it would work.
Here's what it said:

    
    
        Here's why this Python code-generating AI won't work. The other
        reason is that it will require you to make something that performs a
        large and complex program. This, after all, is what the function
        call is to do, a "simple function" that returns a function that
        accepts an integer number as its result. Let's say that I say
        "hello" in the code and I get the value by making a change. To do
        this, I divide the value by a few. Next, I increment the value by
        two. The process will fail, because I need a new function which will
        return a result, but in this case I get a value by incrementing the
        value by two. This gives me the answer that is most likely true. We
        need to know exactly which value we want for the computation and to
        implement ways to handle returning it.
    

You said it, GPT-2.

~~~
pimterry
> These tricks will only work for simple things, won't work for complex things

Even complex systems include a lot of simple code.

A practical tool based on this doesn't need to be able to generate large
working systems or solve complex problems hands-free. It just needs to be able
to provide autocompleteable suggestions for the simple code parts. It can do
so only when it's very confident it knows what you're looking for, and you'll
share a meaningful percentage of time off of all software development work.
Never write any boilerplate code for anything ever again.

Accuracy is likely to improve in that case too, since inline suggestions have
much more context than just a function declaration.

That's before you start thinking about running this in reverse, and using it
as AI-powered linting. If I write some code and the AI is very confident that
part of it isn't what it would suggest, flag it for examination. It'll only
catch simple mistakes, of course, but a lot of developers make a lot of simple
mistakes that they miss every day.

------
Jeff_Brown
More interesting to me is the capacity of Idris 2 to complete a function based
on type signatures -- because (when it works) you can be sure it got it's
correct.

