On: TypeChat

[joefreeman]

> It's unfortunately easy to get a response that includes { "name": "grande latte" }

    type Item = {
        name: string;
        ...
        size?: string;

I'm not really following how this would avoid `name: "grande latte"`?

But then the example response:

    "size": 16

> This is pretty great!

Is it? It's not even returning the type being asked for?

I'm guessing this is more of a typo in the example, because otherwise this seems cool.

[DanRosenwasser → joefreeman]

Whoops - thanks for catching this. Earlier iterations of this blog post used an different schema where `size` had been accidentally specified as a `number`. While we changed the schema, we hadn't re-run the prompt. It should be fixed now!

[graypegg → joefreeman]

Their example here is really weak overall IMO. Like more than just that typo. You also probably wouldn’t want a “name” string field anyway. Like there’s nothing stoping you from receiving

    {
        name: “the brown one”,
        size: “the espresso cup”,
    … }

Like that’s just as bad as parsing the original string. You probably want big string union types for each one of those representing whatever known values you want, so the LLM can try and match them.

But now why would you want that to be locked into the type syntax? You probably want something more like Zod where you can use some runtime data to build up those union types.

You also want restrictions on the types too, like quantity should be a positive, non-fractional integer. Of course you can just validate the JSON values afterwards, but now the user gets two kinds of errors. One from the LLM which is fluent and human sounding, and the other which is a weird technical “oops! You provided a value that is too large for quantity” error.

The type syntax seems like the wrong place to describe this stuff.

[mynameisvlad → joefreeman]

I feel like that's just a documentation bug. I'm guessing they changed from number of ounces to canonical size late in the drafting of the announcement and forgot to change the output value to match.

There would be no way for a system to map "grande" to 16 based on the code provided, and 16 does not seem to be used anywhere else.

[hirsin → joefreeman]

The rest of the paragraph discusses "what happens when it ignores type?", so I think that's where they were going with that?