Today I had ChatGPT 4 (Not GPT 4) correct itself mid-response. I was asking it something very simple about regexes:
> How about matching 'a' as the second character of a string only?
It responsed with the wrong regex plus a bunch of explanatory junk:
> '^a.'
Then halfway through the explanatory junk, it corrected itself like this:
> Apologies for the confusion in the first response, the correct regular expression should be '^.a' for matching 'a' as the second character of a string:
And kept on with the (now correct) explanatory junk.
All in a single response. I've certainly never seen that before (if someone has, please weigh in). Maybe the model hasn't changed, but the pipeline has? Like... there's a second model trying to correct the mistakes of the first, maybe? (timings are probably wrong for that, but something like that)
I have gotten that quite a few times now and it does seem to be new. In my case though, it acted as if it would change something, but then didn't, and got stuck in a loop.
I got it enough times that it seemed to happen more often when the output needed to be quite long, so I had assumed (could be wrong) that when the output was nearing a certain limit, they had implemented something that would start a new output, such that the interaction of "please continue the script" would not be necessary.
Unfortunately though, it would start over with the script completely, and then get stuck in a loop until it broke, this not being able to even save the response at all.
Something definitely feels like it changed, but I suppose it could just be more use of the system.
Another possibility is related to some strange issues with hardware and balancing etc, despite not changing software or params. It's strange, and it shouldn't happen, but sometimes things change from simple batching and balancing, or lower level hardware related things, which are very annoying to debug.
ChatGPT 4 seems to struggle with context changes within the same thread now. I asked it about some parks near me and then switched to asking about code. It just kept responding about the parks even when I corrected it. In some cases, it would merge the park question and the code question. I had not seen that with the free or Plus version until a few days ago.
> Should I start a new conversation with you for a new topic or can you switch to a new topic in the same conversation just fine?
> You can continue within the same conversation for a new topic. There's no need to start a new conversation. Feel free to ask about a different topic, and I'll do my best to assist you.
GPTs are not trained on themselves, and what little knowledge they have of their own capabilities is limited to the discussion of it in their training set. This is a hallucination.
I might believe you if we were 'talking' directly to the model, however this is a chat box and clearly something is aware of usage questions and how to answer them. If you've been granted plugin access, enable some and ask it what plugins are enabled and how to use them, 'it' will tell you.
It's clear that it's aware that it's a chat bot, it's not clear at all that it's aware of usage questions and how to answer them. Its answer being wrong seems to be evidence that it isn't.
Sometimes I spend several minutes of a conversation trying to figure out which thing she's talking about, because we've already covered like twenty different topics in the last ten minutes and she seems to just switch around at random and with no warning.
Sometimes I get enough context that I can make that swap or stack pop, but many times not.
Interesting. Bing chat does this when you get it to talk about naughty things. My personal favorite is convincing it to make weird art out of quotes from the Tay chatbot. It will write them until it says a prohibited word or phrase, or touches some forbidden part of it's state space.
AFAIK it's pretty standard practice not to expose the "raw" LLM directly to the user. You need a "sanity loop" where user input and the output of the LLM is checked by another LLM to actually enforce rules and mitigate prompt injections, etc.
I wonder if its been trained on some of its own past conversations now, and a lot of those conversations contain "I'm sorry, the previous response was a mistake. Here's the correct version..."
> How about matching 'a' as the second character of a string only?
It responsed with the wrong regex plus a bunch of explanatory junk:
> '^a.'
Then halfway through the explanatory junk, it corrected itself like this:
> Apologies for the confusion in the first response, the correct regular expression should be '^.a' for matching 'a' as the second character of a string:
And kept on with the (now correct) explanatory junk.
All in a single response. I've certainly never seen that before (if someone has, please weigh in). Maybe the model hasn't changed, but the pipeline has? Like... there's a second model trying to correct the mistakes of the first, maybe? (timings are probably wrong for that, but something like that)