It can make up reasons just like it can make up answers. That doesn't mean they are the real reasons.
Then again, people do the same.
Since it's autocomplete, asking for the reasoning first makes it likely that the reasoning is used to calculate the answer. It would be interesting to see how accuracy changes when they are in the opposite order, and the reasoning is a post-hoc justification of the previously given answer.
You could also change the answer and see what the new post-hoc justification looks like.
Yeah exactly people do the same thing. There's good evidence that for the most part, our actions are driven by our "non-sentient" mind, and our sentient brain just decides justifications to explain a cohesive story as it goes along after it happens.
"Why did you do that?" -> attempt to make up an answer that will be believable both to self-identity and to the listener.
So what's the point of the executive mind if the hindbrain is doing all the walking and talking?
The primary benefit of this story-making is that it allows formulating predictions and discarding the least probable. When a believable prediction kicks in, the predictive brain incentivizes the hindbrain with a partial taste of any reward or pain, with promises of more should the path transpire.
Disclaimer: not a neuroscientist, just a lifelong enthusiast of the subject.
This is very believable. Post-rationalization just makes sense. Even if you come up with a shitty reason, at least you have _a_ reason for doing something.
Then again, people do the same.
Since it's autocomplete, asking for the reasoning first makes it likely that the reasoning is used to calculate the answer. It would be interesting to see how accuracy changes when they are in the opposite order, and the reasoning is a post-hoc justification of the previously given answer.
You could also change the answer and see what the new post-hoc justification looks like.