I prompted o1 with "analyze this problem word-by-word to ensure that you fully u...

hoosieree · 2024-10-11T16:30:57.000000Z

My classifier is not very accurate:

    is_trick(question)  # 50% accurate

To make the client happy, I improved it:

    is_trick(question, label)  # 100% accurate

But the client still isn't happy because if they already knew the label they wouldn't need the classifier!

...

If ChatGPT had "sense" your extra prompt should do nothing. The fact that adding the prompt changes the output should be a clue that nobody should ever trust an LLM anywhere correctness matters.

[edit]

I also tried the original question but followed-up with "is it possible that the doctor is the boy's father?"

ChatGPT said:

Yes, it's possible for the doctor to be the boy's father if there's a scenario where the boy has two fathers, such as being raised by a same-sex couple or having a biological father and a stepfather. The riddle primarily highlights the assumption about gender roles, but there are certainly other family dynamics that could make the statement true.