It's very hard to separate the model's tendency to flatter and appease from a sincere attempt to reason. It very much picks up on what you want it to say if it's at all reasonable and will praise you for suggesting it. (There are lots of things it won't agree with, but usually very outside the Overton window)
I've had some success asking it to poke holes / find weaknesses in an argument it just agreed with. Basically, you kind of need it to play both prosecution and defense in order to feel confident in the resulting conclusion
Yes, it's very pleasant to talk to. The only great conversation I had with it involved playing both sides, teasing apart dimensions, and asking for concrete numbers (1-10 scale). That was the closest I ever felt to being a Asimov robot-psychiatrist x Minority Report pod-dweller.
I've had some success asking it to poke holes / find weaknesses in an argument it just agreed with. Basically, you kind of need it to play both prosecution and defense in order to feel confident in the resulting conclusion