ELIZA and PARRY proved this all the way back in 1966 and 1972. ELIZA was a silly AI hack which mostly just repeated back what users said to it, turning users' statements into questions in the style of Rogerian psychotherapy. People could tell it was fake. But PARRY used the same technique but pretended to be a crazy asshole, repeating users' statements in the style of angry accusations, and people thought it was real.
Take a simple AI technique, add some element to make emotion overwhelm analysis, and you pass the Turing Test (because the Turing Test was only ever meant to be a thought experiment). PARRY convinced a lot of people, and Peter Norvig's AI book in Lisp shows you how to write ELIZA in the first couple chapters. It's trivially easy to make an AI which is more persuasive than smart.
ELIZA and PARRY proved this all the way back in 1966 and 1972. ELIZA was a silly AI hack which mostly just repeated back what users said to it, turning users' statements into questions in the style of Rogerian psychotherapy. People could tell it was fake. But PARRY used the same technique but pretended to be a crazy asshole, repeating users' statements in the style of angry accusations, and people thought it was real.
Take a simple AI technique, add some element to make emotion overwhelm analysis, and you pass the Turing Test (because the Turing Test was only ever meant to be a thought experiment). PARRY convinced a lot of people, and Peter Norvig's AI book in Lisp shows you how to write ELIZA in the first couple chapters. It's trivially easy to make an AI which is more persuasive than smart.