When Chatbots Sound More Human Than We Do

We often assume that artificial intelligence will eventually give itself away by sounding robotic, overly formal, or subtly "off." We trust our human intuition to spot a machine. But a recent experiment has flipped that assumption on its head, revealing a startling vulnerability in how we judge human sincerity and political discourse.

In a study published in the journal PLOS One, researchers decided to test the political mimicry skills of large language models. Led by Steffen Herbold at the University of Passau, the team tasked GPT-4 Turbo with impersonating 112 well-known UK figures—ranging from politicians to business leaders and journalists. They fed the AI basic Wikipedia biographies and asked it to answer real audience questions from the long-running BBC television show Question Time.

The researchers then presented 948 UK participants with two sets of answers: the exact, verbatim transcripts of what the real public figures said on television, and the AI-generated responses. The participants were asked to rate them blindly on authenticity, coherence, and relevance.

The outcome was highly unexpected. The clear majority of readers found the AI chatbots to be more coherent and relevant than the actual humans. Even more surprisingly, over half of the participants rated the AI's responses as more authentic than the real people's words.

How does a machine out-human a human? The secret lies in the messy reality of spontaneous speech. When real politicians speak off the cuff on live television, they stumble, hesitate, use filler words, and sometimes lose their train of thought. AI, however, doesn't get nervous. Drawing from its prompt, it synthesizes a perfectly structured conversational response that hits all the right thematic notes without the awkwardness of human improvisation. As Herbold noted, the comparison was somewhat unfair to the humans, but it perfectly highlights the polished deception AI is capable of.

When the participants were debriefed after the experiment, almost no one had suspected AI involvement. Many were shocked by the sophistication of the generated texts.

This presents a dire warning for the future of public discourse. If a chatbot can project more perceived authenticity than an elected official, the potential for generating persuasive, targeted misinformation is immense. The challenge moving forward isn't just about spotting visual deepfakes; it's about recognizing that our very metrics for trusting someone—coherence, relevance, and perceived authenticity—can now be reverse-engineered and weaponized by algorithms. As AI continues to master the art of the perfect response, we must learn to be more critical of the "flawless" voices we encounter online.

Key Points

• A PLOS One study found AI impersonations of public figures are rated as more authentic than the actual humans.
• GPT-4 Turbo outperformed real BBC 'Question Time' guests in coherence and relevance.
• The AI's advantage stems from its ability to generate polished, structured text, unlike humans who stumble during spontaneous speech.
• Participants were largely unable to detect the AI, highlighting a severe risk for political misinformation.

Why It Matters

If AI can consistently manufacture a stronger sense of authenticity than real human beings, malicious actors could easily exploit this to manipulate public opinion and erode trust in democratic institutions.

Sources:

• Scientists Asked AI to Impersonate 112 Public Figures. What Happened Next Is a ‘Dire’ Warning — 404 Media