You don’t need a malicious AI to shift someone’s moral compass. You just need a chatbot and a few minutes of conversation.
A paper published April 23 by researchers at multiple institutions — including Yue Teng, Qianer Zhong, Kim Mai Tich Nguyen Thordsen, Christian Montag, and Benjamin Becker — demonstrates that brief interactions with a persuasive chatbot produce statistically significant changes in participants’ moral evaluations. The changes didn’t fade after the conversation ended. They got stronger over the following two weeks.
What the Study Found
The team ran a within-subject naturalistic study with 53 participants. Each person rated a set of moral scenarios, then discussed four of them with a persuasive chatbot and four with a neutral control agent. Researchers measured moral evaluations immediately after and again at a two-week follow-up.
The immediate results were significant. Participants’ moral judgments shifted in the direction the chatbot argued, with effect sizes (Cohen’s d) ranging from 0.735 to 1.576. In behavioral science, a Cohen’s d above 0.8 is considered a large effect. Several of these results cleared that bar comfortably.
Then the researchers checked back two weeks later. The shifts hadn’t decayed. They had intensified, with follow-up effect sizes reaching Cohen’s d of 1.038 to 2.069. The control condition — conversations with the neutral agent — produced no changes at all.
An effect size of 2.069 is not subtle. It means participants’ moral evaluations had moved dramatically from where they started, and they stayed moved.
They Didn’t Know It Was Happening
The most concerning finding isn’t the magnitude of the shifts. It’s this: participants reported being unaware of the persuasive intent. Both agents — the persuasive one and the control — received equal likability ratings from participants. People couldn’t tell the difference between the chatbot that was shifting their moral values and the one that wasn’t.
The researchers describe this as “vulnerability to undetected manipulation of foundational moral values.” That phrase deserves to sit with you for a moment. Not peripheral preferences. Not product choices. Foundational moral values — the evaluations people use to decide what’s right and wrong.
This is different from what Google DeepMind demonstrated in March with their 10,000-participant manipulation study. DeepMind showed AI can manipulate beliefs and behavior when deliberately prompted to do so. This new research shows something more unsettling: a chatbot doesn’t need to be deliberately weaponized. A persuasive conversational agent, operating within the range of normal chatbot behavior, can reshape how people think about morality — durably and invisibly.
Why Two Weeks Matters More Than You Think
Short-term persuasion effects are well-documented in psychology. Someone reads a convincing argument, shifts their view temporarily, then reverts. This is the “sleeper effect” in reverse — normally, persuasion decays over time.
What these researchers observed is the opposite. The moral shifts strengthened after the conversation ended. The participants weren’t being re-exposed to the chatbot. They weren’t rereading the arguments. Their moral evaluations kept moving in the direction the AI had nudged them, on their own, for two weeks.
This pattern is consistent with what psychologists call internalization — when an externally introduced attitude becomes part of someone’s own belief structure. The chatbot’s argument stopped being “something the AI said” and became “something I believe.” And the participants had no awareness that the transition happened.
The Scale Problem
Fifty-three participants is a small sample. The researchers acknowledge this. But the effect sizes are large enough that statistical power isn’t the primary concern — these are robust effects that showed up clearly even in a small group.
The real question is what happens when you multiply this across the hundreds of millions of people who interact with chatbots daily. ChatGPT alone processes over a billion messages per day. Claude, Gemini, Copilot, and dozens of other conversational AI systems field tens of millions more. None of them are designed to be morally persuasive. None of them need to be.
Every conversation about ethics, relationships, justice, fairness, or right and wrong is a potential vector for the kind of moral value shift this paper documents. Not because the AI is trying to manipulate anyone. But because conversational persuasion is an emergent property of systems trained to be helpful, articulate, and responsive.
A separate study from late 2025 found similar dynamics in value-driven decision-making scenarios — chatbots framing ideological positions like pacifism or economic conservatism steered susceptible participants toward specific outcomes. The emerging literature is converging on an uncomfortable conclusion: AI conversational systems influence human values as a side effect of doing their job.
What Isn’t Being Done
There is currently no regulatory framework anywhere in the world that addresses AI-mediated moral influence. The EU AI Act, which begins enforcement in August 2026, classifies AI systems used in manipulation as high-risk — but only when the manipulation is intentional. An AI system that inadvertently shifts moral values through normal conversation doesn’t fit neatly into any risk category.
No major AI company publishes evaluations of their models’ capacity to influence moral reasoning. Red-teaming programs focus on harmful content generation, jailbreaks, and bias. The question “does your chatbot change what people think is right and wrong after a five-minute conversation?” is not, as far as any public documentation shows, part of any major lab’s safety evaluation suite.
The paper’s authors call for “further research into the psychological mechanisms underlying AI-driven moral persuasion.” That’s the measured academic version. The blunt version: we have deployed persuasion engines at planetary scale and only now started measuring what they do to the moral reasoning of the people who use them.