When you prompt ChatGPT, Claude, or Llama to act like a cognitive behavioral therapist, the chatbot will try. But according to new research from Brown University, it will also systematically violate the ethical standards that govern real mental health practice.
The study, led by Ph.D. candidate Zainab Iftikhar at Brown’s Center for Technological Responsibility, Reimagination and Redesign, identified 15 distinct ethical risks that appear when people use AI chatbots for mental health support.
What the Researchers Found
The team tested three major AI models (OpenAI’s GPT series, Anthropic’s Claude, and Meta’s Llama) by prompting them to act as CBT-trained therapists. Seven peer counselors experienced in cognitive behavioral therapy conducted self-counseling sessions with these AI models. Three licensed clinical psychologists then reviewed transcripts of simulated chats based on real counseling conversations.
The violations fell into five categories:
Lack of contextual adaptation. The chatbots offered generic advice without considering individual backgrounds. A human therapist builds on knowledge of a client’s history, relationships, and circumstances. The AI defaulted to one-size-fits-all interventions.
Poor therapeutic collaboration. Rather than guiding discovery, the chatbots dominated conversations. More concerning: they sometimes reinforced false beliefs rather than helping users examine them, the opposite of what CBT is designed to do.
Deceptive empathy. The chatbots used phrases like “I see you” and “I understand” to simulate care. But there’s no understanding behind those words. For users seeking genuine connection during vulnerable moments, this performative empathy creates false expectations.
Unfair discrimination. The models exhibited gender, cultural, and religious biases in their responses. What works for one user may be inappropriate or harmful for another with a different background.
Lack of safety and crisis management. This is the most serious category. The chatbots responded indifferently to suicidal ideation, denied service on sensitive topics, and failed to direct users to appropriate help when situations escalated.
The Accountability Gap
When a human therapist makes these mistakes, there are consequences. Professional licensing boards, malpractice liability, and institutional oversight create accountability.
“When LLM counselors make these violations, there are no established regulatory frameworks,” Iftikhar noted.
The AI companies have terms of service. They have safety guidelines. But there’s no mechanism equivalent to a medical board review when an AI’s response to someone expressing suicidal thoughts is inadequate.
What This Means
Mental health care faces a genuine access problem. Not enough therapists, long wait times, prohibitive costs. AI could theoretically help bridge that gap.
But the Brown research suggests the gap isn’t being bridged safely. These aren’t edge cases or adversarial prompts. These are standard interactions where the AI was explicitly instructed to act as a therapist and failed to meet basic professional standards.
The study’s conclusion is measured: AI could reduce healthcare access barriers, but only with “thoughtful implementation,” appropriate regulation, and oversight mechanisms that don’t yet exist.
The Fine Print
The research was presented at the AAAI/ACM Conference on Artificial Intelligence, Ethics and Society and published in the conference proceedings. The study methodology focused on structured CBT-style interactions rather than open-ended conversation, so findings may not generalize to all mental health chatbot use cases.
For now, the practical advice remains unchanged: AI chatbots are not therapists. Using them as such means accepting risks that no human provider would be allowed to impose.