A new report from Stanford and Harvard researchers finds a wide gap between how clinical AI performs in controlled studies and what holds up in everyday care. The State of Clinical AI 2026, released in January 2026 by ARISE, a Stanford-Harvard research network, reviews the most influential clinical AI studies published in 2025. It is led by Peter Brodeur, Ethan Goh, Adam Rodman and Jonathan H. Chen.
The scale of deployment is not in doubt. According to the report, more than 1,200 AI-enabled tools have been cleared by the FDA, and only a minority of them underwent peer-reviewed evaluation. Of 691 FDA-cleared AI/ML devices from 1995 to 2023, more than 95% went through the 510(k) pathway, which rests on equivalence to existing devices, and fewer than 1% of device summaries reported patient outcomes.
Lab Performance Doesn’t Transfer
A systematic review covered in the report looked at 519 studies of large language models in medicine and found that they mostly tested medical knowledge, often with exam-style questions. Only 5% used real patient data. That matters, because much of clinical work looks nothing like answering exam questions.
In one experiment (Bedi et al., JAMA Network Open, 2025), researchers changed standard medical multiple-choice questions so that the correct answer became “none of the other answers.” The clinical reasoning required did not change. Accuracy dropped across leading models, by between 9% and 38% according to the report, or in Stanford Medicine’s summary, “in some cases by more than a third.”
Performance also fell when models had to ask follow-up questions, manage incomplete information, or revise decisions as new details emerged. In one study summarized in the report, GPT-4’s diagnostic accuracy fell from 0.82 to 0.63 when static case vignettes were turned into conversations. On tests of reasoning under uncertainty, AI systems performed closer to medical students than to experienced physicians and tended to commit strongly to an answer even when ambiguity was high.
Workflow Design Matters As Much As Accuracy
The report’s fourth theme is that “human computer workflow design is as important as model capabilities.” Humans working with AI often outperform humans alone, but the report sees much room for improvement in workflow design and in training clinicians on failure modes, to limit automation bias and deskilling.
The risks are concrete. In several studies, clinicians followed incorrect AI recommendations even when the errors were detectable, and ended up making worse decisions than they would have without AI (Qazi et al., 2025). Other research raised concerns about reduced vigilance after prolonged AI use in procedural tasks (Budzyn et al., 2025).
The report also notes that tools are moving faster than the evidence. Stanford Medicine’s write-up points out that OpenEvidence, an AI evidence platform for clinicians, has become a dominant player at the point of care, “underscoring the fact that doctors are often bypassing traditional IT gatekeepers to use AI in clinical care.”
What Actually Works
Not everything in the report is discouraging. Several areas showed consistent value:
In one hospital-based study cited by Stanford Medicine, a model trained on continuous wearable vital signs predicted patient deterioration up to 8 to 24 hours before standard hospital alerts. AI-derived biological age predictions outperformed traditional measures like frailty scores for predicting mortality.
In Germany, radiologists who could optionally consult an AI system detected more breast cancers without increasing false alarms (Eisemann et al., Nature Medicine, 2025), an example of AI supporting rather than replacing clinical judgment. In Kenya, Penda Health and OpenAI deployed a background AI system to review urgent care visits, which reduced diagnostic and treatment errors across tens of thousands of patients (Korom et al., 2025).
The common thread: AI working alongside clinicians, in well-designed workflows, with clear escalation pathways.
What This Means
The report makes a case for recalibrating expectations. As Stanford Medicine’s write-up puts it: “What this report makes clear is that the next phase will not be driven by newer models alone.” For 2026 the report calls for evaluating models in prospective and post-deployment real-world settings, giving human-computer interaction design as much weight as primary outcomes in clinical decision support trials, and measuring uncertainty, bias and harm explicitly.
In other words, the priority is evidence from real care and integration into real workflows, rather than higher benchmark scores.
Correction (2026-10-11): an earlier version of this article included a “less than 15%” usage figure, a “shadow AI” finding, a nearly 20% decline in sepsis models, an anonymous resident’s quote and two quotes credited to a “Dr. Sarah Chen”. None of these appear in the report or in Stanford Medicine’s summary; they came from a secondary write-up whose quoted researcher is not among the report’s authors. They have been removed.