Google-BIDMC study reports no safety interruptions in 98 AMIE visits

Researchers from Google and Beth Israel Deaconess Medical Center (BIDMC) evaluated AMIE, described as a research diagnostic AI chatbot, at BIDMC’s ambulatory primary care clinic. In the study, 98 patients consulted the system ahead of urgent-care visits. Google described the paper as its first-ever publication in The Lancet’s main journal. That placed AMIE in a pre-visit role in this study, with its interactions monitored by supervising physicians.

Supervising physicians watched the interactions as they took place, and no conversation required interruption under the study’s predefined safety criteria. This is a reported result about those monitored conversations and those criteria, not a conclusion that all patient-facing AI is safe in every setting. Google’s account says larger clinical trials are still needed to assess patient-facing AI at scale. For AI builders, the report offers evidence about a narrowly defined workflow—pre-visit consultation with clinician monitoring—rather than evidence of a system operating independently.

Clinicians said AMIE’s summaries helped them prepare for appointments in 75% of cases. They also reported that the system affected their approach to care in more than half of cases. Separately, AMIE’s differential diagnoses agreed with physicians’ final diagnoses 90% of the time. These are three distinct reported outcomes—preparation, influence on care, and diagnostic agreement—and should be read as separate measures.

Google framed the findings as an early indication that AI could assess patient needs before a doctor visit. It said the results point to potential gains in patient-physician relationships and relief for health workers, while also stressing that larger clinical trials are needed to evaluate patient-facing AI at scale. The evidence in this report is tied to 98 patients at BIDMC’s ambulatory clinic, so builders should not treat it as proof of performance across other settings or populations. One concrete implementation detail for builders is the combination of real-time physician oversight and summaries that clinicians said aided visit preparation. The study therefore offers a bounded example of a pre-visit workflow, not a general validation of clinical AI.