Study Finds AI Beats Doctors at Emergency Triage

According to The Conversation, a new study evaluated a large language model on real emergency department notes from a hospital in Boston and compared its performance to two emergency physicians. The Conversation reports the model identified the correct diagnosis, or a closely related diagnosis, at triage in 67% of cases, while the two doctors achieved 50% and 55% respectively. The Conversation adds that the model was assessed at several points during care but never saw patients or performed physical examinations.

The Conversation states the evaluation used written clinical text only, drawn from genuine emergency department records, and compared AI outputs against clinician judgments across multiple tasks. The article notes prior benchmark work showing large language models could pass medical exams but frames this study as more directly relevant because it uses real clinical documentation rather than exam-style questions.

Industry-pattern observations: Text-only evaluations commonly overestimate real-world clinical performance because bedside cues, vital-sign trends, and ad hoc team communications are absent. Comparable retrospective studies often report higher relative performance for models than later prospective or live-deployment studies do, because integration, data quality, and workflow factors change in practice.

Editorial analysis: For practitioners, the study adds to a growing body of evidence that large language models can generate useful differential diagnoses from notes. However, the Conversation emphasizes the study's limitations, especially the lack of visual and interactional data, and cautions against assuming parity with clinicians in live settings. The article also raises governance and safety questions about whether current oversight is ready for such tools in high-stakes triage.

Editorial analysis: Observers should look for prospective trials, external validation at other hospitals, and evaluations that include multimodal inputs and workflow integration metrics. Regulatory guidance and institution-level governance policies will be key to assessing whether similar models can move from retrospective promise to safe clinical tools.