Chatbots Provide Inaccurate Guidance For Medical Decisions

In a recent study, researchers tested widely available large language model chatbots with members of the public on common medical scenarios and found striking results. Participants using chatbots were less likely to identify correct conditions and no better at choosing appropriate care settings than controls. The study shows models retain medical knowledge in isolation but fail in real-world human-machine interactions, urging policymakers to require real-world evaluations.