Researchllmhealthcareopenaimisinformation
Chatbots Underperform Search Engines For Medical Advice
9.3
A peer-reviewed study by researchers at the University of Melbourne, Harvard Medical School and other institutions finds that consumer AI chatbots underperform traditional search engines when laypeople seek medical advice. The study identifies two failure modes—insufficient user input and inaccurate model outputs—and warns this gap can increase misplaced trust and pose public health risks if chatbots replace search for health queries.
Key Points
- 1Finds that ChatGPT, Microsoft Copilot, and Google Gemini yielded poorer medical outcomes than search engines
- 2Identifies two failure modes: incomplete user input and models producing inaccurate, misleading medical responses
- 3Warns that authoritative-sounding chatbot answers may increase misplaced trust and risk delayed or improper care
Scoring Rationale
Strong, peer-reviewed multicenter evidence highlights major user-safety concerns; scope limited to lay-user scenarios not clinical deployment.
Sources
Public references used for this report.
Practice with real Health & Insurance data
90 SQL & Python problems · 15 industry datasets
Used by DS/ML engineers at top companies
250 free problems · No credit card
See all Health & Insurance problems
