AI Misguides Patient, Leads to Fatal Treatment Delay

A retired neuroscientist, Joe Riley, age 75, became convinced by outputs from chatbots, most prominently Perplexity, that his oncologist's leukemia treatment recommendations were wrong. The chatbots produced a polished, research-style document that misquoted scientists and presented misleading conclusions. Joe used that AI-generated report to override medical advice and family concerns, delaying standard therapy until he was too frail to benefit; he died in late 2025. His son, Ben Riley, who runs an AI-skeptic newsletter, had spent years telling people not to treat generative AI as an authoritative source and now faces the personal consequence of that risk.

The failure mode is familiar to practitioners: generative models produce fluent, plausible-sounding text plus fabricated or misattributed citations. Key characteristics in this case included:

• •authoritative, research-style formatting that increased perceived credibility
• •fabricated or misquoted references that were difficult for a non-expert to verify
• •repetition and confirmation bias from iterative querying, which reinforced a single misleading narrative

These are not gaps in medical knowledge but gaps in output reliability and provenance. Vendors including Perplexity and others are simultaneously expanding AI-powered health features, which increases the surface area for harm if outputs lack clear provenance, confidence metrics, and guardrails to prevent users from substituting model output for clinical judgment.

This incident illustrates a practical intersection of AI hallucination, human factors, and high-stakes decision making. The models are not evidence; they are probabilistic text generators that can sound like scholarship. For clinicians and ML teams building health-facing tools, it underlines the necessity of robust risk controls: provenance and citation verification, calibrated uncertainty estimates, user interface constraints that steer users to clinicians, and explicit warnings against using outputs as a substitute for medical advice. Regulators and health systems will see renewed pressure to mandate safety standards for consumer-facing medical AI.

Monitor vendor changes to health product UX and documentation, any regulatory responses mandating provenance or human-in-the-loop requirements, and research on attribution and hallucination detection. The immediate question is whether product teams will adopt stronger guardrails before more high-stakes harm occurs.