A journalism professor specializing in computer science tested seven generative AI chatbots each morning in September 2025, collecting 839 responses to a prompt requesting the day's top five Quebec news items. He found substantial sourcing and accuracy problems: 18% used non-news or fabricated sources, only 37% supplied complete URLs, summaries were fully accurate in 47% while 45% were partially accurate, and 111 items contained unsupported "generative conclusions."
Key Points
- 1Recorded 839 AI responses, with only 37% providing complete, legitimate URLs for verification
- 2Found 47% fully accurate summaries and 45% partially accurate, indicating pervasive factual reliability issues
- 3Showed 111 items with unsupported conclusions, implying misinformation risk for practitioners and readers
Scoring Rationale
Empirical, systematic testing yields high actionability and credibility; limited novelty and regional scope constrain broader impact.
Sources
Public references used for this report.
Practice with real Logistics & Shipping data
90 SQL & Python problems · 15 industry datasets
250 free problems · No credit card
See all Logistics & Shipping problems
