Study Finds Readers Rate AI Stories Higher

The first experiment involved 1,682 adults aged 18 to 81. Participants read a short story and received either correct or incorrect information about whether a human or AI had written it, then rated its quality and how engaging they found it. The study used three human-authored fictional stories drawn from a literary journal or short-story collection, alongside three corresponding stories generated by ChatGPT, the paper reports.

Across that experiment, AI-generated stories scored higher on both writing quality and reader absorption. However, authorship labels also affected evaluations: stories believed to be human-written received higher ratings regardless of their actual origin. The combination meant AI stories labeled as human-written received the strongest evaluations.

Two subsequent experiments tested whether readers could distinguish the sources without an authorship label. In one experiment involving 424 participants, accuracy in identifying AI-generated material was 39.93%, below chance. In another involving 481 participants, accuracy was 51.97%, which the paper reports as no different from chance.

The Guardian's coverage characterizes the result as evidence that simpler AI prose can be easier to read and process, while cautioning that reader preference for such prose does not establish that human authors are obsolete.

The research concerns ratings of a small set of short stories under experimental conditions, rather than a comprehensive comparison of literary achievement, originality, or long-form narrative capability. It nevertheless isolates two issues relevant to generative-AI evaluation: perceived textual quality and provenance perception.

The finding adds evidence that unaided human detection is an unreliable control for AI-authorship assessment.

Outside the study, University of Birmingham creative-writing professor Luke Kennard, who was not involved in the research, told the Guardian that AI's capacity to produce coherent and enjoyable writing should be considered alongside the systems' training data and broader ethical and environmental costs. Those questions are separate from the study's reported measures of quality, absorption, and source detection.