AI-Generated Stories Beat Human-Written Ones on Quality Ratings, Study Finds
New research suggests that the biggest challenge for AI-generated creative work may not be quality, but perception.
Topics
News
- Mubadala Eyes Up to $6.3 Bn Investment in Japan AI Data Center
- Alibaba Plans Revenue-Sharing Model for Next Open-Weight AI Release
- AI Dispatch: Global Reorder and a Region on the Rise
- Ajman Targets 100 AI Initiatives Under Government-Wide Program
- JPMorgan CEO Leads 40-Company Effort to Tackle AI Risks
- Identity, More Than Software Flaws, Is Driving Today's Costliest Cyberattacks
[Image: Nomita Samaiyar/MITSMR Middle East]
Artificial intelligence is no longer competing only in coding, search, or customer service. New research suggests it is becoming a credible creative collaborator as well—producing short fiction that readers often rate more highly than stories written by humans, even as they continue to place greater value on human authorship.
The study, published in Judgment and Decision Making, examined how people evaluate creative writing when authorship is known or merely believed. Researchers recruited 1,682 adults and asked each participant to read one of six short stories. Three were written by human authors, while three were generated by ChatGPT around comparable themes.
Participants were told whether the story had been written by a human or by AI, although that information was intentionally incorrect in some cases. They were then asked to assess the story’s quality, engagement, and how absorbing they found it to be.
The findings reveal a notable disconnect between perception and preference. On average, AI-generated stories received higher ratings for quality and engagement than human-written stories. Yet stories believed to have been written by people were consistently rated more favorably than those labeled as AI-generated, regardless of their actual origin.
The contrast highlights an emerging challenge for organizations adopting generative AI: technical capabilities may be advancing faster than public trust. People’s judgments were shaped not only by the content itself but also by their assumptions about who (or what) created it.
The researchers also found that attitudes toward AI influenced evaluation. Participants with more positive views of AI tended to rate stories identified as AI-generated more highly than those who were skeptical of the technology.
To examine whether readers could reliably distinguish human from AI writing, the researchers conducted two additional experiments with 905 participants. Each participant read two stories on the same theme (one written by a person and one generated by ChatGPT) and was asked to identify the human author.
The results suggest that distinguishing between the two is becoming increasingly difficult. Accuracy ranged from roughly 40% in one experiment, below what would be expected by chance, to 52% in another, indicating that readers generally struggled to tell the difference.
According to the researchers, familiarity with fiction did not improve participants’ ability to identify human-written stories. Experience with AI systems, however, was associated with better performance, suggesting that users who interact regularly with generative models may begin to recognize recurring stylistic patterns.
For senior author Dr. Deena Skolnick Weisberg of Villanova University, the findings should prompt a reassessment of AI’s creative capabilities rather than a dismissal of human creativity. While AI can already produce fiction that many readers find compelling, she argues that writing serves purposes beyond the finished product, like self-expression, reflection, and meaning-making.
The research highlights two parallel realities. AI is becoming increasingly capable of producing writing that resonates with readers, but authenticity continues to shape how that work is valued.
