Skip to the content

The Court of Public Prose

In re Dead Writing Theory

Case No.
DWT-007
Matter
Better stories, more alike
Filed
Verdict
Splitverdict

The stories grew alike. Each one got better.

For the prosecution the stories grew more alike · P-10For the defence each story got better · D-11
The question
Does access to generative AI ideas make stories more creative, and does it make different writers' stories more alike?
The sample
293 writers wrote eight-sentence stories with no AI help, one GPT-4 story idea or up to five; 600 evaluators made 3,519 evaluations; similarity measured with text embeddings
Filed
The study
Generative AI enhances individual creativity but reduces the collective diversity of novel content Anil R. Doshi, Oliver P. Hauser. UCL School of Management; University of Exeter. Science Advances, 2024.

§ IFindings of fact

What the study found.

  1. With up to five AI ideas, novelty rose 8.1% over writers without generative AI access.
  2. With up to five AI ideas, usefulness rose 9.0%.
  3. Less creative writers gained most: 10 to 11% on creativity and 22 to 26% on how enjoyable and well written the story is.
  4. The increase in similarity from one or five AI ideas was 10.7% and 8.9% of the total range.
  5. The authors: "With generative AI, writers are individually better off, but collectively a narrower scope of novel content is produced."

§ IICaveats for the jury

What this study cannot tell the court.

  • A lab experiment with UK participants, short fiction and early GPT-4.
  • Writers got AI ideas, not AI-written text.
  • The similarity effect is modest: 10.7% and 8.9% of the total range.

§ IVThe exhibits entered in this case

Prosecution v. Defence

Any tally counts the evidence items below; it is not a measure of truth.

Evidence for the theory

For the prosecution

1 exhibit
  1. Exhibit P-10Case No. DWT-007

    10.7%

    of the total similarity range, with one AI idea

    Stories written with AI ideas grew more alike

    8.9% with up to five AI ideas. 293 writers in a preregistered experiment; similarity measured with text embeddings.

    Entered for the prosecution · Doshi and Hauser, Science Advances, 2024 · third-party study

Evidence against the theory

For the defence

1 exhibit
  1. Exhibit D-11Case No. DWT-007

    8.1%

    rise in novelty with up to five AI ideas

    Each story got better with AI ideas

    Usefulness rose 9.0%. Less creative writers gained most: 10 to 11% on creativity, 22 to 26% on how enjoyable and well written the story is.

    Entered for the defence · Doshi and Hauser, Science Advances, 2024 · third-party study

§ VThe finding

The finding

In a preregistered experiment, AI story ideas made each story better and the stories as a group more alike: individually better off, collectively narrower.
Split verdictCase No. DWT-007 · Better stories, more alike
Read the original studyDoshi and Hauser, Science Advances, 2024. Not our study.

§ VIQuestions for the court

Asked and answered.

Does AI make writing more alike?

In this experiment, yes, modestly. Doshi and Hauser (Science Advances, 2024) found the increase in similarity between stories from one or five AI ideas was 10.7% and 8.9% of the total range.

Does AI make writing better?

For each writer, yes. With up to five AI ideas, novelty rose 8.1% and usefulness 9.0%, and less creative writers gained most: 10 to 11% on creativity and 22 to 26% on how enjoyable and well written the story is.

Why is the verdict split?

Because both happened at once. In the authors' words, writers are individually better off, but collectively a narrower scope of novel content is produced.

How was it measured?

A preregistered online experiment: 293 writers wrote eight-sentence stories with no AI help, one GPT-4 idea or up to five, and 600 evaluators made 3,519 evaluations. It is a third-party study, not ours.