The literary world isn’t prepared for AI

Since 2012, the British literary magazine Granta has published the regional winners of the annual Commonwealth Short Story Prize. This year, however, there was something off about one of the selections for the prestigious award: It appears to have been written by AI.

The literary world isn’t prepared for AI

Three recent scandals say more about the publishing industry than they do about the quality of LLM-generated writing.

Jamir Nazir’s “The Serpent in the Grove” has many of the hallmarks of LLM-generated prose — mixed metaphors, anaphora, lists of threes. (I’m aware this, too, is a list of threes, and I promise I wrote this post myself, unassisted, as I write all things.) I’ll admit I was initially unconvinced by the allegation that Nazir’s story had been generated by AI. I know people are using LLMs to help them write — or to write for them, period — but I’ve been wary of the sort of AI paranoia that has developed among my peers. Em dashes are supposedly an AI tell, as are the word “delve” and the aforementioned lists. Short, punchy sentences, too, especially when used to punctuate a succession of longer sentences.

But I, a human being, have certainly used all of the above in my writing before. LLMs, after all, are trained on human writing. They mirror what they’ve been fed. And yet there’s an eerie quality to AI-generated prose. There’s something off about it, even if you can’t immediately tell what it is. If there are specific AI tells, and I’m using those tells right now, then how do you know I actually wrote this?

Nabeel S. Qureshi, a former visiting scholar of AI at the Mercatus Center at George Mason University, was among the first to point out the suspected use of AI in Nazir’s story. For Qureshi, the first two sentences were proof enough.

They say the grove still hums at noon. Not the bees’ neat industry or the clean rasp of cutlass on vibe, but a belly sound — as if the earth swallows a shout and holds it there.

“In general, AI writing has a particular rhythm that I’ve learned to pick up on which is hard to describe,” Qureshi told me via email. “There’s a spectrum from ‘AI helped me edit’ to ‘AI wrote this’ — this case reads to me like the latter end of that, though of course I don’t know for sure.”

The problem is that even when AI use is widely suspected, none of us really know for sure. In a statement, Commonwealth Foundation director-general Razmi Farook said the organization is aware of allegations regarding AI in the prizewinning stories, including Nazir’s. Farook said all writers who submitted work for the prize are asked whether they’re sending in original, unpublished work, and that all shortlisted writers have personally stated no AI was used to help them draft their stories.

“Until a sufficient tool or process to reliably detect the use of AI emerges that can also grapple with the challenges pertaining to working with unpublished fiction, the Foundation and the Commonwealth Short Story Prize must operate on the principle of trust,” Farook said.

Granta, for its part, ran Nazir’s story through Claude “and asked whether it was AI-generated,” publisher Sigrid Rausing said in a statement. “The response was long, concluding that it was ‘almost certainly not produced unaided by a human.’” But Claude isn’t an AI detection tool, it’s a chatbot powered by a large language model. Though AI tools are often better than human readers at detecting LLM-produced prose — or at least those that judge literary prizes — Granta’s statement implied that they had gone to the source to ask whether the story in question had indeed been produced by AI, which demonstrates that perhaps the magazine itself does not understand how AI works either.

“It may be that the judges have now awarded a prize to an instance of AI plagiarism — we don’t yet know, and perhaps we never will know,” Rausing said.

Publications are increasingly being tricked into running AI-generated stories, some of them “written” by “authors” who don’t actually exist. There was even suspicion that Nazir himself was fake — though author Kevin Jared Hosein, a previous Commonwealth Short Story Prize winner, confirmed that Nazir is a real person, and shared messages he recently exchanged with Nazir about the suspicions of AI use in his story. Nazir also published a poetry collection in 2018. Nazir did not respond to The Verge’s request for comment. In March, Hachette pulled the publication of Mia Ballard’s horror novel Shy Girl after its author was accused of using AI, though Ballard denied using it and blamed a for-hire editor.

There’s also the question of whether there’s any acceptable way for authors and journalists to use AI. LLM-generated prose is obviously verboten, but what about using AI for idea generation, or for research? What about AI transcription services? At what point does reliance on these tools mean the work is no longer your own? This week, Polish author Olga Tokarczuk admitted she uses AI to help with her creative process — the other end of the AI-use spectrum Qureshi mentioned, but alarming to readers who admired a writer who had won the Nobel Prize in Literature.

“I often simply throw into the machine an idea with the prompt: ‘Darling, how could we beautifully elaborate this?’” said Tokarczuk, who was awarded literature’s highest honor in 2018:

“Even though I know about its hallucinations and numerous factual errors in the fields of quantitative economics or factual data, I have to admit that in the fluid field of literary fiction, this technology is an asset with unbelievable leverage. At the same time, I feel an acute human grief over an era that is disappearing never to return. I’m heartbroken by the departure of traditional literature written in isolation over months, a work conceived in the mind of a single conscious individual. In all of this, I’m damn mournful for Balzac, Cioran, and the inimitable Nabokov, because in spite of my enthusiasm, I don’t believe that any modern chat has managed to speak in their exquisite manner.”

Tokarczuk’s comments, which were delivered in Polish at a recent event in Poznań, had the misfortune of going viral around the same time of the Commonwealth Prize controversy. (We had her remarks translated into English by a human.) But she’s far more ambivalent about AI than the headlines surrounding the event would suggest. Tokarczuk clarified her AI use in a three-point statement shared with Lit Hub in which she explained that she didn’t use AI to write her forthcoming book but does use it for “faster documenting and checking of facts,” though she independently verifies the information herself.

“I am sometimes inspired by dreams,” she continued, “but before this sentence too is cornered and torn to pieces by the experts, I hasten to report that they are my own dreams.”

The uproar over Tokarczuk’s initial comments — and the need she felt to explain herself — speaks to a greater, not entirely unjustified paranoia in publishing over the use of AI. LLM-generated prose may be the new normal, but is that what anyone wants? Thousands of people threatened to boycott Barnes & Noble after CEO James Daunt said he had no problem selling AI-written books, so long as the books contained disclaimers specifying they hadn’t been written by a person. Daunt later walked back his comments, but not entirely. “Book banning is a clear and present danger, so we are very careful with demands to ban any books,” he told the Los Angeles Times, while also making sure “not to sell AI generated books that masquerade to be by real authors.”

None of this, however, explains the uncanny quality of AI-generated work, or what distinguishes bad LLM-produced prose from bad human writing. When I ran Nazir’s story through Pangram, an AI- and plagiarism-detection software, it came back as 100 percent AI-generated. According to Pangram...