The 100k Whys of AI

The 100,000 whys of AI

One of the most painful arguments I keep having with fellow techies is the question of whether you can distinguish between human-written and AI-generated text.

Their skepticism is rooted in reason: at their core, LLMs are state-of-the-art statistical models of how humans talk. If so, the output from the model should be almost by definition indistinguishable from human language under any statistical test.

I don’t think this is always argued in good faith; at least some of the debates are started by folks who wish to maintain deniability for their own underhanded use of the tech. But if you sincerely hold this belief, I present you the following collage:

The image shows about 150 Amazon book covers that appear if you search the site for “100000 whys”. Some of these books are category bestsellers in children literature.

There’s nothing inhuman about any of these titles or covers. At the same time, I probably don’t need to convince you that you’re staring at the purest form of AI slop that now fills up many nonfiction book categories on Amazon. More specifically, what we’re seeing here is the artifact of the tools being quasi-deterministic: if a hundred “authors” give their favorite AI tool a similar prompt — say, *“generate a reference book for children” — *the model will produce functionally identical output perhaps 80% of the time.

The similarities in the collage go far beyond the choice of titles: for example, all the covers in the top row feature a roaring dinosaur in the top left corner of the design. There are many other clusters in the data, too. Look for a recurring red-and-white cartoon rocket, a golden retriever, a lion, and so forth.

This is precisely what makes LLM writing distinctive: it’s not that the models’ individual mannerisms are different from ours. It’s that they resort to the same, complex set of mannerisms in response to almost any normal prompt. This is a fuzzy signal, so you shouldn’t fire your intern when they say “it’s not this — it’s that”. But in more casual settings, it’s OK to trust your gut. In fact, these instincts are becoming increasingly important because traditional models of online interactions fall apart if it takes much less effort to produce content than to engage with it.

PS. If you’re using an LLM to automate blogging: yes, the tech is amazing, but chances are, your publication could be renamed to “100,000 Whys”.

Three postscripts:

1. Someone on Mastodon pointed out that the title is probably coming from a 1929 book: "One Hundred Thousand Whys" - apparently largely unknown in the West, but popular for weird political reasons in China.

2. Yes, covers aside, these books are about what you'd expect.

3. No, it's not just this title.

Discussing AI writing with a friend recently, I came up with 3 points that can be applied to virtually all AI use situations - 1) we need to remind people who ask AI to do the entire task that this behavior pretty much guarantees that they will be the first person replaced by AI, 2) AI is a tool as much as a piece of heavy equipment or a doctor's scalpel and equally dangerous, and we absolutely need to decide how to train and certify people to use AI properly, and 3) perhaps more realistic pricing on AI use will encourage people to use it more intelligently.

I still don't think we need AI, but it is here, so let's try to use it better.