Substack switched on Pangram-powered AI detection on July 21, scanning any post over 100 words. Six days earlier, Epoch AI tested Pangram, GPTZero, and Originality.ai and found that when a model imitates a specific author's style, they miss 10% to 18% of AI text overall and 24% to 29% of scientific writing. Pangram alone missed 48% of Gemini-written academic passages in the worst case.

Give an AI text detector a passage written by a lazy prompt, and it almost never misses. Give it a passage written to imitate a specific human, and it starts to fail.

That is the finding of a study Epoch AI's Jaeho Lee published on July 15. His team ran three of the most widely used detectors, Pangram, GPTZero, and Originality.ai, against a corpus of 495 human passages from 99 authors and a matching set of AI-generated ones. On plain AI text produced from a simple prompt, the detectors were near-perfect, missing at most 0.7%. On genuine human writing, they rarely cried wolf.

Then the researchers changed one thing. They handed each of three frontier models, Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro, five real samples of an author's work and asked it to write something new in that voice. The detection rates collapsed.

Six days after that result went public, Substack turned one of the three tools on for its entire platform.

The Numbers Get Worse the Closer You Look

Across the style-imitation set, an average of 38 of every 297 AI passages slipped through, a false-negative rate of roughly 13%. Broken out by tool, the spread runs wider, and the tool Substack chose sits at the better end of it.

Detector	Style-imitation misses (overall)	Scientific writing	Worst single case
Pangram (Substack's pick)	10% (30/297)	25% (25/99)	48% (Gemini, science)
GPTZero	11% (32/297)	24% (24/99)	36% (Gemini, science)
Originality.ai	18% (53/297)	29% (29/99)	39% (GPT-5.5, science)

The averages hide the real problem, which lives in the genre breakdown. Fiction was easy: all three detectors missed only 1% to 5% of style-imitated stories. Scientific writing was a different category of failure. Pangram missed 25% of style-imitated academic passages, GPTZero missed 24%, and Originality.ai missed 29%. Push into a single model-and-genre cell and it gets starker still: on Gemini-written scientific passages, Pangram missed 48%, essentially a coin flip.

The reason that specific number matters is where these tools get used. Scientific and academic writing is precisely the setting where institutions lean hardest on AI detection, in classrooms, journals, and grant review. The genre where the stakes of a wrong call are highest is the genre where the detector is least reliable.

Detection Is Losing an Asymmetric Race

Understanding why this happens explains why it is unlikely to get fixed.

Making a model imitate a writing style takes one line of instruction and a few reference samples. Detecting the result requires distinguishing statistical patterns that get subtler with every model generation. The two tasks are not evenly matched. One is a prompt; the other is a hard classification problem trying to hit a target that moves faster than it can aim. The three detectors even work differently, and it did not save them: Pangram is a trained neural classifier, GPTZero measures perplexity and burstiness (how surprising word choices are and how much that surprise varies), and Originality.ai is a supervised classifier. All three showed the same blind spot. If you want the mechanics of why a model can absorb five samples and reproduce a voice, our explainer on how large language models actually work covers the underlying behavior, and the style-conditioning techniques that make it trivial are standard prompt engineering now.

The corpus was built to be clean. All 495 human passages predate ChatGPT's November 2022 release, drawn from blogging, fiction, and arXiv science, which effectively rules out any model contamination. The human-text results were the reassuring part: Pangram and GPTZero flagged zero of 495 human passages as AI, while Originality.ai wrongly flagged 19, a 3.8% false-positive rate, every one of them writing that predates the technology it was accused of using.

What Substack Actually Shipped

Against that backdrop, Substack rolled out AI detection on July 21, powered by Pangram.

The feature scans posts, notes, replies, and comments longer than 100 words and returns an estimated split across three buckets: human-written, AI-assisted, and AI-generated. It is live on the web and iOS, with Android to come. The framing is deliberately soft. Substack presents the output as an estimate rather than a verdict, and it does not automatically penalize anyone based on the score.

That restraint is the responsible version of this. A detection estimate surfaced as context for readers is a weak signal, and a weak signal presented honestly does little harm. The danger is not the label. It is what happens when a platform, a university, or an employer treats the same number as evidence and acts on it, because the research now shows the number carries a false-negative rate that ranges from one in ten to nearly one in two depending on the genre.

The Other Side

Defenders of detection make a fair point, and Epoch's own data supports part of it.

The tools rarely accuse humans. For Pangram and GPTZero, the false-positive rate in this test was zero, which means a writer using them as a reader-facing signal is not going to be wrongly branded on a whim. As a probabilistic hint that invites a closer look, rather than a gavel, detection has a defensible place.

The methodology also cuts the numbers both ways. Pangram and GPTZero sometimes return a "Mixed" verdict, judging a document part-human and part-AI, and Epoch counted those as misses. Count them instead as partial detections and Pangram's style-imitation miss rate falls from 10.1% to 6.1%, and GPTZero's from 10.8% to 8.8%. Epoch is candid about its limits, too: it tested one evasion method, few-shot style imitation, on 500-word passages, using each tool's default setting and the versions available in June 2026. Paraphrasing tools and manual editing were not tested, and detectors update without notice.

None of that rescues the scientific-writing result, and none of it changes the core asymmetry. It is a caution against reading "one in five" as a fixed law rather than a snapshot. Detection is a signal that degrades exactly when someone tries to evade it, which is exactly when it is being asked to work.

The Bottom Line

The story is not that Substack made a mistake. Its implementation is the careful kind: an estimate, not an enforcement action, aimed at readers rather than at punishing writers. The story is the timing, and what it says about an entire class of tools. Universities, publishers, employers, and now platforms are deploying AI detectors whose measured accuracy does not support the weight of the decisions people make with them, and the research documenting the gap is arriving faster than the adoption is slowing down.

The honest use of these tools is narrow. They are a reasonable weak signal for a reader deciding how much to trust a page, and they are close to worthless as proof for a consequence, because the moment a writer prompts a model to sound like a person, the miss rate climbs into double digits and, in the genre where detection is used most, past a quarter. Anyone building assessment or moderation on top of a detector score is building on a number that fails precisely under adversarial pressure.

Substack turned on a detector the same week researchers showed how to walk past it. As the study's data makes plain, a tool that catches lazy AI and misses careful AI is not measuring whether something was written by a machine. It is measuring whether someone bothered to hide it.