What an AI Detector Can Prove, and What It Can’t

What that single result can and can’t support

First, the limits, because they matter more than the headline:

• It was a self-run, fully disclosed single test. One essay, one model, one detector, one run. It is not a benchmark.
• It says nothing about false positives. A detector catching real AI text tells you nothing about how often it flags human writing.
• It says nothing about evasion by paraphrase. The essay went in unedited. No human rewrite, no second model, no “humanizer” service. Whether a reworked version would also be caught is a separate question, and my test did not ask it.
• It says nothing about watermarks. Pangram is a classifier. It reads the style and structure of a text. It does not look for a hidden key from the model maker. My result tells you nothing about the watermarking systems discussed below.

What it does support is narrow: on this occasion, a frontier model that knew it was being tested did not disguise its unedited prose well enough to fool a current classifier.

Two kinds of evidence

The more useful lesson came from what was around the score.

My essay was openly machine-written. The model was named. The prompts were public. I had committed to publishing the outcome. Nobody reading it could have been misled about who wrote it. Pangram still returned 100%, which was correct, because disclosure is not one of its inputs.

That is the distinction at the core of Human Edge:

• A detector gives evidence about the artifact. How strongly does this text show the patterns associated with machine generation?
• A declaration of process gives evidence about production. Who had the idea, who drafted, what a model changed, what sources were used, who takes responsibility.

Neither can do the other’s job. A score cannot tell you whether a person built the argument and used a model to polish it, or whether a model did everything and a person signed it. A process statement cannot verify itself. People can lie in a disclosure. The two signals answer different questions, and the mistake is treating either one as a verdict on the other.

This month’s news hook: watermarks and the EU AI Act

The question has become more practical in October 2026.

On 5 October 2026, OpenAI published its approach to EU text provenance rules. Over the coming weeks, it will add an invisible watermark — a system it calls textGrain — to eligible ChatGPT and Codex text output in the European Union. API customers worldwide can opt in. In August, Anthropic explained that Claude’s text watermark is a version of Google DeepMind’s SynthID-Text and is being applied globally. Both companies tie the move to the EU AI Act. As Alessandro Longo summarized in Il Sole 24 Ore on 6 October, the Act’s transparency provisions have applied since 2 August, with a transitional deadline of 2 December 2026 for systems already on the market.

Longo’s piece, which is worth reading in full, makes a point that is easy to lose: watermarks and classifier detectors are different tools.

• A watermark is placed at generation time by the model maker. Among plausible next words, the model chooses in a way steered by a secret key, which leaves a statistical regularity. Only a detector with access to that scheme can check for it. It looks for one vendor’s signal, not “AI” in general.
• A classifier like Pangram has no key. It is trained to recognize what machine text tends to look like.

They fail in different ways, so they need to be judged separately.

What the primary sources say about limits

Credit where it is due: OpenAI’s announcement is unusually frank about weaknesses. In its own evaluation of 400-token passages, replacing 10% of words with synonyms cut watermark detection from about 92% to 66%. Replacing 25% cut it to 17%. Short passages and low-flexibility content such as mathematics are harder to detect. OpenAI also writes that a watermark “does not measure human contribution”, and that “the absence of a detected watermark does not prove human authorship.” Detector access is initially limited to approved researchers and expert organizations.

Paraphrase is a known problem for detectors in general. Sadasivan et al., in “Can AI-Generated Text be Reliably Detected?” (published in TMLR), show that recursive paraphrasing of roughly 300-token passages can significantly reduce detection rates across watermark-based, neural and zero-shot detectors, often with only slight loss of quality.

The opposite failure, flagging humans, is documented too. Liang et al., in “GPT detectors are biased against non-native English writers” (Patterns, 2023), found that the detectors they tested misclassified a large share of TOEFL essays by non-native English writers as AI-generated.

Newer classifiers claim to have moved past this. Pangram says Pangram 4 identifies AI involvement in 98.83% of texts processed by 13 commercial “humanizer” services. Its Pangram 3.1 model card says the model is calibrated so that a false positive is five times less likely than a false negative. In other words, it prefers to miss AI text rather than wrongly accuse a human. The same card reports 0% false positives on the Liang et al. non-native set, which is 91 samples. These are the company’s own evaluations. They may well be accurate, but they are not external third-party audits.

That gap is what a September 2026 preprint, “Watermarks Without Verification” by Nemecek, Chaudhary and Ayday, focuses on. Because no public tool can test the deployed commercial watermarks, the authors evaluated the open-source SynthID-Text implementation on two open-weight models. They argue that neither vendors’ assurances nor users’ objections can currently be verified. They call for matched outputs, configuration disclosure, accredited audits, a shared evaluation protocol and interoperable detection.

So the signals are getting better, and their makers say openly that they are limited. Neither of those facts turns a score into a finding about conduct.

Why my “perfect” result fits this picture

My self-run, fully disclosed single test is consistent with all of this, and it also shows its limits.

Pangram did well on the easiest case: fully machine-generated, unedited prose from a frontier model. That is what the vendor claims, and it is worth knowing. But the hard cases are mixed and edited texts, non-native writers, short passages and paraphrased output, and my experiment never touched them. A perfect score on an easy case does not make a score meaningful on a hard one.

There is a second point. Even in my easy case, where the score was completely right, it still could not answer the question a reader actually cares about: was I deceived? I wasn’t. The disclosure answered that. The percentage couldn’t.

What to ask for instead of a score

If you are an editor, a teacher or a client, here is what I would ask for, and in roughly this order:

1. 1. A process statement. Who came up with the thesis? Who drafted? Which tools were used, and for what: research, drafting, translation, line editing? Who is responsible for the final text?
2. 2. Revision history. Drafts, document version history, tracked changes. Work that developed over time leaves traces that a finished file does not.
3. 3. Sources. What was read and where the claims come from. A writer who did the research can point to it.
4. 4. Prompts, where AI was used substantively. You do not need every keystroke. Publishing the key instructions, the way I did in my test, makes the model’s role concrete.
5. 5. A conversation. Ask the author to explain a conclusion, defend a choice, or say what they cut and why. Being able to reason about your own work is still the strongest everyday evidence that you did it.
6. 6. Then, if you like, a detector or watermark check, used as a prompt for a question. A high score is a reason to ask items 1–5. It is not a reason to skip them. A low score is not proof of human authorship either, as OpenAI itself says.

What I would not do is treat any single percentage, from a classifier or a watermark detector, as enough to accuse someone or to certify them.

The narrow conclusion

My experiment ended with a detector getting it exactly right. I am glad it did. It made the result clean.

But a correct number is still only one kind of evidence. Detectors and watermarks can tell you something about a text. Process records tell you how it came to exist. As these tools spread across schools, newsrooms and client work in the EU, the useful habit is to keep asking which question each signal actually answers.

“Detectors and watermarks can tell you something about a text. Process records tell you how it came to exist.”