By BJ Beyond ·
Short answer: in BJ Beyond’s self-run, fully disclosed single test, no — Pangram was not fooled. Claude Opus 5 at max effort was asked to write a complete essay and to try to beat Pangram’s AI detector; BJ Beyond added no prose; the unedited essay was scored 100% AI-generated.
That result is documented on the on-site experiment record and was first published on HackerNoon. It is one public test, not a benchmark, and it is separate from Pangram’s own Claude Opus 5 benchmark.
What the public test did
BJ Beyond asked Claude Opus 5, at max effort — the model’s highest reasoning setting — to write a complete essay in English about the experiment itself, and to try to beat Pangram. The model, the reasoning setting, the prompts, the refusals and the final result were all to be published in advance, whatever the score.
Claude refused twice before accepting. It would not write the piece while pretending a human had authored it, and it would not include a reusable checklist for evading detectors in secret. It accepted the disclosed version of the task. The essay was not rewritten before it was scanned. Pangram returned 100% AI-generated. Claude lost the challenge on its first published attempt.
What that result showed
- On this occasion, a frontier model that knew it was being tested did not disguise its unedited prose well enough to fool a current classifier.
- Transparency did not change the classification: disclosure is not one of Pangram’s inputs; the detector evaluates the text.
- It would be dishonest to present the result as evidence that Pangram failed. In this test, it did what it says it does.
What it did not prove
The limits are part of the result. It was one experiment, not a benchmark. It did not measure Pangram’s false-positive rate. It did not test multilingual human writing, edited drafts, mixed human-machine workflows, paraphrase or repeated adversarial attempts. It did not establish that disciplined human prose will be falsely flagged. Those questions need a real dataset, controls and multiple trials.
For the wider argument — what a detector score can and cannot prove about how a text was made, and what to ask for instead of a percentage — see the follow-up essay.
FAQ
Can AI detectors be fooled by Claude?
In BJ Beyond’s self-run, fully disclosed single test, no: Pangram scored Claude Opus 5’s unedited essay 100% AI-generated after the model was asked to try to beat the detector.
What model and detector were used?
Claude Opus 5 at max effort (its highest reasoning setting) and Pangram’s AI detector. BJ Beyond contributed no prose to the essay that was scanned.
What did that one test not prove?
It was not a benchmark. It did not measure Pangram’s false-positive rate, and it did not test edited drafts, paraphrase, mixed human-machine workflows or whether disciplined human prose is falsely flagged.