OpenAI shut down its own AI detector six months after launch

Start here

Anyone handed a percentage score from an AI detector — as evidence a student cheated, or that a piece of writing is fake — is looking at a number with a documented failure rate, not a verdict. The company that built one of the earliest public detectors withdrew it within six months, and a separate study found the failures were not evenly distributed.

What the documents say

OpenAI's own announcement of its AI classifier, dated 31 January 2023, stated the tool ‘correctly identifies 26% of AI-written text (true positives) as “likely AI-written,” while incorrectly labeling human-written text as AI-written 9% of the time (false positives).’ A notice later added to the same page states that ‘as of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy.’ A Stanford study, ‘GPT detectors are biased against non-native English writers’ (Liang, Yuksekgonul, Mao, Wu and Zou, first posted April 2023), tested seven widely used detectors on 91 human-written TOEFL essays from non-native English speakers alongside a set of US eighth-grade essays. It reports near-perfect accuracy for the US eighth-grade essays but an average false-positive rate of 61.22 percent on the TOEFL essays; all seven detectors unanimously flagged 18 of the 91 TOEFL essays as AI-generated, and at least one detector flagged 89 of the 91. Having ChatGPT itself enhance the word choices of the same TOEFL essays cut the false-positive rate to 11.77 percent, suggesting the detectors were responding to simpler, more formulaic phrasing rather than to any real signature of AI generation. OpenAI's own educator guidance states plainly, ‘our research into detectors didn't show them to be reliable enough,’ notes its own tool misclassified the Declaration of Independence, and recommends reviewing a student's actual conversation log instead.

Check this

If a detector score is presented as evidence, ask its stated false-positive rate and sample — the Stanford study shows that rate can differ by a factor of six or more between writer populations. A percentage without a named sample and detector is not a measurement anyone can check.

What holds and what fails

What holds is the documented pattern: detectors tuned on typical native fluency flag simpler, less varied prose, which is not unique to AI writing. What fails is treating any detector score as proof of AI authorship, a use OpenAI's own materials warn against for its retired tool. Editorially, a detector score is better read as a prompt to ask for the writing process — drafts, notes, a conversation log — than as a conclusion in itself.

• Ask what false-positive rate a detector reports, and on which sample, before trusting a score.
• If you are a non-native English writer, know that formulaic or simplified phrasing is the pattern most often misflagged.
• Keep drafts or a conversation log for your own writing as evidence a detector score cannot provide.

No detector tested here reached a reliability its own maker or an independent study called dependable. That is not a claim AI-generated text is undetectable, only that a percentage score, alone, does not establish it.