Skip to content
Sign in
Text Humanizer Forum

What does an AI detector score actually mean?

Staff answerWritten by Desmond Achterberg · Reading AI detector scores0 replies

Short answer

An AI detector score is a statistical estimate, not a finding of fact. It reports how likely a model thinks a passage is to resemble AI-generated text. The vendors themselves say the number should open a conversation, not close one. A high score is a reason to look closer, never proof of who wrote the text.

Written by Desmond Achterberg · Style and disclosure editor ·
CiteText Humanizer Forum, "What does an AI detector score actually mean?", https://texthumanizer.com/forum/discuss/what-does-an-ai-detector-score-actually-mean/, accessed 2026-11-16.
  1. Style and disclosure editor ·

    What the number is

    A detector is a classifier. It was trained on examples of human and machine writing and it returns a probability or a percentage for new text. It does not see who typed the words, whether a draft was edited, or what tools sat behind it.

    That makes the score a guess about style, and styles overlap. Careful academic prose, boilerplate and text in a second language can all look "predictable" to a model. OpenAI said this about its own classifier: text that is very predictable cannot be reliably identified, and its own example was a list of the first 1,000 prime numbers, where the right answer is always the same.

    What the vendors say about their scores

    I read the interpretation pages rather than the marketing pages. Three statements are worth knowing.

    • OpenAI built a classifier, published its limits, and then withdrew it. The page now says the tool was retired on July 20, 2023 "due to its low rate of accuracy." Before that, it reported that it correctly flagged 26% of AI-written text and wrongly flagged human-written text 9% of the time on its test set. It also said the tool "should not be used as a primary decision-making tool."
    • Turnitin's chief product officer wrote that there is no "right" or "target" score, and that highlighted sentences should be used "to initiate a conversation, not to draw a conclusion." Turnitin puts its sentence-level false positive rate at around 4%, and says its document-level false positive rate is under 1% for documents with 20% or more AI writing.
    • GPTZero's FAQ says its results "should not be used to punish students," that there are edge cases in both directions, and that accuracy rises with more text. Its sentence-level judgements are the least reliable part.

    Note what these have in common: each vendor tells you the score is one input.

    How to read a report

    What you see What it can tell you What it cannot tell you
    A percentage for the whole document The model's estimate of how much text resembles AI output That any particular sentence was machine-written
    Highlighted sentences Places worth a human read Who wrote them. Turnitin says false positives cluster near real AI text
    A "low" or "high" confidence label How sure the model is about itself Whether the model is right
    A very short sample Almost nothing Anything. OpenAI called its classifier very unreliable under 1,000 characters

    A passage can look machine-made for ordinary reasons

    Here is a human-written sentence that a model could find predictable:

    In conclusion, it is important to note that there are many factors to consider when evaluating the effects of social media on teenagers.

    Nothing about it is machine-specific. People write this sentence every day, because it is safe. It is also a weak sentence, and the fix is an editing fix:

    Social media affects teenagers in at least three ways: sleep, comparison with peers, and attention. The evidence is strongest for sleep.

    That is not a trick for a score. It is a sentence that says something. Edit for the reader, and you have a better paragraph whatever any detector says. For the editing method see how to edit an AI-generated draft and why AI writing sounds robotic.

    What to do with a score

    1. Treat it as a prompt to read the text yourself, not a verdict.
    2. Check how long the sample is. Short samples give weak signals.
    3. Ask what else you know: drafts, notes, version history, the writer's earlier work.
    4. Check the policy. Whether AI help is allowed, and whether it must be disclosed, depends on your school, employer or publisher. See do I have to disclose AI use.
    5. If you are the writer, keep your drafts. A dated history is better evidence than any percentage.

    If you want the next step in the workflow, a step-by-step editing workflow covers where checks fit.

    0
    ReplyLink

Plain text or simple Markdown. Links are nofollow. Your email is never shown.

Text Humanizer Forum. Questions and answers on editing AI-assisted drafts so they read naturally: workflow, tone and voice, and disclosure.