"Was this written by AI?" is now a daily question for teachers, editors, recruiters and platform moderators. And a small industry of AI detectors promises a yes-or-no answer. This guide explains how those tools really work, the signals they rely on, and the hard truth about how reliable they really are.
What an AI detector is trying to do
An AI text detector estimates the odds that a passage came from a language model rather than a person. Crucially, it doesn't understand the text or check whether it's true. It looks at surface statistics - the shape and predictability of the words - and outputs a likelihood. That gap matters. It's the root of every limit that follows.
To see why these statistics exist, it helps to know how the text was made. An LLM writes by guessing the most likely next token, again and again. That very process leaves a faint statistical trace, and detectors hunt for it.
The three core techniques
1. Perplexity and burstiness
The oldest and most common approach measures two things:
- Perplexity - how surprised a language model is by each word. An LLM writes by choosing high-odds words. So AI text tends to be very predictable, and it scores low perplexity. Human writing is messier and less predictable.
- Burstiness - how much sentence length and complexity vary across a passage. People write in bursts: a long winding sentence, then a short one. Machine text is often flatter and more even.
A detector blends low perplexity and low burstiness into a "this looks machine-written" signal. The idea makes sense. But it is also just why plain, well-built human writing gets misjudged.
2. Trained classifiers
The modern approach is a machine-learning classifier. The tool is shown many human-written and AI-written samples. On its own, it learns the patterns that tell them apart. Then it outputs a score for new text. This is the same kind of method behind spam filters, but aimed at who wrote the text.
The catch: a classifier is only as good as its training data. It learns the styles of the models and topics it saw. So it can be sure but wrong on anything outside that set - new models, edited text, or writers whose natural style looks like the "AI" patterns it learned.
3. Watermarking
This is a whole different idea. Instead of guessing after the fact, the AI provider gently skews the model's word choices as it writes, following a secret pattern. A matching detector that knows the pattern can then spot it. In theory this is the most robust method. But it only works if the provider actually watermarks the output and the watermark survives. Copying, rewording or even mild editing tends to wash it out.
How reliable are they, really?
This is where the marketing and the evidence part ways. AI detectors make two kinds of errors, and both are common:
- False positives - flagging real human writing as AI. Detectors reward "plainness." So clear, by-the-book, well-ordered human writing can score as machine-made.
- False negatives - missing real AI text, above all after a human lightly edits or rewords it.
Two public facts anchor the doubt:
- OpenAI shut down its own AI Text Classifier in July 2023, citing its low rate of accuracy. The firm that builds the leading models could not ship a reliable detector for them.
- Researchers have raised the alarm on bias. A widely-cited 2023 Stanford study (Liang et al., published in Patterns) found that detectors flag writing by non-native English speakers more often. Their simpler, more predictable phrasing reads as "low perplexity," which risks unfair claims.
The deeper problem is built in. Detection is a guess about surface patterns. So anything that changes those patterns defeats it - including the plain editing every careful writer does anyway.
Why detectors are easy to fool
The signal is statistical, not based on meaning. So lots of plain actions lower a detector's confidence. You can reword sentences, vary their length, or swap a few words. You can ask the model to write in a more "human" or varied style. Or you can run the text through a rewording tool. Watermark detection only helps when a watermark was added and survived. Often it wasn't, or didn't. This is a classic cat-and-mouse race, and the cat is losing.
What to do instead
For anything with real stakes - grades, jobs, publication, moderation - a single detector score is the wrong tool. Better signals come from process and context:
- Look at draft history and version control, not just the final text.
- Ask follow-up questions about the work, or compare against a known writing sample.
- Judge whether the content is correct, fresh and useful. An LLM's real weakness isn't that it's easy to detect. It's hallucination - stating false things with full confidence. Checking facts catches more real problems than any detector.
- If you must use a detector, treat its output as one weak input. Note the false-positive risk. And never auto-run a decision or claim on it alone.
For more on how these models handle your data and where the real risks lie, see whether ChatGPT is safe to use.
The bottom line
AI detectors work by measuring the statistical fingerprints of machine text - low perplexity, low burstiness, learned classifier patterns, or provider watermarks. They never understand meaning. That design makes them a matter of odds, not proof. They are prone to false positives, above all against plain or non-native writing. They are easy to defeat with light editing. And they are unreliable enough that even OpenAI pulled its own detector. Use them, if at all, as a faint hint. Base real decisions on process, context and whether the writing is any good.
Related guides: What Is AI Hallucination? Why Chatbots Make Things Up (2026).
