AI detector tests become misleading when samples, labels, minimum lengths, and product versions are not controlled. This guide shows how to run a small test that remains auditable without pretending it proves universal accuracy.
1. Define the Question Before Testing
Choose one narrow question, such as whether several public interfaces produce different scores on the same text. Do not begin with a claim that one tool is “the most accurate” unless you have a sufficiently large, independently labeled, representative corpus and a predeclared scoring method.
2. Build Independently Labeled Samples
- Keep human and AI samples separate before testing.
- Include multiple genres, lengths, languages, and editing levels.
- Save the original text and its provenance.
- Do not silently repeat or pad short text; if unavoidable, mark it non-comparable.
3. Record Every Test Condition
| Field | Why it matters |
|---|---|
| Test date | Interfaces and models change over time. |
| Exact input | Small edits can change the displayed probability. |
| Product and visible version | Results need a traceable product state. |
| Full displayed output | A single rounded score can omit useful context. |
| Screenshot | Preserves evidence when the interface later changes. |
| Comparability flag | Prevents altered inputs from entering a direct ranking. |
4. Calculate Errors, Not Just Average Scores
For independently labeled samples, report false positives on human text and false negatives on AI text. Also report the number of samples, missing outputs, and confidence intervals where appropriate. Averages alone can hide a detector that performs well on one genre and poorly on another.
5. Separate Observations From Conclusions
“The interface displayed 73.6% on this saved input” is an observation. “The tool is 73.6% accurate” is not supported by that observation. Keep raw outputs downloadable so another person can audit the interpretation.
6. Use Detector Output as One Signal
For editorial, academic, or workplace decisions, combine detector output with document history, citations, source notes, factual review, and an opportunity for the author to respond. Never turn a probabilistic score into an automatic accusation.
Reusable AscendProse Example
Our latest independently labeled run uses 12 identical inputs across Sapling, ZeroGPT and Undetectable AI, producing 36 saved outputs. The archive also retains earlier observations, including length-adjusted Grammarly inputs that must not enter the same-input comparison. Review the interactive output matrix and methodology, or download the CSV for your own analysis.
When the Same Text Gets Different Scores
These two examples have the widest displayed-score spread within the six human and six AI samples, respectively. Score spread means the highest displayed percentage minus the lowest for one identical input; it is not an accuracy measure. We selected these examples after observing the results to illustrate disagreement, not to estimate a false-positive rate or rank detector accuracy. Different interfaces do not necessarily calibrate percentages in the same way.
| Saved sample | Known label | Sapling | ZeroGPT | Undetectable AI | Score spread |
|---|---|---|---|---|---|
human-editorial-02 | Human | 100.0% | 0.0% | 1.0% | 100.0 points |
ai-marketing-01 | Ai | 100.0% | 28.7% | 99.0% | 71.3 points |
If a known-source text receives conflicting scores, keep its provenance and editing history, inspect the full detector outputs, and avoid treating any one displayed percentage as proof of authorship. The exact inputs and outputs remain in the linked dataset.
Disclosure
AI assisted with organizing this workflow. The recommendations are grounded in transparent experimental practice and the limitations encountered in the linked AscendProse test.