---
title: "AI Text Detection Accuracy and Public Benchmark Results | Stylome"
description: "Read dated public benchmark reports for each Stylome model, with sample counts, false-positive rates, training-overlap checks, and measurement limits."
url: "https://stylome.io/transparency"
---

> [Documentation index](https://stylome.io/llms.txt)

# How well does Stylome work?

See exactly how each Stylome model performs across public benchmarks, writing types, and source domains. Every result is attributed to a model version, fixed threshold, test population, and overlap audit.

## Choose a model

Published benchmark

### Stylome 1.2 Large

Our newest and best model.

Published results**6 benchmark reports available**

Retired

### Stylome 1.0

Benchmark record for the previous standard model.

Published results**5 benchmark reports available**

Retired

### Stylome 1.0 Fast

Benchmark record for the retired fast model.

Published results**6 benchmark reports available**

Stylome 1.2 Large · Pangram Labs**EditLens**Human-written, AI-edited, and AI-generated text across three test sections.False-positive rate · 0.01% FPR target**0%**0 of 6,062 human rows flaggedAI-generated detected · 0.01% FPR target**99.00%**5,865 of 5,924 generated passagesFull results

Stylome 1.2 Large cleared every human row across all three EditLens test sections and detected 99.00% of fully generated passages at the 0.01% FPR target.

[EditLens](https://github.com/pangramlabs/EditLens) contains human-written, AI-edited, and AI-generated text. This evaluation covers 18,192 rows in its standard, Enron email, and Llama test sections after excluding 27 rows due to dataset contamination. The detection tables count any AI-touched decision, whether edited or generated. The separate three-class table measures exact class matches using the dataset’s original labels.

Stylome 1.2 Large on EditLens
| Calibration FPR target | Human false positives | Generated detected | Edited detected | All-AI recall |
| --- | --- | --- | --- | --- |
| 0.1% | 0 / 6,062 (0%) | 5,918 / 5,924 (99.90%) | 3,900 / 6,206 (62.84%) | 9,818 / 12,130 (80.94%) |
| 0.01% · service default | 0 / 6,062 (0%) | 5,865 / 5,924 (99.00%) | 3,647 / 6,206 (58.77%) | 9,512 / 12,130 (78.42%) |

### Three-class results

The September 29 run also recorded exact class attribution with the frozen full-embedding KNN and five-neighbor fallback. These are exact-class recalls, not the detection recalls in the tables above. Confidence tiers were not evaluated in this rerun.

EditLens exact-class recall
| True class | 0.1% FPR target | 0.01% FPR target |
| --- | --- | --- |
| Human written | 6,062 / 6,062 (100%) | 6,062 / 6,062 (100%) |
| AI edited | 3,296 / 6,206 (53.11%) | 3,161 / 6,206 (50.93%) |
| AI generated | 5,885 / 5,924 (99.34%) | 5,837 / 5,924 (98.53%) |

Three-class confusion · 0.01% FPR target
| True class | Called human | Called edited | Called generated |
| --- | --- | --- | --- |
| Human written | 6,062 | 0 | 0 |
| AI edited | 2,559 | 3,161 | 486 |
| AI generated | 59 | 28 | 5,837 |

Most missed edits were classified as human. These results do not establish performance on unseen sources.

### Results by test section, source, and producer

Every section had zero human false positives at both operating points: 0 of 2,035 standard-test human rows, 0 of 1,992 Enron rows, and 0 of 2,035 Llama rows. Each detection cell gives the count, denominator, and recall.

EditLens by test section
| Test section | Generated at 0.1% FPR target | Edited at 0.1% FPR target | Generated at 0.01% FPR target | Edited at 0.01% FPR target |
| --- | --- | --- | --- | --- |
| Standard test | 2,038/2,042 (99.80%) | 1,268/2,020 (62.77%) | 2,032/2,042 (99.51%) | 1,213/2,020 (60.05%) |
| Enron email | 1,845/1,847 (99.89%) | 1,377/2,308 (59.66%) | 1,798/1,847 (97.35%) | 1,240/2,308 (53.73%) |
| Llama | 2,035/2,035 (100.00%) | 1,255/1,878 (66.83%) | 2,035/2,035 (100.00%) | 1,194/1,878 (63.58%) |

EditLens by source
| Source | Generated at 0.1% FPR target | Edited at 0.1% FPR target | Generated at 0.01% FPR target | Edited at 0.01% FPR target |
| --- | --- | --- | --- | --- |
| amazon\_reviews | 544/546 (99.63%) | 303/549 (55.19%) | 541/546 (99.08%) | 284/549 (51.73%) |
| enron\_email | 1,845/1,847 (99.89%) | 1,377/2,308 (59.66%) | 1,798/1,847 (97.35%) | 1,240/2,308 (53.73%) |
| fineweb\_edu | 986/986 (100.00%) | 626/973 (64.34%) | 986/986 (100.00%) | 598/973 (61.46%) |
| google\_reviews | 419/420 (99.76%) | 259/389 (66.58%) | 417/420 (99.29%) | 247/389 (63.50%) |
| news | 1,009/1,010 (99.90%) | 640/977 (65.51%) | 1,008/1,010 (99.80%) | 605/977 (61.92%) |
| reddit\_writing\_prompts | 1,115/1,115 (100.00%) | 695/1,010 (68.81%) | 1,115/1,115 (100.00%) | 673/1,010 (66.63%) |

EditLens by producer
| Producer | Generated at 0.1% FPR target | Edited at 0.1% FPR target | Generated at 0.01% FPR target | Edited at 0.01% FPR target |
| --- | --- | --- | --- | --- |
| claude-sonnet-4-20250514 | 1,374/1,378 (99.71%) | 727/1,428 (50.91%) | 1,340/1,378 (97.24%) | 648/1,428 (45.38%) |
| gemini-2.5-flash | 1,094/1,094 (100.00%) | 580/964 (60.17%) | 1,090/1,094 (99.63%) | 559/964 (57.99%) |
| gpt-4.1-2025-04-14 | 1,263/1,263 (100.00%) | 1,061/1,472 (72.08%) | 1,249/1,263 (98.89%) | 982/1,472 (66.71%) |
| meta-llama/Llama-3.3-70B-Instruct-Turbo | 2,187/2,189 (99.91%) | 1,532/2,342 (65.41%) | 2,186/2,189 (99.86%) | 1,458/2,342 (62.25%) |

### Evaluation details

The 6,062 human rows contain 4,022 distinct whitespace-normalized texts. None was flagged in any occurrence. Related human and AI texts appear across sections, so row counts are not counts of independent documents.

Evaluated September 16–17, 2026, and rescored September 29, 2026. Both thresholds were fixed before scoring. All inputs were scored without truncation or length exclusion. The exact two-sided 95% false-positive-rate interval for the 4,022 unique human texts is 0–0.0917%, under the independent-text assumption; shared authors and prompts limit that assumption.

Stylome 1.2 Large · PAN at CLEF 2025**PAN 2025**Human and AI-labeled essays, fiction, and news from an independent detection challenge.False-positive rate · 0.01% FPR target**0%**0 of 1,277 human passages flaggedAI recall · 0.01% FPR target**98.01%**2,266 of 2,312 AI passages detectedFull results

At the 0.01% calibration FPR target selected for the service default, Stylome 1.2 Large detected 2,266 of 2,312 AI passages and cleared all 1,277 human passages.

The complete [PAN 2025](https://pan.webis.de/clef25/pan25-web/generated-content-analysis.html) validation set contains 3,589 passages across essays, fiction, and news. Both thresholds were fixed before scoring. The percentages below distinguish the calibration FPR target from the measured false-positive rate on PAN.

Stylome 1.2 Large on PAN 2025 at both operating points
| Calibration FPR target | Human false positives | AI detected | AI recall | 95% recall interval |
| --- | --- | --- | --- | --- |
| 0.1% | 0 / 1,277 | 2,279 / 2,312 | 98.57% | 98.00–99.02% |
| 0.01% · service default | 0 / 1,277 | 2,266 / 2,312 | 98.01% | 97.35–98.54% |

PAN 2025 by genre
| Genre | Human false positives, both targets | AI recall at 0.1% FPR target | AI recall at 0.01% FPR target |
| --- | --- | --- | --- |
| Essays | 0 / 132 | 531 / 533 (99.62%) | 531 / 533 (99.62%) |
| Fiction | 0 / 928 | 883 / 909 (97.14%) | 876 / 909 (96.37%) |
| News | 0 / 217 | 865 / 870 (99.43%) | 859 / 870 (98.74%) |

PAN 2025 by producer
| Producer | AI passages | AI recall at 0.1% FPR target | AI recall at 0.01% FPR target |
| --- | --- | --- | --- |
| deepseek-r1-distill-qwen-32b | 106 | 106/106 (100.00%) | 106/106 (100.00%) |
| falcon3-10b-instruct | 113 | 113/113 (100.00%) | 113/113 (100.00%) |
| gemini-1.5-pro | 145 | 136/145 (93.79%) | 136/145 (93.79%) |
| gemini-2.0-flash | 139 | 139/139 (100.00%) | 139/139 (100.00%) |
| gemini-pro | 59 | 59/59 (100.00%) | 59/59 (100.00%) |
| gemini-pro-paraphrase | 70 | 70/70 (100.00%) | 68/70 (97.14%) |
| gpt-3.5-turbo | 200 | 199/200 (99.50%) | 199/200 (99.50%) |
| gpt-4-turbo | 63 | 63/63 (100.00%) | 63/63 (100.00%) |
| gpt-4-turbo-paraphrase | 59 | 59/59 (100.00%) | 58/59 (98.31%) |
| gpt-4.5-preview | 45 | 45/45 (100.00%) | 45/45 (100.00%) |
| gpt-4o | 209 | 209/209 (100.00%) | 209/209 (100.00%) |
| gpt-4o-mini | 214 | 214/214 (100.00%) | 214/214 (100.00%) |
| llama-2-70b-chat | 65 | 65/65 (100.00%) | 65/65 (100.00%) |
| llama-2-7b-chat | 72 | 72/72 (100.00%) | 72/72 (100.00%) |
| llama-3.1-8b-instruct | 158 | 139/158 (87.97%) | 133/158 (84.18%) |
| llama-3.3-70b-instruct | 54 | 54/54 (100.00%) | 53/54 (98.15%) |
| ministral-8b-instruct-2410 | 139 | 139/139 (100.00%) | 139/139 (100.00%) |
| mistral-7b-instruct-v0.2 | 68 | 68/68 (100.00%) | 68/68 (100.00%) |
| mixtral-8x7b-instruct-v0.1 | 70 | 66/70 (94.29%) | 64/70 (91.43%) |
| o3-mini | 132 | 132/132 (100.00%) | 132/132 (100.00%) |
| qwen1.5-72b-chat-8bit | 63 | 63/63 (100.00%) | 63/63 (100.00%) |
| text-bison-002 | 69 | 69/69 (100.00%) | 68/69 (98.55%) |

Producer names and binary labels follow PAN 2025, including its separately named paraphrase producers.

At the 0.01% FPR target, Llama-3.1-8B accounted for 25 of the 46 missed AI passages, Gemini-1.5-Pro for 9, and Mixtral-8x7B for 6. Fiction accounted for 33 misses.

### Evaluation details

All 3,589 responses were verified. Twenty passages were shortened to the first 10,000 normalized characters to meet the API input limit: 17 human and 3 AI passages, all classified correctly at both operating points. The remaining 3,569 passages were scored in full.

PAN 2025 full-text subset
| Calibration FPR target | Human false positives | AI detected | AI recall |
| --- | --- | --- | --- |
| 0.1% | 0 / 1,260 | 2,276 / 2,309 | 98.57% |
| 0.01% | 0 / 1,260 | 2,263 / 2,309 | 98.01% |

Scoring completed in 24.4 minutes at 32 concurrent requests, averaging 1,972 full-request input tokens per second, including the shared prefix. Development compute cost was approximately $0.21. There were no request errors or checked training, reference, or calibration overlaps.

Evaluated September 16, 2026, and rescored September 29, 2026, on the established PAN validation set. The exact two-sided 95% interval for the observed human false-positive rate is 0–0.2885%, using the independent-passage assumption.

Stylome 1.2 Large · StoryScope**StoryScope**AI-generated fiction from five models.AI-generated recall · 0.01% FPR target**99.73%**6,892 of 6,911 outputs classified as AI-generatedFull results

Stylome 1.2 Large correctly classified 6,892 of 6,911 eligible [StoryScope](https://huggingface.co/datasets/jjrussell10/storyscope) test outputs as AI-generated at the fixed 0.01% calibration FPR target. Six were labeled AI-edited and 13 human; neither counts as correct here.

StoryScope exact AI-generated recall · 0.01% FPR target
| Producer | Correct / scored | AI-generated recall |
| --- | --- | --- |
| GPT-5.4 | 1,380 / 1,384 | 99.71% |
| Claude Sonnet 4.6 | 1,376 / 1,383 | 99.49% |
| DeepSeek V3.2 | 1,377 / 1,380 | 99.78% |
| Kimi K2.5 | 1,380 / 1,384 | 99.71% |
| Gemini 3 Flash | 1,379 / 1,380 | 99.93% |

Evaluated September 19, 2026, and rescored September 29, 2026. Long stories (6,253) use their first 10,000 normalized characters; 658 shorter outputs were scored in full. Three missing outputs and six under 200 characters were excluded. Human story texts are not supplied, so false-positive rate cannot be measured here.

Stylome 1.2 Large · TOEFL11, ICNALE, TOEFL91**ESL writing**Essays written by people learning English as an additional language, from three independent collections.False-positive rate · 0.01% FPR target**0.016%**3 of 18,664 human essays flaggedFull results

Stylome 1.2 Large correctly cleared 18,661 of 18,664 essays by non-native English writers, with the same result at both fixed operating points. Before scoring, every essay was checked against our training data and reference bank; none appeared in either. The thresholds were fixed before scoring.

Stylome 1.2 Large on ESL writing
| Collection | Human texts | Flagged · 0.01% target | False-positive rate · 0.01% target | Flagged · 0.1% target | False-positive rate · 0.1% target |
| --- | --- | --- | --- | --- | --- |
| TOEFL11 | 13,175 | 0 | 0% | 0 | 0% |
| ICNALE written essays | 5,398 | 1 | 0.019% | 1 | 0.019% |
| TOEFL91 | 91 | 2 | 2.20% | 2 | 2.20% |
| All ESL essays | 18,664 | 3 | 0.016% | 3 | 0.016% |

[TOEFL11](https://catalog.ldc.upenn.edu/LDC2014T06) contains real TOEFL exam essays from writers with 11 different native languages. The [ICNALE](https://language.sakura.ne.jp/icnale/) essays were written by university students in 10 Asian countries and regions between 2007 and 2013, plus a United Arab Emirates module. TOEFL91 is the 91-essay panel from [Liang et al. (2023)](https://pmc.ncbi.nlm.nih.gov/articles/PMC10382961/), in which seven earlier GPT detectors labeled an average of 61.3% of the essays as AI-generated. Stylome 1.2 Large flagged 2 of the 91. The panel is small, so each essay changes its rate by about 1.1 percentage points.

**Native-speaker comparison.** ICNALE also includes 400 essays by native English speakers, written to the same two prompts under the same conditions. Stylome 1.2 Large flagged 0 of them at the 0.01% target and 2 at the 0.1% target, compared with 1 of 5,398 learner essays at both targets. We see no sign that ESL writing is flagged more often than native writing.

**Known limitation: professionally proofread essays.** ICNALE also contains 795 learner essays revised by professional proofreaders. Stylome 1.2 Large labeled 32 of them (4.0%) as AI-edited at the 0.01% target, and 36 (4.5%) at the 0.1% target. We count these as human-written, so they are false positives. Heavy human editing of learner writing can resemble AI editing to the model.

TOEFL11 and ICNALE evaluated October 19, 2026; TOEFL91 re-evaluated October 20, 2026, with the current reference bank (unchanged: 2 of 91). The TOEFL11 copy is the 13,200-essay NLI Shared Task 2017 release; 25 essays under the 200-character minimum were not scored. ICNALE uses the Written Essays 2.6 and UAE 1.0 learner essays; the Written Essays Plus module was excluded because it was collected after 2022.

Stylome 1.2 Large · ELLIPSE Corpus**ELLIPSE**Standardized-test essays written by English learners in grades 8–12.False-positive rate · 0.01% FPR target**0%**0 of 1,713 human essays flaggedFull results

All 1,713 essays were classified as human at both fixed operating points. No essays were excluded or truncated, and the thresholds were fixed before scoring.

This evaluation uses the established frozen held half of the [ELLIPSE corpus](https://doi.org/10.1075/ijlcr.22026.cro). The same 1,713 human-written essays were evaluated for earlier Stylome versions.

Stylome 1.2 Large on the frozen ELLIPSE half
| Nominal calibration FPR target | Human essays | False positives | Observed false-positive rate |
| --- | --- | --- | --- |
| 0.1% | 1,713 | 0 | 0% |
| 0.01% | 1,713 | 0 | 0% |

Evaluated September 16, 2026. The exact two-sided 95% binomial interval is 0–0.2151%, using the independent-essay assumption.

Stylome 1.2 Large · PERSUADE 2.0**PERSUADE**Argumentative essays written by United States students in grades 6–12.False-positive rate · 0.01% FPR target**0%**0 of 893 human essays flaggedFull results

All 893 essays were classified as human at both fixed operating points. No essays were excluded or truncated, and the thresholds were fixed before scoring.

This evaluation uses the established frozen held half of the [PERSUADE corpus](https://doi.org/10.1016/j.asw.2024.100865). The same 893 human-written essays were evaluated for earlier Stylome versions.

Stylome 1.2 Large on the frozen PERSUADE half
| Nominal calibration FPR target | Human essays | False positives | Observed false-positive rate |
| --- | --- | --- | --- |
| 0.1% | 893 | 0 | 0% |
| 0.01% | 893 | 0 | 0% |

Evaluated September 16, 2026. The exact two-sided 95% binomial interval is 0–0.4122%, using the independent-essay assumption.

Stylome 1.2 Large · ICNALE Spoken**Transcribed speech**Transcripts of spoken English by learners and native speakers: one-minute monologues and interview answers.False-positive rate · 0.01% FPR target**0.012%**1 of 8,557 human transcripts flaggedFull results

Stylome 1.2 Large correctly cleared 8,556 of 8,557 transcripts at the 0.01% target and 8,555 at the 0.1% target. Transcribed speech is rare in our training data, so this tests how the model handles an unfamiliar kind of text. None of the transcripts appeared in our training data or reference bank.

Stylome 1.2 Large on transcribed speech
| Collection | Human texts | Flagged · 0.01% target | False-positive rate · 0.01% target | Flagged · 0.1% target | False-positive rate · 0.1% target |
| --- | --- | --- | --- | --- | --- |
| ICNALE spoken monologues | 4,358 | 1 | 0.023% | 2 | 0.046% |
| ICNALE spoken dialogues | 4,199 | 0 | 0% | 0 | 0% |
| All transcripts | 8,557 | 1 | 0.012% | 2 | 0.023% |

The monologues were recorded by phone around 2013–2014. The dialogues are the interviewee's answers from interviews recorded around 2017–2019; the interviewer's questions are not included. Both come from the [ICNALE](https://language.sakura.ne.jp/icnale/) project.

Evaluated October 19, 2026. Transcripts under the 200-character minimum were not scored (37 monologues, 36 dialogues). Each speaker contributes several transcripts.

## How to read a result

Stylome 1.2 Large distinguishes authorship composition: human writing, human writing with an AI contribution, and fully machine-derived text. These classes are not confidence levels. Low, medium, or high confidence reports relative KNN support separately; it is not a probability of correctness. A result is not proof of authorship.

What each result means
| Result | Meaning |
| --- | --- |
| Human written | The classifier found no material AI contribution. |
| AI edited | The classifier found retained human authorship with an AI contribution. This does not estimate the percentage written by AI. |
| AI generated | The classifier found fully machine-derived text, including machine-generated text rewritten by another model. |

## What we test

A model can look excellent overall while struggling with a particular kind of writing. Each report will therefore show separate results for essays, fiction, web articles, social posts, conversations, and writing by people who learned English as an additional language.

### For every model

-   Overall detection results
-   False positives on human writing
-   Results for each kind of writing
-   Performance at different passage lengths
-   Test size and evaluation date

### Before the numbers appear

We test on writing kept separate from model training and check for accidental duplicates. That gives the published results a fairer chance of reflecting new, unseen text.

## Comparing model versions

Each report lists the benchmarks completed for that model. Stylome 1.2 Large currently has EditLens, PAN 2025, StoryScope, ESL writing (TOEFL11, ICNALE, and TOEFL91), ELLIPSE, PERSUADE, and transcribed speech results. Compare models on the same test population and at matched false-positive operating points.

Available Stylome models
| Model | Designed for | Status | Results |
| --- | --- | --- | --- |
| Stylome 1.2 Large | Our newest and best model. | Published benchmark | 6 benchmark reports available |
| Stylome 1.0 | Benchmark record for the previous standard model. | Retired | 5 benchmark reports available |
| Stylome 1.0 Fast | Benchmark record for the retired fast model. | Retired | 6 benchmark reports available |

## Known limits

Heavily edited or translated passages can be harder to classify, as can templates, highly unusual writing styles, and text near the 200-character minimum. Stylome evaluates the passage itself; it does not identify who wrote it.

Technical reporting details

## Included with each report

-   Frozen holdout provenance and SHA-256 overlap audit
-   Matched-FPR model comparison with confidence intervals
-   Per-register and per-source tail metrics
-   TOEFL, Nairaland, and ELLIPSE ESL audits
-   Calibration method, sample counts, model version, and evaluation date

## Statistics kept separate

Calibrated false-positive rate

The measured error rate on a fixed evaluation set.

Live bounty findings

Examples found by people actively searching for mistakes. Useful for improving the model, but not an overall error-rate estimate.

[Read the evaluation methodology](https://stylome.io/methodology) for the rules governing holdouts, overlap audits, calibration, and comparisons between models.