How does lyc8503's AI text detector work?

The AITextDetector project (GitHub lyc8503/AITextDetector) uses TF-IDF vectorization plus LinearSVC from scikit-learn — seven separate binary classifiers, one per training LLM family, with majority voting (≥2 flags = AI). It is deliberately classical ML, not a neural detector. The author reports ~85% sentence-level accuracy on held-out data and ships a browser demo at lyc8503.github.io/AITextDetector/ implemented in JavaScript (~500k features, 107MB JSON weights).

How accurate is it on models not seen during training?

On unseen frontier models — Claude Sonnet 4.6 and GPT 5.2 in the author's tests — detection remains around ~70% sentence-level accuracy. That is lower than in-distribution scores but non-trivial for a TF-IDF baseline. The author notes Claude Opus 4.8 fiction is now harder to detect, acknowledging an ongoing arms race.

What is the human false positive rate?

On ~10k pre-2022 human web fiction samples, false positives above 70% AI score were under 0.01% at the chosen threshold — far lower than many neural detectors that flag formal human prose. That does not eliminate false positives in other genres (legal, academic, ESL writing). Never auto-penalize from score alone.

Can users bypass this detector easily?

Author-tested bypasses were weak: translate roundtrip and "reduce AI flavor" prompts still often flagged. HN skeptics argue adaptive paraphrase and newer models will erode margins. Treat detection as a triage signal, not proof — especially for high-stakes academic or employment decisions without human review.

How does this compare to commercial detectors like Pangram?

HN commenters cited Pangram and similar commercial stacks as state-of-the-art neural/proprietary approaches with broader model coverage and enterprise APIs. lyc8503's stack wins on transparency, reproducibility, and offline demo size — not necessarily peak accuracy on every genre. Pick by auditability vs managed SLA.

Should schools use this to punish AI writing?

No — not without human review. explainx.ai aligns with the author and HN ethicists: classical or neural scores are probabilistic; false positives harm students (especially non-native English writers). Use detectors for editorial triage, research, or platform moderation with appeals — not automated academic punishment. Pair with provenance tools like C2PA where media is concerned, not prose alone.

LLM Text Detection — TF-IDF SVM (July 2026) | explainx.ai Blog

July 2026 — A Hacker News thread (~162 points) resurfaced a counter-narrative to billion-parameter AI detectors: classical machine learning still catches a lot of LLM prose. Researcher lyc8503 trains TF-IDF + LinearSVC (scikit-learn) with seven binary classifiers and majority voting, reports ~85% sentence-level accuracy, and ships a browser demo at lyc8503.github.io/AITextDetector/. The stack is fully inspectable — unlike black-box SaaS — which makes it worth understanding even if you disagree with automated academic punishment.

TL;DR — classical ML detector at a glance

table · 2 cols

Question	Answer
Algorithm?	TF-IDF features + LinearSVC (scikit-learn)
Ensemble?	7 binary classifiers — one per training LLM; ≥2 votes = AI
Sentence accuracy?	~85% (author-reported, in-distribution)
Unseen models?	~70% on Claude Sonnet 4.6, GPT 5.2
Human false positives?	Under 0.01% at 70% threshold on 10k pre-2022 fanfics
Web demo?	JS port — ~500k features, 107MB JSON
Training data?	~10k human web fiction (pre-2022) + LLM-regenerated twins
Training LLMs (7)?	Gemini, Qwen, GLM, Kimi K2.5, Doubao, DeepSeek, + one more family
Real-world signal?	Lofter trending tags — 32% scored above 50% AI
Bypasses?	Translate roundtrip, "reduce AI flavor" — still often flagged
Open source?	GitHub lyc8503/AITextDetector

Why TF-IDF + SVM in 2026?

Most 2024–2026 detectors market neural embeddings or perplexity from a reference LM. lyc8503's bet: bag-of-words statistics still separate LLM-regenerated text from human web fiction because:

Training objective artifacts — function-word ratios, sentence length variance, dialogue tag patterns
Twin regeneration — same plot, different function words → classifier learns distribution shift, not plagiarism
Interpretability — linear weights inspectable; no GPU farm to retrain a demo

snippet

Human fiction (pre-2022)
         │
         ├─► TF-IDF vector ──► LinearSVC #1 (vs Gemini regen)
         ├─► TF-IDF vector ──► LinearSVC #2 (vs Qwen regen)
         ├─► ... (7 models)
         │
New text ─► same TF-IDF ──► 7 scores ──► majority vote (≥2) ──► AI / human

This is not magic — it is supervised text classification on a narrow genre (web fiction). Performance will drop on legal briefs, STEM homework, or Claude Opus 4.8 fiction (author admits this arms-race gap).

Training setup — twins, not crawlers alone

table · 2 cols

Component	Detail
Human corpus	~10k web fiction chapters pre-2022 (before ChatGPT-era contamination)
Positive class	Same stories regenerated by 7 LLMs (Gemini, Qwen, GLM, Kimi K2.5, Doubao, DeepSeek, etc.)
Features	TF-IDF — high-dimensional sparse vectors
Classifier	LinearSVC per source model
Decision rule	Majority voting — ≥2 classifiers flag AI → label AI

Why pre-2022 human text? Reduces label noise from humans pasting ChatGPT into fanfic platforms — a common failure mode for detectors trained on "everything after 2023."

Kimi K2.5 in the training set connects to explainx.ai's Chinese open-model coverage — detectors must track non-English-West generators, not just GPT clones.

Accuracy claims — in-distribution vs unseen models

table · 2 cols

Test slice	Approx. sentence accuracy
Held-out same distribution	~85%
Unseen Claude Sonnet 4.6	~70%
Unseen GPT 5.2	~70%
Human fanfic @ 70% threshold	FP < 0.01% (10k samples)

70% on unseen frontier models is remarkable for linear bag-of-words — and simultaneously too low for automatic expulsion from school. A 30% miss rate on adversarial or novel-model prose is a feature, not a bug, if your policy requires human review.

Author honesty: Claude Opus 4.8 fiction is harder now — expect retrains or score drift monthly.

Web demo — 107MB of JSON in the browser

The public demo lyc8503.github.io/AITextDetector/ ports the sklearn pipeline to JavaScript:

table · 2 cols

Spec	Value
Feature space	~500k TF-IDF dimensions
Artifact size	~107MB JSON weights
Runtime	Client-side — no server upload (privacy win)

Paste a paragraph → get per-sentence scores and ensemble vote. Good for journalists, moderators, and researchers auditing slop — bad as a sole disciplinary record.

For platform-scale slop (PRs, feeds), see Slopocalypse / SlopGuard patterns.

The author scanned Lofter (Chinese creative blogging) trending tags — 32% of sampled posts scored above 50% AI by the ensemble.

table · 2 cols

Reading	Caveat
Platform slop influx is real	Tag trending ≠ random sample
Genre match helps detector	Fiction-like posts align with training
Cultural context	Western educators should not import Lofter stats blindly

Ties to AI copying creativity — when copying is free, detection becomes an arms race, not a morality substitute.

Bypass attempts — weaker than HN feared

Author-tested evasions:

table · 2 cols

Bypass	Result
Translate roundtrip (e.g. EN→ZH→EN)	Still often flagged
"Reduce AI flavor" prompt	Still often flagged

HN skeptics note stronger paraphrase models and humanization services will adapt. Classical detectors age faster when attackers optimize against public weights — the GitHub repo is a double-edged sword (auditability vs evasion).

HN debate — skeptics, Pangram, and harm

Thread themes:

Arms race — every public method becomes training noise for the next generator
False positives harm — formal ESL prose, neurodivergent writing styles flagged by brittle detectors
Pangram cited — commercial SOTA neural detector with enterprise API (name only — no competitor link per explainx.ai policy)
Author humility — Opus 4.8 fiction evades; retrains required

explainx.ai position: Use scores as triage, like SEO/GEO audits — not verdicts. For images, prefer C2PA content credentials where signing exists; prose lacks universal credentials today.

Responsible use matrix

table · 3 cols

Use case	OK?	Notes
Editorial queue prioritization	Yes	Human confirms before action
Research on slop prevalence	Yes	Report methods + genre limits
Moderation with appeals	Careful	Disclose detector version
Automatic essay grading penalty	No	FP ruins lives — see HN ethicists
Legal evidence	No	Not forensic-grade
Marketing "100% AI free" badge	No	Misleading — scores are probabilistic

Aligns with ThoughtWorks zero-cost fallacy — "free detection" still has social cost when wrong.

Reproduce locally

bash

git clone https://github.com/lyc8503/AITextDetector.git
cd AITextDetector
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Follow README for train/eval scripts — sklearn pipeline

Extend with your domain corpus (internal docs, support tickets) — do not assume fanfic weights transfer to engineering RFCs without retraining.

Classical vs neural detectors — when to pick which

table · 3 cols

Criterion	TF-IDF + SVM (lyc8503)	Neural / commercial SOTA
Auditability	High — sparse weights	Low — black box
Offline demo	107MB JSON in browser	Usually API-only
Genre sensitivity	High — fiction-tuned	Varies by vendor
Unseen model robustness	~70% (author)	Often higher — unverified here
Maintenance	You retrain	Vendor chases models
Cost	Open source	Subscription

For multimodal provenance, pair prose scores with C2PA on LinkedIn/X — different problem, complementary signal.

Summary

lyc8503's AITextDetector proves TF-IDF + LinearSVC with 7-model majority voting still hits ~85% sentence accuracy on fiction-like text and ~70% on unseen Claude Sonnet 4.6 / GPT 5.2 — with under 0.01% human false positives on pre-2022 fanfic at a 70% threshold. The 107MB browser demo is transparent and privacy-friendly; Lofter trending data shows real-world slop pressure. Do not auto-punish students or employees from scores alone — use human review, disclose limits, and expect Opus 4.8-class fiction to evade. See AI copying debate for why detection ≠ creativity policy.

Hanover Institute — GPTZero flagged 11/12 reports — why a detector score still isn’t the investigation
Top 10 signs of AI-generated text — the free, tool-free reading checklist this classifier's features overlap with
What AI Watermarking Actually Changes for Students and Teachers — how classifiers like this compare to real watermarking in classrooms, and the documented bias against non-native English writers
How AI text watermarking actually works — the keyed-sampling approach that replaces classifier guessing, with a derivable false-positive rate
Will every AI model watermark its output?
Anthropic is watermarking Claude text: what it marks and what it misses — a stronger signal than a classifier, still not proof of authorship
AI copying creativity — shadcn debate
LinkedIn C2PA content credentials for AI images
Slopocalypse — AI slop on the internet
ThoughtWorks zero-cost fallacy in the agentic era
SEO & GEO agent skill — measurement not verdicts
Hallmark — anti-AI-slop design skill (Jul 17) · Install
Top Chinese AI companies — Kimi, Qwen, DeepSeek
AI benchmarks complete guide
What is generative AI? — complete guide

Official sources: GitHub lyc8503/AITextDetector · Web demo

Accuracy figures and Lofter statistics accurate as of July 17, 2026 per author blog and HN thread. Retrain on your genre before high-stakes use; model vendors ship weekly.

TL;DR — classical ML detector at a glance

table · 2 cols

Question	Answer
Algorithm?	TF-IDF features + LinearSVC (scikit-learn)
Ensemble?	7 binary classifiers — one per training LLM; ≥2 votes = AI
Sentence accuracy?	~85% (author-reported, in-distribution)
Unseen models?	~70% on Claude Sonnet 4.6, GPT 5.2
Human false positives?	Under 0.01% at 70% threshold on 10k pre-2022 fanfics
Web demo?	JS port — ~500k features, 107MB JSON
Training data?	~10k human web fiction (pre-2022) + LLM-regenerated twins
Training LLMs (7)?	Gemini, Qwen, GLM, Kimi K2.5, Doubao, DeepSeek, + one more family
Real-world signal?	Lofter trending tags — 32% scored above 50% AI
Bypasses?	Translate roundtrip, "reduce AI flavor" — still often flagged
Open source?	GitHub lyc8503/AITextDetector

Why TF-IDF + SVM in 2026?

Training objective artifacts — function-word ratios, sentence length variance, dialogue tag patterns
Twin regeneration — same plot, different function words → classifier learns distribution shift, not plagiarism
Interpretability — linear weights inspectable; no GPU farm to retrain a demo

snippet

Human fiction (pre-2022)
         │
         ├─► TF-IDF vector ──► LinearSVC #1 (vs Gemini regen)
         ├─► TF-IDF vector ──► LinearSVC #2 (vs Qwen regen)
         ├─► ... (7 models)
         │
New text ─► same TF-IDF ──► 7 scores ──► majority vote (≥2) ──► AI / human

Training setup — twins, not crawlers alone

table · 2 cols

Component	Detail
Human corpus	~10k web fiction chapters pre-2022 (before ChatGPT-era contamination)
Positive class	Same stories regenerated by 7 LLMs (Gemini, Qwen, GLM, Kimi K2.5, Doubao, DeepSeek, etc.)
Features	TF-IDF — high-dimensional sparse vectors
Classifier	LinearSVC per source model
Decision rule	Majority voting — ≥2 classifiers flag AI → label AI

Why pre-2022 human text? Reduces label noise from humans pasting ChatGPT into fanfic platforms — a common failure mode for detectors trained on "everything after 2023."

Kimi K2.5 in the training set connects to explainx.ai's Chinese open-model coverage — detectors must track non-English-West generators, not just GPT clones.

Accuracy claims — in-distribution vs unseen models

table · 2 cols

Test slice	Approx. sentence accuracy
Held-out same distribution	~85%
Unseen Claude Sonnet 4.6	~70%
Unseen GPT 5.2	~70%
Human fanfic @ 70% threshold	FP < 0.01% (10k samples)

Author honesty: Claude Opus 4.8 fiction is harder now — expect retrains or score drift monthly.

Web demo — 107MB of JSON in the browser

The public demo lyc8503.github.io/AITextDetector/ ports the sklearn pipeline to JavaScript:

table · 2 cols

Spec	Value
Feature space	~500k TF-IDF dimensions
Artifact size	~107MB JSON weights
Runtime	Client-side — no server upload (privacy win)

Paste a paragraph → get per-sentence scores and ensemble vote. Good for journalists, moderators, and researchers auditing slop — bad as a sole disciplinary record.

For platform-scale slop (PRs, feeds), see Slopocalypse / SlopGuard patterns.

The author scanned Lofter (Chinese creative blogging) trending tags — 32% of sampled posts scored above 50% AI by the ensemble.

table · 2 cols

Reading	Caveat
Platform slop influx is real	Tag trending ≠ random sample
Genre match helps detector	Fiction-like posts align with training
Cultural context	Western educators should not import Lofter stats blindly

Ties to AI copying creativity — when copying is free, detection becomes an arms race, not a morality substitute.

Bypass attempts — weaker than HN feared

Author-tested evasions:

table · 2 cols

Bypass	Result
Translate roundtrip (e.g. EN→ZH→EN)	Still often flagged
"Reduce AI flavor" prompt	Still often flagged

HN debate — skeptics, Pangram, and harm

Thread themes:

Arms race — every public method becomes training noise for the next generator
False positives harm — formal ESL prose, neurodivergent writing styles flagged by brittle detectors
Pangram cited — commercial SOTA neural detector with enterprise API (name only — no competitor link per explainx.ai policy)
Author humility — Opus 4.8 fiction evades; retrains required

Responsible use matrix

table · 3 cols

Use case	OK?	Notes
Editorial queue prioritization	Yes	Human confirms before action
Research on slop prevalence	Yes	Report methods + genre limits
Moderation with appeals	Careful	Disclose detector version
Automatic essay grading penalty	No	FP ruins lives — see HN ethicists
Legal evidence	No	Not forensic-grade
Marketing "100% AI free" badge	No	Misleading — scores are probabilistic

Aligns with ThoughtWorks zero-cost fallacy — "free detection" still has social cost when wrong.

Reproduce locally

bash

git clone https://github.com/lyc8503/AITextDetector.git
cd AITextDetector
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Follow README for train/eval scripts — sklearn pipeline

Extend with your domain corpus (internal docs, support tickets) — do not assume fanfic weights transfer to engineering RFCs without retraining.

Classical vs neural detectors — when to pick which

table · 3 cols

Criterion	TF-IDF + SVM (lyc8503)	Neural / commercial SOTA
Auditability	High — sparse weights	Low — black box
Offline demo	107MB JSON in browser	Usually API-only
Genre sensitivity	High — fiction-tuned	Varies by vendor
Unseen model robustness	~70% (author)	Often higher — unverified here
Maintenance	You retrain	Vendor chases models
Cost	Open source	Subscription

For multimodal provenance, pair prose scores with C2PA on LinkedIn/X — different problem, complementary signal.

Summary

Hanover Institute — GPTZero flagged 11/12 reports — why a detector score still isn’t the investigation
Top 10 signs of AI-generated text — the free, tool-free reading checklist this classifier's features overlap with
What AI Watermarking Actually Changes for Students and Teachers — how classifiers like this compare to real watermarking in classrooms, and the documented bias against non-native English writers
How AI text watermarking actually works — the keyed-sampling approach that replaces classifier guessing, with a derivable false-positive rate
Will every AI model watermark its output?
Anthropic is watermarking Claude text: what it marks and what it misses — a stronger signal than a classifier, still not proof of authorship
AI copying creativity — shadcn debate
LinkedIn C2PA content credentials for AI images
Slopocalypse — AI slop on the internet
ThoughtWorks zero-cost fallacy in the agentic era
SEO & GEO agent skill — measurement not verdicts
Hallmark — anti-AI-slop design skill (Jul 17) · Install
Top Chinese AI companies — Kimi, Qwen, DeepSeek
AI benchmarks complete guide
What is generative AI? — complete guide

Official sources: GitHub lyc8503/AITextDetector · Web demo

Accuracy figures and Lofter statistics accurate as of July 17, 2026 per author blog and HN thread. Retrain on your genre before high-stakes use; model vendors ship weekly.

LLM Text Detection with Classical ML — TF-IDF + SVM That Still Works (2026)

TL;DR — classical ML detector at a glance

Why TF-IDF + SVM in 2026?

Training setup — twins, not crawlers alone

Accuracy claims — in-distribution vs unseen models

Web demo — 107MB of JSON in the browser

Bypass attempts — weaker than HN feared

HN debate — skeptics, Pangram, and harm

Responsible use matrix

Reproduce locally

Classical vs neural detectors — when to pick which

Summary

LLM Text Detection with Classical ML — TF-IDF + SVM That Still Works (2026)

TL;DR — classical ML detector at a glance

Why TF-IDF + SVM in 2026?

Training setup — twins, not crawlers alone

Accuracy claims — in-distribution vs unseen models

Web demo — 107MB of JSON in the browser

Bypass attempts — weaker than HN feared

HN debate — skeptics, Pangram, and harm

Responsible use matrix

Reproduce locally

Classical vs neural detectors — when to pick which

Summary

Related posts

What Is Bias in AI? Types, Examples, and How to Fix It [2026]

C2PA Android Cameras Broken: Pixel Assurance Level 2 Forged Anyway

What Is AI Ethics? A Complete Guide

Related posts

What Is Bias in AI? Types, Examples, and How to Fix It [2026]

C2PA Android Cameras Broken: Pixel Assurance Level 2 Forged Anyway

What Is AI Ethics? A Complete Guide

TL;DR — classical ML detector at a glance

Why TF-IDF + SVM in 2026?

Training setup — twins, not crawlers alone

Accuracy claims — in-distribution vs unseen models

Web demo — 107MB of JSON in the browser

Lofter field test — 32% trending tags above 50% AI

Bypass attempts — weaker than HN feared

HN debate — skeptics, Pangram, and harm

Responsible use matrix

Reproduce locally

Classical vs neural detectors — when to pick which

Summary

Related on explainx.ai

TL;DR — classical ML detector at a glance

Why TF-IDF + SVM in 2026?

Training setup — twins, not crawlers alone

Accuracy claims — in-distribution vs unseen models

Web demo — 107MB of JSON in the browser

Lofter field test — 32% trending tags above 50% AI

Bypass attempts — weaker than HN feared

HN debate — skeptics, Pangram, and harm

Responsible use matrix

Reproduce locally

Classical vs neural detectors — when to pick which

Summary

Related on explainx.ai

Related posts

What Is Bias in AI? Types, Examples, and How to Fix It [2026]

C2PA Android Cameras Broken: Pixel Assurance Level 2 Forged Anyway

What Is AI Ethics? A Complete Guide

Related posts

What Is Bias in AI? Types, Examples, and How to Fix It [2026]

C2PA Android Cameras Broken: Pixel Assurance Level 2 Forged Anyway

What Is AI Ethics? A Complete Guide