LLM Text Detection with Classical ML — TF-IDF + SVM That Still Works (2026)

July 2026 — A Hacker News thread (~162 points) resurfaced a counter-narrative to billion-parameter AI detectors: classical machine learning still catches a lot of LLM prose. Researcher lyc8503 trains TF-IDF + LinearSVC (scikit-learn) with seven binary classifiers and majority voting, reports ~85% sentence-level accuracy, and ships a browser demo at lyc8503.github.io/AITextDetector/. The stack is fully inspectable — unlike black-box SaaS — which makes it worth understanding even if you disagree with automated academic punishment.

TL;DR — classical ML detector at a glance

Why TF-IDF + SVM in 2026?

Most 2024–2026 detectors market neural embeddings or perplexity from a reference LM. lyc8503's bet: bag-of-words statistics still separate LLM-regenerated text from human web fiction because:

1. Training objective artifacts — function-word ratios, sentence length variance, dialogue tag patterns
2. Twin regeneration — same plot, different function words → classifier learns distribution shift, not plagiarism
3. Interpretability — linear weights inspectable; no GPU farm to retrain a demo

Human fiction (pre-2022)
         │
         ├─► TF-IDF vector ──► LinearSVC #1 (vs Gemini regen)
         ├─► TF-IDF vector ──► LinearSVC #2 (vs Qwen regen)
         ├─► ... (7 models)
         │
New text ─► same TF-IDF ──► 7 scores ──► majority vote (≥2) ──► AI / human

This is not magic — it is supervised text classification on a narrow genre (web fiction). Performance will drop on legal briefs, STEM homework, or Claude Opus 4.8 fiction (author admits this arms-race gap).

Training setup — twins, not crawlers alone

Why pre-2022 human text? Reduces label noise from humans pasting ChatGPT into fanfic platforms — a common failure mode for detectors trained on "everything after 2023."

Kimi K2.5 in the training set connects to explainx.ai's Chinese open-model coverage — detectors must track non-English-West generators, not just GPT clones.

Accuracy claims — in-distribution vs unseen models

70% on unseen frontier models is remarkable for linear bag-of-words — and simultaneously too low for automatic expulsion from school. A 30% miss rate on adversarial or novel-model prose is a feature, not a bug, if your policy requires human review.

Author honesty: Claude Opus 4.8 fiction is harder now — expect retrains or score drift monthly.

Web demo — 107MB of JSON in the browser

The public demo lyc8503.github.io/AITextDetector/ ports the sklearn pipeline to JavaScript:

Paste a paragraph → get per-sentence scores and ensemble vote. Good for journalists, moderators, and researchers auditing slop — bad as a sole disciplinary record.

For platform-scale slop (PRs, feeds), see Slopocalypse / SlopGuard patterns.

Lofter field test — 32% trending tags above 50% AI

The author scanned Lofter (Chinese creative blogging) trending tags — 32% of sampled posts scored above 50% AI by the ensemble.

Ties to AI copying creativity — when copying is free, detection becomes an arms race, not a morality substitute.

Bypass attempts — weaker than HN feared

Author-tested evasions:

HN skeptics note stronger paraphrase models and humanization services will adapt. Classical detectors age faster when attackers optimize against public weights — the GitHub repo is a double-edged sword (auditability vs evasion).

HN debate — skeptics, Pangram, and harm

Thread themes:

1. Arms race — every public method becomes training noise for the next generator
2. False positives harm — formal ESL prose, neurodivergent writing styles flagged by brittle detectors
3. Pangram cited — commercial SOTA neural detector with enterprise API (name only — no competitor link per explainx.ai policy)
4. Author humility — Opus 4.8 fiction evades; retrains required

explainx.ai position: Use scores as triage, like SEO/GEO audits — not verdicts. For images, prefer C2PA content credentials where signing exists; prose lacks universal credentials today.

Responsible use matrix

Aligns with ThoughtWorks zero-cost fallacy — "free detection" still has social cost when wrong.

Reproduce locally

git clone https://github.com/lyc8503/AITextDetector.git
cd AITextDetector
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Follow README for train/eval scripts — sklearn pipeline

Extend with your domain corpus (internal docs, support tickets) — do not assume fanfic weights transfer to engineering RFCs without retraining.

Classical vs neural detectors — when to pick which

For multimodal provenance, pair prose scores with C2PA on LinkedIn/X — different problem, complementary signal.

How should you evaluate a detector outside its training genre?

Keep a labeled evaluation set from the genre you actually moderate, and split related texts together. Sentences from the same story should not leak across training and test partitions in a way that makes the test artificially easy. Record the labeling process and the detector version before comparing results.

Report false positives and false negatives separately. A single accuracy figure depends on the balance of human and generated examples; it cannot tell an editor how many legitimate submissions may be flagged in a mostly human queue. Inspect cases by language, length, and editing history rather than assuming one threshold transfers unchanged.

Use the score to prioritize review only when the workflow includes independent evidence and a way to contest mistakes. The project's reported fiction results are evidence for its studied setting, not proof of authorship for an individual essay, support message, or legal document.

Summary

lyc8503's AITextDetector proves TF-IDF + LinearSVC with 7-model majority voting still hits ~85% sentence accuracy on fiction-like text and ~70% on unseen Claude Sonnet 4.6 / GPT 5.2 — with under 0.01% human false positives on pre-2022 fanfic at a 70% threshold. The 107MB browser demo is transparent and privacy-friendly; Lofter trending data shows real-world slop pressure. Do not auto-punish students or employees from scores alone — use human review, disclose limits, and expect Opus 4.8-class fiction to evade. See AI copying debate for why detection ≠ creativity policy.

Update — September 1, 2026: Mollick argues detectors like this and Pangram are now well-known enough to have ended AI writing's free-pass era — see is AI writing a safe job? Mollick, Demirbas, and the wicked-problem debate.

Related on explainx.ai

• Is AI writing a safe job? Mollick, Demirbas, and the wicked-problem debate

• Hanover Institute — GPTZero flagged 11/12 reports — why a detector score still isn’t the investigation

• Top 10 signs of AI-generated text — the free, tool-free reading checklist this classifier's features overlap with

• What AI Watermarking Actually Changes for Students and Teachers — how classifiers like this compare to real watermarking in classrooms, and the documented bias against non-native English writers

• How AI text watermarking actually works — the keyed-sampling approach that replaces classifier guessing, with a derivable false-positive rate

• Will every AI model watermark its output?

• Anthropic is watermarking Claude text: what it marks and what it misses — a stronger signal than a classifier, still not proof of authorship

• AI copying creativity — shadcn debate

• LinkedIn C2PA content credentials for AI images

• Slopocalypse — AI slop on the internet

• ThoughtWorks zero-cost fallacy in the agentic era

• SEO & GEO agent skill — measurement not verdicts

• Hallmark — anti-AI-slop design skill (Jul 17) · Install

• Top Chinese AI companies — Kimi, Qwen, DeepSeek

• AI benchmarks complete guide

• What is generative AI? — complete guide

Official sources: GitHub lyc8503/AITextDetector · Web demo

Accuracy figures and Lofter statistics accurate as of July 17, 2026 per author blog and HN thread. Retrain on your genre before high-stakes use; model vendors ship weekly.