July 2026 — A Hacker News thread (~162 points) resurfaced a counter-narrative to billion-parameter AI detectors: classical machine learning still catches a lot of LLM prose. Researcher lyc8503 trains TF-IDF + LinearSVC (scikit-learn) with seven binary classifiers and majority voting, reports ~85% sentence-level accuracy, and ships a browser demo at lyc8503.github.io/AITextDetector/. The stack is fully inspectable — unlike black-box SaaS — which makes it worth understanding even if you disagree with automated academic punishment.
TL;DR — classical ML detector at a glance
| Question | Answer |
|---|---|
| Algorithm? | TF-IDF features + LinearSVC (scikit-learn) |
| Ensemble? | 7 binary classifiers — one per training LLM; ≥2 votes = AI |
| Sentence accuracy? | ~85% (author-reported, in-distribution) |
| Unseen models? | ~70% on Claude Sonnet 4.6, GPT 5.2 |
| Human false positives? | Under 0.01% at 70% threshold on 10k pre-2022 fanfics |
| Web demo? | JS port — ~500k features, 107MB JSON |
| Training data? | ~10k human web fiction (pre-2022) + LLM-regenerated twins |
| Training LLMs (7)? | Gemini, Qwen, GLM, Kimi K2.5, Doubao, DeepSeek, + one more family |
| Real-world signal? | Lofter trending tags — 32% scored above 50% AI |
| Bypasses? | Translate roundtrip, "reduce AI flavor" — still often flagged |
| Open source? | GitHub lyc8503/AITextDetector |
Why TF-IDF + SVM in 2026?
Most 2024–2026 detectors market neural embeddings or perplexity from a reference LM. lyc8503's bet: bag-of-words statistics still separate LLM-regenerated text from human web fiction because:
- Training objective artifacts — function-word ratios, sentence length variance, dialogue tag patterns
- Twin regeneration — same plot, different function words → classifier learns distribution shift, not plagiarism
- Interpretability — linear weights inspectable; no GPU farm to retrain a demo
Human fiction (pre-2022)
│
├─► TF-IDF vector ──► LinearSVC #1 (vs Gemini regen)
├─► TF-IDF vector ──► LinearSVC #2 (vs Qwen regen)
├─► ... (7 models)
│
New text ─► same TF-IDF ──► 7 scores ──► majority vote (≥2) ──► AI / human
This is not magic — it is supervised text classification on a narrow genre (web fiction). Performance will drop on legal briefs, STEM homework, or Claude Opus 4.8 fiction (author admits this arms-race gap).
Training setup — twins, not crawlers alone
| Component | Detail |
|---|---|
| Human corpus | ~10k web fiction chapters pre-2022 (before ChatGPT-era contamination) |
| Positive class | Same stories regenerated by 7 LLMs (Gemini, Qwen, GLM, Kimi K2.5, Doubao, DeepSeek, etc.) |
| Features | TF-IDF — high-dimensional sparse vectors |
| Classifier | LinearSVC per source model |
| Decision rule | Majority voting — ≥2 classifiers flag AI → label AI |
Why pre-2022 human text? Reduces label noise from humans pasting ChatGPT into fanfic platforms — a common failure mode for detectors trained on "everything after 2023."
Kimi K2.5 in the training set connects to explainx.ai's Chinese open-model coverage — detectors must track non-English-West generators, not just GPT clones.
Accuracy claims — in-distribution vs unseen models
| Test slice | Approx. sentence accuracy |
|---|---|
| Held-out same distribution | ~85% |
| Unseen Claude Sonnet 4.6 | ~70% |
| Unseen GPT 5.2 | ~70% |
| Human fanfic @ 70% threshold | FP < 0.01% (10k samples) |
70% on unseen frontier models is remarkable for linear bag-of-words — and simultaneously too low for automatic expulsion from school. A 30% miss rate on adversarial or novel-model prose is a feature, not a bug, if your policy requires human review.
Author honesty: Claude Opus 4.8 fiction is harder now — expect retrains or score drift monthly.
Web demo — 107MB of JSON in the browser
The public demo lyc8503.github.io/AITextDetector/ ports the sklearn pipeline to JavaScript:
| Spec | Value |
|---|---|
| Feature space | ~500k TF-IDF dimensions |
| Artifact size | ~107MB JSON weights |
| Runtime | Client-side — no server upload (privacy win) |
Paste a paragraph → get per-sentence scores and ensemble vote. Good for journalists, moderators, and researchers auditing slop — bad as a sole disciplinary record.
For platform-scale slop (PRs, feeds), see Slopocalypse / SlopGuard patterns.
Lofter field test — 32% trending tags above 50% AI
The author scanned Lofter (Chinese creative blogging) trending tags — 32% of sampled posts scored above 50% AI by the ensemble.
| Reading | Caveat |
|---|---|
| Platform slop influx is real | Tag trending ≠ random sample |
| Genre match helps detector | Fiction-like posts align with training |
| Cultural context | Western educators should not import Lofter stats blindly |
Ties to AI copying creativity — when copying is free, detection becomes an arms race, not a morality substitute.
Bypass attempts — weaker than HN feared
Author-tested evasions:
| Bypass | Result |
|---|---|
| Translate roundtrip (e.g. EN→ZH→EN) | Still often flagged |
| "Reduce AI flavor" prompt | Still often flagged |
HN skeptics note stronger paraphrase models and humanization services will adapt. Classical detectors age faster when attackers optimize against public weights — the GitHub repo is a double-edged sword (auditability vs evasion).
HN debate — skeptics, Pangram, and harm
Thread themes:
- Arms race — every public method becomes training noise for the next generator
- False positives harm — formal ESL prose, neurodivergent writing styles flagged by brittle detectors
- Pangram cited — commercial SOTA neural detector with enterprise API (name only — no competitor link per explainx.ai policy)
- Author humility — Opus 4.8 fiction evades; retrains required
explainx.ai position: Use scores as triage, like SEO/GEO audits — not verdicts. For images, prefer C2PA content credentials where signing exists; prose lacks universal credentials today.
Responsible use matrix
| Use case | OK? | Notes |
|---|---|---|
| Editorial queue prioritization | Yes | Human confirms before action |
| Research on slop prevalence | Yes | Report methods + genre limits |
| Moderation with appeals | Careful | Disclose detector version |
| Automatic essay grading penalty | No | FP ruins lives — see HN ethicists |
| Legal evidence | No | Not forensic-grade |
| Marketing "100% AI free" badge | No | Misleading — scores are probabilistic |
Aligns with ThoughtWorks zero-cost fallacy — "free detection" still has social cost when wrong.
Reproduce locally
git clone https://github.com/lyc8503/AITextDetector.git
cd AITextDetector
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
# Follow README for train/eval scripts — sklearn pipeline
Extend with your domain corpus (internal docs, support tickets) — do not assume fanfic weights transfer to engineering RFCs without retraining.
Classical vs neural detectors — when to pick which
| Criterion | TF-IDF + SVM (lyc8503) | Neural / commercial SOTA |
|---|---|---|
| Auditability | High — sparse weights | Low — black box |
| Offline demo | 107MB JSON in browser | Usually API-only |
| Genre sensitivity | High — fiction-tuned | Varies by vendor |
| Unseen model robustness | ~70% (author) | Often higher — unverified here |
| Maintenance | You retrain | Vendor chases models |
| Cost | Open source | Subscription |
For multimodal provenance, pair prose scores with C2PA on LinkedIn/X — different problem, complementary signal.
Summary
lyc8503's AITextDetector proves TF-IDF + LinearSVC with 7-model majority voting still hits ~85% sentence accuracy on fiction-like text and ~70% on unseen Claude Sonnet 4.6 / GPT 5.2 — with under 0.01% human false positives on pre-2022 fanfic at a 70% threshold. The 107MB browser demo is transparent and privacy-friendly; Lofter trending data shows real-world slop pressure. Do not auto-punish students or employees from scores alone — use human review, disclose limits, and expect Opus 4.8-class fiction to evade. See AI copying debate for why detection ≠ creativity policy.
Related on explainx.ai
- AI copying creativity — shadcn debate
- LinkedIn C2PA content credentials for AI images
- Slopocalypse — AI slop on the internet
- ThoughtWorks zero-cost fallacy in the agentic era
- SEO & GEO agent skill — measurement not verdicts
- Hallmark — anti-AI-slop design skill (Jul 17) · Install
- Top Chinese AI companies — Kimi, Qwen, DeepSeek
- AI benchmarks complete guide
- What is generative AI? — complete guide
Official sources: GitHub lyc8503/AITextDetector · Web demo
Accuracy figures and Lofter statistics accurate as of July 17, 2026 per author blog and HN thread. Retrain on your genre before high-stakes use; model vendors ship weekly.