VRSD: A Teacher-Only Prompt That Stops Self-Distillation From Killing Reasoning

A self-distilling model can learn to stop thinking. That is the finding behind VRSD, short for Self-Distillation with Verbal Regularization, a paper posted to Hugging Face on October 14, 2026 with code on GitHub. The authors show that a common training trick, giving a model's own "teacher" copy privileged information, can make the student collapse into short, overconfident answers. Their fix is a single sentence-level change: a system prompt that only the teacher sees during training.

This post explains the setup in plain terms, what the authors report, how to try the code, and where the evidence is thin. If you fine-tune small reasoning models, it is one of the cheaper ideas of the week to test.

Quick reference

What is on-policy self-distillation?

In ordinary distillation, a big teacher model trains a small student. In on-policy self-distillation, the teacher and student are the same model. The trick is that the teacher gets information the student cannot see, called privileged information (PI), such as a verified correct solution. The student generates its own attempts, and the teacher, who knows the answer, scores them by providing next-token distributions. Training pulls the student toward what a better-informed version of itself would say.

This is attractive because it needs no larger model and no human labels beyond the solutions. It is part of a broader wave of work in which models improve using signals they produce themselves; we covered one example in Sakana's MASS, a 27B model that trains itself through agent teams.

What goes wrong

The authors report a specific failure. When the teacher sees the answer, it tends to jump straight to it. Its distributions put little weight on tokens like "Wait" and "However" that signal a model double-checking its work. The student learns from those distributions and imitates the shortcut: it stops re-checking, its token entropy collapses (near zero in some runs), and downstream accuracy falls. In the authors' words, the model stops thinking.

This matters because the behaviors that look like inefficiency, hesitation and backtracking, are often where reasoning models catch their own errors. Removing them makes outputs shorter and cheaper but less reliable, which is the opposite of what you want from a reasoning model.

How VRSD works

VRSD leaves the pipeline as it is and changes what the teacher is told. The teacher receives the question, a correct solution and a training-only system prompt that asks it to verify intermediate steps, consider alternative reasoning paths and revisit earlier judgments. The student still sees only the question. The authors model the prompt as attenuating the shift in log-probabilities that PI causes, and derive conditions under which teacher entropy is preserved.

The repository provides two prompt designs. Single uses one task-level reflection prompt. Multi splits the rollouts of each question across aspect-specific prompts. Everything runs through scripts/train.sh, and the code extends verl and SDPO.

The reported results

Across three models and four tasks, the authors report:

• Pass@1 beats OPSD in 11 of 12 settings, by 3.9 percentage points on average.
• Maj@16 is never below OPSD, and Worst@16 also improves.
• Entropy recovers; in a SQL setting it moves from 0.03 to 0.54.
• The rate of misleading reasoning drops from 3.0% to 1.6%, which the authors present as more faithful reasoning.

The four tasks are deliberately varied: grading short answers (ASAP-SAS), chemistry questions (SciKnowEval), text-to-SQL (Spider) and Python coding (LiveCodeBench). That spread suggests the effect is not specific to one domain, though all three models are small open models, so nothing here speaks to frontier-scale training.

How to try it

The README lists the setup. A Python 3.12 environment, then pip install -r install/requirements.txt and the CUDA requirements for flash-attention. It was tested with torch 2.7.1 on CUDA 12.6, vLLM 0.10.0, transformers 4.57.1 and Ray 2.55.1. Each run uses two GPUs; the text tasks fit on 80 GB cards, while the Python runs used 141 GB cards. Data scripts write train and test files with the same prompts, answers and splits as the paper, which makes reproduction straightforward in principle.

If you already run on-policy self-distillation, the practical test is cheap: add the reflection prompt on the teacher side only, keep everything else identical, and plot token entropy and the frequency of re-checking tokens alongside accuracy. If entropy was collapsing before, you should see it recover. If it was not, you may not need VRSD.

A worked picture of the collapse

Imagine training a small model on text-to-SQL. For each question, the teacher copy is shown the correct query. A teacher that knows the answer has little reason to hedge, so its next-token distribution is sharply peaked: it writes the query directly. The student, trained to match those distributions, learns to emit queries with the same confidence, but it does not have the answer in front of it. Where the teacher could be confident because of the hint, the student is confident without justification.

The authors measure this as entropy, a number describing how spread out the model's token probabilities are. In their SQL runs, entropy sat near 0.03, effectively a model that has stopped considering alternatives. With the VRSD prompt on the teacher side, entropy recovered to 0.54 in the same setting. Higher entropy here is not noise; it reflects the student retaining tokens such as "Wait" and "However," which mark verification steps. The authors' misleading-reasoning measure, which captures reasoning that looks plausible but leads away from the answer, fell from 3.0% to 1.6%.

The idea is conceptually simple: the hint gives the teacher too much confidence, and a prompt that tells the teacher to behave as if it still needed to verify cancels part of that effect. It resembles how a tutor who already knows the solution can still be asked to walk through it carefully, so the learner sees the checking process, not just the final line.

A checklist if you want to test it

1. Reproduce the baseline first. Run plain OPSD on your task and log accuracy, token entropy and the frequency of re-checking words. You need the collapse to be present before a fix can matter.
2. Add the teacher-only prompt. Start with the single, task-level reflection prompt from the repo before trying the multi-aspect variant.
3. Keep everything else fixed. Same data, same seeds where possible, same student input. The paper's claim is about that one change.
4. Compare more than Pass@1. Look at Maj@16 and Worst@16 as the authors do; a method that raises the average while worsening the worst case is a different trade.
5. Read samples. Check a handful of outputs for reasoning that is longer but not better. Entropy can rise for the wrong reasons.
6. Budget for compute. The repo uses two GPUs per run, and the Python experiments used 141 GB cards, so small-scale replication is within reach of a lab but not a laptop.

Why this matters for builders

Three takeaways, even if you never run this code:

1. Self-generated training signals have failure modes. A cheap way to improve a model with its own outputs can quietly remove the behavior you want. Monitor entropy, not only accuracy.
2. Prompts are a training lever, not only an inference lever. The change here is a sentence in a system prompt that never reaches the student. Similar teacher-side prompting could matter for other distillation setups.
3. Small models keep getting better at reasoning through training tricks. For context on how far small models have come, see VibeThinker 3B and the Agent-Reflex work on reflective tool agents with Qwen3-8B. The labs' side of the story, protecting against copying of model outputs, is in Anthropic's distillation defenses.

Limits and open questions

• Small models, four tasks. Gains of 3.9 points on average across Qwen3-4B, Qwen3-8B and OLMo-3-7B-Instruct do not guarantee anything at 70B or on open-ended tasks.
• Reported by the authors. We read the paper page and README; we have not reproduced the numbers, and there is no independent replication yet.
• Prompt sensitivity. The method depends on a handwritten prompt. How much the wording matters, and whether a worse prompt could hurt, is not something the abstract answers.
• Verified-solution assumption. OPSD with PI needs correct solutions. VRSD does not remove that requirement.
• Entropy is a proxy. Higher entropy is not always better; the paper ties recovery to better Pass@1 and fewer misleading traces, but the link should be checked on new tasks.

Related reading

• Sakana MASS: a 27B model that trains itself through agent teams
• VibeThinker 3B: small-model reasoning
• Agent-Reflex: reflective tool agents on Qwen3-8B
• Anthropic's distillation defenses
• OpenAI's research acceleration post
• DeepMind says self-improving AI is not close: claimed vs verified

Results are the authors' reported numbers as of October 14, 2026.