WikiSkill: Compiling Agent Experience Into a Persistent Wiki for Skill Evolution

If you maintain a skill library for your agents, you have probably hit this wall: the skills themselves are versioned and readable, but the reasoning behind each edit — why a step was added, which failure it fixed, what the model kept getting wrong — lives in a sprawl of optimization logs nobody reads twice. A new paper from Google Research and Virginia Tech, WikiSkill (arXiv 2608.27454, posted August 27-28, 2026), names that problem and proposes a fix: compile agent experience into a persistent wiki, and rebuild the skills from the wiki every time.

The authors — Liyan Tang, Cyrus Rashtchian, Chun-Sung Ferng, Andrew Tomkins, Da-Cheng Juan, and Tu Vu — frame it as a separation-of-concerns argument. Skill-evolution systems usually collapse three distinct things into one blob. WikiSkill pulls them apart.

TL;DR

The three things skill-evolution systems collapse into one

Most auto-discovery pipelines take agent trajectories, run an optimizer over them, and emit updated skill files. The optimization history — the intermediate judgments, the "this prompt variant scored 0.71, this one 0.79" trail — is a byproduct. It is kept for provenance but not treated as a first-class asset. So the guiding insights stay scattered, and the next round of skill updates has to re-derive them or ignore them.

WikiSkill treats the accumulated knowledge as the point. Its three layers:

1. Raw execution traces — what actually happened when the agent ran. High volume, low signal per token, discarded or archived.
2. The persistent wiki — experience continuously consolidated into durable, deduplicated knowledge. This is the thing that survives and grows.
3. The executable skills — the SKILL.md-style artifacts the agent actually loads at runtime.

The mechanism is that every later skill update builds on the wiki rather than on the last optimization run. Experience flows into the wiki continuously; skills are regenerated from a knowledge base that only gets richer. It is the difference between editing a document from its git blame and editing it from a maintained design doc.

This maps cleanly onto the thin-skill / thick-artifact framing: the skill file stays lean and executable, while the heavy, accumulated context lives in a separate persistent store the skill is compiled against.

The numbers

Across diverse benchmarks and models, WikiSkill consistently beats EvoSkill and SkillOpt — the two named state-of-the-art skill-evolution baselines — and beats no-skill baselines in most model-benchmark settings.

Figure 1 reports no-skill vs WikiSkill accuracy (approximate, read from the chart):

Two things stand out. First, skill evolution complements model scaling — larger models generally benefit more from evolved skills, so this is not a technique that only rescues weak models. Second, and more useful in practice: smaller models with skills can outperform substantially larger models without them. A Qwen3.5-9B at ~47% with a wiki edges past a no-skill Gemini 3.5 Flash near that range, and the 27B-with-skills number sits well above where a much larger no-skill model would land. If your cost model favors a smaller deployed model, a well-maintained skill library is a lever on that trade-off — the same economic argument SkillOpt made about optimizing with frontier models and deploying on cheap ones.

Skills transfer across models — and self-evolved is not always best

WikiSkill's evolved skills transfer across models and model families. More surprising: skills evolved by another model can outperform skills a model evolved for itself. That inverts a natural assumption — that each model should tune its own skills against its own failure modes. If cross-model skills win, a shared, well-consolidated wiki becomes a team asset rather than a per-model artifact, which is roughly the bet Hiten Shah made about treating a skill library as company strategy.

For anyone running a multi-model agent harness, this is the operational takeaway: you may not need N separate skill-evolution loops. One wiki, evolved by whichever model does it best, distributed to the rest.

What the ablations say

The paper's ablations confirm that persistent knowledge accumulation in the wiki is critical and carries much of the gain. Strip the durable, consolidated store and revert to a scattered optimization history, and performance falls. That is the whole thesis in one experiment: the value is not in the optimizer or the skill format, it is in keeping the knowledge in a form the next update can stand on.

Where it sits relative to ReasoningBank and memory systems

The discussion positions WikiSkill against a few neighbors:

• ReasoningBank (arXiv 2509.25140, Google, ICLR 2026) distills generalizable reasoning strategies from self-judged success and failure, retrieves them at test time, and adds memory-aware test-time scaling (MaTTS). It is a test-time retrieval story about reasoning traces.
• Systems that store raw trajectories keep everything and pay the retrieval-noise cost.
• Systems that keep only successful routines throw away the failure signal that often teaches the most.

WikiSkill's claimed niche is the consolidated middle: not raw traces, not a highlight reel of wins, but a continuously deduplicated knowledge base that executable skills are regenerated from. It is closest in spirit to Microsoft's EvoLib, which also separates distilled skills from raw memory and evolves a library across tasks — and to Self-Harness, which does the analogous thing one layer down at the harness level. The 2026 research direction is consistent: agents that keep getting better after deployment without touching weights, by compounding what they learn into a durable artifact.

What this changes if you maintain a skill library

• Separate your stores. If your pipeline emits skill files and keeps only the run logs, you have the anti-pattern the paper targets. Add a consolidated knowledge layer between traces and skills.
• Consolidate, do not append. The wiki works because it is deduplicated and generalized, not because it is a bigger log. Merge similar lessons into stronger ones.
• Evolve once, distribute widely. Cross-model transfer means one good evolution loop can feed several deployed models.
• Keep the failures. The signal that a durable knowledge base preserves — and a success-only routine store discards — is where much of the gain comes from.
• Browse a curated set first. Before building an evolution loop, see what a maintained library looks like on the explainx.ai skills registry.

Related reading

• What are agent skills? Complete guide
• Agent markdown files — SKILL.md, CLAUDE.md, and more
• Thin prompts, thick artifacts, thin skills — Thariq's framework
• What is an agent harness? Complete guide
• Microsoft SkillOpt — self-improving agent skills
• Microsoft EvoLib — test-time learning with an evolving library
• Self-Harness — agents that improve their own harness
• Hiten Shah — AI skill library as company strategy

Primary source: WikiSkill, arXiv 2608.27454 (Liyan Tang, Cyrus Rashtchian, Chun-Sung Ferng, Andrew Tomkins, Da-Cheng Juan, Tu Vu — Google Research + Virginia Tech). Related: ReasoningBank, arXiv 2509.25140.

Figures and claims follow the WikiSkill preprint (arXiv 2608.27454) as posted August 27-28, 2026. Figure 1 accuracy values are approximate, read from the published chart. Benchmark deltas and method details may change as the paper is revised — verify against the latest version before citing numbers in production planning.