Hugging Face published a recipe for tuning an open model to the coding agent that runs it, and a single merged adapter that works across three agents. The result: a Qwen3.8-27B super-adapter lifts the mean score across six harness and benchmark pairs from 55.5% to 63.5% with thinking on, per the Hugging Face post on harness adapters, published November 20, 2026.
The idea is simple. A harness is the wrapper that runs a model on tasks: its tools, prompts and loop. Models are not tuned for every harness. Train a small LoRA with reinforcement learning inside your harness, on your tasks, and you recover accuracy and efficiency. The authors write that combining RL on a custom task distribution with your favorite agent harness "yields more accurate and efficient models."
This post explains the method, the numbers, how to run the adapter, and where the claims stop. It builds on our earlier coverage of the individual adapters: the pi harness SWE-Bench Pro claim, the OpenCode adapter and the Hermes adapter.
Harness adapters at a glance
| Question | Answer |
|---|---|
| What is it? | A LoRA trained with RL (GRPO) inside a real agent harness |
| Base model | Qwen/Qwen3.8-27B |
| Harnesses | pi 1.5.2, OpenCode 2.2.7, Hermes Agent v0.22.6 |
| Headline result | Thinking-on mean 55.5% to 63.5% across six pairs |
| Thinking off | Mean 39.5% to 49.0%; 53 to 73% fewer tool calls |
| License | Apache-2.0 (super-adapter model card) |
| Rank | 96 (three rank-32 adapters concatenated) |
| Where | Hugging Face collection harness-hub/qwen38-27b-harness-adapters |
What is a harness adapter?
A harness adapter is a LoRA adapter, a small set of extra weights added on top of a frozen base model, trained with reinforcement learning while the model runs inside the real harness. Our fine-tuning explainers cover the general idea. What is new here is the loop.
Hugging Face did not re-implement the harness for training. A proxy sits between the harness and vLLM and records the exact tokens the model produced. The trainer then learns from those tokens. That removes the usual mismatch between a training sandbox and the tool the developer actually runs.
The post puts the motivation plainly: "Models are not fully optimized for all harnesses and the tasks you're doing."
How the training pipeline works
Tasks
Task pipeline: Tasksmith, SWE-smith and SWE-Gym sources pass through an eligibility screen, Harbor packaging, openenv validate and a difficulty check with Qwen3.8-27B
Image: Hugging Face
Tasks come from three sources: SWE-smith (injected bugs in Python and Go), SWE-Gym via TaskTrove (real GitHub issues) and Tasksmith (generated from merged pull requests). Each task passes a metadata screen, openenv validate checks and a difficulty check against the base model.
Qwen3.8-27B results on 442 scored tasks, split by attempts solved and by source
Image: Hugging Face
The difficulty filter matters. Of 442 scored tasks, the base model solved 151 every time and 120 never. The 171 it solved only sometimes became the qualified set, because GRPO needs variation between attempts to learn anything. Our coverage of Termgrade terminal RL environments shows where part of the mix came from.
Training settings
- LoRA on all linear layers.
- GRPO via TRL's
AsyncGRPOTrainer, with thinking off during rollouts. - Reward: the verifier's pass or fail, plus a 0.1 tool-efficiency bonus.
- Learning rate 2e-5, chosen after a sweep over 1e-5, 5e-5 and 1e-4.
- Scale: 598 cumulative steps, 78,114 samples and about 167M trained tokens, on a mixture of 750 harness-adapters tasks and 300 Termgrade tasks.
The tool-efficiency bonus explains a result that looks odd at first: the adapters are not only more accurate, they also use far fewer tokens. For the pi adapter, average tokens per task fell from 2.9K to 1.5K.
Safety plugin
Agents running RL rollouts can do damage. Hugging Face used a safeagents plugin that blocked suspicious tool calls. Across the runs, 647 of about 1.3M tool calls were blocked. The plugin is published on the Hub.
What are the results?
Reinforcement learning as a ball following a track toward a cup, for Qwen3.8-27B harness adapter training
Specialist adapters, one per harness, thinking off, pass@1, as reported in the Hugging Face post:
| Harness | Benchmark | Base | Specialist adapter |
|---|---|---|---|
| pi | SWE-bench Pro (hard 51) | 40.0% | 64.0% |
| Hermes Agent | SWE-bench Pro (hard 51) | 29.4% | 43.1% |
| OpenCode | SWE-bench Pro (hard 51) | 35.3% | 51.0% |
| pi | Terminal-Bench 2.1 | 40.0% | 56.5% |
| Hermes Agent | Terminal-Bench 2.1 | 44.7% | 58.8% |
| OpenCode | Terminal-Bench 2.1 | 47.6% | 52.4% |
The text also quotes slightly different pi figures from another run (39.2% to 64.7%). Treat the table as one run, not a fixed constant.
The super-adapter
The three specialists were concatenated along the rank dimension: three rank-32 LoRAs make one rank-96 adapter. Each update was scaled by one half, which gave the best Terminal-Bench result. The adapters had cosine similarity of about 0.01 to 0.02, so they barely overlap, which is why a plain merge works. The model card says "The merge itself involves no training."
Results for the super-adapter, per the post and the model card:
| Harness | Benchmark | Base, thinking on | Super-adapter, thinking on |
|---|---|---|---|
| pi | SWE-bench Pro (hard 51) | 74.0% | 74.0% |
| Hermes Agent | SWE-bench Pro (hard 51) | 45.1% | 60.8% |
| OpenCode | SWE-bench Pro (hard 51) | 47.1% | 66.7% |
| pi | Terminal-Bench 2.1 | 52.9% | 57.6% |
| Hermes Agent | Terminal-Bench 2.1 | 58.8% | 63.5% |
| OpenCode | Terminal-Bench 2.1 | 54.9% | 58.5% |
Thinking on, the mean goes from 55.5% to 63.5%. It wins five of six pairs and ties one (pi on SWE-bench Pro). It uses 19 to 48% fewer tool calls per task. Thinking off, the mean goes from 39.5% to 49.0%; it newly solves 71 tasks and loses 34.
Notice what the tie says. Where the base model with thinking was already strong (pi at 74.0%), the adapter adds nothing. The biggest gains land where the base is weakest: Hermes Agent and OpenCode on SWE-bench Pro.
Which harness gets the biggest gain?
Hermes Agent and OpenCode gain most on SWE-bench Pro with thinking on: Hermes from 45.1% to 60.8%, OpenCode from 47.1% to 66.7%. For a view of why harness choice moves scores this much, see our piece on harness leaderboard reliability with Hermes Agent. The Hugging Face post warns that results are only comparable inside the same harness, because agent settings differ across harnesses.
How to run the Qwen3.8-27B super-adapter
The model card gives a vLLM recipe. Serve the base model with LoRA enabled and a rank limit of 128, the next supported size above 96:
vllm serve Qwen/Qwen3.8-27B \
--enable-lora --max-lora-rank 128 \
--lora-modules qwen3.8-super=harness-hub/Qwen3.8-27B-super-adapter \
--max-model-len 131072
Thinking is on by default. To turn it off, pass --default-chat-template-kwargs '{"enable_thinking": false}'. Then point pi, Hermes Agent or OpenCode at http://localhost:8000/v1.
You can also load it with PEFT, merge it into a standalone checkpoint, or use the GGUF version for llama.cpp. Our vLLM vs SGLang comparison helps you choose a server, and the DSpine drafter post covers another way to speed up the same base model.
Which adapter should I pick?
| Your situation | Pick |
|---|---|
| One harness, thinking off | That harness specialist adapter |
| Several harnesses, or thinking on | The super-adapter |
| Laptop or llama.cpp | The OpenCode GGUF build |
| Unsure about outside tasks | Test the base model next to the adapter first |
What are the limits?
The authors list several caveats, and they are worth reading before you adopt anything.
- Merge weight picked on the test set. The super-adapter scale was chosen on Terminal-Bench 2.1, so those numbers are slightly optimistic.
- Out-of-distribution results are flat. On AutomationBench (300 tasks), base scored 23.0%, pi 27.0%, OpenCode 25.3%, merged 23.7% and Hermes 17.7%. Differences are mostly within noise, except Hermes, which trails.
- Reasoning leakage. With thinking off, the model often writes short reasoning-like text before tool calls.
- Harness differences. pi and Hermes handle prior reasoning turns differently, which affects SWE-bench Pro results.
- Flawed verifiers. One Terminal-Bench 2.1 verifier accepted known-wrong answers.
- Thinking-off specialists can lose to base with thinking on in some cases.
- Small test slice. The SWE-bench Pro set is a hard 51-task subset, so one task moves a score by about two points.
No dollar costs are reported. The post gives token and tool-call counts only. If you plan to use the weights commercially, read how open-weight licenses work; the adapter is Apache-2.0 per its card, and the base model carries its own terms.
What this means for what you build or pay
Builders. If you run an open 27B model behind a coding agent, a harness-specific LoRA is now a documented, reproducible step, not a research stunt. The data, tasks and trajectories are public, so you can reproduce the loop on your own harness and task mix. Check the result on your own repositories before trusting a 51-task benchmark. Our guide to reading a Hugging Face model card helps with that.
Founders. The efficiency numbers are the business story: 53 to 73% fewer tool calls with thinking off, and 19 to 48% fewer with thinking on. Fewer calls mean lower latency and lower serving cost per task. Because the gain is tied to a harness, it also raises switching costs if you later change agent tools.
Marketers and tool vendors. Expect "tuned for X harness" to become a selling point. Ask for numbers on the same harness version, as the post lists pi 1.5.2, OpenCode 2.2.7 and Hermes Agent v0.22.6.
What is still open
Hugging Face has not shown that the gain transfers to tasks outside the training mix, and the AutomationBench result says it may not. It has not published a cost per task in dollars. And the merge trick is demonstrated on three adapters with near-orthogonal updates; more harnesses may interfere more. Independent reruns on other repositories would settle these.
Related reading
- Qwen3.8-27B SWE-Bench Pro 63% on pi: claimed vs official
- Qwen3.8-27B OpenCode adapter: 51% SWE-Bench Pro
- Qwen3.8-27B Hermes adapter: Terminal-Bench 2.1 to 58.8%
- Harness leaderboard reliability with Hermes Agent
- Termgrade: 1,004 terminal RL environments
- vLLM vs SGLang inference servers
Official sources: Hugging Face blog, super-adapter model card, base model Qwen/Qwen3.8-27B.
Figures reflect the Hugging Face post and model card as of November 20, 2026 and may change as the authors update them.
