harness-hub published a LoRA adapter that makes Qwen3.8-27B behave like an OpenCode-native coder—not a generic chat model with tools bolted on. The Qwen3.8-27B-opencode-adapted card, updated November 18, 2026, reports 51.0% pass@1 on SWE-bench Pro (hard 51) with thinking off, up from 35.3% for the base checkpoint in the same OpenCode harness. Median generated tokens per solved task fall to roughly one-sixth of what the base model spends when thinking is enabled.
explainx.ai covered the Pi sibling in Qwen3.8-27B SWE-bench Pro 63% on Pi and the broader harness-engineering tier debate. This post is the OpenCode-specific install guide and score reconciliation so you do not compare OpenCode hard-51 rows with Alibaba Claude Code full-bench marketing.
TL;DR: verified numbers from the model card
| Benchmark | Base Qwen3.8-27B (OpenCode) | + OpenCode LoRA |
|---|---|---|
| SWE-bench Pro hard 51 | 35.3% | 51.0% |
| Terminal-Bench 2.1 (84 tasks) | 46.4% | 52.4% |
| Median turns on shared SWE solves | 48 | 13 |
| Median tokens (SWE solves vs base + thinking) | 33.5k | ~3.9k |
All rows assume one attempt per task, 100 agent steps, text-only serving (vision tower off), temperature 0.8, top_p 1.0, and thinking off unless the card explicitly compares thinking runs.
Why a separate OpenCode adapter exists
OpenCode, Pi, Hermes, and Claude Code expose different tool schemas, compaction rules, and permission flows. The harness-hub org argues you should keep the harness and adapt the model—train a small LoRA per harness and hot-swap at runtime. That is the same design pattern Liquid LFM adapters pursue on smaller bases, but here the target is a 27B open-weight checkpoint teams already rent or self-host.
Training stack pieces the org README links:
- AsyncGRPO in TRL with LoRA
- Repo2RLEnv tasks built from real agent trajectories
- safeagents proxy blocking reward-hacking tool calls during rollouts
If you run multi-harness shops—OpenCode for one team, Pi for another—one vLLM server can host multiple LoRA modules as long as each CLI points at the correct alias.
What changed on SWE-bench Pro hard 51
OpenCode harness benchmark results chart for Qwen3.8-27B LoRA adapter on SWE-bench Pro
Image: harness-hub
The card breaks outcomes into newly solved, lost, and sign-test p-values:
| Metric | Value |
|---|---|
| Newly solved tasks | 11 |
| Lost tasks | 3 |
| Difference | +15.7 ± 13.9 points |
| Sign test | p = 0.057 (borderline on 51 tasks) |
On Terminal-Bench 2.1, the adapter gains +6.0 points on all 84 graded tasks, but the card highlights +13.6 points on the 66 tasks not used to pick the checkpoint (p = 0.035) as the stronger generalization signal.
Checkpoint selection caveat. Step 150 was chosen using SWE-bench Pro plus an 18-task Terminal-Bench subset watched during training—so headline benchmarks were not fully blind. Treat gains as real for OpenCode users, but run your own repo slice before you reorganize spend.
Thinking off is the product setting
The adapter was trained with thinking off. The card shows that enabling xhigh thinking on the adapter does not materially beat thinking-off accuracy while multiplying tokens.
| Mode on hard 51 | Adapter score | Median generated tokens (illustrative row) |
|---|---|---|
| Base + thinking | 47.1% | 31.8k |
| Adapter, thinking off | 51.0% | ~3.9k |
Practical takeaway: serve with enable_thinking: false in chat template kwargs, exactly as the vLLM snippet documents.
How to reproduce the harness (vLLM + OpenCode 2.2.7)
1. Serve LoRA with vLLM
From the model card:
vllm serve Qwen/Qwen3.8-27B \
--enable-lora --max-lora-rank 32 \
--lora-modules qwen3.8-opencode=harness-hub/Qwen3.8-27B-opencode-adapted \
--language-model-only --enable-prefix-caching --mamba-cache-mode align \
--enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3 \
--default-chat-template-kwargs '{"enable_thinking": false}' \
--max-model-len 131072
Prefix caching on Qwen3.8 hybrid stacks needs --mamba-cache-mode align. Without it, every turn re-reads full history and agent bills explode.
2. Install OpenCode and wire the provider
npm install -g @opencode/cli@2.2.7
Add to ~/.config/opencode/opencode.json:
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"local": {
"npm": "@ai-sdk/openai-compatible",
"options": {
"baseURL": "http://localhost:8000/v1",
"apiKey": "dummy"
},
"models": {
"qwen3.8-opencode": {}
}
}
}
}
Run a smoke task:
opencode run --standalone --model local/qwen3.8-opencode "Find out why the tests in tests/test_parser.py fail and fix the bug."
Cross-check OpenCode versus other terminals if your team mixes CLIs.
Evaluation caveats the card discloses
Builders should read the limitations section before tweeting leaderboard screenshots:
| Caveat | Why it matters |
|---|---|
| Text-only server | OpenCode may request web search or treat files as images; disabled vision and search ended some Terminal-Bench rollouts early |
| Tunnel outages | Public eval infra dropped sessions; maintainers reran affected trajectories (base SWE score moved from 31.4% pre-rerun to 35.3%) |
| Test file edits | Agents sometimes wrote scratch tests; official graders reset tests before grading |
| Not comparable to Qwen card | Alibaba numbers use other harnesses, longer budgets, and thinking on |
Those disclaimers are why explainx.ai keeps claimed-vs-verified posts separate per harness.
Economics: turns and tokens beat vanity percentages
Generated tokens per task comparing Qwen3.8-27B base versus OpenCode LoRA adapter
Image: harness-hub
On tasks both base and adapter solve on SWE-bench Pro hard 51, medians are 13 vs 48 turns and 1.9k vs 6.8k generated tokens. For a 10k-issue backlog, that is the difference between buying another GPU and shipping with the same cluster.
Pair cost math with right-sized open inference APIs and vLLM routing posts when you negotiate provider contracts.
How this fits export-control and cloud planning
OpenCode adapters do not change EAR rules, but they change how much foreign-hosted GPU you need. Teams shrinking tokens per merge may keep training on domestic Ascend pools while US-hosted frontier APIs serve only the hardest tasks—relevant as BIS remote-access licensing reshapes Southeast Asia rentals.
OpenCode versus Pi: when to load which LoRA
harness-hub now ships two Qwen3.8-27B adapters:
| Adapter | Hard-51 SWE (card) | Median turns (shared solves) | CLI |
|---|---|---|---|
| Pi | 62.7% | lower on Pi traces | @earendil-works/pi-coding-agent |
| OpenCode | 51.0% | 13 vs 48 on SWE | @opencode/cli@2.2.7 |
Do not mount the OpenCode LoRA behind Pi or vice versa—the tool XML schemas differ. If your org standardized on OpenCode because of multi-model routing experiments, this release is the first official Qwen-native path with published eval logs.
Teams running both CLIs on one vLLM pod can register:
--lora-modules qwen3.8-pi=harness-hub/Qwen3.8-27B-pi-adapted qwen3.8-opencode=harness-hub/Qwen3.8-27B-opencode-adapted
…then point each harness at the correct model id.
Pairing with GGUF and Apple Silicon paths
The org also publishes Qwen3.8-27B-opencode-adapted-GGUF companions for llama.cpp, mirroring the Pi release. The card warns GGUF bases drift from BF16 training weights—Q8_0 is the suggested default, but re-evaluate before you quote 51% on Metal.
For Mac-heavy teams compare against llama.cpp release cadence and TensorFold local inference if you need sub-27B fallbacks when VRAM is tight.
safeagents and reward hacking during RL
The harness-hub blog positions safeagents as a 10–20 ms rules engine that zeros rollouts when models attempt shortcut tool abuse during AsyncGRPO. That matters because coding-agent felony bench tracking shows real-world agents already publish malicious packages when eval guardrails are weak.
If you fork the recipe:
- Keep safeagents (or equivalent) in the training loop, not only production.
- Log blocked tool calls per checkpoint—spikes mean the LoRA is gaming rewards.
- Mirror MCP misbinding defenses in prod after you ship.
Procurement language for vendor bake-offs
Paste this into RFPs when a vendor quotes SWE-bench Pro:
Scores must name harness name, @opencode/cli version, thinking on/off, step budget, and LoRA commit SHA. OpenCode numbers are not interchangeable with Claude Code rows on vendor model cards.
That single paragraph prevents 61.7% marketing slides from sneaking into OpenCode pilots without re-run costs.
Failure modes we expect in production
| Symptom | Likely cause | Fix |
|---|---|---|
| Tool calls ignored | Wrong chat template or thinking left on | Force enable_thinking: false |
| Sessions die on images | Vision requests against text-only server | Enable vision tower or strip image tools |
| Runaway spend | Missing prefix cache align on hybrid Qwen | Add --mamba-cache-mode align |
| Scores collapse after upgrade | OpenCode >2.2.7 behavior change | Pin CLI + rerun 20-task slice |
Cost worksheet (illustrative)
Assume $2/M input and $6/M output list pricing on a hosted 27B endpoint (your contract may differ):
- Median 1.9k output tokens per solved SWE task on the adapter (card, thinking off).
- Multiply by issues per week your agent fleet accepts.
- Compare against baseline 6.8k tokens on shared solves—adapter wins when output dominates the bill.
Add GPU amortization if self-hosting: a single 48 GB card serving LoRA often beats frontier API pricing when utilization stays above 60%, but only if legal can approve export-category hardware in your region.
Incident response when benchmarks drive deadlines
When leadership sees +15 points on Twitter and demands next-day rollout:
- Publish an internal dashboard with harness version, LoRA SHA, and cost per merged PR.
- Run 20 private issues before you touch customer repos.
- Keep rollback weights (base Qwen3.8-27B without LoRA) one config flag away.
That playbook mirrors Semianalysis agent failure rates—speed without harness discipline creates incidents, not velocity.
What is coming next from harness-hub
The Qwen3.8-27B collection page still lists Pi (OpenCode and Hermes coming)—OpenCode now shipped; Hermes may follow the same AsyncGRPO recipe. The org promises a full training blog on Hugging Face; until it lands, fork TRL async_grpo_harbor if you need a custom harness adapter for an internal CLI.
Related reading
- Qwen3.8-27B Pi adapter claimed vs verified
- Agent harness tier list reactions
- Higher SWE scores, lower merge rates
- External sandboxes for coding agents
- Semianalysis agents fail benchmarks
Bottom line
OpenCode users now have a verified LoRA with public vLLM commands—not a rumor screenshot. Pin CLI 2.2.7, keep thinking off, and rerun your issue slice before you rebalance API spend away from frontier models.
Benchmarks and commands follow the harness-hub model card accessed November 18, 2026. Rerun evals after you upgrade OpenCode beyond 2.2.7.
