Update — October 3, 2026: One more local server you can point OpenCode at — DwarfStar 4 (ds4), for a short list of DeepSeek, GLM, and Qwen quants rather than every GGUF.
July 1, 2026: With Fable 5 still offline, GLM-5.2 and Qwen 3.6 27B proving local coding is practical, the missing piece for many developers is not which model — it is how to wire inference into a harness.
OpenCode is the open-source agent loop that accepts any OpenAI-compatible API. Run weights on your CPU/GPU, expose http://127.0.0.1:…/v1, point ~/.config/opencode/opencode.jsonc at it, and you have a local coding agent — same tools, LSP, and /init AGENTS.md flow as cloud setups, without sending repo context to a third party.
This is explainx.ai's end-to-end stack guide: pick a model → start a server → configure OpenCode → tier local vs cloud.
TL;DR — the full local + OpenCode path
| Step | What to do | Deep dive |
|---|---|---|
| 1. Hardware | 32GB+ for Q4 7B–27B; 48GB+ for Q8 27B | Mac vs GPU guide |
| 2. Model | Qwen 3.6 27B coding default; GLM-5.2 if you have GPU headroom | Qwen local guide · GLM Unsloth |
| 3. Runtime | llama.cpp (control) or Ollama (ease) | What is llama.cpp? · Personal AI system |
| 4. Server | OpenAI-compatible /v1 on localhost | Sections below |
| 5. OpenCode | opencode.jsonc provider + opencode in repo | OpenCode harness guide |
| 6. Verify | /init, small coding task, check tok/s | Quantization guide |
Architecture — three layers
┌─────────────────────────────────────┐
│ OpenCode (agent harness) │
│ tools · LSP · AGENTS.md · sessions │
└──────────────┬──────────────────────┘
│ OpenAI-compatible HTTP
┌──────────────▼──────────────────────┐
│ Inference server │
│ llama.cpp · Ollama · LM Studio · vLLM│
└──────────────┬──────────────────────┘
│ GGUF / safetensors
┌──────────────▼──────────────────────┐
│ Open-weight model on your disk/GPU │
│ Qwen · GLM · Gemma · DeepSeek · … │
└─────────────────────────────────────┘
OpenCode never loads weights itself — it only talks HTTP. That separation is why one config file can swap Ollama today and llama.cpp tomorrow.
Step 1 — Install OpenCode
curl -fsSL https://opencode.ai/install | bash
# or: npm install -g opencode-ai
# or: brew install anomalyco/tap/opencode
cd your-project
opencode
First-run inside TUI:
/connect— for cloud providers (Z.AI, OpenRouter, Copilot OAuth)/init— generateAGENTS.mdproject memory/models— pick active model after providers exist
Slash reference: OpenCode commands. Harness concepts: What is an agent harness?.
Step 2 — Pick an inference runtime
| Runtime | Best for | OpenAI API default | OpenCode fit |
|---|---|---|---|
| llama.cpp | Max control, MTP, Apple Silicon | http://127.0.0.1:8080/v1 | Recommended for serious local coding |
| Ollama | One-command pull/serve | http://127.0.0.1:11434/v1 | Fastest onboarding |
| LM Studio | GUI + local server toggle | http://127.0.0.1:1234/v1 | Non-terminal users |
| vLLM | Multi-user / production GPU box | custom port /v1 | Team LAN server |
Codex users: same local servers work in Codex OSS mode — different harness, identical inference layer.
Step 3 — Start the local server
Option A — llama.cpp (Qwen 3.6 27B example)
From the Qwen 3.6 local benchmark post:
llama-server -hf unsloth/Qwen3.6-27B-MTP-GGUF:Q8_0 \
--spec-type draft-mtp -ngl 999 -fa on -c 65536 --port 8080
Smoke test: open http://127.0.0.1:8080 or curl http://127.0.0.1:8080/v1/models.
Option B — Ollama
ollama pull qwen3.6:27b # verify tag on ollama.com/library
ollama serve # usually auto-starts on 11434
Other coding pulls teams use: glm-5.2, deepseek-coder-v2, gemma4:12b — check VRAM before pulling 70B-class tags.
Option C — LM Studio
- Download model in GUI
- Local Server tab → Start Server (default 1234)
- Enable OpenAI compatible API
Option D — vLLM (Linux GPU server)
python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen3.6-27B-Instruct \
--port 8000
Point OpenCode at http://127.0.0.1:8000/v1 or your LAN IP for a shared team box.
Step 4 — Configure OpenCode (opencode.jsonc)
Config path: ~/.config/opencode/opencode.jsonc (user-wide) or project-local per OpenCode docs.
llama.cpp provider block
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"llama": {
"name": "llama.cpp (local)",
"npm": "@ai-sdk/openai-compatible",
"options": {
"baseURL": "http://127.0.0.1:8080/v1",
"apiKey": "local"
},
"models": {
"qwen3.6-27b": { "name": "Qwen3.6-27B Q8 +MTP" }
}
}
},
"model": "llama/qwen3.6-27b"
}
Ollama provider block
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"ollama": {
"name": "Ollama (local)",
"npm": "@ai-sdk/openai-compatible",
"options": {
"baseURL": "http://127.0.0.1:11434/v1",
"apiKey": "ollama"
},
"models": {
"qwen3.6-27b": { "name": "qwen3.6:27b" }
}
}
},
"model": "ollama/qwen3.6-27b"
}
LM Studio provider block
"options": {
"baseURL": "http://127.0.0.1:1234/v1",
"apiKey": "lm-studio"
}
Restart OpenCode after editing config, or run opencode fresh from the project directory.
Step 5 — Cloud + local in one config (recommended)
Post-Fable tiering:
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"llama": {
"name": "Local Qwen",
"npm": "@ai-sdk/openai-compatible",
"options": {
"baseURL": "http://127.0.0.1:8080/v1",
"apiKey": "local"
},
"models": {
"qwen3.6-27b": { "name": "Qwen3.6-27B local" }
}
},
"zai": {
"name": "GLM Coding Plan",
"options": {
"baseURL": "https://api.z.ai/api/coding/paas/v4"
}
}
},
"model": "llama/qwen3.6-27b"
}
| Workload | Switch to |
|---|---|
| Private repo, offline flight | llama/qwen3.6-27b |
| Hard refactor, long agent | /models → GLM-5.2 (harness guide) |
| Cheap volume API | Cline $9.99 open weights or OpenRouter |
Connect Z.AI via /connect inside TUI if you prefer not to store API keys in jsonc.
Model picks for local OpenCode (July 2026)
| Model | Local fit | OpenCode notes |
|---|---|---|
| Qwen 3.6 27B dense | Best balance 48GB Mac / 24GB+ Nvidia | Beats MoE 35B A3B on instruction following |
| Gemma 4 12B/31B | Multimodal + Apache 2.0 | Slower than Qwen for pure code |
| GLM-5.2 Unsloth | Workstation / multi-GPU | Closest to frontier local |
| DeepSeek V4 Flash (quant) | High tok/s if RAM allows | Aggressive quants trade quality |
| Kimi K2.7 | Usually API — too large for most locals | Use API via OpenCode /connect |
Alibaba cloud line: Qwen 3.7-Max for when local is not enough.
Hardware reality check
| Machine | Realistic local model |
|---|---|
| 32GB Apple Silicon | Q4 27B or Q8 7B–14B |
| 48–64GB Apple Silicon | Q8 Qwen 3.6 27B (~32 tok/s MTP) |
| 24GB Nvidia (4090/5090) | Q6 27B; Q4 70B tight |
| 64GB+ VRAM / dual GPU | GLM-5.2 class, vLLM team server |
Full economics: Mac vs dedicated GPU · closed vs open cost table.
First session checklist
- Server running —
curl -s http://127.0.0.1:8080/v1/models(or Ollama 11434) - Config saved —
~/.config/opencode/opencode.jsonc opencodein git repo/init— creates AGENTS.md- Smoke prompt — "Read the README and summarize the documented setup without editing files" (same test as Quesma Qwen post)
- Verify model — inspect
/models, the configured provider, and the inference server logs
Add verification loops from explainx.ai loops — e.g. ci-until-green.
Troubleshooting
| Symptom | Fix |
|---|---|
| Connection refused | Inference server not running or wrong port in baseURL |
| Empty / garbage output | Quant too low — bump Q4→Q6/Q8 (quant guide) |
| Slow first token | Model loading; MTP helps decode — see llama.cpp --spec-type draft-mtp |
| Ignores package.json / structure | Try dense over MoE; shorten context if RAM swapping |
| Tool calls fail | Model may lack reliable function-calling — switch to GLM API or GPT-class for agent-heavy runs |
| Wrong provider | echo config path; /models list; check model string matches provider/id |
OpenCode vs other local harnesses
OpenCode wins when you want one harness, terminal + desktop, and swap local/cloud without reinstalling.
How do you verify each layer before blaming the model?
Test the inference server independently before opening OpenCode. Confirm that its model-list response names the checkpoint you intended to load, then send a small chat request using that exact identifier. If the server fails here, editing the harness configuration cannot repair a missing model, an unsupported runtime flag, or an out-of-memory error.
Next test one ordinary chat turn through OpenCode. Then test a bounded tool task, such as reading a named file and explaining it without edits. Finally try a small change with an existing check. These stages distinguish basic connectivity from tool-call compatibility and from actual coding competence.
Do not rely on asking the model what model it is. Generated self-identification can be wrong. Inspect the selected provider, model identifier, and server logs instead. OpenCode's provider documentation describes the configuration layer that determines where a request goes.
What if text works but tool calls fail?
An OpenAI-compatible HTTP endpoint does not guarantee identical support for every feature. A model, its chat template, and the server must agree on tool-call formatting. Inspect whether the response contains structured tool calls or ordinary text that merely looks like a call. The harness cannot safely execute the latter as though it were a validated tool request.
Use a minimal tool test before a large repository task. If a harmless file-read request fails, reduce the setup to one model and one provider. Verify the runtime's documented tool support and chat template. Increasing context length or changing quantization may help some failures, but neither is a universal fix for a mismatched tool protocol.
Does local inference make the entire workflow private?
Local inference keeps model requests on the configured local server, but the rest of the agent workflow can still use networked tools. Package installation, web search, MCP servers, telemetry, and remote repository operations deserve separate inspection. Audit those connections before claiming that no project information leaves the machine.
Bind a single-user inference server to the loopback interface unless you intentionally need LAN access. A shared server needs authentication, access boundaries, and a plan for concurrent requests. The fact that a server speaks a familiar API does not provide those controls automatically.
For long tasks, watch memory use and responsiveness. A larger context can increase resource pressure, so begin with a narrow repository task and grow only when you need more context. Record the model file, quantization, runtime version, and configuration alongside observations. That makes a later regression diagnosable when the same model name points to a different artifact.
Choose local and cloud routing deliberately. If you permit a cloud fallback, say which requests may use it and how users can recognize the active provider. A silent fallback can invalidate the privacy assumption that motivated local setup in the first place.
Related on explainx.ai
- OpenCode Desktop tabs — Jul 15, 2026
- Qwen 3.6 27B — llama.cpp + OpenCode deep dive
- OpenCode adds Qwen3.8-Flash 125B — Qwen4 architecture preview (Aug 2026)
- OpenCode harness guide
- OpenCode slash commands
- GLM-5.2 in OpenCode (cloud API path)
- Build personal local AI system
- Codex + Ollama OSS mode
- Mac vs GPU local LLM economics
- What is llama.cpp? — install, GGUF, llama-server
- LM Studio Bionic — open-model agent app (Jul 17)
- Kimi K3 local desktop prep — open weights July 27
- Kimi K3 API guide — cloud K3 until weights drop
- Kimi K2.7 Code — open weights on localhost today
- Fable enterprise open alternatives
- Top Chinese AI companies — Qwen/DeepSeek/Kimi
Official: OpenCode · OpenCode config schema · llama.cpp · Ollama
Runtime ports, model tags, and OpenCode provider schema reflect July 2026 docs — verify before production. Last updated: July 17, 2026.
Update — October 3, 2026: A free hosted open model is now on OpenCode: Ling-3.1-flash.
