explainx.ai Upskill in AI — 16 free pathways, live workshops & bootcamps, and 50+ courses from practitioners. Plus the skills, tools, and MCP servers to practice on.
newsletter · weekly
Get AI news, tools, and insights in your inbox.
Run Open Source Models Locally in OpenCode (2026) | explainx.ai Blog | explainx.ai
On this page
← Back to blog explainx / blog
How to Run Open Source Models Locally and Wire Them Into OpenCode (2026) Step-by-step: Ollama, llama.cpp, LM Studio, or vLLM → OpenAI-compatible API → OpenCode opencode.jsonc. Model picks (Qwen 3.6, GLM, Gemma), hardware tiers, multi-model routing, and troubleshooting.
Jul 1, 2026 · 6 min read · Yash Thakker OpenCode Local LLM Open Source AI
llama.cpp
Ollama
Agent Harness
copy mdx copy post
request update July 1, 2026: With Fable 5 still offline GLM-5.2 Qwen 3.6 27B which model — it is how to wire inference into a harness .
OpenCode any OpenAI-compatible API . Run weights on your CPU/GPU , expose http://127.0.0.1:…/v1, point ~/.config/opencode/opencode.jsonclocal coding agent — same tools, LSP, and /init AGENTS.md flow as cloud setups, without sending repo context to a third party.
This is explainx.ai's end-to-end stack guide : pick a model → start a server → configure OpenCode → tier local vs cloud.
Weekly digest 3.5k readers
Catch up on AI
Curated AI updates on agents, skills, and MCP — delivered to your inbox. Unsubscribe anytime.
TL;DR — the full local + OpenCode path
Step What to do Deep dive 1. Hardware 32GB+ for Q4 7B–27B; 48GB+ for Q8 27B Mac vs GPU guide 2. Model
Related posts Jun 29, 2026
Qwen 3.6 27B Local Dev Guide: llama.cpp, OpenCode, and Why Dense Beats MoE After HN front-page hype, hands-on tests say Qwen 3.6 27B dense is the local sweet spot — better code than the 35B MoE, runnable at Q8 on 48GB RAM. Full llama.cpp + OpenCode config inside.
Jul 17, 2026
LM Studio Bionic: Open-Model Agent for Code and Work Projects LM Studio shipped Bionic on July 16, 2026 — a dedicated agent app (not LM Studio itself) for code repos and work projects over local models, LM Link, or Secure Cloud with zero data retention. This guide covers what works, HN rough edges, closed-source trade-offs, and how it compares to OpenCode and Unsloth Studio.
Jul 2, 2026
What Is llama.cpp? Install, Run GGUF Models, and Serve OpenAI-Compatible APIs If you run open weights on your own hardware in 2026, you are almost certainly touching llama.cpp — directly or through Ollama and LM Studio. This guide explains what it is, how GGUF fits in, copy-paste install and run commands, and how to expose a local API for coding agents.
Qwen 3.6 27B
GLM-5.2
4. Server OpenAI-compatible /v1 on localhost Sections below
Architecture — three layers ┌─────────────────────────────────────┐
│ OpenCode (agent harness) │
│ tools · LSP · AGENTS.md · sessions │
└──────────────┬──────────────────────┘
│ OpenAI-compatible HTTP
┌──────────────▼──────────────────────┐
│ Inference server │
│ llama.cpp · Ollama · LM Studio · vLLM│
└──────────────┬──────────────────────┘
│ GGUF / safetensors
┌──────────────▼──────────────────────┐
│ Open-weight model on your disk/GPU │
│ Qwen · GLM · Gemma · DeepSeek · … │
└─────────────────────────────────────┘
OpenCode never loads weights itself — it only talks HTTP. That separation is why one config file can swap Ollama today and llama.cpp tomorrow.
Step 1 — Install OpenCode curl -fsSL https://opencode.ai/install | bash
/connect/initAGENTS.md/models
Step 2 — Pick an inference runtime Runtime Best for OpenAI API default OpenCode fit llama.cpp Max control, MTP, Apple Silicon http://127.0.0.1:8080/v1Recommended for serious local codingOllama One-command pull/serve http://127.0.0.1:11434/v1Fastest onboarding LM Studio GUI + local server toggle http://127.0.0.1:1234/v1Non-terminal users vLLM Multi-user / production GPU box custom port /v1 Team LAN server
Codex users: same local servers work in Codex OSS mode
Step 3 — Start the local server
Option A — llama.cpp (Qwen 3.6 27B example) llama-server -hf unsloth/Qwen3.6-27B-MTP-GGUF:Q8_0 \
--spec-type draft-mtp -ngl 999 -fa on -c 65536 --port 8080
Smoke test: open http://127.0.0.1:8080 or curl http://127.0.0.1:8080/v1/models.
Option B — Ollama ollama pull qwen3.6:27b
ollama serve
Other coding pulls teams use: glm-5.2, deepseek-coder-v2, gemma4:12b — check VRAM before pulling 70B-class tags.
Option C — LM Studio
Download model in GUI
Local Server tab → Start Server (default 1234 )Enable OpenAI compatible API
Option D — vLLM (Linux GPU server) python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen3.6-27B-Instruct \
--port 8000
Point OpenCode at http://127.0.0.1:8000/v1 or your LAN IP for a shared team box.
Config path: ~/.config/opencode/opencode.jsoncOpenCode docs .
llama.cpp provider block {
"$schema" : "https://opencode.ai/config.json" ,
"provider" : {
"llama" : {
"name" : "llama.cpp (local)" ,
"npm" : "@ai-sdk/openai-compatible" ,
"options" : {
"baseURL" : "http://127.0.0.1:8080/v1" ,
"apiKey" : "local"
} ,
"models" : {
"qwen3.6-27b" : { "name" : "Qwen3.6-27B Q8 +MTP" }
}
}
} ,
"model" : "llama/qwen3.6-27b"
}
Ollama provider block {
"$schema" : "https://opencode.ai/config.json" ,
"provider" : {
"ollama" : {
"name" : "Ollama (local)" ,
"npm" : "@ai-sdk/openai-compatible" ,
"options" : {
"baseURL" : "http://127.0.0.1:11434/v1" ,
"apiKey" : "ollama"
} ,
"models" : {
"qwen3.6-27b" : { "name" : "qwen3.6:27b" }
}
}
} ,
"model" : "ollama/qwen3.6-27b"
}
LM Studio provider block "options" : {
"baseURL" : "http://127.0.0.1:1234/v1" ,
"apiKey" : "lm-studio"
}
Restart OpenCode after editing config, or run opencode fresh from the project directory.
Step 5 — Cloud + local in one config (recommended) {
"$schema" : "https://opencode.ai/config.json" ,
"provider" : {
"llama" : {
"name" : "Local Qwen" ,
"npm" : "@ai-sdk/openai-compatible" ,
"options" : {
"baseURL" : "http://127.0.0.1:8080/v1" ,
"apiKey" : "local"
} ,
"models" : {
"qwen3.6-27b" : { "name" : "Qwen3.6-27B local" }
}
} ,
"zai" : {
"name" : "GLM Coding Plan" ,
"options" : {
"baseURL" : "https://api.z.ai/api/coding/paas/v4"
}
}
} ,
"model" : "llama/qwen3.6-27b"
}
Connect Z.AI via /connect
Model picks for local OpenCode (July 2026) Model Local fit OpenCode notes Qwen 3.6 27B dense Best balance 48GB Mac / 24GB+ NvidiaBeats MoE 35B A3B on instruction following Gemma 4 12B/31B Multimodal + Apache 2.0 Slower than Qwen for pure code GLM-5.2 Unsloth Workstation / multi-GPU Closest to frontier local DeepSeek V4 Flash (quant) High tok/s if RAM allows Aggressive quants trade quality Kimi K2.7 Usually API — too large for most locals Use API via OpenCode /connect
Alibaba cloud line: Qwen 3.7-Max
Hardware reality check Machine Realistic local model 32GB Apple Silicon Q4 27B or Q8 7B–14B 48–64GB Apple Silicon Q8 Qwen 3.6 27B (~32 tok/s MTP) 24GB Nvidia (4090/5090) Q6 27B; Q4 70B tight 64GB+ VRAM / dual GPU GLM-5.2 class, vLLM team server
First session checklist
Server running — curl -s http://127.0.0.1:8080/v1/models (or Ollama 11434)Config saved — ~/.config/opencode/opencode.jsoncopencode/initSmoke prompt — "Create a pnpm package with a hexagonal minesweeper" (same test as Quesma Qwen post )Verify model — ask "What model are you?" — should not hallucinate Claude/GPT
Troubleshooting Symptom Fix Connection refused Inference server not running or wrong port in baseURL Empty / garbage output Quant too low — bump Q4→Q6/Q8 (quant guide ) Slow first token Model loading; MTP helps decode — see llama.cpp --spec-type draft-mtp Ignores package.json / structure Try dense over MoE; shorten context if RAM swapping Tool calls fail Model may lack reliable function-calling — switch to GLM API or GPT-class for agent-heavy runs Wrong provider echo config path; /models list; check model string matches provider/id
OpenCode vs other local harnesses Harness Local OSS models Notes OpenCode Yes — 75+ providers + custom localDefault open multi-surface Pi Yes — provider plugins Minimal, ownable Codex OSS Yes — --oss + Ollama OpenAI-native tool schema Claude Code Indirect — /config remote to desktop Not weight-local Kilo Code Yes — BYOK + Ollama VS Code extension
OpenCode wins when you want one harness , terminal + desktop , and swap local/cloud without reinstalling .
Runtime ports, model tags, and OpenCode provider schema reflect July 2026 docs — verify before production. Last updated: July 17, 2026.
