Mesh LLM v1.0: Split 235B Models Across Your LAN with iroh P2P

Mesh LLM v1.0 dropped July 11, 2026 — and the pitch from n0's iroh team (Rae McKelvey) is blunt: stop renting racks you don't control. Pool the GPUs already under your desk, expose them as http://localhost:9337/v1, and let the mesh decide whether inference runs locally, routes to a peer, or splits across machines too big for any one box.

No data-center picture. No metered black-box API. Version 1.0 turns that architecture from experiment into something you can install today — ~18 MB node software, 40+ models, and a Hacker News thread already arguing about tokens per second.

TL;DR — Mesh LLM 1.0

The problem Mesh LLM targets

From the iroh blog:

When people picture running a large language model, they picture a data center… You send your prompts off to a black box and hope the price, the model, and the privacy policy all stay the way they were when you signed up.

Teams with GPUs in offices, closets, and home labs lack a way to make those machines act as one inference surface. Cloud APIs scale spend with success; local Ollama stacks cap at single-node VRAM.

Mesh LLM's answer: compose hardware you own into an OpenAI client endpoint — same SDKs, same agents, different economics.

Three ways a request gets served

OpenAI client → localhost:9337/v1
                    │
        ┌───────────┼───────────┐
        ▼           ▼           ▼
   Run local    Route peer   Skippy split
   (this GPU)   (model hot)  (layer pipeline)

The client never sees Skippy. It still talks to localhost.

Skippy pipeline (layer splits)

Internally "Skippy" partitions by layer range:

• Node A: layers 0–15
• Node B: layers 16–31
• Activations flow A → B → … down the pipe

Listed perf: Qwen 235B A22B (MoE 235B/22B active) — 16 tok/s across 2 nodes on the models list.

HN contributor @i386 (Skippy engine author): home lab with ~5ms latency + jitter between machines; GLM 5.2 split at ~10 tok/s. Works well at metro WAN latency; global WAN slower. Goal: machines without RDMA/NVLink still serve giants you own — then share with team or public mesh.

Architecture — plugins, gossip, QUIC streams

Mesh LLM is pluggable: plugins declare capabilities in manifests; runtime routes calls over MCP, HTTP, inference, and mesh events.

iroh transport layer

Every node boots an iroh endpoint = public key identity. No central coordinator.

Three ALPN protocols:

Inside mesh-llm/1, the first stream byte demuxes traffic:

explainx.ai read: iroh makes "peer" and "localhost" the same primitive with different endpoint IDs — networking stops being a custom ops project.

Mesh LLM builds its own gossip on top to control admission, version compatibility, and trust for private meshes.

Performance — HN debate and ballpark math

SwellJoe (HN): consumer networks are slow vs local RAM — "Are we talking 1 token per second?"

Counter-evidence:

woadwarrior01 ballpark: transfer roughly 2 × hidden_size × num_shards bytes per token during decode (divide by chunk size on prefill).

Reality check: 16 tok/s is usable for agents, borderline for chat UX — comparable to why teams still pay for GPT-5.6 Sol cloud tokens. Mesh LLM wins on privacy, lock-in, and marginal cost at scale — not raw interactive speed vs a single H100.

For single-node baselines: MacBook vs dedicated GPU local LLM guide.

Getting started

Per the July 11 announcement:

1. Install ~18 MB Mesh LLM node
2. Join public mesh or configure private deployment
3. Point any OpenAI client at http://localhost:9337/v1
4. Add nodes — laptop, mini PC, workstation, cloud VM

Coming: mobile app on iroh Swift SDK; ACP (agent standard) so more clients join without proprietary lock-in.

Philosophy (blog): "more peer to peer, fewer closed servers, and no lock-in."

Mesh LLM vs alternatives

HN noted cocompute for trusted-pool delegation; model splitting across devices is what Mesh LLM v1.0 ships now.

Who should use this

Pair with Fable 5 local hardware projections for the macro trend: frontier models eventually fit owned silicon — Mesh LLM accelerates today's gap via networked sharding.

Security notes

• Transport: iroh QUIC encryption per endpoint keys
• Trust: Private mesh = you control who gets admitted via gossip policy
• Ops: Split pipelines multiply attack surface across nodes — patch all peers

Not a replacement for MCP security hardening when agents call tools on mesh-served models.

Related on explainx.ai

• Petals resurfaces — why BitTorrent-style LLM inference still struggles
• Ollama $88M funding — open models ecosystem
• MacBook vs dedicated GPU for local LLMs
• Claude Fable 5 local hardware projection 2028
• Kokoro — OpenAI-compatible localhost TTS
• Fix local LLM looping — samplers & sandboxing
• GLM 5.2 agent harness guide
• Fable 5 usage credits from July 13
• US vs Chinese AI startups

Official: Mesh LLM blog — iroh.computer · iroh docs · HN discussion · Mesh LLM website (linked from iroh blog)

Throughput depends on GPU tier, quantization, network latency, and split tuning — treat 16 tok/s as a published data point, not a SLA. Verify hardware requirements on the Mesh LLM models list before production workloads.