explainx / blog
Petals, a 2022 BigScience project for running Llama peer-to-peer, resurfaced on Hacker News in 2026. Why distributed LLM inference remains hard.
Jul 12, 2026
n0's Mesh LLM 1.0 exposes distributed inference as localhost OpenAI API. Run locally, route to peers, or Skippy-split giants across your mesh. explainx.ai architecture, benchmarks, and HN perf debate.
Jul 10, 2026
JustVugg's colibrì streams GLM-5.2 routed experts from a 370 GB int4 container while keeping 9.9 GB dense weights in RAM — 453-point Show HN. explainx.ai maps the physics, honest tok/s, and when disk-streaming beats 256 GB Unsloth setups.
Jul 9, 2026
Docker Desktop founders Jeff and Michael raised $88M for Ollama on July 9, 2026 — 9M+ active builders per @ollama, hybrid cloud scaling, day-one open model support. explainx.ai explains what Ollama is and why it matters now.
TL;DR: Petals, a 2022 open-source project from Hugging Face and the BigScience research workshop, resurfaced on Hacker News in 2026 — a BitTorrent-style system for running large language models peer-to-peer across volunteer consumer GPUs, supporting models up to Llama 3.1 405B, Mixtral 8x22B, Falcon 180B, and BLOOM 176B. Single-batch inference reaches up to 6 tokens/sec for Llama 2 70B. The renewed discussion centers on whether distributed inference is finally practical given 2026-era quantization and smaller high-quality models — and whether network bandwidth, the project's original bottleneck, has actually improved enough to matter.
Petals lets you run a large language model without owning the hardware to run it alone. Each participant loads a portion of the model's layers onto their own GPU (or a free tier like Google Colab) and joins a network of other participants collectively serving the rest. A user's inference request gets routed through the chain of volunteer nodes, each contributing its slice of the model, producing output as if the whole model ran locally — the same basic principle as BitTorrent, applied to model layers instead of file chunks.
Per the project's homepage:
"You load a part of the model, then join a network of people serving its other parts. Single-batch inference runs at up to 6 tokens/sec for Llama 2 (70B) and up to 4 tokens/sec for Falcon (180B) — enough for chatbots and interactive apps."
Petals also supports fine-tuning and custom sampling methods, going beyond a typical hosted API's fixed interface — since participants run actual model code rather than calling a black-box endpoint, users get access to internal hidden states and can execute custom paths through the model.
| Question | Answer |
|---|---|
| What is it? | Peer-to-peer LLM inference, BitTorrent-style |
| Origin? | BigScience research workshop + Hugging Face, 2022 |
| Supported models? | Llama 3.1 (up to 405B), Mixtral 8x22B, Falcon 40B+, BLOOM 176B |
| Speed? | Up to 6 tok/s (Llama 2 70B), up to 4 tok/s (Falcon 180B) |
| Hardware needed? | Consumer GPU, or Google Colab (free tier) |
| Fine-tuning? | Yes — supports custom fine-tuning and sampling |
| 2026 activity level? | Limited recent GitHub activity, per HN commenters |
| Why resurfacing now? | HN submission of the homepage reignited the "is P2P inference viable now?" debate |
Petals launched at a moment when running a 176B-parameter model like BLOOM locally was essentially impossible for anyone without datacenter-scale hardware. Distributed peer-to-peer inference was a genuinely creative answer to that constraint. The 2026 resurfacing prompted commenters to ask whether the underlying premise still holds, given how much has changed in the intervening years:
"Petals is from 2022. Nowadays intelligence of smaller models, quantization techs, and optimizations to run models faster on consumer GPUs have improved a lot." — @woctordho
This is the crux of the debate: in 2022, the only way to access frontier-scale capability was a model too large for any consumer GPU. In 2026, techniques like aggressive quantization and smaller, highly capable models mean a single consumer GPU can now run models that would have required distribution across a Petals-style network four years ago. That shrinks Petals' original use case even as the underlying networking technology theoretically could support larger models today.
The most consistent technical objection across the Hacker News discussion was network bandwidth and latency, not GPU availability:
"It's an interesting concept but the timing is probably too early. If more people had reliable low latency gigabit or ideally 10gbit throughput, then it might start to approach feasibility." — @N_Lens
"Not gonna work, will be starved by inter node bandwidth." — @lostmsu
Splitting a model's layers across multiple physically separate machines requires passing activation tensors between nodes on every forward pass — a fundamentally different communication pattern than downloading a static file via BitTorrent. Residential internet connections, even good ones, add latency and bandwidth constraints that a single machine's internal GPU-to-GPU interconnect (or a datacenter's dedicated fabric) doesn't have. One reply pointed to the underlying infrastructure gap directly:
"Why Switzerland has 25 Gbit internet and America doesn't" — @woctordho, linking to a related HN discussion
This is a genuinely regional problem: distributed inference's practicality depends heavily on what "typical" residential bandwidth looks like in a given country, which varies enormously worldwide.
Petals wasn't the only attempt to solve this problem, and the ecosystem has diversified into different architectural answers to "how do you share LLM compute across a network":
| Project | Approach | Best fit |
|---|---|---|
| Petals | Split one large model's layers across many nodes | Running models too big for any single participant's GPU |
| AI Horde (formerly Stable Horde) | Route each full request to a single volunteer node with a resident model | Avoiding cross-node latency; powers SillyTavern by default |
| Mesh LLM | Splits large MoE models (up to 235B) across a local area network using the iroh P2P library | LAN-scale distributed inference with lower latency than open internet P2P |
AI Horde's design sidesteps Petals' core bottleneck entirely — instead of splitting one model across many nodes and paying a per-hop latency cost, it routes a complete request to a single node that already has a compatible model loaded, and uses a "kudos" economy to incentivize volunteers. That's a meaningfully different tradeoff: no cross-node communication overhead during generation, but you're limited to whichever models volunteers happen to be running.
Mesh LLM, which explainx.ai covered in its own iroh-based distributed inference guide, targets a narrower and more tractable version of Petals' original problem: splitting large mixture-of-experts models across a local network rather than the open internet, where bandwidth and latency are dramatically more favorable than a global volunteer swarm. That LAN-scoped framing is arguably the more realistic near-term answer to the bandwidth objections raised against Petals in 2026 — same layer-splitting idea, smaller and faster network.
Not everyone in the thread dismissed the concept outright. One commenter sketched a smaller-scope alternative that sidesteps the cross-machine tensor-passing problem entirely:
"I have been thinking of something on these lines but with much smaller models. The entire model has to fit on a single computer... Peer to peer, consumers get to route their request to a host with compatible model." — @brainless
This is architecturally closer to AI Horde than to Petals — routing to a complete model on a single machine rather than splitting layers across several — and reflects a broader shift in how the community thinks about distributed inference in 2026 compared to 2022: given that quantized models now fit on consumer hardware more often, routing to whole models is often simpler and faster than splitting models across nodes, even though the latter is the more technically ambitious idea.
If you're evaluating distributed inference for your own use case:
If you're building distributed AI infrastructure:
Primary sources: petals.dev · github.com/bigscience-workshop/petals · huggingface.co/bigscience · stablehorde.net
Project details reflect Petals' public documentation and 2026 Hacker News discussion as of publication. The project originated in 2022 — verify current maintenance status and feature support directly on GitHub before depending on it.
