Local AI

Oct 27

Oct 27

On October 27, 2026 Opera began shipping WebAI, a proposed web standard that lets websites and extensions call AI models running on your own computer. Here is what is confirmed, what is still unclear, and what it means for developers.

Oct 27

Sentient OS is an open-source, AGPL-3.0 Mac app from a YC F26 team that analyzes your mail, messages and files overnight with an on-device model, then offers to do the work. We checked the README and release notes for what is truly local.

Oct 27

Survival LLM, trending on Hacker News on October 27, 2026, packages a local model and an offline emergency library. explainx.ai covers what it does, the author's own test numbers, install costs, and why commenters question whether an LLM belongs in an emergency kit.

Oct 22

Underdog released Bark 3.0, a 0.6B English speech recognition model in a 163 MB download under CC-BY 4.0. Its card says it averages 4.94% word error on seven Open ASR Leaderboard sets, edging NVIDIA Parakeet and beating Whisper large-v3. The numbers are self-scored.

Oct 20

A Hugging Face post by two PyTorch MPS contributors details what changes for Apple Silicon in PyTorch 2.15: FP8 tensors, faster dense and mixture-of-experts decoding, Metal kernels in place of MPSGraph, and a preview of 2.16.

Oct 16

Magnitude, a Y Combinator-backed open-source inference engine, says it decodes about 2.2x faster than Ollama on a Mac M4 Pro and on an NVIDIA DGX Spark by tuning kernels to each model and chip. Here is how it works, how to try it, and why the benchmark needs a careful read.

Oct 16

A new open-source app, PS5LM, runs llama.cpp on a jailbroken PlayStation 5 GPU and serves OpenAI and Anthropic-style APIs to your network. We go through the claimed speeds, the setup, and what to be careful about.

Oct 15

PrismML added three small ternary models to the Bonsai 2 family, each more than 9x smaller than its full-precision counterpart. Here is what the company claims, what is unverified, and how to try them.

Oct 14

Hugging Face and the ggml team shipped Llama Windows, a free open-source tray app that runs llama.cpp locally, downloads models from the Hub in one click and exposes an OpenAI-compatible API on localhost.

Oct 13

Janas is a new GPL-3.0 inference engine written in C that keeps mixture-of-experts weights on an SSD and streams only the experts each token needs. Its author reports an 80B-parameter model answering on a 32 GB laptop. Here is how it works, what the numbers do and do not show, and how to try it safely.

Oct 13

Liquid AI released LFM2.5-24B-A2B, a 24B-parameter reasoning model that activates only 2B parameters per token. It targets local agents in OpenClaw, Hermes Agent, Pi and OpenCode. explainx.ai walks through the numbers, the architecture, how to run it and where it is weaker.

Oct 13

Qualcomm CEO Cristiano Amon said on October 8 that AI companies want phones that continuously run models around 100 billion parameters by 2028. It is a customer request, not a product promise. Here is what is verified and the memory and quantization math behind it.

Oct 12

OpenBMB has published MiniCPM-V 4.7, its first mixture-of-experts vision model. The 35B-A3B checkpoint is Apache-2.0 and, per the model card, runs on phones using Lookahead Expert Scheduling. Here is what is documented, what is self-reported, and how to try it.

Oct 11

Oct 10

llama.cpp build b11404 adds Metal matrix kernels tuned for 2 to 16 rows, the shape speculative decoding produces when a model verifies draft tokens. A follow-up PR extends them to nearly every quantization type. Here is what the benchmarks actually show and what they do not.

Oct 9

On October 2, 2026, Cactus Compute released Whistle, a 16.9 MB Apache 2.0 speech recognition model that runs on one CPU engine beside the Needle tool-calling model. It beats Whisper base on several benchmarks and loses on others. Here is the full picture.

Oct 9

Samsung Research released LittleBit code that compresses LLMs to 0.1 bits per weight by factorizing and binarizing each weight matrix. The headline says a 13B model fits under 1 GB. The paper says so too, but the method needs training, and the license blocks commercial use.

Oct 8

Oct 7

YC-backed Oki launched Oki Home on October 6, 2026: a 5-liter PC with an RTX 5060 Ti, 32 GB RAM and a credit-card-sized 2 TB Memchip that holds your photos, messages and files on one searchable timeline, with a local Qwen 3.8 27B model claimed at 106 tokens per second. Reservations are $59 and shipping is planned for mid-December. Here is what is claimed, what is not shown, and how it compares with doing it yourself.

Oct 5

Ghost launched Core, a $3,499 personal AI computer that runs open-weight models entirely on device, ingests your apps, files and wearables, and acts without being asked. Batch 1 is already sold out. Here is what Ghost has said, what it has not, and how it compares with building your own local AI box.

Oct 3

Salvatore Sanfilippo's ds4 is back on the Hacker News front page. The useful question is whether your Mac, Spark, or Strix Halo can hold a project quant this week, and whether Claude Code or Codex can talk to it.

Oct 3

On October 2, 2026 the ggml team announced decision-model support in llama-server. You POST state plus typed questions to /v1/systemone and get option probabilities in one forward pass. Five open models ship as GGUF today — and the local runner map just got a third path next to Ollaya and Python/MLX stacks.

Oct 3

Feed headlines said NVIDIA "halved" DGX Spark memory for a $4,999 Oct 23 launch. The official story is a new 64GB OEM SKU beside the 128GB model — same GB10, up to 100B params solo, 128GB when you pair two. Here is what that means for local LLM buyers.

Sep 18

PrismML released Ternary Bonsai 2 27B on September 17, 2026: a 1.76-bit compression of Qwen3.8 27B that fits in 5.9GB while retaining 98.2% of the full-precision model's aggregate benchmark score — up from 95% in the first Bonsai release two months earlier. It runs coding agents locally at 143 tokens/second on an RTX 5090, but Hacker News users report inconsistent real-world throughput and at least one clear reasoning failure in testing.

Sep 17

A September 14, 2026 paper from Intel Labs researchers challenges an assumption baked into every ternary LLM deployed today — that the three weight values are roughly equally likely. They measured 29 real ternary models and found zeros can account for over half of all weights, then built BITCOS, a packing format that exploits this to beat the standard five-trit format on 26 of 29 models.

Sep 10

Edge0, founded by Samuel Zeng, open-sourced a framework that runs a 35-billion-parameter language model using only 1-2.5GB of peak memory — small enough for a phone, in principle. The technique is "SSD expert offload": stream only the parameters a given step needs from storage instead of loading the whole model into RAM. explainx.ai covers how it works, what's actually shipped today versus the demo, and what it means for on-device AI.

Sep 5

Sep 3

Tether — the company behind the USDT stablecoin — released TranslatePsy-AfriSLM on September 2, 2026, a family of open-source, on-device translation models covering 19 Sub-Saharan African languages, claiming the 800-million-parameter version beats much larger models like Qwen3.5-122B on translation benchmarks. Here's what's actually in it.

Sep 2

Google's official Gemma account credited a public MLX leaderboard for making Gemma 4 26B A4B run dramatically faster on Apple Silicon. The precise figure is 130.3%, not a clean 2x—here's what actually changed, who did the work, and what it means for running MoE models locally on a Mac.

Aug 25

On August 25, 2026, Apple launched the M6 Mac mini — its first 2 nm chip with a Dual Neural Engine and Neural Accelerators in every GPU core. explainx.ai breaks down what that hardware actually runs locally, how it compares to the M5 Pro option and M5 Ultra Studio, and whether the $899 entry price still makes sense for AI builders after Namespace spent a week putting MacBooks in racks.

Aug 25

On August 25, 2026, Apple launched Mac Studio with M5 Max (128GB, from $2,499) and M5 Ultra (512GB, 4.3× peak AI vs M3 Ultra). explainx.ai breaks down what that silicon runs locally — MLX, LM Studio, Thunderbolt 5 clustering — and whether the fully specced $18,299 box beats a dedicated GPU for your workload.

Aug 25

On August 24, 2026, @namespacelabs posted a video of MacBooks going into server racks (~2.8M views). The internet asked one question: why laptops? explainx.ai answers whether MacBooks can be servers, what Namespace actually builds, and when Mac minis, Studios, or Nvidia boxes win instead.

Aug 25

On August 24, 2026, Daniel Lemire unpacked Xiaomi's Xring O3 — ~3,945 Geekbench single-core, ~15,221 multi-core, 44MB cache, ARM C1-Ultra cores on TSMC N3P. explainx.ai maps what is real, what is lab-only, and why Qualcomm should worry.

Aug 24

The viral version says FreeToken runs a 753B model on a single GPU. The paper does — at 14.9 tok/s — but on a workstation with a 96GB RTX PRO 6000, 512GB of system RAM, and a 433GB checkpoint. This guide explains the whole machine, why its bandwidth-adaptive MoE runtime matters, and which smaller tier is realistic for your hardware.

Aug 24

On August 24, 2026, Liquid AI and Artificial Analysis released Pipette — an open platform measuring on-device AI as a full system (model × quant × runtime × device), not model cards alone. 1,000+ configs, iOS/Android clients, public data.

Aug 24

Half the documents you want in an LLM cannot be selected: scanned books, slide decks, locked PDF viewers. OCR It pins a rectangle once, captures each page with a hotkey or auto-run, and runs bundled Tesseract entirely offline. explainx.ai walks through install, auto-pagination, and the MV3 tricks.

Aug 20

Superwhisper open-weighted S1-mini, a 0.6B Qwen3 fine-tune that sits after speech-to-text and rewrites messy ASR into clean English. It is not a replacement for Parakeet TDT v3 or Cohere Transcribe. explainx.ai covers the pipeline, the license catch, and how to run the GGUF locally.

Aug 19

bstnxbt/dflash-mlx ports DFlash block-diffusion speculative decoding to MLX on Apple Silicon with lossless greedy verification. On an M5 Max 64GB, Qwen3.5-4B jumps from 54 to 189 tok/s at 2048 tokens, while Qwen3.5-27B-4bit lands at 70 tok/s — the headline "~70 tok/s" figure — at roughly 2.1x over baseline. Speedup varies sharply by model size, architecture, and context length.

Aug 19

128GB of DDR5 now costs $3,399, roughly 10x the lowest price ever tracked, and average DDR5 kit prices are up 350-485% year-over-year — driven by AI datacenter demand locking up global memory production. For anyone building a local-inference rig, this changes the math significantly, and not for a year or two.

Aug 17

Aug 16

Aug 12

Aug 5

Aug 3

Jul 30

Jul 28

Jul 26

Jul 23

Jul 19

Jul 17

Jul 14

Jul 12

Jul 9

Jul 4

Jul 4

Jul 3

Jul 2

Jun 29

Jun 29

Jun 26

Jun 26

Jun 17

Jun 16

Jun 15

May 28

May 22

May 17

May 5