MiniCPM-V 4.7: A 35B MoE Vision Model That Runs on an iPhone

OpenBMB has published MiniCPM-V 4.7, and its headline claim is unusual: a 35-billion-parameter mixture-of-experts (MoE) vision model, released under Apache-2.0, that the lab says decodes at 13+ tokens per second on an iPhone 17 Pro Max. The checkpoint is openbmb/MiniCPM-V-4.7-35B-A3B on Hugging Face, and the model card calls it the first MoE model in the MiniCPM-V series. The speed and benchmark numbers are OpenBMB's own; this post separates what the card documents from what still needs independent testing.

If you follow local AI, this is the interesting corner of the open-weight release cycle: not another bigger model, but a bigger model that is engineered to fit where small ones used to be the only option. We have been tracking that trend through Liquid AI's open edge decision models, Cactus Whistle's 16 MB speech model and EmbeddingGemma 2 on device. MiniCPM-V 4.7 pushes the same direction into multimodal MoE.

TL;DR: the questions people ask first

What is MiniCPM-V 4.7?

MiniCPM-V is OpenBMB's line of efficient multimodal models. The 4.7 release introduces a single variant, 35B-A3B, which the card describes as building on the family's single-image, multi-image and video understanding while adding stronger foundation capabilities and efficient visual encoding. The pipeline_tag on Hugging Face is image-text-to-text and the library is Transformers.

The "35B-A3B" naming follows a convention you will recognise from other open MoE models: 35B parameters in total, about 3B activated for each token. That matters for speed, because per-token compute scales with the active parameters, while memory footprint scales with the total. Keep that distinction in mind, because it is the whole story of the on-device claim below.

MiniCPM-V 4.7 35B-A3B mixture-of-experts funnel routing each token to one of several expert paths

Why a 35B model on a phone is hard, and what OpenBMB did

The model card spells out the problem clearly. An MoE model can have far more total parameters than it activates for one token, but its storage needs are not limited to the active ones. Different tokens pick different experts at different layers, and a phone rarely has enough memory to hold every expert. If the weights stay on SSD and are loaded on demand, you can run the model, but the repeated loading stalls make decoding too slow for real-time chat.

The obvious shortcut is to restrict the pool of experts, for example fixing a subset for a whole prompt. The card notes this reduces loading but limits later token-level routing and can reduce capability. OpenBMB's stated goal is "smooth inference without compromising performance", and it combines three techniques:

1. Lookahead Expert Scheduling (LES). It uses cross-layer routing prediction to preload experts likely needed by the next layer. Among the experts the real router selects, it prioritises ones already prefetched and overlaps their computation with loading the rest. Predictions only guide prefetching and scheduling; computation still follows the model's original routing, and any expert the prediction missed is loaded and evaluated as usual. OpenBMB documents it in a dedicated LES write-up.
2. Fine-grained layer-wise quantization, which shrinks the storage and memory traffic of the expert weights while, per the card, preserving capability.
3. DSpark speculative sampling integration, which folds speculative decoding into the on-device path so that scheduling, quantization and drafting work together.

The combined result, according to OpenBMB, is 13+ tokens per second on an iPhone 17 Pro Max, with screen recordings shown for an iPhone and an iPad M4. Thirteen tokens per second is comfortably readable speed for chat, but note that this is a decode figure from the lab's own stack. Prefill time for large images or long video, thermals under sustained load, and battery draw are not quantified on the card.

On-device MiniCPM-V 4.7 routing expert layers to a phone without fixing a restricted expert subset

Efficient vision: early compression inside the encoder

Speed on a phone is not only about the language model. The card says MiniCPM-V 4.7 uses the intra-ViT early compression technique from LLaVA-UHD v4, compressing visual tokens early inside the vision encoder so later encoder layers do less work. It also supports mixed 4x and 16x visual token compression.

In practice you see this as the downsample_mode parameter. The default "16x" merges tokens for efficiency, while "4x" keeps four times as many tokens for finer detail. The card lists a few more knobs worth knowing:

A practical tip that follows from this: for reading dense documents or small text, try 4x with more slices; for casual photo questions on a battery-powered device, stay on 16x.

How does it compare with Qwen3.6 35B-A3B?

OpenBMB frames the release as a new Pareto frontier for multimodal models: in Instruct mode, the card says 35B-A3B reaches a higher overall score in less inference time than Qwen3.6 35B-A3B, and publishes a performance-versus-time chart plus vision-language and language comparison charts. The charts are images on the lab's GitHub, so we are not reproducing individual benchmark numbers here.

Treat that as a claim to test, for three reasons. The comparison is chosen by the publisher, "overall score" is an aggregate whose composition you should inspect, and inference time depends on the serving stack. If you already run Qwen-family MoE models locally, as in our coverage of Strata running a Qwen3 flash model on a consumer GPU, the sensible move is to run your own images, documents and clips through both and compare on your task.

How do I run it?

The model card documents Transformers inference with version 5.18.0 or newer:

pip install "transformers[torch]>=5.18.0" torchvision torchcodec

The card notes that torchcodec can break on some CUDA setups, for example torch 2.11 bundling CUDA 13.1 on a machine with CUDA 12.x, and offers two workarounds: replace it with PyAV (pip install "transformers[torch]>=5.18.0" torchvision av) or pin the torch CUDA build to match your environment. Loading follows the standard pattern:

from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "openbmb/MiniCPM-V-4.7-35B-A3B"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, torch_dtype="auto", device_map="auto"
)

The weights are bf16 and large, so plan for roughly 70 GB of disk for the full-precision checkpoint. Quantized GGUF, BNB, AWQ and GPTQ variants plus FP8 support are listed for people with smaller machines, and the card points to a live demo Space and an OpenVINO path for Intel hardware. Serving options include transformers serve, SGLang, vLLM, llama.cpp and Ollama; check the linked deployment doc for current status per framework, since support for a brand-new architecture often lands unevenly. For llama.cpp users, our notes on recent llama.cpp model support show how quickly new architectures get wired in, and the Metal speculative decoding work is relevant background for the DSpark-style drafting OpenBMB mentions.

Fine-tuning: Self-Replay Fine-Tuning

OpenBMB also announces Self-Replay Fine-Tuning, described as a way for developers to extend model capabilities at low cost while reducing forgetting of existing abilities. The card supports fine-tuning through SWIFT and LLaMA-Factory. Details beyond that one-line description are in the project docs, and we have not tested it, so treat it as a promising option rather than a proven recipe.

What is verified and what is not

Documented by the publisher: the Apache-2.0 license, the MoE design, the A3B naming, the techniques (LES, layer-wise quantization, DSpark integration, intra-ViT compression), the supported frameworks and quantization formats, and the platform list covering iOS, Android and HarmonyOS plus Rockchip, Houmo, Intel, AMD and Arm.

Self-reported, not independently reproduced: 13+ tokens per second on iPhone 17 Pro Max, the better-than-Qwen3.6 35B-A3B trade-off, and "lossless" on-device behaviour. Words like "lossless" and "without compromising performance" are the lab's framing; the card does not publish a bit-exact comparison between the on-device and server outputs in text form.

Unknown: memory use on the phone, battery and thermal behaviour over long sessions, prefill latency for video, and how the model behaves on safety and hallucination tests. Early release-day numbers on Hugging Face were small (low downloads, tens of likes), so community feedback is still thin.

Why this matters

Three reasons stand out. First, MoE has been a server-side story, and a credible phone-class MoE would change what "on-device" can mean: a 3B-active, 35B-total model has far more knowledge capacity than a dense 3B. Second, the technique is general. Predicting and prefetching experts across layers is not specific to MiniCPM, and if it generalises, other open MoE models could follow, which feeds the broader shift described in Microsoft's hybrid local and cloud agent plans. Third, the license: Apache-2.0 means you can build a private, offline image-and-video assistant without a per-token bill or sending photos to a cloud API.

What to do this week

• Pull the model card and read the LES doc before judging the speed claim.
• Test on your own data: receipts, screenshots, handwriting, short clips. Compare 16x and 4x.
• If you target phones, prototype with the vendor's on-device deployment guide rather than assuming llama.cpp parity.
• Wait for third-party reproductions before betting a product on 13 tokens per second.

Specs and numbers are as published by OpenBMB on October 12, 2026 and may change as the repository is updated.

Related reading

• Qualcomm: 100B-parameter phones by 2028, claimed vs verified

• Liquid AI open edge decision models

• Cactus Whistle: 16 MB on-device speech-to-text

• EmbeddingGemma 2 on-device guide

• Strata: Qwen3 flash model on a consumer GPU

• llama.cpp model support roundup

• Microsoft hybrid local and cloud intelligence

• Official: MiniCPM-V 4.7 model card · MiniCPM-o repository docs