SenseNova-Loop-9B-Preview: A Looped 9B Multimodal Model That Runs 24 Layers Twice

SenseNova-Loop-9B-Preview is a multimodal model that stores 32 decoder layers and runs 56. SenseNova released it on Hugging Face on October 30, 2026 under the Apache 2.0 license. A block of 24 layers runs twice with shared weights, so the model gets more depth at the memory cost of a roughly 9B model.

The model card sells it for long-context and agentic work: documents, coding and tool use. It also leaves out what many readers want first. The benchmark section is not readable as text, and the technical report is marked as not yet announced. This post covers what the card states, how to run the model, and what to hold back on until independent tests exist. As of our read it had 220 likes and 2,704 downloads on Hugging Face.

TL;DR: questions people ask first

How a looped model works

An ordinary transformer sends tokens through N different layers once. A looped transformer sends them through some layers more than once, reusing the same weights. You pay for the repeated compute at inference time but not for extra stored parameters, which matters for memory-bound setups such as laptops, phones and small GPUs.

For SenseNova-Loop, the card describes the path this way: layers 0 to 3 run once; layers 4 to 27 form a block that runs twice in sequence; layers 28 to 31 and the output head finish. In the card's notation the looped part is B(B(h)). That makes 32 distinct layers and 56 layer executions per forward pass. The hidden size is 4,096, the feed-forward size is 12,288, and the vocabulary has 248,320 tokens.

We have seen the same idea in other recent releases. NVIDIA's Dory 4B looped model lets you choose one, two or four loops at inference. Our explainer on recurrent looped transformers and reasoning depth covers why researchers like the approach, and The Information has reported that OpenAI is exploring a looped design for its next flagship, which we treat as a claim until OpenAI says more.

Layers and nodes in a network, illustrating how SenseNova-Loop reuses a block of decoder layers in a loop

What mHC adds

Looping the same weights twice can make the signal drift. SenseNova pairs the loop with two-stream manifold-constrained hyper-connections. In the card's words, learned mappings govern how a layer reads from two residual streams, mixes information between them and writes the layer output back. A Sinkhorn normalization constrains the mixing map, and the documented configuration uses 20 iterations.

For background, a separate paper, Hyperloop Transformers by Zeitoun, Torroba-Hennigen and Kim, puts hyper-connections on a looped middle block and reports matching depth-matched baselines with about 50% fewer parameters. That paper is not from SenseNova, and we are not claiming the Loop model follows it. We cite it because it shows why researchers combine loops with hyper-connections: to keep a small, repeated block expressive.

One practical consequence from the card: the loop range, pass count and mHC routing parameters are part of the checkpoint's execution configuration. An inference engine must load them and reproduce the same order. A runtime that ignores them will give wrong output, which is why stock engines do not run this model yet.

Hybrid attention for long context

The decoder interleaves linear and full attention in groups of four layers: three linear, one full. The loop boundaries line up with whole groups, so the pattern holds across both passes. The card lists 16 full-attention heads and 4 key-value heads. Training recipes combine long sequences, images, code and supervised task trajectories, with a maximum sequence length of 262,144 tokens. The card says "documented recipes" reach 256K, but it gives no long-context retrieval scores in text.

How to run it

The card's tested environment is Transformers 5.18.0 and PyTorch 2.11.0 with CUDA 13.0.

python -m pip install "transformers==5.18.0" accelerate safetensors

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sensenova/SenseNova-Loop-9B-Preview"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, trust_remote_code=True, dtype=torch.bfloat16, device_map={"": "cuda:0"}
).eval()

The card's example uses greedy decoding and 256 new tokens, and enable_thinking=True switches on thinking mode. trust_remote_code=True loads a custom class from the repository, so read the code first and run it in a sandbox. The weights ship as about 38 GB of safetensors (the card lists a BF16 backbone with FP32 mHC routing parameters), so plan disk and memory accordingly.

Serving. The card gives a four-GPU LightLLM Docker example. For SGLang it points to pull request 47281, and for vLLM to pull request 64832, because neither mainline includes the implementation. Both are pinned to unmerged code, so expect rough edges. The card also lists a torchtitan-based fine-tuning recipe for eight GPUs and a repository for training speculative-decoding draft models.

What the numbers imply for memory and speed

The card gives the arithmetic. Storing 32 layers keeps the footprint near a 9B model, while 56 executions per forward pass push the compute toward a 14.6B model. In practice that means the weights fit where a 9B model fits, but each generated token costs more than a 9B model would. The gap shows up as lower tokens per second on the same GPU, unless the speculative drafts or multi-token prediction recover it.

The card says practical speedups depend on workload, hardware and serving configuration, and it gives no measured throughput. So the honest summary is: memory like a 9B model, compute like a 14.6B model, quality unknown. The trade is attractive only if the extra depth buys quality that a plain 9B model cannot reach, and that is the exact claim a technical report would need to support.

Two other practical points follow from the design. First, the KV cache for the full-attention layers still grows with context length, though one full-attention layer in every four, with linear attention elsewhere, should keep that growth lower than in a model that uses full attention in every layer. The card does not quantify the saving. Second, because the loop is part of the checkpoint configuration, quantized community builds must preserve it. The Hugging Face page lists 10 quantizations and one fine-tune of the model; we did not test any of them, so check each one's notes before relying on it.

What the model card does not tell you

• No readable benchmark numbers. There is a Key Benchmarks heading, but we could not read scores from it as text. Do not quote figures until you have seen them yourself.
• No technical report. The resources table lists it as not yet announced and the Transformers implementation "publication details to be announced".
• No independent test. We found no outside evaluation, and a web search turned up no coverage beyond the repository.
• Showcase, not proof. The card's demos (motion design, slide design, data analysis) are labeled "Created by Raccoon Work". They show example outputs, not a controlled comparison.
• Preview label. The name says Preview. Treat behavior and weights as subject to change.

Compare this with how small models get credibility elsewhere: Coinbase published measured results when its fine-tuned Qwen3.5 9B beat larger models on fraud, and PrismML shipped Bonsai models for phones with stated sizes and tests. SenseNova-Loop has not reached that bar yet.

Why it is worth watching anyway

Memory is the constraint that keeps good models off consumer hardware. A looped design trades extra compute for fewer stored weights, which is the right trade on devices where bandwidth and capacity are scarce, and it fits the agent workloads the card names, where the same model is called many times. The Apache 2.0 license, working serving instructions and open training scripts make it easy for others to test the claim. Speculative-decoding drafts, now training, could offset the extra layer executions.

For another model-research item from SenseTime, see our post on MT-Forcing for multi-turn image editing. We would expect more detail in the technical report, and we will update this post when it appears.

Should you try it now?

A fair first test: run your own 50 prompts on this model and on a plain 9B model, measure quality and tokens per second, and note the memory use. Looping should cost compute, so the question is whether the quality gain is worth it for your task.

Related reading

• NVIDIA Dory 4B looped model with adjustable loops
• Recurrent looped transformers and reasoning depth
• OpenAI's next flagship and looped transformers: claimed vs verified
• Coinbase's fine-tuned Qwen3.5 9B on fraud
• PrismML Ternary Bonsai for phones and glasses
• MT-Forcing multi-turn image editing from SenseTime

Details are accurate as of November 4, 2026. The model card may change as the preview develops.