Iris-3B: An Open Pixel-Space Image Model With No VAE, and an Honest Negative Result

Almost every popular image model works in a latent space. A VAE squeezes the image into a small grid, the diffusion model draws in that grid, and a decoder turns the result back into pixels. On October 8, 2026, Sperid Labs released Iris-3B, an open 3B model that skips the VAE and paints every pixel itself.

The release is interesting for a second reason. The authors tested the idea that pixel-space models should help on detail-heavy tasks, and their paper reports that the gain did not show up. This post covers how Iris-3B works, what it can do today, how to run it, and how to read the paper's claims. We read the Hugging Face model card and the arXiv abstract. We did not run the model, so the quality statements are the authors'.

TL;DR: the questions people are asking

What Iris-3B is

The model card describes "a 3-billion-parameter model that paints every pixel directly — no VAE, no latent space — and, fine-tuned, estimates depth and restores and upscales images."

The argument for pixel space is simple. A latent encoder is lossy. It can blur fine texture and bias the model toward smooth detail. If the network outputs pixels, that loss never happens. The card says the authors "explore pixel-space generative models as an alternative to vision foundation models such as DINOv2."

The paper is "Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning," by Hanqiu Li Cai and Chema Garabito of Sperid Labs, submitted to arXiv on October 7, 2026 (arXiv 2610.09450).

How it works

The model card gives these specs:

The arXiv abstract adds that the pixel head follows the pixel-transformer (PiT) design from PixelDiT. The authors first ablated the prediction target and representation alignment at 256 by 256 pixels to decide what to scale.

Two routes to a pixel backbone

The abstract says the team tested both routes to a pixel-space model:

1. Pretrain from scratch. This is Iris-3B.
2. Convert a latent model. They converted a pretrained latent model, FLUX.2 Klein base 4B, to pixel space.

They fine-tuned both families for monocular depth and for restoration and super-resolution.

The negative result, in the authors' words

The abstract is unusually frank. It says: "We find no significant improvement from using a pixel-space generative prior."

The specifics:

• Depth. Fine-tuned with one matched direct-regression recipe, Iris-3B is "level with the latent FLUX.2 Klein." The converted pixel FLUX.2 Klein falls behind it.
• Restoration. On 4x DIV2K restoration, neither pixel model beats a latent FLUX.2 Klein fine-tune. The converted one trails slightly.
• Text-to-image. "Iris-3B shows that pixel-space pretraining ... scales to 3B parameters and to text-to-image quality competitive with latent models, matching Qwen-Image on OneIG under the official evaluators at 1024 by 1024."

So the release does not prove that dropping the VAE wins. It shows that it works at 3B and costs nothing in OneIG quality against one strong open model. For readers who pick models, that is the useful reading: pixel space is a viable design, not yet a clear upgrade. The authors say they release weights and code "in the hope that they help pave the way for further work on pixel-space" models.

More from the full paper: ablations, architecture and the conversion recipe

The sections above rely on the abstract and model card. The full text adds the recipe choices behind Iris-3B, which are the most reusable part for anyone training a pixel-space model. We read the paper's introduction through the architecture section; the later sections on training data, depth numbers and restoration numbers were not in the text we had, so we do not report them.

Headline text-to-image scores. Under the official evaluators at 1024 by 1024 pixels, the paper reports GenEval 0.798 and DPG 86.52, and says this shows pixel-space pretraining is practical at 3B.

How the recipe was chosen: ablations at 256 pixels

Before scaling, the authors ran controlled ablations on a 1.3B dual-stream PixelDiT reproduction at 256 by 256 pixels. All arms used a fixed subset of the pretraining data at global batch 256 and one scoring protocol: EMA weights, 25-step FlowDPM++ sampling, CFG 4.5, GenEval, DPG-Bench, FID-10K against a held-out split, and CLIP score. Each arm was compared with its own control at matched training steps.

Two details matter for reading these results.

• The x-versus-v test is not clean. Earlier work (JiT) argues that predicting the clean image is necessary once patch tokens are high-dimensional, and another group found x-prediction consistently stronger when converting latent models. The authors saw no such benefit, but the x arm also used JiT's logit-normal timestep distribution (0.8, 0.8) instead of their default (0.0, 1.0), which shifts training toward noisier timesteps. They say the contrast "does not isolate the prediction target alone."
• The alignment ranking has nuance. Switching to a DINOv3 teacher lowered GenEval at every milestone and worsened FID throughout. iREPA was the one arm with a real gain on any metric: better FID than control at 150K and 200K steps, but behind on GenEval and CLIP at nearly every milestone, and its FID edge had almost gone by 250K. Self-Flow started near zero on GenEval, closed much of the gap with training, and still sat well below control at 250K.

The authors also note that the architecture changes below were chosen for efficiency at scale and were not selected by these ablations.

What Iris-3B looks like inside

The patch size is a real design lever. The paper cites a concurrent unified model, PixelUMM, which found that 32 by 32 patches train to a higher loss than 16 by 16 even when they see four times as many images.

The conversion route: what blocked FLUX.2 Klein

For the second route, the authors converted FLUX.2 Klein base 4B from latent to pixel space. They identify the input interface as the obstacle, and say a gain-matched ridge initialization and a scaled trunk learning rate remove it. That is useful as a recipe even though the converted model did not beat its latent source on depth or restoration.

It also sets this paper apart from earlier conversion work the authors cite. L2P replaces the VAE with large-patch tokenization and trains only shallow layers, approaching the source model's quality at a small fraction of the cost. Another study reports converted models that match or exceed their latent counterparts while running faster, and AsymFlow reports a converted model that surpasses its FLUX.2 Klein 9B base. Iris's converted 4B model trailing slightly on restoration and falling behind on depth is therefore a less flattering result than that literature suggests, which the authors present openly: "None of these comparisons is free of confounds, and we document those that remain rather than claim to have resolved them."

What it can do

Text to image

The card shows examples generated at native aspect ratios of about one megapixel with the default settings. Prompts range from studio portraits and film-grain photography to a Byzantine mosaic and night landscapes.

Iris-3B pixel-space image generation of Petra's Treasury carved into pink sandstone in morning light, a 3B model without a VAE

Sample output for the prompt "The ancient city of Petra with the Treasury carved into pink sandstone, morning light." Source: the Iris-3B model card by Sperid Labs (Apache 2.0), used with credit.

Depth estimation

Fine-tuned for monocular depth, the model predicts relative log depth in a single forward pass with an empty prompt, so no text encoder is needed. The card reports AbsRel 0.071 and delta1 0.946, as the mean over NYUv2, KITTI, ETH3D, ScanNet and DIODE, using the Marigold V2 protocol zero-shot. Depth is affine-invariant, not metric. You must fit a scale and shift to compare with ground truth.

Iris-3B monocular depth estimation turning a photo into a relative depth map, an image model without a VAE used as a vision learner

Left: input photo. Right: Iris-3B depth output. Source: Iris-3B model card, Sperid Labs.

Restoration and upscaling

A second fine-tune restores and upscales images in one adversarial step, using the HYPIR recipe, 10K steps and EMA weights. The card reports LPIPS 0.292 and PSNR 20.68 on DIV2K validation at 4x. Large outputs are tiled into overlapping 1024 by 1024 windows.

Iris-3B image restoration and upscaling comparing a degraded input to the restored output from a pixel-space diffusion model

Left: degraded input. Right: Iris-3B restoration. Source: Iris-3B model card, Sperid Labs.

How to try it

git clone https://github.com/speridlabs/iris-3b.git
cd iris-3b
pip install -e .          # Python 3.11+, PyTorch 2.7.1+

hf download speridlabs/iris-3b --local-dir iris-3b --exclude "depth/*" "upscaler/*"

python scripts/sample.py --checkpoint iris-3b \
    --prompt "a red fox sleeping in fresh snow, golden hour"

The weights for text-to-image are about 12 GB. The depth and upscaler folders add about 12 GB each. The Qwen3-VL-4B-Instruct text encoder downloads on the first run, and the card says you need an NVIDIA GPU with CUDA.

The card's tuning tips:

Write prompts as plain descriptive English sentences. For depth and upscaling, run scripts/depth.py and scripts/upscale.py. A hosted demo exists on Hugging Face Spaces for trying both without a GPU.

A cost note: 100 steps at one megapixel with a 3B transformer plus a 4B text encoder is not light. The card gives no timing or VRAM figure, so measure on your card before you plan a batch job.

How it compares

If you need production images today, a hosted model or a mainstream latent model still has more tooling around it: LoRAs, ControlNets, ComfyUI nodes. Iris-3B is for researchers, people who want to fine-tune a pixel-space backbone, and teams who care about fine texture and want an Apache 2.0 base.

What this means for what you build or pay

1. No licensing fee. Apache 2.0 allows commercial use of the weights. Check the terms on any data you fine-tune with.
2. A new fine-tuning base. The same backbone supports generation, depth and restoration with no architecture change. A team that needs all three can start from one model.
3. Do not expect a quality jump. The authors found no significant gain from pixel space on depth or restoration. Choose Iris-3B for the design, not for a promised quality edge.
4. Plan for GPU cost. A 3B model at 100 steps and about 12 GB of weights is a datacenter or high-end desktop job, not a laptop one.

Limits and open questions

• Text and counts. The card says text rendering inside images and exact object counts are not always reliable.
• Safety. Like every image generator, it "can produce inaccurate, biased or unsafe content." Review outputs.
• Resolution. About one megapixel. Use an upscaler for larger prints.
• Evaluation scope. The OneIG match is against Qwen-Image only. We have not seen a broad head-to-head with FLUX or other models.
• Early days. The model page showed 18 downloads on the day we looked, so there is little community testing yet.
• Hacker News. A story on the paper appeared on October 8 with no comments when we checked, so we have no developer reports to add.

Summary

Iris-3B is a clean, open, Apache 2.0 proof that a 3B text-to-image model can work without a VAE. Its own paper says the pixel-space prior did not improve depth or restoration. Treat it as a research base and a useful second option beside latent models, and run your own prompts before you commit.

Related reading

• RoAR-VAE: a tokenizer recipe aiming at FLUX.2-VAE

• Qwen-Image 3.0 and richer authentic detail

• FLUX 3 from Black Forest Labs

• Moebius: 0.2B inpainting versus FLUX

• Google Nano Banana 2.5 image model

• PixelRAG: visual retrieval over screenshots

Primary sources: the Iris-3B model card, the arXiv paper 2610.09450, the code repository and the project page. The HuggingNews summary is here.

Specs, benchmarks and download counts are accurate as of October 9, 2026. Model cards change, so re-check before you build on them.