BlenderWorld: Qwen's Benchmark for AI Agents That Build 3D Scenes in Blender

QwenLM has put a new repository on GitHub called BlenderWorld, a benchmark that asks whether AI agents can build real, editable 3D scenes in Blender from a brief and reference images or video. It is a release from a top lab's org, it is MIT licensed, and at the time of writing the repository had one star and no published leaderboard, so this is an early look at the design rather than a ranking of models.

TL;DR

What BlenderWorld measures

Most 3D benchmarks grade a picture. BlenderWorld grades a file. The agent receives a task, reference media and optional spec files, and it has to leave behind a .blend project a human artist could open and keep editing. The task areas named in the README are products, architecture, environments, characters, animation and scientific visualization.

The "from real-world references" part matters. Tasks start from photos or video of real objects and places rather than a text prompt alone, so an agent has to read proportions, materials and layout from the evidence and then reproduce them with Blender's tools. That is much closer to what a working 3D artist does than text-to-mesh generation, and it is why the benchmark is aimed at agents that can run tools, inspect renders and iterate.

Earlier academic work such as BlenderGym tested vision-language models on editing existing Blender scenes. BlenderWorld shifts the question to building from scratch against references.

How the scoring works

The README describes a two-part score. Half comes from a weighted binary rubric (the 2,651 criteria, each a yes or no check such as whether a feature exists or a dimension matches). The other half comes from eight visual-quality ratings. To judge, the harness renders the project from eight adaptive views using Cycles at 1024 by 1024 with 64 samples.

Missing or failed submissions score zero but stay in the denominator, so an agent cannot raise its average by skipping hard tasks. The paper setup, according to the README, uses GPT-5.6-terra in all three evaluator roles through a vision-capable Chat Completions endpoint. Two caveats are stated: the public evaluator runs Cycles on CPU while the paper setup used GPU, and the public protocol applies no score cap, so numbers you get may differ from published ones.

How to run it

The repo is designed around any generation harness, with an optional Harbor adapter. The flow, from the README:

python -m pip install -e .
bw download
bw verify
bw list
bw prepare            # build a workspace for a task, e.g. bw_0001
# your agent writes model-output/final.blend
bw pack-submission    # one ZIP per task
bw evaluate --dry-run # writes a plan without calling Blender or model services
bw evaluate --resume

Requirements listed: Linux, Python 3.10 or newer, Blender 4.2.10, Node.js 22.17.0 and ffmpeg/ffprobe. The bundled coding-agent environment pins Codex CLI 0.147.0 and Claude Code 2.1.220, and Claude Code is also used for rendering and scene inspection. The README points to a Hugging Face dataset under the Qwen org for the task data, which we could not open without authentication when checking, so confirm access before planning a run.

Why this release matters

Agents that drive creative software are one of the fastest-moving areas. Blender is the common target because it is open source and scriptable through Python, and the ecosystem around it keeps growing: see our coverage of Higgsfield's Blender MCP for AI blockouts, the Voyager creative agent harness for Blender, After Effects, Resolve and Ableton, and Tencent's HunyuanWorld Claw for agentic 3D open worlds. Most of those are demos. A benchmark with binary criteria is what lets people compare them.

It also fits Qwen's own agent push, such as the Qwen multimodal plugins harness. For the protocol that lets agents talk to Blender in the first place, see what is MCP.

What is missing

• No results. There is no leaderboard and no per-model table in the README, so claims like "model X is best at 3D" cannot be sourced to this release yet.
• No paper link. The README refers to a paper setup but links none.
• Pinned, heavy environment. Exact CLI versions and a vision-capable judge mean runs cost money and are hard to reproduce as tools change.
• Judge dependence. Half the score comes from model-rated visual quality, which inherits the biases of the judge model.
• Linux only for documented install.

What a task looks like in practice

The repo's own example uses task bw_0001. You run bw prepare for it, which builds a workspace holding the brief and any reference media. Your agent works inside that workspace and must write one file, model-output/final.blend. Then bw pack-submission bundles the output into a ZIP per task, and bw evaluate renders and scores it.

Of the 223 tasks, 78 are text-only, 130 pair an image with text and 15 pair a video with text. Fifty-nine include specification attachments with hard numbers such as dimensions, materials or camera settings. Those are the tasks where binary criteria shine: a table is either 75 cm high or it is not, and a camera is either set to the stated focal length or it is not. The open-ended tasks, such as a stylised character, lean more on the eight visual-quality ratings.

Because the unit of scoring is the .blend file, an agent cannot win by producing a pretty single render. The renderer looks from eight adaptive views, so geometry hidden from the reference photo still has to exist and look plausible from other angles.

Questions people will ask

Is this a text-to-3D model? No. BlenderWorld is an evaluation set. It scores whatever agent or model you connect, and it does not generate anything itself.

Do I need a Qwen model to use it? No. The README says any generation harness works, and an optional Harbor integration is provided. The pinned Codex CLI and Claude Code versions in the bundled environment show that rival coding agents are expected to be run against it.

Why binary criteria? Yes or no checks are easier to audit than a single holistic grade. When an agent scores low, you can list exactly which of the 2,651 criteria it missed. That makes the benchmark useful for debugging an agent, not just ranking it.

Can I trust the visual score? Half the grade comes from a model judge, so treat small differences with suspicion. Re-running with a different judge endpoint, which the README's setup supports in principle, is a sensible sanity check.

How expensive is a run? Not stated. Each task involves agent time, Blender rendering from eight views and several judge calls, so budget for 223 of each, and use bw evaluate --dry-run first to see the plan without spending anything.

How to build an agent that scores well

The README does not publish agent recipes, so this is our reading of the design rather than Qwen guidance. The scoring implies a few habits:

1. Read the spec files first. Fifty-nine tasks have exact numbers. Turn each into an explicit assertion your agent checks inside Blender before it finishes.
2. Render and look. The judge sees eight views. An agent that renders its own scene from several angles and compares to the reference will catch floating parts and wrong proportions.
3. Keep the project editable. The output is meant to be opened by a human, so named objects, sensible hierarchies and real materials beat a pile of joined meshes.
4. Never submit nothing. A missing file scores zero and still counts in the denominator, so even a rough scene beats a skipped task.
5. Budget for failure. Use --resume so one crashed render does not force a full rerun.

Agents that already speak to Blender through MCP servers, like the ones covered in our posts on creative agent harnesses, have a head start because the tool interface is solved and the remaining work is judgement.

A short history of Blender-agent evaluation

Blender has been a testbed for model-driven graphics for a while. BlenderGym, published on arXiv in 2025, benchmarked 13 closed and open vision-language models on graphics editing and included a human Blender-user baseline, reporting a large gap between people and models. Other projects fine-tuned small code models to emit Blender Python scripts from natural language. BlenderWorld is the next step in that line: longer tasks, references from the real world, a full project as the output, and a rubric large enough to see where agents break.

Who should care

If you build Blender agents or MCP servers, BlenderWorld gives you a fixed set of 223 tasks to regress against, instead of cherry-picked demos. If you evaluate frontier models for creative work, wait for the dataset access and a published baseline. Community 3D tests such as what people built with GPT-6 Astra's spatial reasoning show the appetite, but they are anecdotes, not a rubric.

For procedural-generation approaches that skip agents entirely, see iCity's Blender procedural city guide.

Bottom line

BlenderWorld is a sensible, open and reproducible way to ask a hard question: can an agent make a usable 3D project from real references? Whether it becomes a standard depends on what Qwen publishes next, namely baseline scores, the paper and open dataset access. Until then, treat it as infrastructure to watch, not evidence about any model.

Related reading

• Higgsfield Blender MCP and AI blockout
• Voyager creative agent harness
• Tencent HunyuanWorld Claw
• Qwen multimodal plugins
• GPT-6 Astra 3D spatial reasoning builds
• iCity Blender procedural city
• What is MCP

Details reflect the QwenLM repository README as of October 14, 2026 and may change.