NVIDIA MotionBricks: Real-Time Motion From Characters to Unitree G1

The viral version of NVIDIA MotionBricks is easy to summarize: one neural model animates a character in Unreal Engine 5, then drives a Unitree G1 humanoid in the physical world. The more consequential result is underneath that demo. MotionBricks turns more than 350,000 motion clips into a runtime system with 2ms latency, while preserving the controls a game or robot application actually needs.

That is why MotionBricks deserved more than the short entry in our NVIDIA SIGGRAPH 2026 roundup. The paper, “MotionBricks: Scalable Real-Time Motions with Modular Latent Generative Model and Smart Primitives,” is not another prompt-to-dance demo. It is an attempt to replace brittle animation graphs with a generative model that still accepts velocity, heading, style, and exact keyframe constraints.

The keynote's other real-time generative system raises a different control question: our DLSS 5 neural-rendering deep dive examines whether masks, model selection and intensity sliders can preserve a game's art direction while changing its final pixels.

Primary sources: arXiv:2604.24833 · MotionBricks project page

TL;DR — what people ask after the MotionBricks demo

What MotionBricks is — and what it is not

MotionBricks is best understood as a learned motion in-betweening system with a runtime behavior interface.

At runtime, a game event or user command does not ask the network to “make a cool sword pickup” in natural language. A smart primitive converts intent into sparse target keyframes. The backbone then predicts how long the transition should take, where the character root should travel, which latent pose tokens fit that path, and the continuous joint motion between the known frames.

The system runs autoregressively through a motion buffer. It replans when controls change or future frames run low, rather than regenerating every frame. In the paper's UE5 and robot demos, replanning is triggered at 10 Hz or when commands change.

This makes MotionBricks different from three adjacent ideas:

• It is not merely text-to-motion; text is too ambiguous for exact runtime control.
• It is not a clip database search; the model synthesizes transitions and motion.
• It is not an end-to-end robot torque policy; it produces kinematic targets that another controller tracks.

Which two production problems does MotionBricks solve?

Problem 1: generative motion usually misses the real-time budget

Motion generation has improved quickly, but quality often depends on iterative sampling. A diffusion model may denoise a motion sequence over many network evaluations. The paper notes that standard motion diffusion can take seconds or minutes per clip; accelerated variants still trade output quality against runtime.

Games and robotics do not have that budget. A 60 Hz game has 16.7ms for the entire frame: gameplay, physics, rendering, networking, audio, and animation. A motion system that consumes most of that window—or returns after the input has changed—is not interactive.

Problem 2: a prompt is not a production control surface

“Walk nervously to the table and pick up the cup” describes intent, but a runtime needs parameters:

• What velocity and heading should the character follow?
• Which gait or style should persist?
• Where must a hand make contact?
• Which keyframes are exact, and which can vary?
• How should a new command interrupt buffered motion?

Text- and tag-conditioned systems generally do not expose all of those controls with predictable precision. Traditional animation graphs do, but at enormous authoring cost. The paper cites a modern AAA example with 15,000 animations, 5,000 states, and graphs nested 12 levels deep.

MotionBricks' answer is a universal keyframe constraint interface: production logic decides what must happen; the model synthesizes how the body gets there.

How the modular latent backbone works

The “modular latent” phrase carries most of the paper's technical weight. The model avoids generating every joint value directly in one monolithic pass. Instead it compresses motion and solves a sequence of narrower problems.

Stage 1: structured multi-head motion tokens

A conditional tokenizer encodes T frames into T/4 discrete latent tokens. Rather than one huge codebook or manually assigning codebooks to arms, legs, and torso, MotionBricks splits the latent feature dimension across multiple learned codebooks. The network discovers useful composition by itself.

This multi-head design gives a much larger combinatorial vocabulary without making inference proportional to the raw number of motion clips. It also fails more gracefully: one mistaken codebook choice need not corrupt the whole pose.

The tokenizer separates root motion from pose. That is an important inductive bias. The same gait can move along slightly different trajectories, and the same route can be traversed with a walk, crouch, or injured style.

Stage 2: timing and root trajectory first

The root module predicts:

1. how many frames should exist between constraints; and
2. the character's coarse root trajectory through that interval.

MotionBricks accepts variable constraint density. Navigation might provide only one or two future keyframes; hand-object contact may need several consecutive positions. Missing constraints become learned mask embeddings, so the same backbone can process both.

Stage 3: pose tokens conditioned on the route

The pose module is a 150M-parameter transformer with 16 layers and 16 attention heads. It predicts the distribution of latent pose tokens conditioned on root trajectory and available keyframes.

The model is trained with masked-token prediction. Unlike diffusion, which normally requires repeated denoising, the authors report that one forward pass typically produces high-quality motion. This is a primary reason the design can satisfy runtime latency.

Stage 4: decode and refine

Finally, the decoder expands latent tokens into continuous joint positions, rotations, velocities, contact labels, and root values. Root conditions and sparse keyframes enter at multiple decoder layers, preserving spatial precision instead of hoping a global prompt survives compression.

The pipeline is deliberately inspectable:

commands / game events
        ↓
smart primitive → target keyframes
        ↓
root module → timing + trajectory
        ↓
pose module → multi-head latent tokens
        ↓
decoder → continuous full-body motion

That coarse-to-fine split matters operationally. A technical animator can inspect timing or trajectory separately instead of debugging one opaque motion blob.

What “smart primitives” really mean

“Assemble like bricks” does not mean the final motion is assembled from canned clips. The control logic is modular; the neural backbone still generates the transition and full-body result.

Smart locomotion

Smart locomotion takes continuous velocity and heading commands plus selected style examples. It builds proxy keyframes, smooths the commanded root, and lets the root module refine that mechanical path into one compatible with the style.

The paper shows walking, running, crouching, strafing, stealth, crawling, and expressive movement from the shared backbone. The same keyboard command can produce different root details because “crouch” and “happy” imply different timing and body mechanics.

Smart object

Smart object stores proxy interaction keyframes relative to a scene object: approach, hand placement, contact, and follow-through constraints. Move or rotate the object, and those targets move with it; the backbone fills the motion between current state and interaction frames.

The UE5 demo covers sitting, jumping over a bench, falling, and picking up a sword. According to the paper, non-experts authored smart navigation and smart object setups in under 10 minutes each. That is a research result, not a guarantee for a shipped game, but it reveals the intended workflow: author a small spatial contract, not hundreds of transitions.

Conceptual comparison: MotionBricks vs existing runtime approaches

This is a conceptual workflow comparison, not a claim that one method wins every production benchmark.

Why 15,000 FPS and 2ms latency are a big deal

The two numbers sound contradictory because 15,000 FPS does not mean one character receives 15,000 sequential control updates per second.

• Latency (2ms) measures how long one inference request takes on the tested desktop RTX 5090 path.
• Throughput (15,000 FPS) measures how many output animation frames the system can produce per second under efficient processing.

MotionBricks generates short future segments—12 to 64 frames at 30 FPS—and buffers them. A 2ms replan leaves room inside a 16.7ms 60 Hz frame budget. High throughput matters for crowds, multiple NPCs, previewing variants, or sharing a GPU with rendering and other inference.

The benchmark is still hardware- and implementation-specific. NVIDIA exports the model through ONNX, runs it with TensorRT, and reports the headline numbers on an RTX 5090. The paper's Jetson Orin robot path takes 5ms per inference, which is slower but still compatible with 10 Hz replanning.

Training is not cheap. The reported setup used 32 H100 GPUs, with roughly seven days for the tokenizer, three for the root module, and seven for the pose module. “Real-time” describes deployment, not a low-cost training recipe.

Why the same model reaching Unitree G1 matters

The important transfer is not “animation code directly becomes motor torque.” It is that the same motion-generation architecture and control vocabulary span two embodiments:

1. In UE5, generated motion is retargeted at runtime to demo characters.
2. For Unitree G1, the motion corpus is retargeted offline to the G1 skeleton.
3. MotionBricks generates kinematic target motion on Jetson Orin.
4. A separate physical tracking controller follows those targets on hardware.

This separation is healthy. The planner answers “what whole-body movement should happen next?” The low-level controller answers “which feasible actions keep this robot balanced while tracking it?”

The bridge matters because animation corpora contain much broader movement diversity than most robot datasets. Compare the paper's 350,000 clips with HIW-500's real-home Unitree G1 trajectories: MotionBricks supplies a broad motion prior; teleoperation data supplies sensor-grounded task execution. They solve complementary layers.

It also makes graphics infrastructure relevant to humanoid teams. Retargeting, contact labels, in-betweening, root trajectories, and animation authoring tools become pieces of the robot-learning stack—not just character polish.

Honest limits and open questions

The motion is not fully natural yet

The project videos are unusually complete for neural runtime animation, but viewers can still notice stiffness, foot artifacts, imperfect contact, or transitions that lack a human animator's intent. The paper adds foot-sliding and velocity losses precisely because retargeted data introduces artifacts.

“State of the art” means the model wins the paper's comparisons across selected open and proprietary datasets. It does not mean every generated frame is production-ready without review.

Kinematic generation is not physical understanding

The authors explicitly list self-collision and hardware-infeasible motion as limitations. A kinematic planner can request a dangerous flip or place the body in a configuration the robot cannot realize. The tracking controller may reject or fail to follow it; MotionBricks itself does not certify safety.

The real-world G1 demo also lacks privileged object poses and terrain geometry. That disables the paper's smart object primitive outside controlled settings unless perception—RGB-D, reconstruction, or SLAM—supplies reliable geometry. This is where systems such as LingBot-Map address a different but adjacent layer.

Retargeting remains expensive

Runtime retargeting in UE5 is fast but lower quality. Offline G1 retargeting is better, but the paper says it required months of iteration. Different limb lengths, joint limits, balance regions, and contact geometry make “same model” more nuanced than “same weights on any skeleton.”

What is actually open?

The current project page now describes an initial preview under NVIDIA's GR00T Whole-Body Control effort: an interactive G1 demo plus a self-contained synthetic training pipeline, with instructions for incorporating the BONES-SEED dataset. It says a fuller robotics-integrated model and complete training pipeline are targeted later.

That is more than “nothing is open,” but it is not confirmation that the exact 350K production model, full pretrained weights, UE5 production plugin, and complete training corpus are all downloadable under production-friendly terms. Teams should verify artifacts, license, hardware requirements, and checkpoint scope on the project page before budgeting an integration.

Who should care about MotionBricks?

Game studios

Large studios could reduce graph complexity and synthesize transitions across huge motion libraries. Smaller teams gain more: smart primitives could provide high-quality runtime behavior without a dedicated army of technical animators. The caveat is tooling maturity—shipping teams need deterministic fallbacks, profiling, QA, and art-directable overrides, not only a paper demo.

Robotics teams

MotionBricks offers scalable motion priors and a unified command layer for humanoids. Pairing those priors with real-world data such as HIW-500 and compact on-robot models like the Gemma 4 Open Duck Mini stack illustrates the emerging division of labor: motion generation, perception and reasoning, then low-level control.

Virtual production and previs

Directors could block navigation and interaction using spatial primitives, generate variations immediately, and preserve editable key constraints. That is more useful for previs than waiting on polished mocap, while final shots can still graduate to hand-authored animation.

How MotionBricks fits NVIDIA's physical-AI stack

Our SIGGRAPH 2026 roundup describes NVIDIA's larger thesis: graphics research becomes robot infrastructure when rendering, simulation, world models, and control share representations and hardware.

MotionBricks occupies the motion planning and authoring layer:

• Cosmos models environments and possible futures; see our Cosmos 3 physical-AI guide.
• Omniverse and simulation provide scenes, physics, synthetic data, and validation before hardware deployment.
• MotionBricks converts commands and spatial constraints into full-body kinematic trajectories.
• A physical tracking controller executes feasible targets on G1.
• Real robot corpora such as HIW-500 provide grounded household trajectories for policy learning and evaluation.

The July 20 keynote used the MotionBricks-to-G1 demo as a compact proof of that stack. But the paper's strongest contribution may be less cinematic: a common interface between high-level behavior design and low-level motion synthesis that can be inspected, constrained, and replanned in milliseconds.

Related on explainx.ai

• Unitree AS2-W wheel-legged Super Athlete
• NVIDIA DLSS 5 — real-time neural rendering and the AI-slop debate
• NVIDIA SIGGRAPH 2026 — Cosmos Edge, MCP, and 21 graphics papers
• HIW-500 — 500 hours of Unitree G1 teleoperation in real homes
• LingBot-Map — streaming 3D reconstruction at roughly 20 FPS
• China URKL robot fight — humanoid control under contact
• NVIDIA Cosmos 3 — open physical-AI world model guide
• What are world models? A practical physical-AI guide
• Xiaomi Robotics U0 — embodied synthesis and robot data
• Xiaomi-Robotics-1 — 100K hours UMI VLA foundation model
• Gemma 4 on Open Duck Mini — on-device robot AI

Academic sources: MotionBricks paper, arXiv:2604.24833 · MotionBricks project page

Paper results, benchmark hardware, code availability, and project-page claims are accurate as of July 21, 2026. MotionBricks is active SIGGRAPH research; verify release scope, licenses, checkpoints, and robot-safety requirements before treating the preview as a production animation or control stack.