NVIDIAMotionBricksSIGGRAPHCharacter AnimationRoboticsPhysical AI
The viral version of NVIDIA MotionBricks is easy to summarize: one neural
model animates a character in Unreal Engine 5, then drives a Unitree G1
humanoid in the physical world. The more consequential result is underneath
that demo. MotionBricks turns more than 350,000 motion clips into a runtime
system with 2ms latency, while preserving the controls a game or robot
application actually needs.
That is why MotionBricks deserved more than the short entry in our
NVIDIA SIGGRAPH 2026 roundup.
The paper, “MotionBricks: Scalable Real-Time Motions with Modular Latent
Generative Model and Smart Primitives,” is not another prompt-to-dance demo.
It is an attempt to replace brittle animation graphs with a generative model
that still accepts velocity, heading, style, and exact keyframe constraints.
The keynote's other real-time generative system raises a different control
question: our DLSS 5 neural-rendering deep
dive
examines whether masks, model selection and intensity sliders can preserve a
game's art direction while changing its final pixels.
NVIDIA reports 15,000 FPS throughput and 2ms latency on an RTX 5090; 5ms inference on Jetson Orin.
What are smart primitives?
Smart locomotion turns velocity, heading, and style into keyframes; smart object authors object/scene interactions with proxy keyframes.
What did the robot demo prove?
The same motion architecture can generate targets for UE5 characters and a Unitree G1, after skeleton retargeting and with a separate physical tracking controller.
Is it open source?
Only partially today. The project page now lists an initial GR00T Whole-Body Control code preview; it does not confirm a complete pretrained-weight/production release.
What MotionBricks is — and what it is not
MotionBricks is best understood as a learned motion in-betweening system with
a runtime behavior interface.
At runtime, a game event or user command does not ask the network to “make a
cool sword pickup” in natural language. A smart primitive converts intent into
sparse target keyframes. The backbone then predicts how long the transition
should take, where the character root should travel, which latent pose tokens
fit that path, and the continuous joint motion between the known frames.
The system runs autoregressively through a motion buffer. It replans when
controls change or future frames run low, rather than regenerating every frame.
In the paper's UE5 and robot demos, replanning is triggered at 10 Hz or when
commands change.
This makes MotionBricks different from three adjacent ideas:
It is not merely text-to-motion; text is too ambiguous for exact runtime control.
It is not a clip database search; the model synthesizes transitions and motion.
It is not an end-to-end robot torque policy; it produces kinematic targets
that another controller tracks.
Which two production problems does MotionBricks solve?
Problem 1: generative motion usually misses the real-time budget
Motion generation has improved quickly, but quality often depends on iterative
sampling. A diffusion model may denoise a motion sequence over many network
evaluations. The paper notes that standard
motion diffusion can take seconds or minutes per clip; accelerated variants
still trade output quality against runtime.
Games and robotics do not have that budget. A 60 Hz game has 16.7ms for the
entire frame: gameplay, physics, rendering, networking, audio, and animation.
A motion system that consumes most of that window—or returns after the input
has changed—is not interactive.
Problem 2: a prompt is not a production control surface
“Walk nervously to the table and pick up the cup” describes intent, but a
runtime needs parameters:
What velocity and heading should the character follow?
Which gait or style should persist?
Where must a hand make contact?
Which keyframes are exact, and which can vary?
How should a new command interrupt buffered motion?
Text- and tag-conditioned systems generally do not expose all of those controls
with predictable precision. Traditional animation graphs do, but at enormous
authoring cost. The paper cites a modern AAA example with 15,000 animations,
5,000 states, and graphs nested 12 levels deep.
MotionBricks' answer is a universal keyframe constraint interface: production
logic decides what must happen; the model synthesizes how the body gets there.
How the modular latent backbone works
The “modular latent” phrase carries most of the paper's technical weight. The
model avoids generating every joint value directly in one monolithic pass.
Instead it compresses motion and solves a sequence of narrower problems.
Stage 1: structured multi-head motion tokens
A conditional tokenizer encodes T frames into T/4 discrete latent tokens.
Rather than one huge codebook or manually assigning codebooks to arms, legs,
and torso, MotionBricks splits the latent feature dimension across multiple
learned codebooks. The network discovers useful composition by itself.
This multi-head design gives a much larger combinatorial vocabulary without
making inference proportional to the raw number of motion clips. It also fails
more gracefully: one mistaken codebook choice need not corrupt the whole pose.
The tokenizer separates root motion from pose. That is an important
inductive bias. The same gait can move along slightly different trajectories,
and the same route can be traversed with a walk, crouch, or injured style.
Stage 2: timing and root trajectory first
The root module predicts:
how many frames should exist between constraints; and
the character's coarse root trajectory through that interval.
MotionBricks accepts variable constraint density. Navigation might provide only
one or two future keyframes; hand-object contact may need several consecutive
positions. Missing constraints become learned mask embeddings, so the same
backbone can process both.
Stage 3: pose tokens conditioned on the route
The pose module is a 150M-parameter transformer with 16 layers and 16
attention heads. It predicts the distribution of latent pose tokens conditioned
on root trajectory and available keyframes.
The model is trained with masked-token prediction. Unlike diffusion, which
normally requires repeated denoising, the authors report that one forward pass
typically produces high-quality motion. This is a primary reason the design
can satisfy runtime latency.
Stage 4: decode and refine
Finally, the decoder expands latent tokens into continuous joint positions,
rotations, velocities, contact labels, and root values. Root conditions and
sparse keyframes enter at multiple decoder layers, preserving spatial precision
instead of hoping a global prompt survives compression.
That coarse-to-fine split matters operationally. A technical animator can
inspect timing or trajectory separately instead of debugging one opaque motion blob.
What “smart primitives” really mean
“Assemble like bricks” does not mean the final motion is assembled from canned
clips. The control logic is modular; the neural backbone still generates
the transition and full-body result.
Smart locomotion
Smart locomotion takes continuous velocity and heading commands plus
selected style examples. It builds proxy keyframes, smooths the commanded root,
and lets the root module refine that mechanical path into one compatible with
the style.
The paper shows walking, running, crouching, strafing, stealth, crawling, and
expressive movement from the shared backbone. The same keyboard command can
produce different root details because “crouch” and “happy” imply different
timing and body mechanics.
Smart object
Smart object stores proxy interaction keyframes relative to a scene object:
approach, hand placement, contact, and follow-through constraints. Move or
rotate the object, and those targets move with it; the backbone fills the
motion between current state and interaction frames.
The UE5 demo covers sitting, jumping over a bench, falling, and picking up a
sword. According to the paper, non-experts authored smart navigation and smart
object setups in under 10 minutes each. That is a research result, not a
guarantee for a shipped game, but it reveals the intended workflow: author a
small spatial contract, not hundreds of transitions.
Conceptual comparison: MotionBricks vs existing runtime approaches
This is a conceptual workflow comparison, not a claim that one method wins
every production benchmark.
Current pose, trajectory query, events, graph state
Motion source
Generated from learned latent distribution
Generated, often with iterative sampling
Retrieved/blended recorded clips
Fine contact control
Explicit proxy keyframes
Usually weak or indirect
Strong when authored into clips/graphs
Runtime latency
2ms reported on RTX 5090
Often too slow for tight loops
Usually fast
Scaling burden
Train backbone; author reusable primitives
Train model; prompt and guidance design
Capture/tag clips; maintain databases and transitions
Novel transitions
Synthesized between constraints
Synthesized, but less deterministic
Limited by database and transition coverage
Best fit
Interactive navigation + object interaction
Offline ideation and broad semantic generation
Deterministic shipped systems with mature assets
Why 15,000 FPS and 2ms latency are a big deal
The two numbers sound contradictory because 15,000 FPS does not mean one
character receives 15,000 sequential control updates per second.
Latency (2ms) measures how long one inference request takes on the tested
desktop RTX 5090 path.
Throughput (15,000 FPS) measures how many output animation frames the
system can produce per second under efficient processing.
MotionBricks generates short future segments—12 to 64 frames at 30 FPS—and
buffers them. A 2ms replan leaves room inside a 16.7ms 60 Hz frame budget. High
throughput matters for crowds, multiple NPCs, previewing variants, or sharing a
GPU with rendering and other inference.
The benchmark is still hardware- and implementation-specific. NVIDIA exports
the model through ONNX, runs it with TensorRT, and reports the headline
numbers on an RTX 5090. The paper's Jetson Orin robot path takes 5ms per
inference, which is slower but still compatible with 10 Hz replanning.
Training is not cheap. The reported setup used 32 H100 GPUs, with roughly
seven days for the tokenizer, three for the root module, and seven for the pose
module. “Real-time” describes deployment, not a low-cost training recipe.
Why the same model reaching Unitree G1 matters
The important transfer is not “animation code directly becomes motor torque.”
It is that the same motion-generation architecture and control vocabulary
span two embodiments:
In UE5, generated motion is retargeted at runtime to demo characters.
For Unitree G1, the motion corpus is retargeted offline to the G1 skeleton.
MotionBricks generates kinematic target motion on Jetson Orin.
A separate physical tracking controller follows those targets on hardware.
This separation is healthy. The planner answers “what whole-body movement should
happen next?” The low-level controller answers “which feasible actions keep this
robot balanced while tracking it?”
The bridge matters because animation corpora contain much broader movement
diversity than most robot datasets. Compare the paper's 350,000 clips with
HIW-500's real-home Unitree G1 trajectories:
MotionBricks supplies a broad motion prior; teleoperation data supplies
sensor-grounded task execution. They solve complementary layers.
It also makes graphics infrastructure relevant to humanoid teams. Retargeting,
contact labels, in-betweening, root trajectories, and animation authoring tools
become pieces of the robot-learning stack—not just character polish.
Honest limits and open questions
The motion is not fully natural yet
The project videos are unusually complete for neural runtime animation, but
viewers can still notice stiffness, foot artifacts, imperfect contact, or
transitions that lack a human animator's intent. The paper adds foot-sliding and
velocity losses precisely because retargeted data introduces artifacts.
“State of the art” means the model wins the paper's comparisons across selected
open and proprietary datasets. It does not mean every generated frame is
production-ready without review.
Kinematic generation is not physical understanding
The authors explicitly list self-collision and hardware-infeasible motion as
limitations. A kinematic planner can request a dangerous flip or place the body
in a configuration the robot cannot realize. The tracking controller may reject
or fail to follow it; MotionBricks itself does not certify safety.
The real-world G1 demo also lacks privileged object poses and terrain geometry.
That disables the paper's smart object primitive outside controlled settings
unless perception—RGB-D, reconstruction, or SLAM—supplies reliable geometry.
This is where systems such as
LingBot-Map
address a different but adjacent layer.
Retargeting remains expensive
Runtime retargeting in UE5 is fast but lower quality. Offline G1 retargeting is
better, but the paper says it required months of iteration. Different limb
lengths, joint limits, balance regions, and contact geometry make “same model”
more nuanced than “same weights on any skeleton.”
What is actually open?
The current project page now describes an initial preview under NVIDIA's
GR00T Whole-Body Control effort: an interactive G1 demo plus a self-contained
synthetic training pipeline, with instructions for incorporating the
BONES-SEED dataset. It says a fuller robotics-integrated model and complete
training pipeline are targeted later.
That is more than “nothing is open,” but it is not confirmation that the exact
350K production model, full pretrained weights, UE5 production plugin, and
complete training corpus are all downloadable under production-friendly terms.
Teams should verify artifacts, license, hardware requirements, and checkpoint
scope on the project page
before budgeting an integration.
Who should care about MotionBricks?
Game studios
Large studios could reduce graph complexity and synthesize transitions across
huge motion libraries. Smaller teams gain more: smart primitives could provide
high-quality runtime behavior without a dedicated army of technical animators.
The caveat is tooling maturity—shipping teams need deterministic fallbacks,
profiling, QA, and art-directable overrides, not only a paper demo.
Robotics teams
MotionBricks offers scalable motion priors and a unified command layer for
humanoids. Pairing those priors with real-world data such as
HIW-500 and
compact on-robot models like the
Gemma 4 Open Duck Mini stack
illustrates the emerging division of labor: motion generation, perception and
reasoning, then low-level control.
Virtual production and previs
Directors could block navigation and interaction using spatial primitives,
generate variations immediately, and preserve editable key constraints. That
is more useful for previs than waiting on polished mocap, while final shots can
still graduate to hand-authored animation.
How MotionBricks fits NVIDIA's physical-AI stack
Our SIGGRAPH 2026 roundup
describes NVIDIA's larger thesis: graphics research becomes robot
infrastructure when rendering, simulation, world models, and control share
representations and hardware.
MotionBricks occupies the motion planning and authoring layer:
Omniverse and simulation provide scenes, physics, synthetic data, and
validation before hardware deployment.
MotionBricks converts commands and spatial constraints into full-body
kinematic trajectories.
A physical tracking controller executes feasible targets on G1.
Real robot corpora such as
HIW-500
provide grounded household trajectories for policy learning and evaluation.
The July 20 keynote used the MotionBricks-to-G1 demo as a compact proof of that
stack. But the paper's strongest contribution may be less cinematic: a common
interface between high-level behavior design and low-level motion synthesis
that can be inspected, constrained, and replanned in milliseconds.
Paper results, benchmark hardware, code availability, and project-page claims
are accurate as of July 21, 2026. MotionBricks is active SIGGRAPH research;
verify release scope, licenses, checkpoints, and robot-safety requirements
before treating the preview as a production animation or control stack.
