NVIDIA MetricTrail: Camera Motion and Metric Depth From Long Video, Explained

NVIDIA has published MetricTrail, a research model that estimates where a camera moved and how far away things are, frame by frame, from long ordinary video. The Hugging Face model card lists a release date of November 5, 2026. It describes a system that stitches together single-image metric depth, learned sparse correspondences and local geometric optimization. The catch: the checkpoint is not downloadable yet, and the license is noncommercial.

This post explains what MetricTrail does, what NVIDIA reports, what is missing, and how it fits the company's growing set of robotics and 3D models, such as the Metatron humanoid policy that runs in a browser and the Long-WAM world action model.

TL;DR: MetricTrail at a glance

What problem does MetricTrail solve?

Visual odometry answers a simple question: given a video, how did the camera move? SLAM adds a map. Classical systems work, but they struggle with scale. A single camera sees the world up to an unknown size, so a trajectory can be correct in shape and wrong by a factor of two. Robots, cars and AR headsets need real meters.

MetricTrail attacks scale by borrowing a depth model that already predicts metric distances from a single image. The card says it uses MoGe-2 as a frozen "metric-depth provider," which predicts camera-Z depth for each frame. Those predictions initialize the 3D patches that the tracker follows and then act as soft constraints during optimization. If several views disagree with a depth guess, the multi-view evidence can override it.

That design choice matters. The system does not retrain a giant depth network. It trains a small adapter and lets frozen foundation models carry most of the visual knowledge.

How the architecture works

NVIDIA describes the architecture as a "learned recurrent visual odometry with frozen visual foundation-model features and explicit bundle adjustment." In plain terms, there are four moving parts.

1. DPVO as the base. MetricTrail builds on DPVO, short for Deep Patch Visual Odometry. It tracks sparse image patches across frames and refines camera poses and patch depths in a bounded window. The base DPVO paper is on arXiv.
2. DINOv3 features. A frozen DINOv3 ViT-S/16 encoder produces 384-dimensional features. Learned residual adapters project them to 64 channels, split them into two 32-channel branches, and add them to the matching stream (128 channels) and the recurrent context (384 dimensions). The final projections start at zero, so training begins from the official DPVO checkpoint's behavior. The DINOv3 method is described in its arXiv paper.
3. MoGe-2 depth prior. The frozen MoGe-2 ViT-L model gives metric depth per frame. See the MoGe-2 paper. It is not used during adaptation training, only at inference.
4. Explicit optimization. Poses and sparse patch inverse depths are optimized jointly in a bounded active window. An optional pose-graph step, loop correction, runs separately on CPU.

The result is a hybrid. Learned components match patches and supply priors. Classical geometry decides the final pose.

How was it trained?

The training recipe is small by modern standards. NVIDIA reports 15,000 optimizer steps at a global batch size of 16 on 16 A100 80GB GPUs, about seven hours. A warm-up stage trains the fusion branches on 5,000 TartanAir clips with the odometry network frozen. Then odometry and fusion branches are fine-tuned together on the remaining 235,000 clips.

All fine-tuning data is synthetic or game-derived:

The card is careful about what this number means. These are sampled clip exposures, not unique clips. It also says that a fine-tuning validation split, exact dataset revisions and sample-level provenance are not documented. Training clips never exceed 15 frames, and no long-trajectory pose corpus is used. Yet the evaluation runs on sequences with tens of thousands of frames. That gap is the central empirical claim, and it is worth watching when independent tests arrive.

What results does NVIDIA report?

All numbers below come from NVIDIA's own model card. The repository ships prediction generation but not the benchmark evaluation scripts, so readers cannot rerun the tables directly from the release. Trajectory error is absolute trajectory error (ATE) as RMSE after one Sim(3) alignment per sequence.

Long outdoor trajectories

The VBR line carries a caveat that NVIDIA states itself. The refinement step improves base-calibration error from 33.51 m to 16.36 m without loop closure, and from 27.95 m to 10.88 m with it. But it is retrospective and averages 7.1 minutes per routed sequence, so it should not be described as the same online system that runs at KITTI speed.

Indoor and dynamic scenes

The card says the shipped configuration does not expose the indoor global-BA and refinement variants. So the indoor numbers describe a research workflow, not the command you can run.

Dense reconstruction and depth

For dense reconstruction, with RGB-estimated intrinsics and no loop closure, NVIDIA reports a Chamfer distance of 0.04 m and an F1 of 97.60 percent at 0.25 m on 7Scenes. On TUM-Dynamic it reports 0.13 m and 89.58 percent F1; on Oxford Spires 0.98 m and 95.12 percent at a 4 m threshold. On KITTI video depth, with one multiplicative scale fitted per video, it reports Abs Rel 0.057 and delta-1 accuracy of 97.8 percent. The card notes that dense depth comes from the frozen frame-level provider and is not a densely optimized output of the tracker.

The card also admits a weak spot: on TUM-Dynamic, "scores trail leading evaluated methods," and the model uses no explicit dynamic-object mask.

Speed and memory

On KITTI-02, with an A100:

The long RGB-only startup is not a bug. The system estimates intrinsics from a prefix of 32 frames (or 128 keyframes in the robust128 mode), fixes the camera model, and replays that prefix before continuing. If you know your camera's intrinsics, supply them and start in under a second.

NVIDIA is clear that these figures are tied to this configuration. A memory footprint "does not by itself establish a supported hardware requirement," and "compatibility with other GPUs and a minimum memory requirement have not been established."

What is missing from the release

Reading the card closely, several pieces are absent or unproven. This is the part most coverage of a model card skips.

• No weights yet. The checkpoint file, named ckpt/dino_dpvo_multidataset15k.pth, "will be released separately on Hugging Face." Until then, the pipeline cannot run end to end.
• Several downloads on top. You need the DINOv3 weights, MoGe-2, and optional calibration and RoMa dependencies, each under its own terms.
• No prebuilt CUDA extensions. You must compile cuda_corr, cuda_ba and lietorch_backends from source with nvcc and Eigen 3.4.0 headers.
• No dense export. The tool writes poses and depth caches only.
• No evaluation scripts. The tables cannot be reproduced from the repository alone.
• An unvalidated postprocessor. The card says the new loop-correction postprocessor "has not been benchmark-validated," and that the paper numbers use the original SALAD and PyPose workflow instead.
• No uncertainty. Learned correspondence weights are not calibrated pose or depth uncertainty, and the model offers no guarantee of geometric correctness.

NVIDIA lists known weaknesses too: dependence on depth-provider accuracy, trouble with low parallax, rapid rotation, rolling shutter and motion blur, and sensitivity to the shared camera model. Loop edges that come from retrieval assume near-duplicate views are almost coincident, with no pairwise geometric check, so a wrong match can bend the trajectory.

License: what you can and cannot do

The license section is dense, so here it is in a table.

If you plan a product, the weights license is the blocker. A startup that wants camera tracking in a commercial robot or app should treat MetricTrail as a reference design, not a drop-in dependency. The Apache-2.0 source may be reusable, but the trained checkpoint is not.

Who should care, and why

Robotics teams. Many robot stacks need a pose and a depth estimate from a cheap camera. A method that gives metric scale from a single RGB stream, at 12.7 FPS on one data-center GPU, is a useful data point. It is not yet an embedded solution: the only tested hardware is an A100, and no TensorRT or Triton export is provided.

3D and simulation researchers. NVIDIA lists use cases such as reconstructing indoor and outdoor scenes for graphics, simulation and robotics research. Metric-scale reconstructions from ordinary video can feed digital twins and training worlds. See how NVIDIA already packages such assets in its SimReady Central OpenUSD library, and how Meta tackles a neighboring problem in GenIA, which reconstructs 3D objects with SAM 3D.

People who follow open model releases. MetricTrail is another sign of the pattern in NVIDIA's recent Hugging Face activity: small adapters on top of frozen foundation models, with long model cards that spell out risks. The company's data-generation side shows up in Flora, its open models for a robot data engine. The distribution side changed in September when NVIDIA agreed to buy the platform that hosts these cards; see our Hugging Face acquisition coverage.

Privacy note for anyone testing on real footage

The card warns that benchmark footage "may include identifiable people, private interiors, readable text, or locations," and that users need rights and permissions for any input video. It also says the model will not blur or preserve proportions of people in images. The fine-tuning data is synthetic and holds no real personal information, but the frozen foundation models carry their own, less documented, pretraining histories. If you run MetricTrail on video of private spaces, treat the cached depth and pose files as sensitive data.

How to try it once the weights land

The card documents a command-line flow. In outline:

1. Prepare a Python 3.10 environment with PyTorch 2.5.1 and CUDA 12.4, plus xformers, yacs, OpenCV, Pillow and SciPy. Run with PYTHONNOUSERSITE=1.
2. Build the three CUDA extensions from source.
3. Download the MetricTrail checkpoint, the DINOv3 ViT-S/16 file and the MoGe-2 model, and check the published SHA256 hashes.
4. Convert your video to prepared PNG frames or KITTI-style NPZ files. The CLI does not decode arbitrary video containers.
5. Write a JSON config and run python -m metrictrail.cli --config config.json. Use --preflight first to check files and imports, and --stop-after to run only calibration or the local stage.
6. Check the successful-completion record before using any output. The card warns that failed runs can leave caches or older files behind.

Optional loop correction is a separate command that consumes a candidate file of frame pairs and its SHA256. MetricTrail does not do place retrieval itself.

What to watch next

• The checkpoint release. Without it, nobody outside NVIDIA can verify the headline numbers.
• Independent KITTI and TUM runs. The training clips are 15 frames long and the tests run for thousands. Third-party results will show whether the long-horizon claim holds on unseen footage.
• The license for the weights. A move from noncommercial to a permissive license would change who can use it.
• Edge versions. An export for Jetson-class hardware would turn this from a research artifact into something a robot could carry.

Bottom line

MetricTrail is a well-documented research release with a small learned core, strong self-reported numbers on KITTI, Oxford Spires, 7Scenes and TUM, and a candid list of limits. Today, though, it is a model card without downloadable weights, under a noncommercial license, with evaluation scripts left out. Read it for the design: frozen foundation models supply metric priors, and classical optimization keeps the geometry honest. Wait for the checkpoint and for outside tests before you plan around it.

Related reading

• NVIDIA Metatron: a Unitree G1 humanoid policy in your browser
• NVIDIA Long-WAM: a world action model with long context
• Meta GenIA: generative 3D reconstruction with SAM 3D Objects
• NVIDIA SimReady Central: OpenUSD assets on Hugging Face
• NVIDIA Flora: open models for a robot data engine
• NVIDIA to acquire Hugging Face for $12.9 billion
• Official sources: MetricTrail model card, DPVO paper, MoGe-2 paper, DINOv3 paper

Details are accurate as of November 5, 2026, from the model card on that date. NVIDIA may update the card, license or weights.