NVIDIA Flora: Open Robot-Data Models for Captioning, Segmenting and Rewarding Video

NVIDIA has put the data-labeling models behind its robot foundation model work on Hugging Face. Flora is a suite of three models, a temporal segmenter, a captioner and a reward model, that turn raw robot and human manipulation video into training signal. The model cards describe Flora as "the data engine used for training GR00T-2," released "to help researchers scale up custom robot datasets."

The weights are open for research, but not for commercial use. And the headline cost claim circulating on social media, a 125x reduction, does not appear on the cards we read. This post explains what each model does, the numbers NVIDIA reports, the license catch, and how to check the claims. For context on NVIDIA's wider robotics push, see our coverage of the Cosmos data factory blueprints and SimReady assets on Hugging Face.

TL;DR: the three Flora models

What does a robot data engine do?

Training a robot policy needs video of manipulation paired with language ("place the blue mug on the tray") and a sense of whether the attempt worked. Raw teleoperation and human egocentric footage has neither at scale. A data engine labels it automatically. Flora chains three steps.

1. Segment. The segmenter reads overlapping video windows, 16 seconds at 4 Hz in the standard profile (64 frames), or 32 seconds in the long profile, and outputs JSON intervals in 20 Hz ticks plus flags for operations that continue beyond the window. The pipeline projects these onto the source clock and stitches them into episode-level events.
2. Caption. For each stitched interval, the captioner sees the segment plus the neighboring BEFORE and AFTER context and writes one concise imperative, such as "Place the blue mug on the tray with your right hand." It can take a single camera or a three-camera mosaic (main camera plus two wrist cameras).
3. Reward. The rewarder scores episodes. The cards describe an outcome reward from 1 (complete failure) to 5 (complete success), a dense reward giving an action-quality score for each 0.5-second step in a 16-second window, and a rubric reward with a score per step.

The cards say the captioner text is used for balancing the pretraining data distribution, and the rewarder supports monitoring real-world deployments and reinforcement learning post-training. This is annotation infrastructure, not a robot controller: the cards state outputs are annotations and not intended to directly control robots or to be the sole safety monitor for a physical system.

How the models were built

The captioner and rewarder are 27B-parameter models post-trained from Qwen3.5-27B. The captioner uses supervised fine-tuning, then reinforcement learning with GRPO and a reference-grounded caption-quality reward. The segmenter is a rank-32 LoRA adapter on Qwen3.5-9B, also refined with GRPO using a verifiable temporal-matching reward. Training data is a mix of open-source datasets, vendor-licensed data and NVIDIA's own collection, covering fewer than 10,000 hours of video per model according to the cards. GRPO is the same reinforcement-learning recipe we saw applied to a 3D critic in Scene-Designer.

What the numbers say, and what they do not

All figures below are NVIDIA-reported, citing the Flora paper; we have not reproduced them.

• WGO-Bench: 100 human and robot manipulation episodes (25 HomER, 50 DROID, 25 Galaxea) with 743 gold segments. Captioning is scored on gold intervals with the episode instruction supplied, so 64.7% is accuracy under favorable conditions, not end-to-end pipeline accuracy. Segmentation at IoU 0.75 gives 50.3 F1: roughly half of strict boundary matches, which is a reminder that automatic segmentation is far from solved.
• RoboRewardBench: a mean absolute error of 0.40 on the 1 to 5 outcome scale (the card reports it multiplied by 100 as 40.0).
• RobometerBench: policy ranking on 567 trajectories from six out-of-distribution datasets, Kendall tau-a of 0.72.

The cards list no comparison against other labeling systems, and no test dataset separate from the benchmarks. Treat these as the vendor's own evaluation.

The 125x claim: claimed vs verified

An aggregator summary of posts by the Flora researchers says the suite "delivers performance comparable to Astra at 125x lower cost," labels about 200,000 hours of video a day on 500 GPUs, and is used to train GR00T. The posts are from NVIDIA researchers on X, who are primary voices on their own work.

To check the 125x number yourself, you would need the paper's cost table: what hardware price, what hours labeled, and which baseline model and prompt budget. We could not find that comparison in the model cards, so we do not repeat the figure as established. If you use it, attribute it to the researchers.

The license catch

Both card sets say the models are governed by the NVIDIA OneWay Noncommercial License (22 March 2022) and that "Flora is not intended or supported for commercial or production use." The Qwen base models are Apache 2.0, but the post-trained weights are not. In practice:

• Fine for academic labs, benchmarks and prototyping data pipelines.
• Not fine for a startup annotating customer robot data in production.
• Check the license text before including Flora outputs in a dataset you plan to release commercially, since derivative terms matter.

How to try it

1. Pull the repos: nvidia/Flora-TempSeg (load the LoRA adapter on the Qwen3.5-9B base), nvidia/Flora-Captioner and nvidia/Flora-Rewarder.
2. Use PyTorch with Hugging Face Transformers on a Linux GPU. The cards list Ampere, Hopper, Lovelace and Blackwell, with testing on H100. The captioner and rewarder are 27B models, so expect a high-memory GPU or multi-GPU setup; the captioner's checkpoint is about 109 GB in FP32 per the page.
3. Follow the reference inference profile: the trained Qwen3.5 processor and chat template, thinking disabled, greedy decoding, and frame sampling prepared as the cards describe, with do_sample_frames=False.
4. Preserve the mixed dtypes: BF16 base weights with FP32 Gated DeltaNet parameters for the captioner and segmenter.

Questions people will ask

Is Flora a robot model? No. It does not output actions. It produces text and numbers about video, which downstream training uses.

Why build on Qwen? The cards state the base models are Qwen3.5-27B and Qwen3.5-9B, both Apache 2.0, with NVIDIA's post-training on top. That is a notable choice for a US chipmaker's flagship robotics pipeline and shows how widely open Chinese base models are used as raw material.

Does it work on any robot? The training data covers "multiple embodiments" plus human egocentric video, and WGO-Bench draws from HomER, DROID and Galaxea data. Generalization to your own robot is untested by us; run the rewarder on a held-out set of your episodes before trusting its scores.

What about people on camera? The cards warn that you need rights and permissions for all input video, and that if footage includes people or personal information, the models will not blur or alter them. Egocentric human video raises privacy questions that the license does not solve for you.

How does it compare to prompting a frontier VLM? NVIDIA's cards give no head-to-head table, and the 125x claim is the only comparison that circulates. If you already caption video with a general-purpose model, a fair test is to label 50 of your own episodes both ways and compare against human labels.

Why this matters

A verifier checkmark, illustrating how the NVIDIA Flora reward model scores robot attempts

Data, not model architecture, is a main bottleneck in robot learning. Open labeling tools lower the barrier for labs building their own datasets, which matters for the humanoid and manipulation work we follow, such as the LIBERO-PERM personalization benchmark and agent-driven robot use in Hao AI Lab's ORA. A reward model is particularly useful: dense step-level scores are what reinforcement learning post-training needs.

The caution is the usual one. A rewarder that scores 0.40 MAE on a 1 to 5 scale is useful for filtering and ranking, but should not be the sole safety monitor, which NVIDIA itself says.

Related reading

• NVIDIA Cosmos data factory blueprints
• NVIDIA SimReady Central on Hugging Face
• GTC Berlin 2026 keynote
• LIBERO-PERM robot benchmark
• ORA: open robot-use agents
• NVIDIA MotionBricks on Unitree G1
• Official: Flora-Captioner, Flora-Rewarder, Flora-TempSeg and the GR00T N1 white paper

Details are accurate as of October 22, 2026 and come from the Hugging Face model cards. Scores are NVIDIA-reported.