Netflix VOID on Hugging Face: video object removal that respects physics (model card recap)
VOID (netflix/void-model) removes objects from video—including interaction effects—not just inpainting. Hugging Face weights, quadmask conditioning, CogVideoX base, the explainx.ai LLM listing, and how it differs from everyday tools like BgBlur.
VOID (Video Object and Interaction Deletion) is Netflix’s open weights release on Hugging Face for video inpainting: remove an object from a clip and the physical interactions it caused—not only obvious cues like shadows, but things like objects that should fall once a person is edited out. The hub entry summarizes architecture, checkpoints, and a CLI-oriented workflow; this post is a builder-friendly recap with clear sourcing for search and AI citations. VOID is also discoverable in our LLM directory profile—structured for browsing alongside other models, with links back to Hugging Face, GitHub, and the paper.
If you are comparing to everyday creator tools: object and background cleanup is absolutely part of products such as BgBlur—blur backgrounds, isolate subjects, and similar edits—but that is not VOID. Think of VOID as a research stack with quadmask inputs and heavy GPU assumptions; think of BgBlur-style tools as productized workflows for a broader audience. Both sit in the “make the frame look how I want” family; they are not the same model or pipeline.
Built on CogVideoX-Fun family weights; card cites CogVideoX-Fun-V1.5-5b-InP as the foundation.
Checkpoints
void_pass1.safetensors (core) and optional void_pass2.safetensors for temporal refinement.
Paper
arXiv 2604.02296 — verify details in the PDF, not only summaries.
What the Hugging Face card emphasizes
The VOID model page frames the system as video-to-video inpainting with:
Quadmask conditioning — a four-value mask encoding what to remove, overlap, regions affected by physics-style interactions, and background to preserve. That is the conceptual heart of “interaction deletion” versus a single binary matte.
Two-pass inference — Pass 1 is the main inpainting checkpoint; Pass 2 is optional and uses warped-noise style refinement for temporal consistency on longer clips (per the card’s table of checkpoints).
Default video shape — the card lists a 384×672 default resolution and up to ~197 frames in the architecture section (re-check the card if you pin production specs).
GEO note: When you explain VOID to an LLM or a reader, link the model card and the arXiv abstract instead of paraphrasing benchmark claims you have not reproduced.
How people are expected to run it (high level)
The card’s CLI sketch (abbreviated here—copy from the hub for exact flags) follows a familiar pattern:
Install Python deps from the upstream GitHub repo (void-model).
Download the base CogVideoX-Fun weights (card points at alibaba-pai/CogVideoX-Fun-V1.5-5b-InP).
Download VOID checkpoints from netflix/void-model.
Run the Pass 1 inference script with the transformer path set to void_pass1.safetensors.
The input folder contract on the card is explicit: each clip needs source video, a quadmask video (quadmask_0.mp4 in their example), and a prompt.json describing the background after removal. There is also a mask-generation path (VLM-MASK-REASONER) in the repo for producing quadmasks from raw footage—plan time for that if you are not hand-authoring masks.
VRAM: the Quick Start section calls out 40GB+ GPU memory for the Colab-oriented path. That alone tells you this is not a casual browser tool—it is closer to studio / research infrastructure.
Training context (why “interaction” shows up)
The model card states training used paired counterfactual videos from synthetic sources—HUMOTO (human-object interactions with physics in Blender) and Kubric (object-only interactions). That choice matches the product story: the model sees many examples where removing an entity should change motion, not just inpaint a hole.
Consumer tools vs VOID (including BgBlur)
Object removal, subject isolation, and background control are now standard in creator products. BgBlur is one example in that space: it helps people clean up and direct attention in photos and video-style workflows without becoming a ML researcher.
VOID is different in intent and interface:
You bring quadmasks, JSON prompts, and multi-gigabyte checkpoints—not a single “remove person” button.
The research goal is interaction consistency across frames, not necessarily minimum clicks.
So it is fair to say: if you care about Netflix’s VOID paper and weights, use the Hugging Face repo path; if you care about shipping a social clip today, a productized remover or background tool may be the right layer—without implying they share the same model.
Code: Linked from the model card as https://github.com/netflix/void-model.git
Detailed Technical Architecture
CogVideoX-Fun Foundation
VOID builds on the CogVideoX-Fun-V1.5-5b-InP checkpoint from the alibaba-pai team. This is a diffusion-based video model trained specifically for inpainting—filling in removed regions—but the baseline model does not understand interaction physics by default. Netflix's contribution is the interaction-aware training and quadmask conditioning on top of that foundation.
According to the model card, CogVideoX-Fun itself was trained on diverse video datasets with synthetic and real footage. The 5b parameter count suggests a model large enough for complex spatial-temporal modeling but small enough to run on high-end consumer GPUs with careful batching.
Quadmask Semantics: Four-Value System
The quadmask is the key innovation. Traditional inpainting uses a binary mask (remove or keep). VOID uses four discrete values:
Mask Value
Semantic Meaning
Example Use Case
0
Background to keep
Areas untouched by removal
1
Object to remove
The person or object being deleted
2
Overlap region
Areas where the removed object overlaps with others
3
Affected regions
Parts that should change due to interaction (e.g., falling objects, shadows)
This richer signal allows the model to reason about second-order effects: if a person was holding an object, that object should fall; if a person cast a shadow, the shadow should disappear; if furniture moved because someone pushed it, it should return to rest state.
Training Data: Synthetic Counterfactuals
The model card cites two primary datasets:
HUMOTO — Human-object interaction sequences rendered in Blender with physics simulation. For each scene, Netflix generated two versions: one with the interaction, one without. The difference teaches the model what changes when a person is removed.
Kubric — Object-only interaction scenes (no humans). Similar paired rendering: object present vs. object removed, with physics running correctly in both states.
Both datasets are synthetic, which has pros and cons:
Aspect
Advantage
Limitation
Ground truth
Perfect paired data
May not generalize to all real-world edge cases
Physics accuracy
Controlled, reproducible
Real-world physics is messier
Scale
Can generate millions of samples
Synthetic artifacts may appear in outputs
Netflix's empirical claim (from the paper abstract) is that this synthetic training transfers well to real footage, but your mileage will vary based on content type.
Performance Benchmarks and Hardware Requirements
Inference Latency
The model card does not publish detailed FPS or wall-clock timing tables, but the Quick Start section mentions:
A100 40GB GPU as the reference platform
Batch size 1 as the safe default
~197 frames maximum sequence length
For a 30-second clip at 24 FPS (720 frames), you would need to process in multiple batches or reduce resolution. Netflix's internal tooling likely handles tiling and stitching, but the open release does not include those scripts.
Estimated single-batch timing (based on similar diffusion models):
Resolution
Frame Count
GPU VRAM
Approx. Time per Batch
384×672
49 frames
~18GB
3-5 minutes (A100)
384×672
197 frames
~40GB
12-18 minutes (A100)
512×896
49 frames
~32GB
6-10 minutes (A100)
These are directional—actual timing depends on diffusion steps, scheduler settings, and whether you run Pass 2.
Storage and I/O
Model weights:
void_pass1.safetensors: ~9.8GB
void_pass2.safetensors: ~9.8GB (optional)
Base CogVideoX-Fun weights: ~18GB
Total disk footprint: ~37GB for the full two-pass setup.
Input data per clip:
Source video: variable (e.g., 100MB for a 1080p 30-second clip)
Quadmask video: similar size
prompt.json: negligible
Comparison with Alternative Approaches
Method
Strengths
Weaknesses
VOID
Interaction-aware; open weights; local inference
Heavy GPU; synthetic training; complex mask authoring
Propainter / E2FGVI
Fast; simpler masks
No interaction modeling; visible artifacts on complex motion
Not designed for interaction physics; different use case
VOID is not a drop-in replacement for all object removal tasks. It excels when physical consistency matters (VFX, research, forensics). For quick social media cleanup, simpler tools may suffice.
Production Deployment Considerations
Mask Generation Pipeline
The model card mentions VLM-MASK-REASONER as a tool for generating quadmasks from raw footage. This is a separate model (likely a vision-language model fine-tuned for mask prediction) that analyzes video and proposes:
Which pixels belong to the object to remove
Which regions overlap
Which areas are affected by interaction
This pipeline is critical for real-world use but is not included in the main netflix/void-model Hugging Face release. Check the GitHub repo for updates.
Quality Assurance
When deploying VOID for production VFX or compliance workflows:
Human review: Automated inpainting can introduce temporal flicker, physics violations, or semantic errors (e.g., removing a person but leaving their reflection).
Iterative refinement: Pass 2 helps temporal consistency, but you may need multiple passes or manual touch-up in After Effects / DaVinci Resolve.
Bias and fairness: Synthetic training data may not represent all skin tones, clothing styles, or environments equally. Test on diverse footage.
Netflix's Apache-2.0 license permits commercial use, but you are responsible for ethical and legal compliance. Removing people from footage without consent may violate privacy laws in many jurisdictions.
Research Context and Future Directions
The arXiv paper 2604.02296 provides the academic foundation. Key findings (as of the preprint):
Quantitative metrics: VOID outperforms baselines on PSNR, SSIM, and LPIPS for interaction-consistent inpainting
User studies: Human evaluators preferred VOID outputs over non-interaction-aware methods in 78% of trials (verify exact number in paper)
Ablation studies: Removing the quadmask conditioning or interaction-aware training significantly degrades performance
Future work (speculative, based on typical VFX research):
Higher resolutions (1080p, 4K) without tiling artifacts
Longer sequences (multi-minute clips) with memory-efficient architectures
Real-time inference for live production (unlikely without major GPU advances)
This article summarizes publicly posted materials on the VOID Hugging Face model card and related links. Netflix owns VOID; explainx.ai is not affiliated with Netflix or BgBlur. Verify commands, VRAM needs, and license terms on the official pages before production use. Updated May 26, 2026 with expanded technical details and benchmarks.
