← Key papers

Paper review · November 17, 2026

VibePrevisBench: A Benchmark for Evaluating Coding Agents on 4D Scene Generation in Blender

VibePrevisBench: Evaluating Agentic Coding for 4D Scene Generation in Blender

Data & evaluationAgentsGenerative mediaMedia Intermediate Upvotes 83 cs.CV Kling-Keye

The authors build 242 tasks in which coding agents use Blender to create multi-shot 4D scenes and proxy videos, and diagnose the weaknesses of 11 agents using rule-based scene checks and VLM-based screen checks.

Problem

Controllable video generation models take a gray-box proxy video (previs) as a condition to fix layout, motion and camera. As a result, errors in the proxy video carry straight into the final video. Yet no benchmark evaluated both the ability to build long multi-shot scenes in code, with people and the camera moving together, and whether the result is actually visible on screen.

Method

The tasks cover four input types (script, photo, continuation and reference video) and span seven scene domains, for a total of 242. The Blender program an agent submits is re-run in a fresh sandbox to extract the 4D scene and proxy video for every frame. Scoring then happens in two ways. The Rule-based Scene Check compares the output with hidden GT facts across nine dimensions (objects, layout, movement paths, events, camera, cuts and others), correcting for random-placement levels and missing objects. In the VLM-based Screen Check, for items that passed the rule check, a VLM judge (Gemini 4) assesses recognition, action, visibility and story from a viewer's perspective.

Results

Limitations

Appearance is not evaluated. Some GT was estimated with a 4D reconstruction pipeline, so facts that cannot be verified are left out of scoring. The screen check relies on a single VLM judge (the appendix states it was validated against three judges and human evaluation). Because tasks were filtered to keep only those that separate agent performance well, absolute scores should be interpreted with care.

Why it matters

As coding agents are increasingly used as the 'director' in video production pipelines, this work sets up a standard that measures scene structure and on-screen visibility separately. It shows concretely that spatial and camera understanding, along with item-by-item self-verification, are the next areas to improve.

Key terms

Proxy video (previs)
A draft video of a scene rendered as gray boxes without textures. Video generation models take it as a condition so they follow its layout, motion and camera.
4D scene
A scene in which the placement of objects, people and the camera in 3D space changes over time. Position and orientation are defined for every frame.
Rule-based Scene Check
A deterministic scoring method that re-runs the submitted code and checks the exported scene against hidden ground-truth facts using rules.
VLM judge
A vision-language model that watches a video and answers evaluation questions. In place of a human viewer, it judges whether content can be recognized on screen.

Original

Authors: Yuzhe Chen, Muxi Diao, Lele Yang, Wenbin Xu, Xuyang Liu, Qile Su, Changyi Liu, Jiankang Chen, et al. · Posted 2026-11-11 · License CC BY 4.0 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.