4Director

Controlling Video World Models with Rigid 3D Geometry

Anonymous
Object Motion Rigid 3D trajectories
New Object Insertion Reference-guided composition
Joint Camera and Object Control Jointly authored trajectories

Rebuttal

W1/Q1 (DAVIS-NewTraj: trajectories authored independently of the source videos)

Inputa DAVIS first frame
Input image: a camel in a sandy enclosure
Input imageinput image (first frame)
Source videosource video: its motion is not used
180° object turnstatic camera
Trajectorystatic camera
Camera orbitaround a static object
Trajectory−120°
Trajectory+120°
Trajectory−180°
Trajectory+180°
Object translation±3 body widths, following camera
Trajectory−3 body widths
Trajectory+3 body widths

DAVIS-NewTraj: 61 trajectories on DAVIS first frames, specified independently of the source videos, in the nine types (examples above, on one first frame, in its point cloud). Without ground-truth videos, we evaluate camera control and recognition rate: 4Director 90.4% vs 86.7% for the strongest baseline.

W2/Q2/Q4 (user study: does the identity gate of IG-IoU agree with people?)

User study form header: Same object? Left: a frame from a real video with one object outlined in pink. Right: the same moment in a generated video.
Case 01 of the form: real frame with the object outlined, generated frame, and Yes / No. Case 07 of the form: real frame with the object outlined, generated frame, and Yes / No. Case 10 of the form: real frame with the object outlined, generated frame, and Yes / No. Case 14 of the form: real frame with the object outlined, generated frame, and Yes / No.
Screenshots of the user study form: 50 frames, one Yes / No question each; method names and judge verdicts are hidden.

W3 (rigid control freezes articulated subjects, e.g. the breakdance of Appendix G)

1Sliding onlyAppendix G, as submitted
Source videoonly frame 0 is used
Depth control
Ours4Director
Outputkeeps the handstand
2Original motionfrom the source video
Source videomotion source
Motion controlSMPL-X body from the source video (GVHMR)
Depth control
Ours4Director
Outputfollows the source moves
3Sliding + original motion2 + the slide of 1
Source videomotion source
Motion controlSMPL-X body of 2, sliding 3 body widths right; camera follows
Depth control
Ours4Director
Outputfollows the source moves; deforms near the end
4Shorter slide + original motion2 + half the slide of 1
Source videomotion source
Motion controlSMPL-X body of 2, sliding 1.5 body widths right; camera follows
Depth control
Ours4Director
Outputfollows the source moves throughout

Sliding only (1): the dancer freezes in the handstand. With the original motion (2), the dancer performs the source moves instead of freezing. Adding the 3-body-width slide of 1 (3), the moves hold at first, but the body deforms near the end: sliding this far while breakdancing is unrealistic. With a 1.5-body-width slide (4), the dancer performs the source moves throughout.

TL;DR: 4Director uses complete object meshes and rigid trajectories in a shared 3D space to jointly control the camera, existing objects, and inserted objects in video world models.

Abstract

Precise control over camera and object motion is essential for professional video production. Existing methods support either camera control alone or coarse object control, through image-plane cues that are ambiguous in depth and rotation or through 3D tracks and blobs that lack complete geometry and lose consistency across viewpoint changes. We introduce 4Director, a video world model conditioned on an explicit 4D scene representation: each object is reconstructed once as a canonical mesh and moved by one prescribed rigid transformation per frame. This representation provides an intuitive 3D control interface and prevents unobserved geometry from being regenerated independently in every frame. We render the controlled scene as a depth video and introduce a Motion Adapter that transforms this geometric scaffold into video while synthesizing view-consistent appearance, illumination, and non-rigid dynamics. For training, we construct RealCOD-Rigid, a new dataset of 20,774 clips annotated with rigid 3D scenes by our automatic pipeline. We further introduce Identity-Gated IoU (IG-IoU), which jointly evaluates adherence to prescribed object motion and preservation of object identity. Experiments demonstrate that 4Director consistently outperforms prior methods in visual quality and in camera and object control.

Method

4Director pipeline: an input image becomes rigid 3D geometry with user-defined object and camera trajectories; depth rendering drives a Motion Adapter alongside a frozen Wan-DiT to produce the generated video.

Qualitative Comparison

What happens if the object leaves and returns?

Input Image
Input image of a camel in an enclosure
Point CloudInteractive
4Director (Ours)

Previous Works Native control above, generated video below

MotionControl
Perception as Control
SymphoMotion
VerseCrafter