Long multi-shot video generation

Balancing Global Planning and Local Rendering for Long Multi-Shot Video Generation

Coherence across Time, for movies longer than any generation window.

Jinpeng Lu* · Haoyuan Shi* · Yuanshen Guan · Jiyuan Xia · Bin Wu · Yinda Chen · Zhiwei Xiong

University of Science and Technology of China · * Equal contribution

Diffusion transformers generate only a few seconds per pass, so minute-long stories must be stitched from local windows where either identity drifts or camera cuts fail. CaT-Movie connects a training-free global story planner with a local window renderer through one structured storyboard that decouples persistent identity from per-shot state and framing.

SHOT 01 / 12Close-upPass 1 / 5
00:00:00:00
s01Close-up shot. Character 1 looks at Character 2 while standing near a metallic shock absorber.
Cable Car · held-out test story · CaT-Movie output12 shots · 27.8 s · 5 generation passes · 7 s window cap
one film strip = one generation passanchor frames recalled from earlier passesclick any shot to jump
Film Storyboard Dataset358,302annotated film shots across 75,461 scene blocks and 332.8 hours
Camera Cut Execution0.974of 674 planned cuts executed across window boundaries
Character Consistency0.651highest among 8 methods, holding 61–63% past 20 seconds
Overall Benchmark Score0.597#1 overall on CaT-MBench across all four evaluation aspects

The trade-off

Existing generators either execute camera cuts or preserve identity across them.

Per-shot pipelines cut cleanly but rebuild the cast in every shot. Joint multi-shot generators coordinate cuts inside a window but lose identity at window seams. Memory-augmented generators carry raw frames forward, bundling identity with prior camera framing so planned cuts repeat old setups.

Consistency against transition control

per-shotjointCaT-Movie

92 held-out test stories. CaT-Movie reaches 0.985 in transition control and 0.593 in inter-shot consistency (0.651 on Character), advancing beyond the baseline Pareto frontier (dotted line).

Method

One structured storyboard decouples identity from per-shot state and framing.

Given a story request at any granularity, a training-free global story planner writes an entity registry and per-shot plans. Deterministic code maintains cross-shot state, compiles self-contained shot prompts, and routes anchor views into a local window renderer that denoises each window jointly.

System overview · Cable Car, pass 4

One window, end to end

Request · three granularities
Storyboard · Qwen3.8-27B, no fine-tuninghover an entity to highlight its shots
Window packing · greedy, 7 s cap
Anchor-view recall
Window attention
SHOT 07
PASS 4
01 · Request

Three input granularities, one contract.

A unified prompt guides frozen Qwen3.8-27B to expand a one-sentence synopsis, paragraph, or numbered shot list into valid storyboard JSON: preserve stated facts, infer implied presence and outcomes, and complete open visual details.

Retains 100% of requested events and 90–98% of outcomes across 70 stories.
02 · Storyboard

Entity registry & state propagation.

The storyboard links a global entity registry (fixed IDs, appearance, default attire) with per-shot plans. Deterministic code propagates attire changes and held items across shots to compile one self-contained prompt per shot.

Code state propagation lifts post-change attire accuracy by +14.1 pts.
03 · Windows

Duration scaling & greedy packing.

Predicted shot durations are scaled to the target video length and greedily packed into multi-shot windows under a 7 s cap. Shots inside each window are denoised jointly to coordinate local camera cuts directly.

91.4% of entity returns in test storyboards cross a window boundary.
04 · Recall

Storyboard-indexed anchor views.

Rather than taking the latest shot (which chains errors across windows), code queries the storyboard for each entity's clearest anchor view after its latest state change, routing each recalled frame only to shots where that entity appears.

Anchor views beat recent-shot recall by +9.3 pts on Character and +15.4 pts at ≥2 windows apart.
05 · Render

Routed window attention.

Each shot is VAE-encoded separately so no latent straddles a cut. Under shot-wise cross-attention and reference-routed sparse self-attention, each shot reads its own tokens, co-generated first frames, and its routed anchor views.

Entity routing lifts cut execution from 0.869 to 0.970 when hiding absent-character frames.

Videos

Eight held-out test stories. Four paradigms, in sync.

Watch the exact test-set renders evaluated in Table 1 in a 2×2 grid, single-method focus, or interactive A/B wipe. Baselines render the film-annotated reference storyboard in their native prompt formats, whereas CaT-Movie renders from the shot-list request via our planner.

0.0 / 0.0 s
Shot 1/12

CaT-MBench

Consistency counts only when the planned cut is executed.

Feature-similarity metrics such as ViCLIP and DINOv2 reward videos that repeat the previous camera framing across a planned cut. CaT-MBench pairs entity-specific binary checks with an explicit cut gate: a returning character, object, or scene is credited as consistent only if TransNetV2 and the VLM judge verify that the planned cut opened a new camera setup.

Shot 3 Close-up
The woman looks at him and speaks.
Shot 4 Medium close-up
The man listens to the woman.
Shot 6 Medium close-up
The man speaks to the woman.
Joint generationMultiShotMaster
Memory-augmentedStoryMem
Cut gate offKitchen test story (11 shots). Toggle to apply the cut-execution gate.
Consistency without cut gateJoint 0.85<1.00 Memory
CaT-MBench (cut-gated)Joint 0.85>0.20 Memory
inconsistent (judged) new camera setup same camera setup
Benchmark scale · held-out films
122stories (30 dev + 92 test), 6–12 shots and up to 60 s each
1,012binary test checks (1,361 total) linked by registry ID
674of 740 test cuts require a new location, shot size, angle, or cast
Human calibration · 557 blind-annotated items
κ = 0.70Gemini 3.8 Flash judge vs. adjudicated human labels
κ = 0.85two human annotators vs. adjudicated ground truth
0.49–0.70ViCLIP frame-similarity AUC on the same human labels

Unedited cuts fail all cross-shot consistency and state-persistence checks across them. Human rankings and CaT-MBench agree on all 7 evaluated baselines.

Question bank

Entity-linked binary verification.

Each story request is compiled into binary checks anchored to shot time ranges.

Native input adaptation · one adapter per baseline
Cut-gated judging · Gemini 3.8 Flash (2 fps)
Finding 1 · Cross-Window Drift

Joint generators reset identity at window seams.

28% / 30% character pass rate across windows

HoloCine and MultiShotMaster pass 62–83% of character checks within a single window, but drop to 28% and 30% as soon as a character crosses a window boundary.

Finding 2 · Copied Framing

Raw frame memory copies prior camera setups.

114of 674 planned setup changes repeated by StoryMem

StoryMem fails the cut gate on 15.7% of consistency checks (vs. 0.7% for CaT-Movie). Omitting the cut gate inflates its consistency score by over 15.3 points.

Finding 3 · Per-Shot Amnesia

Independent per-shot pipelines cut cleanly but forget.

0.17–0.30character consistency across per-shot baselines

Per-shot Wan2.1/2.2 and keyframe pipelines (StoryDiffusion, IC-LoRA) execute over 90% of cuts, but independent sampling collapses cross-shot identity.

Quantitative results

Highest overall score, most consistent characters, and cuts on schedule.

Across 92 held-out test stories, CaT-Movie executes 0.974 of planned cuts while achieving the highest overall consistency (0.593), Character consistency (0.651), State persistence (0.785), and benchmark average (0.597). Click any column header to sort.

Table 1 · 92 held-out test stories. CaT-Movie renders from the shot-list request via our planner; baselines render the reference storyboard in their native formats.best · second

Character consistency by return time

<10 s10–20 s≥20 s

CaT-Movie holds 61–63% character consistency even when characters return more than 20 s later.

Consistency by window distance

CaT-Movierecent-shot recallno references

Anchor-view recall avoids cross-window error chaining, beating recent-shot recall by 15.4 points at ≥2 windows apart.

Ablation studies

30 development stories over three seeds. Toggle to view point differences from CaT-Movie († marks 95% bootstrap intervals excluding zero).

Ablation 1 · Renderer & Attention

Routing prevents framing copy.

Dropping references collapses Character (−29.2 pts) and Object (−24.3 pts). Removing entity routing drops cut execution from 0.970 to 0.869 on cuts where routing hides absent-character frames.

Ablation 2 · Reference Recall

Anchor views beat recent & CLIP recall.

Recalling from recent shots degrades Execution (−3.0 pts), Character (−9.3 pts), and Scene (−9.5 pts) due to error chaining across windows. CLIP retrieval drops the overall average by 3.4 pts.

Ablation 3 · Storyboard Source

Planner state tracking lifts rendering.

Our planner's storyboard beats directly compiled requests across every column (+5.4 Character, +16.0 Establish, +15.1 Persist) and surpasses the reference storyboard by +9.3 pts on Persist.

Qualitative comparison & planner analysis

Police Station (11 shots, 29 s). Hover or tap any frame to play its shot clip.

Information stated in compiled text

request aloneadded by planner

Across 70 stories (synopsis, paragraph, shot list), the planner expands stated opening states to 45%–73% and held items to 22%–68%.

Written versus executed state changes

stated & shownstated, not shownunstated

On 30 rendered development stories, our planner states 91% of required state changes and the renderer executes 69% of stated changes.

Film storyboard dataset

358K film shots annotated in the exact schema the planner writes.

We segment 19,535 films from CineDance, CinePile, and MovieChat into 75,461 scene blocks (358,302 shots, 332.8 hours) and annotate each block in a single Qwen3.7-Flash pass with entity registries, per-shot states, transitions, and pixel grounding.

Source videos19,535

CineDance, CinePile, MovieChat

Scene blocks75,461

up to 12 shots each, median 4

Total shots358,302

332.8 hours of narrative film

Usable shots290,267

shot-first filtering (+28.1% yield)

Train · held out66,573

train blocks · 916 held out by film

Dataset & benchmark composition

hover any ring segment

Upper half (rose): 288,083 labelled training shots. Lower half (blue): 1,361 CaT-MBench questions and 122 stories.

Source corpora and visual style

290,267 usable shots

Registered entities per scene block

0123+

58.3% of the 192,815 registered characters recur across two or more shots within their block.

Scene block length distribution

75,461 scene blocks

Shot-first vs. window-first curation yield

window-firstshot-first
Entity registryFixed IDs, appearance, and default attire for characters, objects, and places.
Per-shot statesLocation, camera setup, present entities, posture, held items, and attire.
State transitionsWithin-shot changes recording target entity, attribute, and visible end state.
Entity groundingRegistry-linked Qwen3.8-27B boxes, SAM 3.1 masks, and VGGT-Omega depth.
Three request tiersSynopsis, paragraph, and shot-list requests paired with every block.

Citation & Open Release

Cite CaT-Movie

Code is on GitHub, and weights, datasets, benchmark files, videos, and human labels are in the CaT-Movie Hugging Face Collection.

@misc{catmovie2026,
  title         = {CaT-Movie: Balancing Global Planning and Local Rendering for Long Multi-Shot Video Generation},
  author        = {Jinpeng Lu and Haoyuan Shi and Yuanshen Guan and Jiyuan Xia and Bin Wu and Yinda Chen and Zhiwei Xiong},
  year          = {2026},
  url           = {https://jinplu.github.io/CaT-Movie/}
}
Copied