Virtual Dynamic Storyboard runs on a “propose-simulate-discriminate” mode: given a formatted story script and a camera script as input, it generates several character animation and camera movement proposals following predefined story and cinematic rules⚡️.

A screenwriter types in a script and selects their story’s fixed and variable attributes. ScriptViz retrieves sequences for the input script. The screenwriter iterates on the script based on the proposed sequences. The visualization helps to enrich the details of the existing dialogue and write the unfinished dialogue.

Given a film shot, a series of visual continuous frames, containing complex camera movement and character motion, we present an approach that estimates the camera trajectory and character motion in a coherent global space. The extracted camera and characters' behavior can be applied to new 2D/3D content through our cinematic transfer pipeline. The 2D cinematic transfer replicates character motion, camera movement, and background with new characters.The 3D cinematic transfer result is similar to 2D results, but have more freedom than 2D, such as changing the lighting and environment.

The ability to choose an appropriate camera view among multiple cameras plays a vital role in TV shows delivery. But it is hard to figure out the statistical pattern and apply intelligent processing due to the lack of high-quality training data. To solve this issue, we first collect a novel benchmark on this setting with four diverse scenarios including concerts, sports games, gala shows, and contests, where each scenario contains 6 synchronized tracks recorded by different cameras. It contains 88-hour raw videos that contribute to the 14-hour edited videos. Based on this benchmark, we further propose a new approach temporal and contextual transformer that utilizes clues from historical shots and other views to make shot transition decisions and predict which view to be used. Extensive experiments show that our method outperforms existing methods on the proposed multi-camera editing benchmark.