Cinematic Behavior Transfer via NeRF-based Differentiable Filming

Given a film shot, a series of visual continuous frames, containing complex camera movement and character motion, we present an approach that estimates the camera trajectory and character motion in a coherent global space. The extracted camera and characters' behavior can be applied to new 2D/3D content through our cinematic transfer pipeline. The 2D cinematic transfer replicates character motion, camera movement, and background with new characters.The 3D cinematic transfer result is similar to 2D results, but have more freedom than 2D, such as changing the lighting and environment.