DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes

This paper focuses on 360-degree panoramic video generation, a crucial part of spatial intelligence for immersive dynamic scenes. Challenges in this area include difficulties in collecting complete 360-degree panorama video data, limitations in existing methods such as the generalization ability of models trained on small datasets (e.g., 360-degree DVDs) and the limited motion range in inversion strategies (like 4K4DGen), as well as the overlooked issue of achieving continuous and loopable scene-level dynamic effects. To overcome these, we propose DynamicScaler, a novel framework capable of generating high-resolution dynamic effects in infinite spatial dimensions and creating 360-degree dynamic panoramas. It supports text-conditioned and text-image-conditioned generation and can produce theoretically infinite-length or loopable motion effects without the need for training data. By integrating concepts from prior works, it offers a robust, training-free, and data-free solution that addresses data limitations and quality constraints in dynamic effect generation. Our exploration led to unexpected and significant breakthroughs. Notably, our framework can directly generate 360-degree panoramas from text, eliminating the need for large datasets of field-of-view video panoramas, which is a major advancement considering the challenges in obtaining such data and the quality loss in field-of-view transitions. Additionally, the shift mechanism allows for the generation of near-infinite or loopable scene-level dynamic effects, enhancing the immersiveness in AR/VR environments. Overall, our framework sets a better stage for 360-degree panorama generation, showing great potential in applications such as 3D gaming and film design for creating more immersive 4D spatial experiences.