AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies

Li Siyao, Yuhang Li, Bo Li, Chao Dong, Ziwei Liu, Chen Change Loy

Existing correspondence datasets for two-dimensional (2D) cartoon suffer from simple frame composition and monotonic movements, making them insufficient to simulate real animations. In this work, we present a new 2D animation visual correspondence dataset, AnimeRun, by converting open source three-dimensional(3D) movies to full scenes in 2D style, including simultaneous moving background and interactions of multiple subjects. Our analyses show that the proposed dataset not only resembles real anime more in image composition, but also possesses richer and more complex motion patterns compared to existing datasets. With this dataset, we establish a comprehensive benchmark by evaluating several existing optical flow and segment matching methods, and analyze shortcomings of these methods on animation data. In conclusion, the contributions of our work are three-fold: (1) we make the first full-scene animation correspondence dataset by converting open source 3D movies into 2D style. (2) We provide fine grained correspondence labels in both pixel-wise and region-wise levels. (3) We establish the first benchmark to correspondence in 2D animation.