- Published on
MagicAnimate is a framework for generating new animation data given a reference image and a motion sequence.
The authors use a conditional diffusion model for generation, with an additional encoder used to preserve the identity of the reference image in the output.
A simple but effective method of producing smooth transitions between video frames is also introduced, which is necessary for producing animations of reasonable length.
Comparisons with other state-of-the-art methods on two benchmark datasets show that MagicAnimate produces more temporally consistent animations, while also better preserving the appearance of the reference image.
The method performs well on both short and long animations, and when animating reference images with different identities to the motion sequence, showing the robustness and versatility of the approach.