GitHub radar
Wan-Dancer: AI Video Model for Music-to-Dance
Wan-Dancer is an open-source 14 billion parameter model developed by the Wan-AI team, capable of producing 1+ min long high resolution (720p/30fps) videos from a music track and an image prompt. This is the first open model that can generate videos over 20 seconds using video diffusion.
Wan-Dancer is a 14B diffusion model released by the Wan-AI team, who has also developed the Wan2.1 open video generation series related to the HumanAIGC research of Alibaba. By taking a music track and an image of a person as input, it outputs videos longer than 1 minute with 5 types of dance styles (Chinese traditional, K-POP, street, tap, latin). The core idea is to decouple the global keyframe generation and local temporal detail generation through a hierarchical framework, which addresses the problem of identity loss and movement repetition in the prior arts, thus enabling more than 20 seconds of video generation.
Why a vibe-coder should care
This model is very practical for those who are interested in trying out AI generated videos. You can use the weight on Hugging Face and the source code is open source. However, you would need pretty powerful hardware to run the experiments, since we used 8x A800 80G GPU. So if you don’t have access to such computing resources, you can still use the cloud service on ModelScope.
▌ More finds