DongCi DaCi

An automatic lyric animation generation system

Introduction

DongCi DaCi [1] is an automatic lyric animation generation system that allows users to generate lyric animations within few clicks, and an unique editing tool for users to customize their animations. It aims to improve creativity and experiences of creating lyric animations through tool innovation, and to lower the learning threshold of lyric animation production.

The system is divided into two components, mood board and storyboard, which control the overall visualization style and the flow of animations respectively. We use Muisc Information Retrieval (MIR) technologies to extract relevant information of songs as the input of these two compononets, including emotional features such as valence, loudness and acousticness and basic information such as beats, tempos and sections.

Inside the mood board, we handcrafted mapping rules from emotional features to the visual components of animations, such as fonts, hues, geometry shapes, etc. On the other hand, the storyboard controls when a word should appear according to lyric-music alignment information and when a change between sections should happen (also based on input music’s sections, such as verse and chorus).

Figure 1: Product Conceptual Model of DongCi DaCi.

At the end of the day, users can choose between creating their own lyric animations with default settings or fine-tune the parameters used to generate the animations to their satisfication.

[1] DongCi DaCi is the Pinyin of Chinese word "动词打次", which imitates the sound of people playing the drum sets. The product got its name beacause we believe the groove and the rhythm of music are big parts of creating lyric animations.

Usage

Work Flow

The work flow of DongCi DaCi is in following steps:

  1. Type in the name of the song for which you are creating animations.
  2. Choose the mood board from the templates generated by the system.
  3. Adjust the borders between sections, and mark the important sections (usually that would be choruses).
  4. Adjust the typography of each section.
  5. Export your animation.

(Users can simply choose a random template in step(2), and leave the step(3) and step(4) in default settings to make the generation automatic.)

The work flow of DongCi DaCi.
Demo of editing the typography.

Demo

Demo of multiple lyric animations created with DongCi DaCi.

Acknowledgments

This work was done as a group while I was doing my Student Research Training Project in Zhejiang University in 2020.

Other group members are Haichuan Zhang, Hongguang Xiao, and the project is lead by Yitong Chen.