Hierarchical attention based long short-term memory for Chinese lyric generation

Automating the process of lyric generation should face the challenge of being meaningful and semantically related to a scenario. Traditional keyword or template based lyric generation systems always ignore the patterns and styles of lyricists, which suffer from improper lyric construction and maintenance. A Chinese lyric generation system is proposed to learn patterns and styles of certain lyricists and generate lyrics automatically. A long short-term memory network is utilized to process each lyric line and generate the next line word by word. A hierarchical attention model is designed to capture the contextual information at both sentence and document level, which could learn high level representations of each lyric line and the entire document. Furthermore, the LSTM decoder decodes all the semantic contextual information into lyric lines word by word. The results of the automatically generated lyrics show that the proposed method can correctly capture the patterns and styles of a certain lyricist and fit into certain scenarios, which also outperforms state-of-the-art models.