DopeLearning: A Computational Approach to Rap Lyrics Generation

Writing rap lyrics requires both creativity to construct a meaningful, interesting story and lyrical skills to produce complex rhyme patterns, which form the cornerstone of good flow. We present a rap lyrics generation method that captures both of these aspects. First, we develop a prediction model to identify the next line of existing lyrics from a set of candidate next lines. This model is based on two machine-learning techniques: the RankSVM algorithm and a deep neural network model with a novel structure. Results show that the prediction model can identify the true next line among 299 randomly selected lines with an accuracy of 17%, i.e., over 50 times more likely than by random. Second, we employ the prediction model to combine lines from existing songs, producing lyrics with rhyme and a meaning. An evaluation of the produced lyrics shows that in terms of quantitative rhyme density, the method outperforms the best human rappers by 21%. The rap lyrics generator has been deployed as an online tool called DeepBeat, and the performance of the tool has been assessed by analyzing its usage logs. This analysis shows that machine-learned rankings correlate with user preferences.

[1]  Geoffrey E. Hinton,et al.  Generating Text with Recurrent Neural Networks , 2011, ICML.

[2]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[3]  Pyry Takala Word Embeddings for Morphologically Rich Languages , 2016, ESANN.

[4]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[5]  Matthew D. Zeiler ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[6]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[7]  Quoc V. Le,et al.  A Neural Conversational Model , 2015, ArXiv.

[8]  Simon Colton,et al.  Computational Creativity: The Final Frontier? , 2012, ECAI.

[9]  Razvan Pascanu,et al.  Theano: A CPU and GPU Math Compiler in Python , 2010, SciPy.

[10]  Tony Veale,et al.  Creative Language Retrieval: A Robust Hybrid of Information Retrieval and Linguistic Creativity , 2011, ACL.

[11]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[12]  M. Boden The creative mind : myths & mechanisms , 1991 .

[13]  Dekai Wu,et al.  Learning to Rap Battle with Bilingual Recursive Neural Networks , 2015, IJCAI.

[14]  A. Bradley Book of Rhymes: The Poetics of Hip Hop , 2009 .

[15]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[16]  Daniel G. Brown,et al.  Automatic Detection of Internal and Imperfect Rhymes in Rap Lyrics , 2009, ISMIR.

[17]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[18]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[19]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[20]  Eric Brill,et al.  Beyond PageRank: machine learning for static ranking , 2006, WWW '06.

[21]  Lei Yu,et al.  Deep Learning for Answer Sentence Selection , 2014, ArXiv.

[22]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[23]  Daniel G. Brown,et al.  Using Automated Rhyme Detection to Characterize Rhyming Style in Rap Music , 2010 .

[24]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[25]  Hannu Toivonen,et al.  Corpus-Based Generation of Content and Form in Poetry , 2012, ICCC.

[26]  Dekai Wu,et al.  Learning to Freestyle: Hip Hop Challenge-Response Induction via Transduction Rule Segmentation , 2013, EMNLP.

[27]  Li Deng,et al.  An Overview of Deep-Structured Learning for Information Processing , 2011 .