Automatically Generating Rhythmic Verse with Neural Networks

We propose two novel methodologies for the automatic generation of rhythmic poetry in a variety of forms. The first approach uses a neural language model trained on a phonetic encoding to learn an implicit representation of both the form and content of English poetry. This model can effectively learn common poetic devices such as rhyme, rhythm and alliteration. The second approach considers poetry generation as a constraint satisfaction problem where a generative neural language model is tasked with learning a representation of content, and a discriminative weighted finite state machine constrains it on the basis of form. By manipulating the constraints of the latter model, we can generate coherent poetry with arbitrary forms and themes. A large-scale extrinsic evaluation demonstrated that participants consider machine-generated poems to be written by humans 54% of the time. In addition, participants rated a machine-generated poem to be the best amongst all evaluated.

[1]  Yoshua Bengio,et al.  Neural Probabilistic Language Models , 2006 .

[2]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[3]  Aaron E. Rosenberg,et al.  Performance tradeoffs in dynamic time warping algorithms for isolated word recognition , 1980 .

[4]  Gerald Penn,et al.  An Exact A* Method for Deciphering Letter-Substitution Ciphers , 2010, ACL.

[5]  Pablo Gervás,et al.  WASP: Evaluation of Different Strategies for the Automatic Generation of Spanish Verse , 2002 .

[6]  Kevin Knight,et al.  Automatic Analysis of Rhythmic Poetry with Applications to Generation and Translation , 2010, EMNLP.

[7]  J. Chall,et al.  A FORMULA FOR PREDICTING READABILITY , 1948 .

[8]  Hannu Toivonen,et al.  Harnessing Constraint Programming for Poetry Composition , 2013, ICCC.

[9]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[10]  François Pachet,et al.  Markov Constraints for Generating Lyrics with Style , 2012, ECAI.

[11]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[12]  Marc Dymetman,et al.  Natural Language Generation through Character-based RNNs with Finite-state Prior Knowledge , 2016, COLING.

[13]  Christopher D. Manning,et al.  Better Word Representations with Recursive Neural Networks for Morphology , 2013, CoNLL.

[14]  Jean-Luc Gauvain,et al.  Training Neural Network Language Models on Very Large Corpora , 2005, HLT.

[15]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[16]  Matthew Purver,et al.  Process Based Evaluation of Computer Generated Poetry , 2016, CC-NLG.

[17]  Hideaki Aoyama,et al.  Word Length Frequency and Distribution in English: Observations, Theory and Implications for the Construction of Verse Lines , 1998, ArXiv.

[18]  Bipin Indurkhya,et al.  Poetry generation system with an emotional personality , 2014, ICCC.

[19]  Mirella Lapata,et al.  Chinese Poetry Generation with Recurrent Neural Networks , 2014, EMNLP.

[20]  Kevin Knight,et al.  Learning Phoneme Mappings for Transliteration without Parallel Data , 2009, HLT-NAACL.

[21]  Grzegorz Kondrak,et al.  N-Gram Similarity and Distance , 2005, SPIRE.

[22]  Geoffrey E. Hinton,et al.  Generating Text with Recurrent Neural Networks , 2011, ICML.

[23]  Kevin Knight,et al.  Unsupervised Analysis for Decipherment Problems , 2006, ACL.

[24]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[25]  Henry S. Thompson,et al.  Towards A Computational Model Of Poetry Generation , 2000 .

[26]  Alan W. Black,et al.  Issues in building general letter to sound rules , 1998, SSW.

[27]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[28]  Simon Colton,et al.  Full-FACE Poetry Generation , 2012, ICCC.

[29]  Yoav Goldberg,et al.  Gaiku : Generating Haiku with Word Associations Norms , 2009 .

[30]  Alexander M. Rush,et al.  Character-Aware Neural Language Models , 2015, AAAI.

[31]  Yejin Choi,et al.  Generating Topical Poetry , 2016, EMNLP.

[32]  Maosong Sun,et al.  Generating Chinese Classical Poems with RNN Encoder-Decoder , 2016, CCL.