Deep-speare: A joint neural model of poetic language, meter and rhyme

In this paper, we propose a joint architecture that captures language, rhyme and meter for sonnet modelling. We assess the quality of generated poems using crowd and expert judgements. The stress and rhyme models perform very well, as generated poems are largely indistinguishable from human-written poems. Expert evaluation, however, reveals that a vanilla language model captures meter implicitly, and that machine-generated poems still underperform in terms of readability and emotion. Our research shows the importance expert evaluation for poetry generation, and that future research should look beyond rhyme/meter and focus on poetic language.

[1]  Stephen Adams,et al.  Poetic Designs: An Introduction to Meters, Verse Forms, and Figures of Speech , 1997 .

[2]  Kevin Knight,et al.  Unsupervised Discovery of Rhyme Schemes , 2011, ACL.

[3]  Luka Crnkovic-Friis,et al.  Generative Choreography using Deep Learning , 2016, ICCC.

[4]  Pablo Gervás,et al.  WASP: Evaluation of Different Strategies for the Automatic Generation of Spanish Verse , 2002 .

[5]  Kevin Knight,et al.  Automatic Analysis of Rhythmic Poetry with Applications to Generation and Translation , 2010, EMNLP.

[6]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[7]  Christopher D. Manning,et al.  Get To The Point: Summarization with Pointer-Generator Networks , 2017, ACL.

[8]  Bob L. Sturm,et al.  Music transcription modelling and composition using deep learning , 2016, ArXiv.

[9]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[10]  Hakan Inan,et al.  Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling , 2016, ICLR.

[11]  Graeme Hirst,et al.  GutenTag: an NLP-driven Tool for Digital Humanities Research in the Project Gutenberg Corpus , 2015, CLfL@NAACL-HLT.

[12]  Mirella Lapata,et al.  Chinese Poetry Generation with Recurrent Neural Networks , 2014, EMNLP.

[13]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[14]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[15]  Douwe Kiela,et al.  Automatically Generating Rhythmic Verse with Neural Networks , 2017, ACL.

[16]  Richard Socher,et al.  A Deep Reinforced Model for Abstractive Summarization , 2017, ICLR.

[17]  Ming Zhou,et al.  Selective Encoding for Abstractive Sentence Summarization , 2017, ACL.

[18]  Wiebke Wagner,et al.  Steven Bird, Ewan Klein and Edward Loper: Natural Language Processing with Python, Analyzing Text with the Natural Language Toolkit , 2010, Lang. Resour. Evaluation.

[19]  Dong Wang,et al.  Chinese Song Iambics Generation with Neural Attention-Based Model , 2016, IJCAI.

[20]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[21]  Simon Colton,et al.  Full-FACE Poetry Generation , 2012, ICCC.

[22]  Yoav Goldberg,et al.  Gaiku : Generating Haiku with Word Associations Norms , 2009 .

[23]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[24]  Yann LeCun,et al.  Feature learning and deep architectures: new directions for music informatics , 2013, Journal of Intelligent Information Systems.

[25]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[26]  Mark B. Sandler,et al.  Text-based LSTM networks for Automatic Music Composition , 2016, ArXiv.

[27]  Yejin Choi,et al.  Generating Topical Poetry , 2016, EMNLP.

[28]  Hannu Toivonen,et al.  Harnessing Constraint Programming for Poetry Composition , 2013, ICCC.

[29]  Sebastian Risi,et al.  Creative Generation of 3D Objects with Deep Learning and Innovation Engines , 2016, ICCC.

[30]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[31]  Ryohei Nakatsu,et al.  New Hitch Haiku: An Interactive Renku Poem Composition Supporting Tool Applied for Sightseeing Navigation System , 2009, ICEC.

[32]  Lior Wolf,et al.  Using the Output Embedding to Improve Language Models , 2016, EACL.