Algorithmic approach to sonification of classical Chinese poetry

The classical Chinese poetry is a remarkable form of art in traditional Chinese character. However, it is difficult for people who are unfamiliar with ancient Chinese to experience the artistic content of the poetry. In this study, a sonification scheme, Tx2Ms (Text-to-Music), is proposed to extract the poetry features between lines in verses; moreover, dynamics and interval relations are modeled to map those features to the movement of multi-dimensional musical elements such as durations. This conversion is based on poetry intonation and acoustic analysis of the pronunciations of poems; and a stochastic compositional algorithm is created by applying a Markov chain. In addition, the best pentatonic mode for a specific poem is recommended according to the formants analysis. Therefore, the sonification of classical Chinese poetry not only provides a novel way for people to appreciate Chinese poetry but also enriches the state of mind and imagery in the delivery process, and the experiment results show that the proposed system is successfully accepted by most people.

[1]  Eduardo Miranda,et al.  Making Music with Algorithms: A Case-Study System , 1999, Computer Music Journal.

[2]  D. Franz.  Markov Chains as Tools for Jazz Improvisation Analysis , 1998 .

[3]  Myounghoon Jeon,et al.  Enhanced auditory menu cues improve dual task performance and are preferred with in-vehicle technologies , 2009, AutomotiveUI.

[4]  Hans G. Kaper,et al.  Data sonification and sound visualization , 1999, Comput. Sci. Eng..

[5]  D. Cope.  Virtual Music: Computer Synthesis of Musical Style , 2001 .

[6]  Terri L. Bonebright,et al.  Sonific ation Report: Status of the Field and Research Agenda , 2010 .

[7]  Man-Kwan Shan,et al.  Algorithmic compositions based on discovered musical patterns , 2009, Multimedia Tools and Applications.

[8]  Pietro Polotti,et al.  Elementary Gestalts for Gesture Sonification , 2008, NIME.

[9]  Ian Oakley,et al.  Urban Sound Gardens: Supporting Overlapping Audio Landmarks in Exploratory Environments , 2010 .

[10]  Ulrich Schimmack,et al.  Mixed affective responses to music with conflicting cues , 2008 .

[11]  Joel Chadabe,et al.  Electric Sound: The Past and Promise of Electronic Music , 1996 .

[12]  F. Lerdahl.  The Sounds of Poetry Viewed as Music , 2001, Annals of the New York Academy of Sciences.

[13]  Reginald Bain.  The AIMS Project: Creative Experiments in Musical Sonification , 2006, ICMC.

[14]  Composition Jon McCormack,et al.  Grammar Based Music , 1996 .

[15]  Paul Doornbusch.  A Brief Survey of Mapping in Algorithmic Composition , 2002, ICMC.

[16]  Joseph Rothstein,et al.  MIDI: A Comprehensive Introduction , 1992 .

[17]  Bradley J. Clement.  Learning Harmonic Progression Using Markov Models , 1998 .

[18]  Johan Fagerlönn,et al.  AWESOME SOUND DESIGN TOOL: A WEB BASED UTILITY THAT INVITES END USERS INTO THE AUDIO DESIGN PROCESS , 2009 .

[19]  John Rahn,et al.  Basic Atonal Theory , 1980 .

[20]  Xiaoying Wu,et al.  A study of image-based music composition , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[21]  Bruce N. Walker,et al.  Spearcon Performance and Preference for Auditory Menus on a Mobile Phone , 2009, HCI.

[22]  George Frederick McKay.  Experimental Music , 1959 .

[23]  Donald W. Mitchell.  Gustav Mahler: Songs and Symphonies of Life and Death. Interpretations and Annotations , 2008 .

[24]  J. Mccormack.  Grammar-based music composition , 1996 .

[25]  John G. Neuhoff,et al.  Sonification Report: Status of the Field and Research Agenda Prepared for the National Science Foundation by members of the International Community for Auditory Display , 1999 .

[26]  G. Schlaug,et al.  Brain structures differ between musicians and non-musician , 2001, NeuroImage.

[27]  Stephen Barrass,et al.  Using sonification , 1999, Multimedia Systems.

[28]  Moonis Ali,et al.  Innovations in Applied Artificial Intelligence , 2005 .

[29]  Richard Kronland-Martinet,et al.  Analysis/synthesis and spatialization of noisy environmental sounds , 2009 .

[30]  Eamonn O'Neill,et al.  Auditory icon and earcon mobile service notifications: intuitiveness, learnability, memorability and preference , 2009, CHI.

[31]  Bruce N. Walker,et al.  SONIFICATION SANDBOX: A GRAPHICAL TOOLKIT FOR AUDITORY GRAPHS , 2003 .

[32]  Phil Winsor.  Automated Music Composition , 1989 .

[33]  Peter S. Langston.  Six Techniques for Algorithmic Music Composition (Extend Abstract) , 1989, ICMC.

[34]  Georgios Kouroupetroglou,et al.  Acoustic Rendering of Data Tables Using Earcons and Prosody for Document Accessibility , 2009, HCI.

[35]  David Cope,et al.  Computer Modeling of Musical Intelligence in EMI , 1992 .

[36]  G. May.  STAT 992: Topics in Statistics Extracting Patterns in Music for Composition via Markov Chains , 2012 .

[37]  Eli Tilevich,et al.  Sonification design guidelines to enhance program comprehension , 2009, 2009 IEEE 17th International Conference on Program Comprehension.

[38]  Stephen A. Brewster,et al.  An evaluation of earcons for use in auditory human-computer interfaces , 1993, INTERCHI.

[39]  Sigurd Saue,et al.  A model for interaction in exploratory sonification displays , 2000 .

[40]  Gregory Kramer,et al.  Auditory Display: Sonification, Audification, And Auditory Interfaces , 1994 .

[41]  Bernd Schöner,et al.  Analysis and Synthesis of Palestrina-Style Counterpoint Using Markov Chains , 2001, ICMC.

[42]  Gregory Kramer,et al.  An introduction to auditory display , 1994 .