The Music Paint Machine: Stimulating Self-monitoring Through the Generation of Creative Visual Output Using a Technology-enhanced Learning Tool

Abstract In this paper, we discuss the pedagogically grounded and research-based design of a technology-enhanced learning tool, the Music Paint Machine. This interactive music system introduces a musical experience in which the musician creates a digital painting by playing an acoustic musical instrument and by moving the body on a coloured pressure mat. As a learning tool it aims at the development of musical creativity, at the stimulation of embodied understanding of music and at the development of an intimate relationship with the musical instrument. First, the methodological approach is outlined and pedagogical and theoretical backgrounds are discussed. Then, we report on an experiment in which 51 amateur musicians participated. The experiment aimed at probing the application's potential to induce a flow experience and to learn about how participants evaluate the didactic relevance of the Music Paint Machine. Results suggest that the Music Paint Machine has the potential to evoke a flow experience. Furthermore participants acknowledged its didactic relevance with regard to learning to improvise, to developing understanding of musical parameters and to stimulating creativity.

[1]  J. Woollard.  Psychology for the classroom , 2013 .

[2]  David H. Jonassen,et al.  Constructivism and the Technology of Instruction : A Conversation , 2013 .

[3]  Mehmet Buldu.  Making learning visible in kindergarten classrooms: Pedagogical documentation as a formative assessment technique , 2010 .

[4]  Marc Leman,et al.  User-oriented studies in embodied music cognition research , 2010 .

[5]  J. Klein.  International handbook of research in arts education , 2010 .

[6]  Pao-Ta Yu,et al.  Using a Multimodal Learning System to Support Music Instruction , 2010, J. Educ. Technol. Soc..

[7]  John Woollard,et al.  Psychology for the Classroom: Constructivism and Social Learning , 2010 .

[8]  Isabella Poggi,et al.  Gestures in performance , 2009 .

[9]  G. Welch.  Ecological Validity and Impact: Key Challenges for Music Education Research , 2009 .

[10]  Beatriz Ilari,et al.  Music in the early years: research, theory and practice , 2009 .

[11]  Liselott Mariett Olsson,et al.  Movement and Experimentation in Young Children's Learning: Deleuze and Guattari in Early Childhood Education , 2009 .

[12]  P. Janata,et al.  Embodied music cognition and mediation technology , 2009 .

[13]  Sibylle C. Herholz,et al.  Cortical Plasticity Induced by Short-Term Unimodal and Multimodal Musical Training , 2008, The Journal of Neuroscience.

[14]  Rinie Geenen,et al.  Development and Validation of a Dutch Translation of the Big Five Inventory (BFI) , 2008, Journal of personality assessment.

[15]  Petri Toiviainen,et al.  :Musical creativity: Multidisciplinary research in theory and practice , 2008 .

[16]  Tillman Weyde,et al.  3d augmented mirror: a multimodal interface for string instrument learning and teaching with gesture support , 2007, ICMI '07.

[17]  Margaret MacDonald,et al.  Toward formative assessment: The use of pedagogical documentation in early elementary classrooms , 2007 .

[18]  Peter Desain,et al.  Development of real-time visual feedback assistance in singing training: a review , 2006, J. Comput. Assist. Learn..

[19]  Niels Birbaumer,et al.  From music perception to creative performance: Mapping cerebral differences between professional and amateur musicians , 2006 .

[20]  Scott D. Lipscomb,et al.  How different is good? How good is different? The assessment of children's creative musical thinking , 2006 .

[21]  Joelle L. Lien.  Praxial Music Education: Reflections and Dialogues , 2005 .

[22]  Densil Cabrera,et al.  Seeing sound: real-time sound visualisation in visual feedback loops used for training musicians , 2005, Ninth International Conference on Information Visualisation (IV'05).

[23]  L. Custodero.  Observable indicators of flow experience: a developmental perspective on musical engagement in young children from infancy to school age , 2005 .

[24]  David M. Howard,et al.  Real-time feedback in the singing studio: an innovatory action-research project using new voice technology , 2005 .

[25]  Jude Brereton,et al.  WinSingad: A real-time display for the singing studio* , 2004, Logopedics, phoniatrics, vocology.

[26]  R. Eklund,et al.  The Flow Scales Manual , 2004 .

[27]  Ryan Daniel,et al.  Peer assessment in musical performance: the development, trial and evaluation of a methodology for the Australian tertiary environment , 2004, British Journal of Music Education.

[28]  R. Parncutt,et al.  The Science and Psychology of Music Performance Creative Strategies for Teaching and Learning , 2002 .

[29]  R. Woody.  Learning From the Experts: Applying Research in Expert Performance To Music Education , 2001 .

[30]  Rolf T. Wigand,et al.  Optimal experience of Web activities , 1999 .

[31]  C. Shea,et al.  Enhancing motor learning through external-focus instructions and feedback , 1999 .

[32]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[33]  C. Shea,et al.  Frequent feedback enhances complex motor skill learning. , 1998, Journal of motor behavior.

[34]  C Palmer,et al.  Monitoring and planning capacities in the acquisition of music performance skills. , 1997, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[35]  Gabriele Wulf,et al.  Continuous Concurrent Feedback Degrades Skill Learning: Implications for Training and Simulation , 1997, Hum. Factors.

[36]  Jan O. Borchers.  WorldBeat: designing a baton-based interface for an interactive music exhibit , 1997, CHI.

[37]  Delores R. Gauthier,et al.  Music Matters: A New Philosophy of Music Education , 1996 .

[38]  Thomas W. Goolsby.  Portfolio Assessment for Better Evaluation , 1995 .

[39]  Richard J. Colwell.  The New Handbook of Research on Music Teaching and Learning: A Project of the Music Educators National Conference , 1992 .

[40]  Richard J. Colwell,et al.  Handbook of Research on Music Teaching and Learning , 1992 .

[41]  R. Snider.  Brain Mechanisms , 1965, Neurology.

[42]  E. Jaques-Dalcroze.  Rhythm, Music and Education , 1921 .

[43]  G. Riva,et al.  From intention to action: The role of presence , 2011 .

[44]  Marc Leman,et al.  The Music Paint Machine - A Multimodal Interactive Platform to Stimulate Musical Creativity in Instrumental Practice , 2010, CSEDU.

[45]  M. Leman,et al.  Musical gestures : sound, movement, and meaning , 2010 .

[46]  David Johnson.  Music Coach : Real-time Evaluation of Music Performance using Nokia N 900 , 2009 .

[47]  Marc Leman,et al.  The musical instrument as a natural extension of the musician , 2009 .

[48]  Thomas A. Regelski,et al.  Music education for changing times : guiding visions for practice , 2009 .

[49]  G. Riva.  Presence and Social Presence: From Agency to Self and Others , 2008 .

[50]  Giuseppe Riva,et al.  Enacting interactivity: The role of presence , 2008 .

[51]  Elaine Chew,et al.  A Musical Improvisation System That Provides Visual Feedback to the Performer , 2007 .

[52]  Dominique Fober,et al.  VEMUS - Feedback and Groupware Technologies for Music Instrument Learning , 2007 .

[53]  Marcelo M. Wanderley,et al.  Visualization of Bowing Gestures for Feedback : The Hodgson Plot , 2007 .

[54]  Kimberly Powell,et al.  The Body in a State of Music , 2007 .

[55]  Alexandra Pierce,et al.  Deepening Musical Performance through Movement: The Theory and Practice of Embodied Interpretation , 2007 .

[56]  P. Webster.  Computer-based Technology and Music Teaching and Learning: 2000–2005 , 2007 .

[57]  L. Bresler.  International handbook of research in arts education , 2007 .

[58]  François Pachet,et al.  Young children's musical experiences with a flow machine , 2006 .

[59]  T. Anguera,et al.  From Communication to Presence: Cognition, emotions and culture towards the ultimate communicative experience , 2006 .

[60]  Gerhard Widmer,et al.  The "Air Worm": an Interface for Real-Time manipulation of Expressive Music Performance , 2005, ICMC.

[61]  G. Riva.  2 The Psychology of Ambient Intelligence : Activity , Situation and Presence , 2004 .

[62]  Kjetil Falkenberg Hansen,et al.  IMUTUS : an interactive system for learning to play a musical instrument , 2004 .

[63]  Gary W. Hill,et al.  A real-time responsive/interactive system for musical conducting using motion capture technology , 2004 .

[64]  Thomas P. Rohrer.  The science and psychology of music performance: creative strategies for teaching and learning , 2004 .

[65]  K. Stanford.  Experiments with a musical machine: musical style replication in 3–5-year-old children , 2004 .

[66]  Nathan Bos,et al.  WHAT DO GAME DESIGNERS KNOW ABOUT SCAFFOLDING ? BORROWING SIMCITY DESIGN PRINCIPLES FOR EDUCATION , 2001 .

[67]  Holger Regenbrecht,et al.  Embodied Presence in Virtual Environments , 1999, Visual Representations and Interpretations.

[68]  Gunilla Dahlberg,et al.  Beyond quality in early childhood education and care : postmodern perspectives , 1999 .

[69]  Miller Puckette,et al.  Real-time audio analysis tools for Pd and MSP , 1998, ICMC.

[70]  J. Russell.  Autism as an executive disorder , 1997 .

[71]  K. A. Ericsson,et al.  Expert and exceptional performance: evidence of maximal adaptation to task constraints. , 1996, Annual review of psychology.

[72]  David N. Perkins,et al.  Technology Meets Constructivism: Do They Make a Marriage? , 1991 .

[73]  James Mainwaring,et al.  Psychology in the classroom , 1956 .