Musicking with an interactive musical system: The effects of task motivation and user interface mode on non-musicians' creative engagement

Abstract Creative engagement with novel musical interfaces can be rewarding for non-musicians. However, designing novel musical interfaces for non-musicians can be challenging because they lack conceptual and technical musical skills. In this paper we explore the effects of task motivation (experiential goal vs utilitarian goal) and user interface mode (whether the content is editable, and whether content can be replayed), on non-musicians’ creative engagement with novel musical interfaces. We show through an empirical study of twenty-four participants that an experiential exploratory goal encourages users’ creative engagement compared to a utilitarian creative goal. We found that being able to replay records is less important when participants have an experiential exploratory goal than when they have a utilitarian creative goal. Results also indicate that allowing people to replay their musical ideas increased some aspects of their creative engagement which was further increased when they were able to edit their creations. We also found that creative engagement increased when the interface supported users in planning ahead. A descriptive model of non-musician’s creative engagement with musical interfaces is proposed including three modes of musicking. An optimal trajectory of creative engagement through these modes is suggested and a description of inferred motivations, output, status and activities during creative processes is discussed. Design implications are proposed for supporting novices’ creative engagement taking into consideration their motivation and skills, and supporting insight and real-time activity.

[1]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[2]  Alexander Refsum Jensenius,et al.  To gesture or Not? An Analysis of Terminology in NIME Proceedings 2001-2013 , 2014, NIME.

[3]  Anna Xambó Embodied Music Interaction: Creative Design Synergies Between Music Performance and HCI , 2017 .

[4]  Sergi Jordà,et al.  The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces , 2007, TEI.

[5]  Todd Lubart,et al.  How can computers be partners in the creative process: Classification and commentary on the Special Issue , 2005, Int. J. Hum. Comput. Stud..

[6]  Andrew M. Webb,et al.  Using Metrics of Curation to Evaluate Information-Based Ideation , 2014, ACM Trans. Comput. Hum. Interact..

[7]  Mounia Lalmas,et al.  Models of user engagement , 2012, UMAP.

[8]  Nathalie Bonnardel,et al.  Creativity in design activities: the role of analogies in a constrained cognitive environment , 1999, Creativity & Cognition.

[9]  R. Sawyer Explaining Creativity: The Science of Human Innovation , 2006 .

[10]  Heather L. O'Brien,et al.  The influence of hedonic and utilitarian motivations on user engagement: The case of online shopping experiences , 2010, Interact. Comput..

[11]  Holger Winnemöller,et al.  Painting with Bob: assisted creativity for novices , 2014, UIST.

[12]  J. Fereday,et al.  Demonstrating Rigor Using Thematic Analysis: A Hybrid Approach of Inductive and Deductive Coding and Theme Development , 2006 .

[13]  Golan Levin Painterly interfaces for audiovisual performance , 2000 .

[14]  Gideon D'Arcangelo,et al.  Creating Contexts of Creativity: Musical Composition with Modular Components , 2001, NIME.

[15]  Teresa M. Amabile,et al.  Motivation and creativity. , 1998 .

[16]  Yongmeng Wu,et al.  Open Symphony: Creative Participation for Audiences of Live Music Performances , 2017, IEEE Multim..

[17]  Berry Eggen,et al.  Musico: Personal Playlists through Peripheral and Implicit Interaction , 2017, Conference on Designing Interactive Systems.

[18]  Ali Momeni,et al.  Facilitating collective musical creativity , 2005, ACM Multimedia.

[19]  Celine Latulipe,et al.  Creativity factor evaluation: towards a standardized survey metric for creativity support , 2009, C&C '09.

[20]  Christina E. Shalley,et al.  Effects of Productivity Goals, Creativity Goals, and Personal Discretion on Individual Creativity , 1991 .

[21]  Lauralee Alben,et al.  Quality of experience: defining the criteria for effective interaction design , 1996, INTR.

[22]  Ernest A. Edmonds Art, Interaction and Engagement , 2011, 2011 15th International Conference on Information Visualisation.

[23]  Linda Candy,et al.  Designing for creative engagement , 2008 .

[24]  Steve Larson,et al.  COMPOSITION VERSUS IMPROVISATION , 2005 .

[25]  Daniel Ullrich,et al.  To do or not to do: Differences in user experience and retrospective judgments depending on the presence or absence of instrumental goals , 2007, Interact. Comput..

[26]  Jason Freeman,et al.  Experience and Ownership with a Tangible Computational Music Installation for Informal Learning , 2017, TEI.

[27]  Elaine Toms,et al.  What is user engagement? A conceptual framework for defining user engagement with technology , 2008, J. Assoc. Inf. Sci. Technol..

[28]  Mark Blythe,et al.  Funology:From Usability to Enjoyment , 2003 .

[29]  Simon Holland,et al.  Music and Human-Computer Interaction , 2013, Springer Series on Cultural Computing.

[30]  Ben Shneiderman,et al.  Creating creativity: user interfaces for supporting innovation , 2000, TCHI.

[31]  Mikael Fernström,et al.  Issues for Designing a flexible expressive audiovisual system for real-time performance & composition , 2004, NIME.

[32]  Dominique Robson PLAY!: Sound Toys for Non-Musicians , 2002, Computer Music Journal.

[33]  Nick Bryan-Kinns,et al.  Mutual engagement and collocation with shared representations , 2013, Int. J. Hum. Comput. Stud..

[34]  Jennifer G. Sheridan,et al.  Designing for performative tangible interaction , 2008, Int. J. Arts Technol..

[35]  Daniel Overholt,et al.  The Musical Interface Technology Design Space , 2009, Organised Sound.

[36]  Christopher Small Musicking: The Meanings of Performing and Listening , 1999 .

[37]  Elaine Toms,et al.  The development and evaluation of a survey to measure user engagement , 2010, J. Assoc. Inf. Sci. Technol..

[38]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[39]  Brian O'Neill,et al.  Creativity support for novice digital filmmaking , 2013, CHI.

[40]  Nick Bryan-Kinns Daisyphone: the design and impact of a novel environment for remote group music improvisation , 2004, DIS '04.

[41]  D. R. Lehman,et al.  What's wrong with cross-cultural comparisons of subjective Likert scales?: The reference-group effect. , 2002, Journal of personality and social psychology.

[42]  Richard Dudas,et al.  Comprovisation: The Various Facets of Composed Improvisation within Interactive Performance Systems , 2010, Leonardo Music Journal.

[43]  R. Friedman,et al.  The effects of promotion and prevention cues on creativity. , 2001, Journal of personality and social psychology.

[44]  Mitchel Resnick,et al.  Pianos not stereos: creating computational construction kits , 1996, INTR.

[45]  Daniel Gómez,et al.  A look at the design and creation of a graphically controlled digital musical instrument , 2007, NIME '07.

[46]  Atau Tanaka Sensor-Based Musical Instruments and Interactive Music , 2011 .

[47]  Celine Latulipe,et al.  Quantifying the Creativity Support of Digital Tools through the Creativity Support Index , 2014, ACM Trans. Comput. Hum. Interact..

[48]  Sidney S. Fels,et al.  Designing for intimacy: creating new interfaces for musical expression , 2004, Proceedings of the IEEE.

[49]  George Frederick McKay Experimental Music , 1959 .

[50]  Stefania Serafin,et al.  A Practical Approach towards an Exploratory Framework for Physical Modeling , 2010, Computer Music Journal.

[51]  Ann Blandford,et al.  “Making my own luck”: Serendipity strategies and how to support them in digital information environments , 2014, J. Assoc. Inf. Sci. Technol..

[52]  Marcelo M. Wanderley,et al.  New Digital Musical Instruments: Control And Interaction Beyond the Keyboard (Computer Music and Digital Audio Series) , 2006 .

[53]  E. Toms,et al.  The development and evaluation of a survey to measure user engagement , 2010, J. Assoc. Inf. Sci. Technol..

[54]  Robert J. Sternberg,et al.  The Cambridge Handbook of Creativity: Constraints on Creativity , 2010 .

[55]  Tina Blaine,et al.  Contexts of Collaborative Musical Experiences , 2003, NIME.

[56]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[57]  Hiroshi Ishii,et al.  The tangible user interface and its evolution , 2008, CACM.

[58]  N. Bonnardel,et al.  Evocation Processes by Novice and Expert Designers: Towards Stimulating Analogical Thinking , 2004 .

[59]  Marcelo M. Wanderley,et al.  The Importance of Parameter Mapping in Electronic Instrument Design , 2002, NIME.

[60]  John C. McCarthy,et al.  Technology as experience , 2004, INTR.

[61]  Erin A. Carroll,et al.  Quantifying the personal creative experience: evaluation of digital creativity support tools using self-report and physiological responses , 2013 .

[62]  Jan C. Schacher,et al.  Gestural electronic music using machine learning as generative device , 2015, NIME.

[64]  Francisco Zamorano Simpletones: A System of Collaborative Physical Controllers for Novices , 2012, NIME.

[65]  Gil Weinberg,et al.  The Squeezables: Toward an Expressive and Interdependent Multi-player Musical Instrument , 2001, Computer Music Journal.

[66]  Ellen Campana,et al.  Amateur Musicians, Long-Term Engagement, and HCI , 2013, Music and Human-Computer Interaction.

[67]  Thomas T. Hewett,et al.  Informing the design of computer-based environments to support creativity , 2005, Int. J. Hum. Comput. Stud..

[68]  Filipa Martins de Abreu,et al.  Creative Engagement: Multimodal digital games in children's learning environment in Macau S.A.R. , 2017 .

[69]  Matthew Wright,et al.  Problems and prospects for intimate musical control of computers , 2001 .

[70]  Christian Dindler Designing infrastructures for creative engagement , 2014, Digit. Creativity.

[71]  Holger Winnemöller,et al.  Toward a cognitive theory of creativity support , 2013, Creativity & Cognition.

[72]  Dale E. Parson Chess-Based Composition and Improvisation for Non-Musicians , 2009, NIME.

[73]  Ted Selker Fostering motivation and creativity for computer users , 2005, Int. J. Hum. Comput. Stud..

[74]  Curtis Roads,et al.  Composing Electronic Music: A New Aesthetic , 2015 .

[75]  D. Hoffman,et al.  The Influence of Goal-Directed and Experiential Activities on Online Flow Experiences , 2003 .

[76]  Gary P. Scavone,et al.  Different Strokes: a Prototype Software System for Laptop Performance and Improvisation , 2006, NIME.

[77]  Csikszentmih Alyi,et al.  Flow and the Foundations of Positive Psychology , 2014 .

[78]  Michael S. Bernstein,et al.  Designing Creativity Support Tools for Failure , 2015, Creativity & Cognition.

[79]  Gil Weinberg,et al.  Interconnected musical networks: bringing expression and thoughtfulness to collaborative group playing , 2003 .

[80]  Tim Murray-Browne,et al.  Interactive music: Balancing creative freedom with musical development. , 2012 .

[81]  Peter C. Wright,et al.  Funology: from usability to enjoyment , 2005 .

[82]  David V. Keyson,et al.  Product behavior and appearance effects on experienced engagement during experiential and goal-directed tasks , 2007, DPPI.

[83]  Anne-Marie Skriver Hansen,et al.  Play Fluency in Music Improvisation Games for Novices , 2011, NIME.

[84]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[85]  Pier Giuseppe Rossi,et al.  Art , 2000, The Lancet.

[86]  Eva Hornecker,et al.  Let's jam the reactable , 2013, ACM Trans. Comput. Hum. Interact..

[87]  Thor Magnusson,et al.  Algorithms as Scores: Coding Live Music , 2011, Leonardo Music Journal.

[88]  Yuta Sugiura,et al.  RoboJockey: real-time, simultaneous, and continuous creation of robot actions for everyone , 2010, UIST '10.

[89]  Linda Candy,et al.  Supporting the creative user: a criteria-based approach to interaction design , 1997 .

[90]  E. Toms,et al.  What is user engagement? A conceptual framework for defining user engagement with technology , 2008, J. Assoc. Inf. Sci. Technol..

[91]  Michael A. Terry,et al.  Creativity Support Tool Evaluation Methods and Metrics , 2005 .

[92]  Andrew P. McPherson,et al.  Radear: A Tangible Spinning Music Sequencer , 2014, NIME.

[93]  M. Csíkszentmihályi,et al.  Creative Insight: The Social Dimension of a Solitary Moment , 2014 .

[94]  Jodi Forlizzi,et al.  Understanding experience in interactive systems , 2004, DIS '04.

[95]  Diemo Schwarz,et al.  A Topo-Phonic Table for Tangible Sonic Interaction , 2017, ICMC.

[96]  Nick Bryan-Kinns,et al.  Designing collaborative musical experiences for broad audiences , 2013, Creativity & Cognition.

[97]  Sergi Jordà,et al.  The reacTable*: A Collaborative Musical Instrument , 2006, 15th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE'06).

[98]  James D. Hollan,et al.  Distributed cognition: toward a new foundation for human-computer interaction research , 2000, TCHI.

[99]  Gary A. Davis,et al.  Faking High or Low Creativity Scores on the Adjective Check List. , 1979 .

[100]  David V. Keyson,et al.  Product features and task effects on experienced richness, control and engagement in voicemail browsing , 2009, Personal and Ubiquitous Computing.

[101]  Kenneth B. McAlpine Shake and Create: Reappropriating Video Game Technologies for the Enactive Learning of Music , 2017, Serious Games and Edutainment Applications.

[102]  S. Koelsch,et al.  Neural correlates of strategy use during auditory working memory in musicians and non‐musicians , 2011, The European journal of neuroscience.

[103]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[104]  K. A. Ericsson,et al.  The Scientific Study of Expert Levels of Performance: general implications for optimal learning and creativity 1 , 1998 .

[105]  Ben Shneiderman,et al.  Creativity support tools: accelerating discovery and innovation , 2007, CACM.

[106]  Atau Tanaka,et al.  Multimodal Interaction in Music Using the Electromyogram and Relative Position Sensing , 2002, NIME.

[107]  Gil Weinberg,et al.  "iltur" - Connecting Novices and Experts Through Collaborative Improvisation , 2005, NIME.

[108]  Marc Hassenzahl,et al.  How motivational orientation influences the evaluation and choice of hedonic and pragmatic interactive products: The role of regulatory focus , 2008, Interact. Comput..

[109]  Yongmeng Wu,et al.  Supporting Non-Musicians? Creative Engagement with Musical Interfaces , 2017, Creativity & Cognition.

[110]  Max V. Mathews,et al.  The PLANK: Designing a simple Haptic Controller , 2002, NIME.

[111]  Linda Candy,et al.  Understanding and evaluating creativity , 2009, C&C '09.

[112]  Ernest Edmonds,et al.  On creative engagement , 2006 .

[113]  Thomas B. Holmes Electronic and experimental music : pioneers in technology and composition , 2002 .

[114]  Effie Lai-Chong Law,et al.  The Influence of Motivation on Emotional Experience in E-commerce , 2015, INTERACT.

[115]  Christopher Frauenberger,et al.  Auditory display design - An investigation of a design pattern approach , 2009, Int. J. Hum. Comput. Stud..

[116]  Nicola Orio,et al.  Evaluation of Input Devices for Musical Expression: Borrowing Tools from HCI , 2001, Computer Music Journal.

[117]  Gil Weinberg,et al.  The Beatbug – evolution of a musical controller , 2008, Digit. Creativity.

[118]  Anna Reid,et al.  Design students' experience of engagement and creativity , 2007 .

[119]  Ge Wang,et al.  Principles of Visual Design for Computer Music , 2014, ICMC.

[120]  Jane Webster,et al.  Audience engagement in multimedia presentations , 1997, DATB.

[121]  Ben Shneiderman,et al.  Creativity Support Tools: A Grand Challenge for HCI Researchers , 2009 .

[122]  Gilbert Beyer,et al.  Music Interfaces for Novice Users: Composing Music on a Public Display with Hand Gestures , 2011, NIME.