Live Music-Making: A Rich Open Task Requires a Rich Open Interface

In live human-computer music-making, how can interfaces successfully support the openness, reinterpretation and rich signification often important in live (especially improvised) musical performance? We argue that the use of design metaphors can lead to interfaces which constrain interactions and militate against reinterpretation, while consistent, grammatical interfaces empower the user to create and apply their own metaphors in developing their performance. These metaphors can be transitory and disposable, yet do not represent wasted learning since the underlying grammar is retained. We illustrate this move with reflections from live coding practice, from recent visual and two-dimensional programming language interfaces, and from musical voice mapping research. We consider the integration of the symbolic and the continuous in the human-computer interaction. We also describe how our perspective is reflected in approaches to system evaluation.

[1]  Dan Stowell,et al.  Learning Timbre Analogies from Unlabelled Data by Multivariate Tree Regression , 2011 .

[2]  Alan F. Blackwell,et al.  A Principled Approach to Developing New Languages for Live Coding , 2011, NIME.

[3]  Dan Stowell,et al.  Making music through real-time voice timbre analysis: machine learning and timbral control , 2010 .

[4]  Alan F. Blackwell,et al.  CHAPTER 5 – Notational Systems—The Cognitive Dimensions of Notations Framework , 2003 .

[5]  Henry Lieberman,et al.  End-User Development , 2015, Lecture Notes in Computer Science.

[6]  Nick Collins,et al.  Live coding in laptop performance , 2003, Organised Sound.

[7]  Christopher Dromey,et al.  Speech interactions with linguistic, cognitive, and visuomotor tasks. , 2005, Journal of speech, language, and hearing research : JSLHR.

[8]  Dan Stowell,et al.  Evaluation of live human-computer music-making: Quantitative and qualitative approaches , 2009, Int. J. Hum. Comput. Stud..

[9]  John M. Carroll,et al.  Interface metaphors and user interface design , 1988 .

[10]  Stephen McAdams,et al.  Structural and affective aspects of music from statistical audio signal analysis , 2006, J. Assoc. Inf. Sci. Technol..

[11]  G. Lakoff,et al.  Metaphors We Live By , 1980 .

[12]  Aniruddh D. Patel.  Music, Language, and the Brain , 2007 .

[13]  T. Landauer,et al.  Handbook of Human-Computer Interaction , 1997 .

[14]  Paul Mulholland,et al.  What Can the Language of Musicians Tell Us about Music Interaction Design? , 2010, Computer Music Journal.

[15]  Dan Stowell,et al.  Delayed Decision-making in Real-time Beatbox Percussion Classification , 2010 .

[16]  Miller Puckette,et al.  The Patcher , 1988, International Conference on Mathematics and Computing.

[17]  Wilson Coker,et al.  Music and Meaning , 1972 .

[18]  Jon McCormack,et al.  Composing with nodal networks , 2006 .

[19]  David G. Hendry,et al.  Creating, comprehending and explaining spreadsheets: a cognitive interpretation of what discretionary users think of the spreadsheet model , 1994, Int. J. Hum. Comput. Stud..

[20]  Brad A. Myers,et al.  Taxonomies of visual programming and program visualization , 1990, J. Vis. Lang. Comput..

[21]  Alex McLean,et al.  Texture: Visual Notation for Live Coding of Pattern , 2011, ICMC.

[22]  Alan F. Blackwell,et al.  The reification of metaphor as a design tool , 2006, TCHI.

[23]  Alistair Stead.  User configurable machine vision for mobiles , 2011, PPIG.

[24]  Alex McLean,et al.  Visualisation of live code , 2010, EVA.

[25]  S. McAdams,et al.  Structural and affective aspects of music from statistical audio signal analysis: Special Topic Section on Computational Analysis of Style , 2006 .

[26]  J. Henderson.  Human gaze control during real-world scene perception , 2003, Trends in Cognitive Sciences.

[27]  Sergi Jordà,et al.  The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces , 2007, TEI.

[28]  Regan L. Mandryk,et al.  A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies , 2007, Int. J. Hum. Comput. Stud..

[29]  M. Thaut,et al.  The Oxford handbook of music psychology , 2011 .

[30]  John Millar Carroll.  HCI Models, Theories, and Frameworks: Toward a Multidisciplinary Science , 2003 .

[31]  Alan F. Blackwell,et al.  Psychological Issues in End-User Programming , 2006, End User Development.

[32]  Nicola Orio,et al.  Evaluation of Input Devices for Musical Expression: Borrowing Tools from HCI , 2001, Computer Music Journal.

[33]  Harold W. Thimbleby,et al.  Equal Opportunity Interactive Systems , 1986, Int. J. Man Mach. Stud..