Evaluating Interactive Music Systems: An HCI Approach

In this paper, we discuss a number of issues related to the design of evaluation tests for comparing interactive music systems for improvisation. Our testing procedure covers rehearsal and performance environments, and captures the experiences of a musician/participant as well as an audience member/observer. We attempt to isolate salient components of system behavior, and test whether the musician or audience are able to discern between systems with significantly different behavioral components. We have applied our testing methodology in comparative studies of our London and ARHS improvisation systems [1], with the help of saxophonists John Butcher and James Fei; we report on preliminary experiences and ongoing design refinements.

[1]  Y. Rogers,et al.  Interaction Design , 2002 .

[2]  George E. Lewis.  Too Many Notes: Computers, Complexity and Culture in Voyager , 2000, Leonardo Music Journal.

[3]  Jason Freeman,et al.  Technology, Real-Time Notation, and Audience Participation in Flock , 2008, ICMC.

[4]  Takehisa Onisawa,et al.  Music composition system based on subjective evaluation , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[5]  Julio d'Escrivan,et al.  Cambridge Companion to Electronic Music , 2007 .

[6]  P. Kam,et al.  : 4 , 1898, You Can Cross the Massacre on Foot.

[7]  Jerry Avorn.  Technology , 1929, Nature.

[8]  Masataka Goto,et al.  Subjective evaluation of common singing skills using the rank ordering method , 2006 .

[9]  Nicola Orio,et al.  Evaluation of Input Devices for Musical Expression: Borrowing Tools from HCI , 2001, Computer Music Journal.

[10]  Alma Mater Studiorum.  Subjective evaluation of common singing skills using the rank ordering method , 2006 .

[11]  Nicola Orio,et al.  Evaluation of input devices for musical expression , 2002 .

[12]  William Hsu,et al.  Managing Gesture and Timbre for Analysis and Instrument Control in an Interactive Environment , 2006, NIME.

[13]  William Hsu.  Two Approaches for Interaction Management in Timbre-Aware Improvisation Systems , 2008, ICMC.

[14]  E. Coiera.  Interaction Design , 2002 .

[15]  Christopher Ariza,et al.  The Interrogator as Critic: The Turing Test and the Evaluation of Generative Music Systems , 2009, Computer Music Journal.