The Self-Supervising Machine

Supervised machine learning enables complex many-to-many mappings and control schemes needed in interactive performance systems. One of the persistent problems in these applications is generating, identifying and choosing input output pairings for training. This poses problems of scope (limiting the realm of potential control inputs), eort (requiring signicant pre-performance training time), and cognitive load (forcing the performer to learn and remember the control areas). We discuss the creation and implementation of an automatic \supervisor," using unsupervised machine learning algorithms to train a supervised neural network on the y. This hierarchical arrangement enables network training in real time based on the musical or gestural control inputs employed in a performance, aiming at freeing the performer to operate in a creative, intuitive realm, making the machine control transparent and automatic. Three implementations of this self supervised model driven by iPod, iPad, and acoustic violin are described.

[1]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[2]  Barbara Tillmann,et al.  Music Cognition: Learning, Perception, Expectations , 2008, CMMR.

[3]  François Pachet,et al.  Musical data mining for electronic music distribution , 2001, Proceedings First International Conference on WEB Delivering of Music. WEDELMUSIC 2001.

[4]  Perry R. Cook,et al.  Play-Along Mapping of Musical Controllers , 2009, ICMC.

[5]  W. Jay Dowling,et al.  Artist: adaptive resonance theory to internalize the structure of tonality (a neural net listening to music) , 1999 .

[6]  Richard Kronland-Martinet,et al.  Sense of Sounds , 2008 .

[7]  François Pachet,et al.  Tools and Architecture for the Evaluation of Similarity Measures : Case Study of Timbre Similarity , 2004, ISMIR.

[8]  P.P. de Leon,et al.  Pattern Recognition Approach for Music Style Identification Using Shallow Statistical Descriptors , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[10]  Belinda Thom,et al.  Interactive Improvisational Music Companionship: A User-Modeling Approach , 2003, User Modeling and User-Adapted Interaction.

[11]  David S. Watson,et al.  A Machine Learning Approach to Musical Style Recognition , 1997, ICMC.

[12]  J. Wolfe,et al.  Spectral centroid and timbre in complex, multiple instrumental textures , 2004 .

[13]  David Wessel.  Connectionist models for musical control of nonlinear dynamical systems , 1992 .

[14]  Robert O. Gjerdingen,et al.  Categorization of Musical Patterns by Self-Organizing Neuronlike Networks , 1990 .

[15]  Jeffrey S Bowers,et al.  Contrasting five different theories of letter position coding: evidence from orthographic similarity effects. , 2006, Journal of experimental psychology. Human perception and performance.

[16]  N. M. Collins,et al.  Towards autonomous agents for live computer music : realtime machine listening and interactive music systems , 2007 .

[17]  I. Peretz,et al.  Brain organization for music processing. , 2005, Annual review of psychology.