Towards an interactive multimedia experience for club music and dance

In this paper, we describe completed and ongoing work towards an interactive multimedia system that will appeal to today's youth culture identified as most likely to adopt such novel mobile applications that combines music, dance and technology. We describe our work in the application of three types of successful movement recognition applied in the field of Tai Chi with the objective being to identify gestural primitives of club dance associated with electronic dance music. In this approach, dance movements are first recognized and classified and then mapped, using multiple levels of complexity, to higher level algorithms that can modify multimedia content in real time. The paper describes the mechanisms supporting an attractive alternative to the now standard Video Disc Jockey (VJ) in which members of the dance public are empowered to create multimedia content in real time as opposed to the VJ.

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  Jan C. Schacher.  Action and Perception in Interactive Sound Installations: An Ecological Approach , 2009, NIME.

[3]  Alex Pentland,et al.  Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  R. Blake,et al.  Perception of human motion. , 2007, Annual review of psychology.

[5]  Paul Lukowicz,et al.  Advances in Expressive Animation in the Interactive Performance of a Butoh Dance , 2008 .

[6]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Antonio Camurri,et al.  A tool for analysis of expressive gestures: The EyesWeb Expressive Gesture Processing Library , 2003 .

[8]  Antonio Camurri,et al.  EXPRESSIVE GESTURAL CONTROL OF SOUND AND VISUAL OUTPUT IN MULTIMODAL INTERACTIVE SYSTEMS , 2004 .

[9]  Hongli Zhu,et al.  3D Motion Recognition based on Ensemble Learning , 2007, Eighth International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS '07).

[10]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[11]  Jan C. Schacher.  Gesture control of sounds in 3D space , 2007, NIME '07.

[12]  Alessandro Lameiras Koerich,et al.  Automatic music genre classification using ensemble of classifiers , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[13]  Paul Lukowicz,et al.  Towards Recognizing Tai Chi - An Initial Experiment Using Wearable Sensors , 2006 .

[14]  Antonio Camurri,et al.  Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques , 2003, Int. J. Hum. Comput. Stud..

[15]  N. Ranieri,et al.  Ergonomic Low Cost Motion Capture for every day health exercise , 2008, 2008 Third International Conference on Pervasive Computing and Applications.

[16]  Aaron F. Bobick,et al.  A state-based technique for the summarization and recognition of gesture , 1995, Proceedings of IEEE International Conference on Computer Vision.

[17]  Sethuraman Panchanathan,et al.  Computational analysis of mannerism gestures , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[18]  W. Freeman.  Societies of Brains: A Study in the Neuroscience of Love and Hate. By W. J. Freeman. Erlbaum: Hillsdale, NJ. 1994. , 1997, Psychological Medicine.

[19]  J. Gutknecht,et al.  Qi energy flow visualisation using wearable computing , 2007, 2007 2nd International Conference on Pervasive Computing and Applications.