Unsupervised dance figure analysis from video for dancing Avatar animation

This paper presents a framework for unsupervised video analysis in the context of dance performances, where gestures and 3D movements of a dancer are characterized by repetition of a set of unknown dance figures. The system is trained in an unsupervised manner using hidden Markov models (HMMs) to automatically segment multiview video recordings of a dancer into recurring elementary temporal body motion patterns to identify the dance figures. That is, a parallel HMM structure is employed to automatically determine the number and the temporal boundaries of different dance figures in a given dance video. The success of the analysis framework has been evaluated by visualizing these dance figures on a dancing avatar animated by the computed 3D analysis parameters. Experimental results demonstrate that the proposed framework enables synthetic agents and/or robots to learn dance figures from video automatically.

[1]  Atsushi Nakazawa,et al.  Synthesize stylistic human motion from examples , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[2]  Atsushi Nakazawa,et al.  Analysis and synthesis of human dance motions , 2003, Proceedings of IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, MFI2003..

[3]  A. Murat Tekalp,et al.  Audio-driven human body motion analysis and synthesis , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[4]  A. Murat Tekalp,et al.  Prosody-Driven Head-Gesture Animation , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[5]  Junichi Hoshino,et al.  Independent component analysis and synthesis of human motion , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[6]  Fabio Remondino,et al.  Human Motion Reconstruction and Animation from Video Sequences , 2004 .

[7]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[8]  Aaron Hertzmann,et al.  Style machines , 2000, SIGGRAPH 2000.

[9]  Ronald Poppe,et al.  Vision-based human motion analysis: An overview , 2007, Comput. Vis. Image Underst..

[10]  A. Murat Tekalp,et al.  Multicamera Audio-Visual Analysis of Dance Figures , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[11]  Jessica K. Hodgins,et al.  Interactive control of avatars animated with human motion data , 2002, SIGGRAPH.

[12]  José M. F. Moura,et al.  Capture and synthesis of human motion in video sequences , 1998, 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No.98EX175).

[13]  Rin-ichiro Taniguchi,et al.  Human figure control software for real-virtual application , 2004, Proceedings. Eighth International Conference on Information Visualisation, 2004. IV 2004..

[14]  Sameer Singh,et al.  Video analysis of human dynamics - a survey , 2003, Real Time Imaging.