Motion Control of a Dancing Character with Music

We present our framework that efficiently synchronizes dancing motion of a virtual character according to music input. Two modules have been developed to achieve this. A music server analyzes audio input on the fly and extracts information on music such as tempo, time signature, and beat time. A motion client chooses and displays motion clips according to the music information, by time-warping and synchronizing the start time of motion clips with the music beat time. As a preprocessing step, we construct a database of motion clips using captured dancing motion of a dancer. A PCA-based method is presented to easily identify transiting parts from each motion clip.

[1]  B. Prabhakaran,et al.  A similarity measure for motion stream segmentation and recognition , 2005, MDM '05.

[2]  Wei Wang,et al.  A system for analyzing and indexing human-motion databases , 2005, SIGMOD '05.

[3]  Daniel Thalmann,et al.  PCA-based walking engine using motion capture data , 2004, Proceedings Computer Graphics International, 2004..

[4]  Hui Gao,et al.  Recognizing human action efforts: an adaptive three-mode PCA framework , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[5]  Maja J. Mataric,et al.  Automated Derivation of Primitives for Movement Classification , 2000, Auton. Robots.

[6]  Sung Yong Shin,et al.  On‐line motion blending for real‐time locomotion generation , 2004, Comput. Animat. Virtual Worlds.

[7]  David A. Forsyth,et al.  Automatic Annotation of Everyday Movements , 2003, NIPS.

[8]  Jernej Barbic,et al.  Segmenting Motion Capture Data into Distinct Behaviors , 2004, Graphics Interface.

[9]  Masataka Goto,et al.  An Audio-based Real-time Beat Tracking System for Music With or Without Drum-sounds , 2001 .

[10]  Ryohei Nakatsu,et al.  Study of Dance Entertainment Using Robots , 2006, Edutainment.

[11]  I. Jolliffe.  Principal Component Analysis , 2002 .

[12]  Lucas Kovar,et al.  Motion Graphs , 2002, ACM Trans. Graph..

[13]  David A. Forsyth,et al.  Motion synthesis from annotations , 2003, ACM Trans. Graph..

[14]  Marc Alexa,et al.  Representing Animations by Principal Components , 2000, Comput. Graph. Forum.

[15]  Daniel Thalmann,et al.  Construction of animation models out of captured data , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[16]  Tao Yu,et al.  Motion retrieval based on movement notation language , 2005, Comput. Animat. Virtual Worlds.

[17]  Dimitrios Gunopulos,et al.  Indexing Large Human-Motion Databases , 2004, VLDB.

[18]  Michael F. Cohen,et al.  Verbs and Adverbs: Multidimensional Motion Interpolation , 1998, IEEE Computer Graphics and Applications.

[19]  Sung Yong Shin,et al.  Rhythmic-motion synthesis based on motion-beat analysis , 2003, ACM Trans. Graph..

[20]  Jehee Lee,et al.  Precomputing avatar behavior from human motion data , 2004, SCA '04.