Motion data retrieval based on statistic correlation between motion symbol space and language

Captured human motion data are used so that humanoid robots or computer graphics (CG) characters can behave naturally. Because the motion capture system is expensive, and time-consuming process is needed to acquire motion data, technology that enables to reuse existing motion data efficiently is required. This paper proposes motion retrieval method with natural language word based on stochastic correlation between motion and language. We construct a space which has maximum correlation with motion pattern features and word features, and we use this space as search space for motion retrieval. Proto-symbol space, which represents the relationship of each symbolized motion patterns, is used as motion feature space. And as word feature, binary features are used which represent whether a word label is attached or not. Because the constructed search space has correlation between motion patterns and words, associative motion retrieval considering similarity of motion pattern or closeness of word meaning becomes possible. We validate proposed motion retrieval method by constructing motion database with captured human motion data.

[1]  Roger Levy,et al.  A new approach to cross-modal multimedia retrieval , 2010, ACM Multimedia.

[2]  Yoshihiko Nakamura,et al.  Statistically integrated semiotics that enables mutual inference between linguistic and behavioral symbols for humanoid robots , 2009, 2009 IEEE International Conference on Robotics and Automation.

[3]  Meinard Müller,et al.  Efficient content-based retrieval of motion capture data , 2005, SIGGRAPH '05.

[4]  Katsu Yamane,et al.  Retrieval and Generation of Human Motions Based on Associative Model between Motion Symbols and Motion Labels , 2010 .

[5]  Vladimir Pavlovic,et al.  A dynamic Bayesian network approach to figure tracking using learned dynamic models , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Yoshihiko Nakamura,et al.  Embodied Symbol Emergence Based on Mimesis Theory , 2004, Int. J. Robotics Res..

[7]  Jun Tani,et al.  Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment , 2003, IEEE Trans. Syst. Man Cybern. Part A.

[8]  Yoshihiko Nakamura,et al.  From Stochastic Motion Generation and Recognition to Geometric Symbol Development and Manipulation , 2003 .

[9]  Mitsuo Kawato,et al.  MOSAIC Model for Sensorimotor Learning and Control , 2001, Neural Computation.

[10]  Yasuo Kuniyoshi,et al.  Image Annotation and Retrieval for Weakly Labeled Images Using Conceptual Learning , 2009, New Generation Computing.

[11]  Hirokazu Kameoka,et al.  SemiCCA: Efficient Semi-supervised Learning of Canonical Correlations , 2010, 2010 20th International Conference on Pattern Recognition.

[12]  Zoran Popovic,et al.  Motion warping , 1995, SIGGRAPH.

[13]  Forrest W. Young,et al.  Nonmetric individual differences multidimensional scaling: An alternating least squares method with optimal scaling features , 1977 .

[14]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[15]  Lucas Kovar,et al.  Motion Graphs , 2002, ACM Trans. Graph..