Classification and translation of style and affect in human motion using RBF neural networks

Human motion can be carried out with a variety of different affects or styles such as happy, sad, energetic, and tired among many others. Modeling and classifying these styles, and more importantly, translating them from one sequence onto another has become a popular problem in the fields of graphics, multimedia, and human computer interaction. In this paper, radial basis functions (RBF) are used to model and extract stylistic and affective features from motion data. We demonstrate that using only a few basis functions per degree of freedom, successful modeling of styles in cycles of human walk can be achieved. Furthermore, we employ an ensemble of RBF neural networks to learn the affective/stylistic features following time warping and principal component analysis. The system learns the components and classifies stylistic motion sequences into distinct affective and stylistic classes. The system also utilizes the ensemble of neural networks to learn motion affects and styles such that it can translate them onto neutral input sequences. Experimental results along with both numerical and perceptual validations confirm the highly accurate and effective performance of the system.

[1]  Barbara Caputo,et al.  Recognizing human actions: a local SVM approach , 2004, ICPR 2004.

[2]  David J. Fleet,et al.  Human attributes from 3D pose tracking , 2010, Comput. Vis. Image Underst..

[3]  Christoph Bregler,et al.  Motion capture assisted animation: texturing and synthesis , 2002, ACM Trans. Graph..

[4]  Fernando De la Torre,et al.  Canonical Time Warping for Alignment of Human Behavior , 2009, NIPS.

[5]  Ronen Basri,et al.  Actions as space-time shapes , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Nikolaus F Troje,et al.  Gaze patterns during perception of direction and gender from biological motion. , 2010, Journal of vision.

[7]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[8]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[9]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[10]  Michelle Karg,et al.  Recognition of Affect Based on Gait Patterns , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Barbara Caputo,et al.  Local velocity-adapted motion events for spatio-temporal recognition , 2007, Comput. Vis. Image Underst..

[12]  David C. Knill,et al.  Response to perturbations of visual feedback during reaching movements , 2010 .

[13]  Pierre Payeur,et al.  Automatic Temporal Location and Classification of Human Actions Based on Optical Features , 2009, 2009 2nd International Congress on Image and Signal Processing.

[14]  Liang Wang,et al.  Learning and Matching of Dynamic Shape Manifolds for Human Action Recognition , 2007, IEEE Transactions on Image Processing.

[15]  B. Wandell,et al.  Mapping Hv4 and Ventral Occipital Cortex: the Venous Eclipse , 2022 .

[16]  Jeremy S. De Bonet,et al.  Multiresolution sampling procedure for analysis and synthesis of texture images , 1997, SIGGRAPH.

[17]  Mubarak Shah,et al.  Chaotic Invariants for Human Action Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[18]  Euntai Kim,et al.  Neural network ensemble with probabilistic fusion and its application to gait recognition , 2009, Neurocomputing.

[19]  D. Massart,et al.  A comparison of two algorithms for warping of analytical signals , 2002 .

[20]  N. Troje.  Decomposing biological motion: a framework for analysis and synthesis of human gait patterns. , 2002, Journal of vision.

[21]  S. Ali Etemad,et al.  Modeling and Transformation of 3D Human Motion , 2010, GRAPP.

[22]  Michelle Karg,et al.  A comparison of PCA, KPCA and LDA for feature extraction to recognize affect in gait kinematics , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[23]  Tido Röder,et al.  Documentation Mocap Database HDM05 , 2007 .

[24]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[25]  Lance Williams,et al.  Motion signal processing , 1995, SIGGRAPH.

[26]  Zoran Popovic,et al.  Motion warping , 1995, SIGGRAPH.

[27]  C. K. Liu,et al.  Learning physics-based motion style with nonlinear inverse optimization , 2005, SIGGRAPH 2005.

[28]  Ludmila I. Kuncheva,et al.  A Theoretical Study on Six Classifier Fusion Strategies , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Mubarak Shah,et al.  Human Action Recognition in Videos Using Kinematic Features and Multiple Instance Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  James D. Keeler,et al.  Layered Neural Networks with Gaussian Hidden Units as Universal Approximations , 1990, Neural Computation.

[31]  Gentiane Venture.  Human characterization and emotion characterization from gait , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[32]  Frans van den Berg,et al.  Correlation optimized warping and dynamic time warping as preprocessing methods for chromatographic data , 2004 .

[33]  S. Ali Etemad,et al.  A Customizable Time Warping Method for Motion Alignment , 2013, 2013 IEEE Seventh International Conference on Semantic Computing.

[34]  Aaron Hertzmann,et al.  Style machines , 2000, SIGGRAPH 2000.

[35]  Rasmus Bro,et al.  Automated alignment of chromatographic data , 2006 .

[36]  Nikolaus F. Troje,et al.  Retrieving Information from Human Movement Patterns , 2008 .

[37]  Yasuo Kuniyoshi,et al.  A self-organizing neural model for context-based action recognition , 2003, First International IEEE EMBS Conference on Neural Engineering, 2003. Conference Proceedings..

[38]  Jessica K. Hodgins,et al.  Interactive control of avatars animated with human motion data , 2002, SIGGRAPH.

[39]  Kari Pulli,et al.  Style translation for human motion , 2005, SIGGRAPH 2005.

[40]  Nadia Magnenat-Thalmann,et al.  A simple footskate removal method for virtual reality applications , 2007, The Visual Computer.

[41]  Michael F. Cohen,et al.  Verbs and Adverbs: Multidimensional Motion Interpolation , 1998, IEEE Computer Graphics and Applications.

[42]  Xiao Yang,et al.  Modeling style and variation in human motion , 2010, SCA 2010.

[43]  Jovan Popovic,et al.  Guided time warping for motion editing , 2007, SCA '07.

[44]  Ramakant Nevatia,et al.  Single View Human Action Recognition using Key Pose Matching and Viterbi Path Searching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Yang Wang,et al.  Unsupervised Discovery of Action Classes , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[46]  Aude Billard,et al.  Stochastic gesture production and recognition model for a humanoid robot , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[47]  Lionel Revéret,et al.  Natural Character Posing from a Large Motion Database , 2011, IEEE Computer Graphics and Applications.

[48]  S. Ali Etemad,et al.  Separation and extraction of energy variants from human motion using temporal minimization , 2011, 2011 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems Proceedings.

[49]  Aaron Hertzmann,et al.  Style-based inverse kinematics , 2004, SIGGRAPH 2004.