Human motion primitive discovery and recognition

We present a novel framework for the automatic discovery and recognition of human motion primitives from motion capture data. Human motion primitives are discovered by optimizing the 'motion flux', a quantity which depends on the motion of a group of skeletal joints. Models of each primitive category are computed via non-parametric Bayes methods and recognition is performed based on their geometric properties. A normalization of the primitives is proposed in order to make them invariant with respect to anatomical variations and data sampling rate. Using our framework we build a publicly available dataset of human motion primitives based on motion capture sequences taken from well-known datasets. We expect that our framework, by providing an objective way for discovering and categorizing human motion, will be a useful tool in numerous research fields related to Robotics including human inspired motion generation, learning by demonstration, and intuitive human-robot interaction.

[1]  Fiora Pirri,et al.  Bayesian Image Based 3D Pose Estimation , 2016, ECCV.

[2]  Dana Kulic,et al.  Incremental learning of full body motion primitives and their sequencing through human motion observation , 2012, Int. J. Robotics Res..

[3]  Pietro Perona,et al.  Decomposition of human motion into dynamics-based primitives with application to drawing tasks , 2003, Autom..

[4]  Michelle Karg,et al.  Movement Primitive Segmentation for Human Motion Modeling: A Framework for Analysis , 2016, IEEE Transactions on Human-Machine Systems.

[5]  Michael J. Black,et al.  Implicit Probabilistic Models of Human Motion for Synthesis and Tracking , 2002, ECCV.

[6]  Aude Billard,et al.  Discriminative and adaptive imitation in uni-manual and bi-manual tasks , 2006, Robotics Auton. Syst..

[7]  Tamim Asfour,et al.  Toward an Unified Representation for Imitation of Human Motion on Humanoids , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[8]  Jessica L. Allen,et al.  Neuromechanical Principles Underlying Movement Modularity and Their Implications for Rehabilitation , 2015, Neuron.

[9]  Lourdes Agapito,et al.  Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Juan Carlos Niebles,et al.  A Hierarchical Pose-Based Approach to Complex Action Understanding Using Dictionaries of Actionlets and Motion Poselets , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Yoshihiko Nakamura,et al.  Embodied Symbol Emergence Based on Mimesis Theory , 2004, Int. J. Robotics Res..

[12]  T. Flash,et al.  Minimum-jerk, two-thirds power law, and isochrony: converging approaches to movement planning. , 1995, Journal of experimental psychology. Human perception and performance.

[13]  Ran Xu,et al.  Human action segmentation with hierarchical supervoxel consistency , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Bernhard Schölkopf,et al.  Probabilistic movement modeling for intention inference in human–robot interaction , 2013, Int. J. Robotics Res..

[15]  Norman I. Badler,et al.  Semantic Segmentation of Motion Capture Using Laban Movement Analysis , 2007, IVA.

[16]  Jason M Wilken,et al.  Range of Motion Requirements for Upper-Limb Activities of Daily Living. , 2015, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[17]  Mehrtash Tafazzoli Harandi,et al.  Going deeper into action recognition: A survey , 2016, Image Vis. Comput..

[18]  Tamim Asfour,et al.  Unifying Representations and Large-Scale Whole-Body Motion Databases for Studying Human Motion , 2016, IEEE Transactions on Robotics.

[19]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[20]  Stefan Schaal,et al.  Learning and generalization of motor skills by learning from demonstration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[21]  Oliver Kroemer,et al.  Interaction primitives for human-robot cooperation tasks , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Radford M. Neal.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[23]  Tamar Flash,et al.  Affine differential geometry analysis of human arm movements , 2007, Biological Cybernetics.

[24]  Jan Peters,et al.  Noname manuscript No. (will be inserted by the editor) Policy Search for Motor Primitives in Robotics , 2022 .

[25]  Jun Morimoto,et al.  Adaptation and coaching of periodic motion primitives through physical and visual interaction , 2016, Robotics Auton. Syst..

[26]  C. Antoniak.  Mixtures of Dirichlet Processes with Applications to Bayesian Nonparametric Problems , 1974 .

[27]  P. Fihl,et al.  View-invariant gesture recognition using 3D optical flow and harmonic motion context , 2010, Comput. Vis. Image Underst..

[28]  Cristian Sminchisescu,et al.  Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Aude Billard,et al.  Task Parameterization Using Continuous Constraints Extracted From Human Demonstrations , 2015, IEEE Transactions on Robotics.

[30]  Ferdinando A. Mussa-Ivaldi,et al.  Toward a neurobiology of coordinate transformations , 1995 .

[31]  Tamar Flash,et al.  Models of human movement: Trajectory planning and inverse kinematics studies , 2013, Robotics Auton. Syst..

[32]  Bernard Ghanem,et al.  ActivityNet Challenge 2017 Summary , 2017, ArXiv.

[33]  Jungdam Won,et al.  Shadow theatre , 2016, ACM Trans. Graph..

[34]  Rahul Sukthankar,et al.  Articulated motion discovery using pairs of trajectories , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Mubarak Shah,et al.  Discovering Motion Primitives for Unsupervised Grouping and One-Shot Learning of Human Actions, Gestures, and Expressions , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Gérard G. Medioni,et al.  Robust unsupervised motion pattern inference from video and applications , 2011, 2011 International Conference on Computer Vision.

[37]  Nikolaos G. Tsagarakis,et al.  On the Kinematic Motion Primitives (kMPs) – Theory and Application , 2012, Front. Neurorobot..

[38]  Tamim Asfour,et al.  The KIT whole-body human motion database , 2015, 2015 International Conference on Advanced Robotics (ICAR).

[39]  Gérard G. Medioni,et al.  Structured Time Series Analysis for Human Action Segmentation and Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Tamar Flash,et al.  Motor primitives in vertebrates and invertebrates , 2005, Current Opinion in Neurobiology.

[41]  Yi Yang,et al.  Unsupervised Video Adaptation for Parsing Human Motion , 2014, ECCV.

[42]  Dmitry Berenson,et al.  A framework for unsupervised online human reaching motion recognition and early prediction , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[43]  Yoshihiko Nakamura,et al.  Spatio-temporal structure of human motion primitives and its application to motion prediction , 2016, Robotics Auton. Syst..

[44]  Radford M. Neal,et al.  A Split-Merge Markov chain Monte Carlo Procedure for the Dirichlet Process Mixture Model , 2004 .

[45]  Stefan Schaal,et al.  Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[46]  Jessica K. Hodgins,et al.  Hierarchical Aligned Cluster Analysis for Temporal Clustering of Human Motion , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Xiaowei Zhou,et al.  Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Dagmar Sternad,et al.  Dynamic primitives of motor behavior , 2012, Biological Cybernetics.

[49]  Roman Filipovych,et al.  Recognizing primitive interactions by exploring actor-object states , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Martin A. Giese,et al.  Segmenting sign language into motor primitives with Bayesian binning , 2013, Front. Comput. Neurosci..

[51]  Diego Torricelli,et al.  Quantitative assessment based on kinematic measures of functional impairments during upper extremity movements: A review. , 2014, Clinical biomechanics.

[52]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[53]  Rajesh P. N. Rao,et al.  A Bayesian Developmental Approach to Robotic Goal-Based Imitation Learning , 2015, PloS one.

[54]  Tamim Asfour,et al.  Imitation Learning of Dual-Arm Manipulation Tasks in Humanoid Robots , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[55]  Adrian Hilton,et al.  Viewpoint invariant exemplar-based 3D human tracking , 2006, Comput. Vis. Image Underst..

[56]  Michael J. Black,et al.  HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion , 2010, International Journal of Computer Vision.

[57]  Reinhard Klein,et al.  Efficient Unsupervised Temporal Segmentation of Motion Data , 2015, IEEE Transactions on Multimedia.

[58]  Felix Polyakov,et al.  Affine differential geometry and smoothness maximization as tools for identifying geometric movement primitives , 2014, Biological Cybernetics.