Online Segmentation and Classification of Manipulation Actions From the Observation of Kinetostatic Data

This paper presents an automated method for segmentation and classification of manipulation tasks. It introduces a method to build and update a dictionary of elementary actions, so as to express observed tasks as a sequence of items. Segmentation is carried out by splitting an observed manipulation task into submaneuvers. It is based on singular value decomposition of data that is gathered from the observation of humans. This observation consists of hand joint angles, the hand pose with respect to a world frame, and fingertip contact forces. The classification step introduces, from a large set of observed maneuvers, new entities called elementary actions that generalize the concept of segments, instances of elementary actions. This paper uses fingertip contact forces in the measured data. In grasping and manipulation tasks, the interaction between the hand and the object in the physical world is necessary to segment and interpret motion. A set of 120 maneuvers involving six tasks have been used to evaluate the methods with dependent measures including metrics of robustness, effectiveness, and repeatability. In such evaluations, the average value of the effectiveness metrics over all the maneuvers is 0.866. The interuser repeatability is equal to 0.8926, while the average repeatability is 0.911.

[1]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[2]  Jernej Barbic,et al.  Segmenting Motion Capture Data into Distinct Behaviors , 2004, Graphics Interface.

[3]  Alberto Cavallo,et al.  Primitive actions extraction for a human hand by using SVD , 2011, 2011 IEEE 9th International Symposium on Intelligent Systems and Informatics.

[4]  Weihua Sheng,et al.  Wearable Sensor-Based Hand Gesture and Daily Activity Recognition for Robot-Assisted Living , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[5]  Danica Kragic,et al.  Primitive-Based Action Representation and Recognition , 2011, Adv. Robotics.

[6]  Pietro Perona,et al.  Decomposition of human motion into dynamics-based primitives with application to drawing tasks , 2003, Autom..

[7]  Volker Krüger,et al.  Primitive Based Action Representation and Recognition , 2009, SCIA.

[8]  Jessica K. Hodgins,et al.  Aligned Cluster Analysis for temporal segmentation of human motion , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[9]  Yinlai Jiang,et al.  Acquisition of Embodied Knowledge on Gesture Motion by Singular Value Decomposition , 2011, J. Adv. Comput. Intell. Intell. Informatics.

[10]  Alberto Cavallo.  Real-Time Recognizing Human Hand Gestures , 2012, ICIRA.

[11]  Tamar Flash,et al.  Motor primitives in vertebrates and invertebrates , 2005, Current Opinion in Neurobiology.

[12]  Tetsuo Sawaragi,et al.  Extraction of Coordinative Structures of Motions by Segmentation Using Singular Spectrum Transformation , 2011, J. Adv. Comput. Intell. Intell. Informatics.

[13]  Rüdiger Dillmann,et al.  Ensuring kinetostatic consistency in observation of human manipulation , 2013, Robotics Auton. Syst..

[14]  Pietro Falco,et al.  Data Fusion Based on Optical Technology for Observation of Human Manipulation , 2012 .

[15]  Rüdiger Dillmann,et al.  Learning of generalized manipulation strategies in the context of Programming by Demonstration , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[16]  Yiannis Aloimonos,et al.  Understanding visuo‐motor primitives for motion synthesis and analysis , 2006, Comput. Animat. Virtual Worlds.

[17]  Alberto Cavallo.  Using SVD for Segmentation and Classification of Human Hand Actions , 2011, 2011 10th International Conference on Machine Learning and Applications and Workshops.