Human motion capture sensors and analysis in robotics

Purpose – The purpose of this paper is to provide a review of various motion capture technologies and discuss the methods for handling the captured data in applications related to robotics.Design/methodology/approach – The approach taken in the paper is to compare the features and limitations of motion trackers in common use. After introducing the technology, a summary is given of robotic‐related work undertaken with the sensors and the strengths of different approaches in handling the data are discussed. Each comparison is presented in a table. Results from the author's experimentation with an inertial motion capture system are discussed based on clustering and segmentation techniques.Findings – The trend in methodology is towards stochastic machine learning techniques such as hidden Markov model or Gaussian mixture model, their extensions in hierarchical forms and non‐linear dimension reduction. The resulting empirical models tend to handle uncertainty well and are suitable for incrementally updating mo...

[1]  Ales Ude,et al.  Programming full-body movements for humanoid robots by observation , 2004, Robotics Auton. Syst..

[2]  Kazuhito Yokoi,et al.  Leg motion primitives for a dancing humanoid robot , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[3]  Javier Ruiz-del-Solar,et al.  Learning to fall: Designing low damage fall sequences for humanoid soccer robots , 2009, Robotics Auton. Syst..

[4]  Norman I. Badler,et al.  Acquiring and validating motion qualities from live limb gestures , 2005, Graph. Model..

[5]  Z. Pan,et al.  Mixture Model Segmentation for Gait Recognition , 2008, 2008 ECSIS Symposium on Learning and Adaptive Behaviors for Robotic Systems (LAB-RS).

[6]  Daniel Grest,et al.  Human Action Recognition in Table-Top Scenarios : An HMM-Based Analysis to Optimize the Performance , 2007, CAIP.

[7]  Jun Nakanishi,et al.  Movement imitation with nonlinear dynamical systems in humanoid robots , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[8]  Giancarlo Ferrigno,et al.  Modeling and driving a reduced human mannequin through motion captured data: a neural network approach , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[9]  L. Rocchi,et al.  A simple test to assess the static and dynamic accuracy of an inertial sensors system for human movement analysis , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[10]  P. Veltink,et al.  Compensation of magnetic disturbances improves inertial and magnetic sensing of human body segment orientation , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[11]  C. S. George Lee,et al.  Whole-body human-to-humanoid motion transfer , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[12]  Wojciech Matusik,et al.  Practical motion capture in everyday surroundings , 2007, SIGGRAPH 2007.

[13]  Christopher G. Atkeson,et al.  Adapting human motion for the control of a humanoid robot , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[14]  Rajesh P. N. Rao,et al.  Learning Shared Latent Structure for Image Synthesis and Robotic Imitation , 2005, NIPS.

[15]  Ahmed M. Elgammal,et al.  Tracking People on a Torus , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Zhaoqin Peng,et al.  Design of humanoid complicated dynamic motion based on human motion capture , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Fazel Naghdy,et al.  Motion segmentation for humanoid control planning , 2008 .

[18]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[19]  Jessica K. Hodgins,et al.  Aligned Cluster Analysis for temporal segmentation of human motion , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[20]  Maja J. Mataric,et al.  Motion capture from inertial sensing for untethered humanoid teleoperation , 2004, 4th IEEE/RAS International Conference on Humanoid Robots, 2004..

[21]  Michael J. Black,et al.  HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion , 2010, International Journal of Computer Vision.

[22]  Greg Welch,et al.  Motion Tracking: No Silver Bullet, but a Respectable Arsenal , 2002, IEEE Computer Graphics and Applications.

[23]  Geoffrey E. Hinton,et al.  Factored conditional restricted Boltzmann Machines for modeling motion style , 2009, ICML '09.

[24]  Yasuo Kuniyoshi,et al.  Wearable motion capture suit with full-body tactile sensors , 2009, 2009 IEEE International Conference on Robotics and Automation.

[25]  Yoshihiko Nakamura,et al.  Making feasible walking motion of humanoid robots from human motion capture data , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[26]  Ankur Agarwal,et al.  Recovering 3D human pose from monocular images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Russ Tedrake,et al.  Efficient Bipedal Robots Based on Passive-Dynamic Walkers , 2005, Science.

[28]  Dana Kulic,et al.  Incremental Learning, Clustering and Hierarchy Formation of Whole Body Motion Patterns using Adaptive Hidden Markov Chains , 2008, Int. J. Robotics Res..

[29]  Cynthia Breazeal,et al.  Action parsing and goal inference using self as simulator , 2005, ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005..

[30]  Thomas Beth,et al.  Analysis of Human Motions for Humanoid Robots , 2005 .

[31]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[32]  Thomas Beth,et al.  Analysis of Human Motion for Humanoid Robots , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[33]  Zicheng Liu,et al.  Expandable Data-Driven Graphical Modeling of Human Actions Based on Salient Postures , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Jernej Barbic,et al.  Segmenting Motion Capture Data into Distinct Behaviors , 2004, Graphics Interface.

[35]  Henry Been-Lirn Duh,et al.  Motion Control of a Robotic Puppet through a Hybrid Motion Capture Device , 2007, 2007 IEEE International Conference on Automation Science and Engineering.

[36]  Stefan Schaal,et al.  http://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained , 2007 .

[37]  Miomir Vukobratovic,et al.  Survey of Intelligent Control Techniques for Humanoid Robots , 2003, J. Intell. Robotic Syst..

[38]  Rajesh P. N. Rao,et al.  Robotic imitation from human motion capture using Gaussian processes , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[39]  Ales Ude,et al.  Stereo-based Markerless Human Motion Capture for Humanoid Robot Systems , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[40]  L. Benini,et al.  Activity recognition from on-body sensors by classifier fusion: sensor scalability and robustness , 2007, 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information.

[41]  Gordon Cheng,et al.  Discovering optimal imitation strategies , 2004, Robotics Auton. Syst..

[42]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[43]  Hans-Peter Seidel,et al.  Markerless motion capture of man-machine interaction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[45]  Tamim Asfour,et al.  Toward an Unified Representation for Imitation of Human Motion on Humanoids , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[46]  S. Münch,et al.  Robot Programming by Demonstration (RPD) - Using Machine Learning and User Interaction Methods for the Development of Easy and Comfortable Robot Programming Systems , 2000 .

[47]  Rajesh P. N. Rao,et al.  Towards a Real-Time Bayesian Imitation System for a Humanoid Robot , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[48]  Philippe Beaudoin,et al.  Motion-motif graphs , 2008, SCA '08.

[49]  David J. Fleet,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE Gaussian Process Dynamical Model , 2007 .

[50]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[51]  Tsutomu Fujinami,et al.  Hierarchical organization of the coordinative structure of the skill of clay kneading. , 2008, Human movement science.

[52]  Yoshihiko Nakamura,et al.  Embodied Symbol Emergence Based on Mimesis Theory , 2004, Int. J. Robotics Res..

[53]  Odest Chadwicke Jenkins,et al.  Interactive Human Pose and Action Recognition Using Dynamical Motion Primitives , 2007, Int. J. Humanoid Robotics.

[54]  Paul Lukowicz,et al.  Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.