Unifying Representations and Large-Scale Whole-Body Motion Databases for Studying Human Motion

Large-scale human motion databases are key for research questions ranging from human motion analysis and synthesis, biomechanics of human motion, data-driven learning of motion primitives, and rehabilitation robotics to the design of humanoid robots and wearable robots such as exoskeletons. In this paper we present a large-scale database of whole-body human motion with methods and tools, which allows a unifying representation of captured human motion, and efficient search in the database, as well as the transfer of subject-specific motions to robots with different embodiments. To this end, captured subject-specific motion is normalized regarding the subject's height and weight by using a reference kinematics and dynamics model of the human body, the master motor map (MMM). In contrast with previous approaches and human motion databases, the motion data in our database consider not only the motions of the human subject but the position and motion of objects with which the subject is interacting as well. In addition to the description of the MMM reference model, we present procedures and techniques for the systematic recording, labeling, and organization of human motion capture data, object motions as well as the subject-object relations. To allow efficient search for certain motion types in the database, motion recordings are manually annotated with motion description tags organized in a tree structure. We demonstrate the transfer of human motion to humanoid robots and provide several examples of motion analysis using the database.

[1]  Katsu Yamane,et al.  Simultaneous tracking and balancing of humanoid robots for imitating human motion capture data , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  T. Asfour,et al.  Human Push-Recovery : Strategy Selection Based on Push Intensity Estimation , 2016 .

[3]  Marko B. Popovic,et al.  Exploiting angular momentum to enhance bipedal center-of-mass control , 2009, 2009 IEEE International Conference on Robotics and Automation.

[4]  Wolfgang Seemann,et al.  A Kinematic Study of Human Torso Motion , 2007 .

[5]  Taku Komura,et al.  Interaction capture using magnetic sensors , 2013, Comput. Animat. Virtual Worlds.

[6]  Oskar von Stryk,et al.  HuMoD - A versatile and open database for the investigation, modeling and simulation of human motion dynamics on actuation level , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[7]  Sung Yong Shin,et al.  Computer puppetry: An importance-based approach , 2001, TOGS.

[8]  B. Buchholz,et al.  Anthropometric data for describing the kinematics of the human hand. , 1992, Ergonomics.

[9]  Carlos Busso,et al.  IEMOCAP: interactive emotional dyadic motion capture database , 2008, Lang. Resour. Evaluation.

[10]  Taku Komura,et al.  Relationship descriptors for interactive motion adaptation , 2013, SCA '13.

[11]  Gentiane Venture,et al.  Real-time identification and visualization of human segment parameters , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[13]  G.F. Harris,et al.  Validation of a multi-segment foot and ankle kinematic model for pediatric gait , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[14]  Michael J. Black,et al.  HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion , 2010, International Journal of Computer Vision.

[15]  Meinard Müller,et al.  Motion templates for automatic classification and retrieval of motion capture data , 2006, SCA '06.

[16]  Tamim Asfour,et al.  ARMAR-4: A 63 DOF torque controlled humanoid robot , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[17]  Tamim Asfour,et al.  Analyzing whole-body pose transitions in multi-contact motions , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[18]  Michael J. Black,et al.  Cardboard people: a parameterized model of articulated image motion , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[19]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Cristian Sminchisescu,et al.  Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Yu Zheng,et al.  Human motion tracking control with strict contact force constraints for floating-base humanoid robots , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[22]  Gentiane Venture,et al.  Motion capture based identification of the human body inertial parameters , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[23]  Daniel Thalmann,et al.  The HUMANOID Environment for Interactive Animation of Multiple Deformable Human Characters , 1995, Comput. Graph. Forum.

[24]  Antonio Morales,et al.  From Robot to Human Grasping Simulation , 2013, Cognitive Systems Monographs.

[25]  J J O'Connor,et al.  Bone position estimation from skin marker co-ordinates using global optimisation with joint constraints. , 1999, Journal of biomechanics.

[26]  R. Marks,et al.  Validation of a multisegment foot and ankle kinematic model for pediatric gait , 2004, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[27]  Tamim Asfour,et al.  Toward an Unified Representation for Imitation of Human Motion on Humanoids , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[28]  Mako Popovic,et al.  Angular momentum primitives for human walking: biomechanics and control , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[29]  Tamim Asfour,et al.  Generation of Human-like Motion for Humanoid Robots Based on Marker-based Motion Capture Data , 2010, ISR/ROBOTIK.

[30]  Norman I. Badler,et al.  Efficient motion retrieval in large motion databases , 2013, I3D '13.

[31]  Gutemberg Guerra-Filho,et al.  The human motion database: A cognitive and parametric sampling of human motion , 2011, Face and Gesture 2011.

[32]  Takashi Minato,et al.  Generating natural posture in an android by mapping human posture in three-dimensional position space , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Ralph Gross,et al.  The CMU Motion of Body (MoBo) Database , 2001 .

[34]  Ayman Habib,et al.  OpenSim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement , 2007, IEEE Transactions on Biomedical Engineering.

[35]  Patrick Lacouture,et al.  An auto-adaptable algorithm to generate human-like locomotion for different humanoid robots based on motion capture data , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[36]  Marko B. Popovic,et al.  Angular momentum in human walking , 2008, Journal of Experimental Biology.

[37]  Rajesh P. N. Rao,et al.  Robotic imitation from human motion capture using Gaussian processes , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[38]  H. Benjamin Brown,et al.  Controlling a marionette with human motion capture data , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[39]  Seong-Whan Lee,et al.  A full-body gesture database for automatic gesture recognition , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[40]  Tido Röder,et al.  Documentation Mocap Database HDM05 , 2007 .

[41]  David J. Fleet,et al.  Stochastic Tracking of 3D Human Figures Using 2D Image Motion , 2000, ECCV.

[42]  W. T. Dempster,et al.  The anthropometry of the manual work space for the seated subject. , 1959, American journal of physical anthropology.

[43]  David A. Winter,et al.  Biomechanics and Motor Control of Human Movement , 1990 .

[44]  Tamim Asfour,et al.  The KIT whole-body human motion database , 2015, 2015 International Conference on Advanced Robotics (ICAR).

[45]  J. Hodgins,et al.  Optimizing Human Motion for the Control of a Humanoid Robot , 2002 .

[46]  Stefan Ulbrich,et al.  Simox: A Robotics Toolbox for Simulation, Motion and Grasp Planning , 2012, IAS.

[47]  P. Leva.  Adjustments to Zatsiorsky-Seluyanov's segment inertia parameters. , 1996 .

[48]  T. Rowan.  Functional stability analysis of numerical algorithms , 1990 .

[49]  Thomas Serre,et al.  HMDB: A large video database for human motion recognition , 2011, 2011 International Conference on Computer Vision.

[50]  F. Pollick,et al.  A motion capture library for the study of identity, gender, and emotion perception from biological motion , 2006, Behavior research methods.

[51]  Florence Billet,et al.  The HuMAnS toolbox, a homogenous framework for motion capture, analysis and simulation , 2006 .

[52]  Torsten Bumgarner,et al.  Biomechanics and Motor Control of Human Movement , 2013 .