Overapproximative Human Arm Occupancy Prediction for Collision Avoidance

Predicting the occupancy of a human in real time is of great interest in human-robot coexistence for obtaining regions that a robot should avoid in safe motion planning. The human body is composed of joints and links, suiting approximation by a kinematic chain, but the control strategy of the human is completely unknown, meaning the potential occupancy grows very fast and it is difficult to compute tightly in real time. As such, most previous work considers only specific, known, or probable movements, and usually does not account for a range of human dimensions. Focusing on the human arm, we analyze archetypal movements performed by test subjects to create a dynamic model. Motion-capture data of subjects are fitted, for modeling purposes, to two abstractions: a 4-degree of freedom (DOF) model and a 3-DOF model, to obtain dynamic parameters. We validate our approach on movements from a publicly available database. The prediction is shown to be computationally fast, and reachable sets of the abstraction are shown to enclose all possible future occupancies of the arm for different subjects, tightly but overapproximatively. The 3-DOF model has advantages over the 4-DOF in terms of speed, though the 4-DOF model is tighter at smaller time horizons. Such an overapproximative representation is intended for certifiable safety-guaranteed collision avoidance algorithms for robots.Note to Practitioners—Motivated by the need to keep humans safe when working alongside robots, our earlier work proposes a method of trajectory planning where the robot certifies each movement as safe before it performs it. For this to prove that unsafe collisions cannot occur, an overapproximative prediction of the human is needed, meaning that no possible future position of the human is outside the predicted region, or reachable occupancy. However, making this prediction both small enough (so that it does not include unreachable regions) and fast enough for real-time use is not straightforward. We find the limits of human motion by asking a range of test subjects to perform movements as fast as possible. We calculate the reachable occupancies based on these limits and show that our predictions are indeed overapproximative, fast, and not wasteful of volume. One can then use the aforementioned approach to guarantee safety; future challenges are reliably sensing the human’s pose and implementing our approach on an industrial robot.

[1]  Andrea Maria Zanchettin,et al.  Path-consistent safety in mixed human-robot collaborative manufacturing environments , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Berthold Bäuml,et al.  Real-time swept volume and distance computation for self collision detection , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Bryan Buchholz,et al.  ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion--Part II: shoulder, elbow, wrist and hand. , 2005, Journal of biomechanics.

[4]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[5]  Sami Haddadin,et al.  Towards Safe Robots - Approaching Asimov's 1st Law , 2013, Springer Tracts in Advanced Robotics.

[6]  N. Manning,et al.  The human arm kinematics and dynamics during daily activities - toward a 7 DOF upper limb powered exoskeleton , 2005, ICAR '05. Proceedings., 12th International Conference on Advanced Robotics, 2005..

[7]  Yadolah Dodge,et al.  Mathematical Programming In Statistics , 1981 .

[8]  Michael Ulbrich,et al.  A bilevel optimization approach to obtain optimal cost functions for human arm movements , 2012 .

[9]  Gordon Cheng,et al.  Realizing whole-body tactile interactions with a self-organizing, multi-modal artificial skin on a humanoid robot , 2015, Adv. Robotics.

[10]  Dmitry Berenson,et al.  A framework for unsupervised online human reaching motion recognition and early prediction , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  G. Frehse Handbook of Hybrid Systems Control: Verification tools for linear hybrid automata , 2009 .

[12]  Dmitry Berenson,et al.  Human-robot collaborative manipulation planning using early prediction of human motion , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[13]  Klaus Bengler,et al.  Human motion behavior while interacting with an industrial robot. , 2012, Work.

[14]  B. Nigg,et al.  Biomechanics of the musculo-skeletal system , 1995 .

[15]  Christine M. Haslegrave,et al.  Hands and Handles , 2018 .

[16]  Matthias Althoff,et al.  An Introduction to CORA 2015 , 2015, ARCH@CPSWeek.

[17]  Ronald Poppe,et al.  Vision-based human motion analysis: An overview , 2007, Comput. Vis. Image Underst..

[18]  Oussama Khatib,et al.  Human Motion Reconstruction and Synthesis of Human Skills , 2010 .

[19]  Matthias Althoff,et al.  Reachability Analysis of Nonlinear Differential-Algebraic Systems , 2014, IEEE Transactions on Automatic Control.

[20]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[21]  Gerardo Lafferriere,et al.  A New Class of Decidable Hybrid Systems , 1999, HSCC.

[22]  Oliver Brock,et al.  Soft Robotics: Transferring Theory to Application , 2015 .

[23]  Matthias Althoff,et al.  Online motion synthesis with minimal intervention control and formal safety guarantees , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[24]  Hema Swetha Koppula,et al.  Anticipating Human Activities Using Object Affordances for Reactive Robotic Response , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Siegfried M. Rump,et al.  INTLAB - INTerval LABoratory , 1998, SCAN.

[26]  Andrea Maria Zanchettin,et al.  Safety-aware trajectory scaling for Human-Robot Collaboration with prediction of human occupancy , 2015, 2015 International Conference on Advanced Robotics (ICAR).

[27]  N Klopcar,et al.  A kinematic model of the shoulder complex to evaluate the arm-reachable workspace. , 2007, Journal of biomechanics.

[28]  Matthias Althoff,et al.  Reachability computation of low-order models for the safety verification of high-order road vehicle models , 2012, 2012 American Control Conference (ACC).

[29]  D. Dowson,et al.  Analysis of elbow forces due to high-speed forearm movements. , 1980, Journal of biomechanics.

[30]  Matthias Althoff,et al.  Overapproximative arm occupancy prediction for human-robot co-existence built from archetypal movements , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[31]  Andrea Maria Zanchettin,et al.  Achieving Humanlike Motion: Resolving Redundancy for Anthropomorphic Industrial Manipulators , 2013, IEEE Robotics & Automation Magazine.

[32]  Matthias Althoff,et al.  Reachability analysis of nonlinear systems with uncertain parameters using conservative linearization , 2008, 2008 47th IEEE Conference on Decision and Control.

[33]  Matthias Althoff,et al.  Probabilistic mapping of dynamic obstacles using Markov chains for replanning in dynamic environments , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  G. Smirnov Introduction to the Theory of Differential Inclusions , 2002 .

[35]  Scott L. Delp,et al.  A Model of the Upper Extremity for Simulating Musculoskeletal Surgery and Analyzing Neuromuscular Control , 2005, Annals of Biomedical Engineering.

[36]  Matthias Althoff,et al.  Reachset Conformance Testing of Hybrid Automata , 2016, HSCC.

[37]  Dana Kulic,et al.  Real-time safety for human - robot interaction , 2005, ICAR '05. Proceedings., 12th International Conference on Advanced Robotics, 2005..

[38]  Oussama Khatib,et al.  Muscle force transmission to operational space accelerations during elite golf swings , 2012, 2012 IEEE International Conference on Robotics and Automation.

[39]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[40]  Ansgar Schwirtz,et al.  Knee and ankle joint torque-angle relationships of multi-joint leg extension. , 2011, Journal of biomechanics.

[41]  Oded Maler,et al.  Recent progress in continuous and hybrid reachability analysis , 2006, 2006 IEEE Conference on Computer Aided Control System Design, 2006 IEEE International Conference on Control Applications, 2006 IEEE International Symposium on Intelligent Control.

[42]  Matthias Althoff,et al.  Road occupancy prediction of traffic participants , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[43]  Mark Whitty,et al.  Robotics, Vision and Control. Fundamental Algorithms in MATLAB , 2012 .

[44]  S. Sathiya Keerthi,et al.  A fast procedure for computing the distance between complex objects in three-dimensional space , 1988, IEEE J. Robotics Autom..

[45]  Matthias Althoff,et al.  Safety control of robots under Computed Torque control using reachable sets , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[46]  Thierry Fraichard,et al.  Safe motion planning in dynamic environments , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[47]  Tadej Bajd,et al.  A Simple Kinematic Model of a Human Body for Virtual Environments , 2010 .

[48]  Matteo Parigi Polverini,et al.  A pre-collision control strategy for human-robot interaction based on dissipated energy in potential inelastic impacts , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[49]  Jingzhou Yang,et al.  Towards understanding the workspace of human limbs , 2004, Ergonomics.

[50]  T. Santner,et al.  Changes in the moment arms of the rotator cuff and deltoid muscles with abduction and rotation. , 1994, The Journal of bone and joint surgery. American volume.

[51]  P. Morasso Three dimensional arm trajectories , 1983, Biological Cybernetics.

[52]  Matthias Althoff,et al.  Reachability Analysis and its Application to the Safety Assessment of Autonomous Cars , 2010 .

[53]  Manfred Morari,et al.  Multi-Parametric Toolbox 3.0 , 2013, 2013 European Control Conference (ECC).