A Framework for Human-Robot-Human Physical Interaction Based on N-Player Game Theory

In order to analyze the complex interactive behaviors between the robot and two humans, this paper presents an adaptive optimal control framework for human-robot-human physical interaction. N-player linear quadratic differential game theory is used to describe the system under study. N-player differential game theory can not be used directly in actual scenerie, since the robot cannot know humans’ control objectives in advance. In order to let the robot know humans’ control objectives, the paper presents an online estimation method to identify unknown humans’ control objectives based on the recursive least squares algorithm. The Nash equilibrium solution of human-robot-human interaction is obtained by solving the coupled Riccati equation. Adaptive optimal control can be achieved during the human-robot-human physical interaction. The effectiveness of the proposed method is demonstrated by rigorous theoretical analysis and simulations. The simulation results show that the proposed controller can achieve adaptive optimal control during the interaction between the robot and two humans. Compared with the LQR controller, the proposed controller has more superior performance.

[1]  Donald E. Kirk,et al.  Optimal control theory : an introduction , 1970 .

[2]  Masayoshi Tomizuka,et al.  Modeling and controller design of cooperative robots in workspace sharing human-robot assembly teams , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Neville Hogan,et al.  Impedance Control: An Approach to Manipulation: Part I—Theory , 1985 .

[4]  Keng Peng Tee,et al.  Continuous Role Adaptation for Human–Robot Shared Control , 2015, IEEE Transactions on Robotics.

[5]  Nicolas Jouandeau,et al.  A Survey and Analysis of Multi-Robot Coordination , 2013 .

[6]  Cagatay Basdogan,et al.  The role of roles: Physical cooperation between humans and robots , 2012, Int. J. Robotics Res..

[7]  Gianluca Antonelli,et al.  A coordination strategy for multi-robot sampling of dynamic fields , 2012, 2012 IEEE International Conference on Robotics and Automation.

[8]  E. Burdet,et al.  A Framework to Describe, Analyze and Generate Interactive Motor Behaviors , 2012, PloS one.

[9]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[10]  H. Kazerooni,et al.  Biomechanical design of the Berkeley lower extremity exoskeleton (BLEEX) , 2006, IEEE/ASME Transactions on Mechatronics.

[11]  Andrea Maria Zanchettin,et al.  Prediction of Human Activity Patterns for Human–Robot Collaborative Assembly Tasks , 2019, IEEE Transactions on Industrial Informatics.

[12]  Frank L. Lewis,et al.  Online adaptive learning for team strategies in multi-agent systems , 2012 .

[13]  Jin Xu,et al.  A Bayesian Framework for Nash Equilibrium Inference in Human-Robot Parallel Play , 2020, Robotics: Science and Systems.

[14]  Jeha Ryu,et al.  Safe physical human robot interaction-past, present and future , 2008 .

[15]  Frank L. Lewis,et al.  Multi-player non-zero-sum games: Online adaptive learning solution of coupled Hamilton-Jacobi equations , 2011, Autom..

[16]  Sandra Hirche,et al.  Human-Oriented Control for Haptic Teleoperation , 2012, Proceedings of the IEEE.

[17]  Zheng Chen,et al.  Adaptive Fuzzy Backstepping Control for Stable Nonlinear Bilateral Teleoperation Manipulators With Enhanced Transparency Performance , 2020, IEEE Transactions on Industrial Electronics.

[18]  John D. Simeral,et al.  An assistive decision-and-control architecture for force-sensitive hand–arm systems driven by human–machine interfaces , 2015, Int. J. Robotics Res..

[19]  Antonio Bicchi,et al.  An atlas of physical human-robot interaction , 2008 .

[20]  F.L. Lewis,et al.  Reinforcement learning and adaptive dynamic programming for feedback control , 2009, IEEE Circuits and Systems Magazine.

[21]  Duc Truong Pham,et al.  Human-Robot Collaborative Manufacturing using Cooperative Game: Framework and Implementation , 2018 .

[22]  D. Campolo,et al.  Differential game theory for versatile physical human–robot interaction , 2019, Nat. Mach. Intell..

[23]  Yildiray Yildiz,et al.  Modeling Cyber-Physical Human Systems via an Interplay Between Reinforcement Learning and Game Theory , 2019, Annu. Rev. Control..

[24]  Martin Buss,et al.  A Survey of Environment-, Operator-, and Task-adapted Controllers for Teleoperation Systems , 2010 .

[25]  Huaguang Zhang,et al.  An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games , 2011, Autom..

[26]  D. Fudenberg,et al.  Noncooperative Game Theory for Industrial Organization: An Introduction and Overview , 1986 .

[27]  Keng Peng Tee,et al.  Adaptive optimal control for coordination in physical human-robot interaction , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[28]  Dirk Wollherr,et al.  Human-Like Motion Planning Based on Game Theoretic Decision Making , 2018, Int. J. Soc. Robotics.

[29]  Shuzhi Sam Ge,et al.  Force tracking control for motion synchronization in human-robot collaboration , 2014, Robotica.

[30]  Derong Liu,et al.  Online Synchronous Approximate Optimal Learning Algorithm for Multi-Player Non-Zero-Sum Games With Unknown Dynamics , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[31]  Shing Chow Chan,et al.  A New Variable Forgetting Factor-Based Bias-Compensated RLS Algorithm for Identification of FIR Systems With Input Noise and Its Hardware Implementation , 2020, IEEE Transactions on Circuits and Systems I: Regular Papers.

[32]  Y. Ho,et al.  Nonzero-sum differential games , 1969 .

[33]  Keng Peng Tee,et al.  A Framework of Human–Robot Coordination Based on Game Theory and Policy Iteration , 2016, IEEE Transactions on Robotics.

[34]  Daniel Neumann,et al.  A problem design and constraint modelling approach for collaborative assembly line planning , 2019, Robotics and Computer-Integrated Manufacturing.

[35]  Veronique Limère,et al.  A structured methodology for the design of a human-robot collaborative assembly workplace , 2019, The International Journal of Advanced Manufacturing Technology.

[36]  Aude Billard,et al.  A dynamical system approach to task-adaptation in physical human–robot interaction , 2019, Auton. Robots.

[37]  Edoardo Battaglia,et al.  A Review of Intent Detection, Arbitration, and Communication Aspects of Shared Control for Physical Human-Robot Interaction , 2018 .

[38]  Tal Shima,et al.  UAV Cooperative Decision and Control: Challenges and Practical Approaches , 2008 .

[39]  Sandra Hirche,et al.  Control sharing in human-robot team interaction , 2017, Annu. Rev. Control..

[40]  Shuzhi Sam Ge,et al.  Human–Robot Collaboration Based on Motion Intention Estimation , 2014, IEEE/ASME Transactions on Mechatronics.

[41]  Allison M. Okamura,et al.  Task-dependent impedance and implications for upper-limb prosthesis control , 2014, Int. J. Robotics Res..

[42]  Yuqiang Wu,et al.  Towards Ergonomic Control of Collaborative Effort in Multi-human Mobile-robot Teams , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[43]  Stefano Chiaverini,et al.  A distributed approach to human multi-robot physical interaction , 2019, 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC).