Safety bounds in human robot interaction: A survey

Abstract In the era of industrialization and automation, safety is a critical factor that should be considered during the design and realization of each new system that targets operation in close collaboration with humans. Of such systems are considered personal and professional service robots which collaborate and interact with humans at diverse applications environments. In this collaboration, human safety is an important factor in the wider field of human-robot interaction (HRI) since it facilitates their harmonic coexistence. The paper at hand aims to systemize the recent literature by describing the required levels of safety during human-robot interaction, focusing on the core functions of the collaborative robots when performing specific processes. It is also oriented towards the existing methods for psychological safety during human-robot collaboration and its impact at the robot behaviour, while also discusses in depth the psychological parameters of robots incorporation in industrial and social environments. Based on the existing works on safety features that minimize the risk of HRI, a classification of the existing works into five major categories namely, Robot Perceptions for Safe HRI, Cognition-enabled robot control in HRI, Action Planning for safe navigation close to humans, Hardware safety features, and Societal and Psychological factors is also applied. Finally, the current study further discusses the existing risk assessment techniques as methods to offer additional safety in robotic systems presenting thus a holistic analysis of the safety in contemporary robots, and proposes a roadmap for safety compliance features during the development of a robotic system.

[1]  Chris Melhuish,et al.  Compliance Control and Human-Robot Interaction: Part 1 - Survey , 2014, Int. J. Humanoid Robotics.

[2]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[3]  Hitoshi Kimura,et al.  Flexible Hermetically-Sealed Mobile Robot for Narrow Spaces Using Hydrostatic Skeleton Driving Mechanism , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Wolfram Burgard,et al.  An accurate and efficient navigation system for omnidirectional robots in industrial environments , 2017, Auton. Robots.

[5]  Gonzalo Ferrer,et al.  Robot social-aware navigation framework to accompany people walking side-by-side , 2016, Autonomous Robots.

[6]  Antonio C. Siochi,et al.  The UAN: a user-oriented representation for direct manipulation interface designs , 1990, TOIS.

[7]  Toru Tsumugiwa,et al.  Variable impedance control based on estimation of human arm stiffness for human-robot cooperative calligraphic task , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[8]  Olivier Gibaru,et al.  Survey of methods for design of collaborative robotics applications- Why safety is a barrier to more widespread robotics uptake , 2018, ICMRE.

[9]  Kerstin Dautenhahn,et al.  Empirical results from using a comfort level device in human-robot interaction studies , 2006, HRI '06.

[10]  Amir Khajepour,et al.  Design, Kinematics, and Control of a Multijoint Soft Inflatable Arm for Human-Safe Interaction , 2017, IEEE Transactions on Robotics.

[11]  Yu Sun,et al.  A dataset of daily interactive manipulation , 2018, Int. J. Robotics Res..

[12]  Wolfram Burgard,et al.  People Tracking with Mobile Robots Using Sample-Based Joint Probabilistic Data Association Filters , 2003, Int. J. Robotics Res..

[13]  Leila Takayama,et al.  Strategies for human-in-the-loop robotic grasping , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[14]  Iwan Ulrich,et al.  VFH/sup */: local obstacle avoidance with look-ahead verification , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[15]  Andrey V. Savkin,et al.  An algorithm for safe navigation of mobile robots by a sensor network in dynamic cluttered industrial environments , 2018, Robotics and Computer-Integrated Manufacturing.

[16]  Francisco A. Candelas,et al.  Safe human-robot interaction based on dynamic sphere-swept line bounding volumes , 2011 .

[17]  Masatoshi Ishikawa,et al.  Safe human-robot-coexistence: emergency-stop using a high-speed vision-chip , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  T. Ogata,et al.  Dynamic communication of humanoid robot with multiple people based on interaction distance , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[19]  Kevin R Duda,et al.  Markov analysis of human-in-the-loop system performance , 2010, 2010 IEEE Aerospace Conference.

[20]  Rafiq Ahmad,et al.  Human-Robot Collaboration: Twofold strategy algorithm to avoid collisions using ToF sensor , 2014 .

[21]  Kerstin Dautenhahn,et al.  A long-term Human-Robot Proxemic study , 2011, 2011 RO-MAN.

[22]  Andrey V. Savkin,et al.  Seeking a path through the crowd: Robot navigation in unknown dynamic environments with moving obstacles based on an integrated environment representation , 2014, Robotics Auton. Syst..

[23]  Marek P. Michalowski,et al.  A spatial model of engagement for a social robot , 2006, 9th IEEE International Workshop on Advanced Motion Control, 2006..

[24]  Reid G. Simmons,et al.  Sensor fusion for human safety in industrial workcells , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  F. Farelo,et al.  Task-oriented control of a 9-DoF WMRA system for opening a spring-loaded door task , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[26]  Ivan Kalaykov,et al.  Semi-active compliant robot enabling collision safety for human robot interaction , 2010, 2010 IEEE International Conference on Mechatronics and Automation.

[27]  Antonio Bicchi,et al.  Fast and "soft-arm" tactics [robot arm design] , 2004, IEEE Robotics & Automation Magazine.

[28]  Clément Gosselin,et al.  A flexible robot skin for safe physical human robot interaction , 2009, 2009 IEEE International Conference on Robotics and Automation.

[29]  André Crosnier,et al.  Collaborative manufacturing with physical human–robot interaction , 2016 .

[30]  Tetsuo Ono,et al.  Robovie: an interactive humanoid robot , 2001 .

[31]  Seonghee Jeong,et al.  Risk management simulator for low-powered human-collaborative industrial robots , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Willem F. G. Haselager,et al.  Do Robot Performance and Behavioral Style a ↵ ect Human Trust ? A Multi-Method Approach , 2014 .

[33]  Andrea Maria Zanchettin,et al.  Safety Assessment and Control of Robotic Manipulators Using Danger Field , 2013, IEEE Transactions on Robotics.

[34]  Aude Billard,et al.  Safety issues in human-robot interactions , 2013, 2013 IEEE International Conference on Robotics and Automation.

[35]  Aftab Ahmad,et al.  A design metric for safety assessment of industrial robot design suitable for power- and force-limited collaborative operation , 2018, International Journal of Intelligent Robotics and Applications.

[36]  Alessandro De Luca,et al.  Collision detection and reaction: A contribution to safe physical Human-Robot Interaction , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Jérémie Guiochet,et al.  Experience with Model-Based User-Centered Risk Assessment for Service Robots , 2010, 2010 IEEE 12th International Symposium on High Assurance Systems Engineering.

[38]  Alin Albu-Schäffer,et al.  Soft-tissue injury in robotics , 2010, 2010 IEEE International Conference on Robotics and Automation.

[39]  Rachid Alami,et al.  Human-aware robot navigation: A survey , 2013, Robotics Auton. Syst..

[40]  Terrence Fong,et al.  A Survey of Methods for Safe Human-Robot Interaction , 2017, Found. Trends Robotics.

[41]  Rachid Alami,et al.  A Human Aware Mobile Robot Motion Planner , 2007, IEEE Transactions on Robotics.

[42]  Vanessa Evers,et al.  BEHAVE-II: The Revised Set of Measures to Assess Users’ Attitudinal and Behavioral Responses to a Social Robot , 2013, International Journal of Social Robotics.

[43]  Wolfram Burgard,et al.  Using EM to learn motion behaviors of persons with mobile robots , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[44]  Antonios Gasteratos,et al.  Identifying Hazardous Emerging Behaviors in Search and Rescue Missions with Drones: A Proposed Methodology , 2017, ISCRAM-med.

[45]  Giuseppe Menga,et al.  Risked-based design for the physical human-robot interaction (pHRI): an overview , 2015 .

[46]  Alessandro De Luca,et al.  Estimation of contact forces using a virtual force sensor , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[47]  Boubekeur Mendil,et al.  Toward safety navigation in cluttered dynamic environment: A robot neural-based hybrid autonomous navigation and obstacle avoidance with moving target tracking , 2015, 2015 3rd International Conference on Control, Engineering & Information Technology (CEIT).

[48]  Antonios Gasteratos,et al.  Safety certification requirements for domestic robots , 2012 .

[49]  Paolo Fiorini,et al.  Switching control approach for stable navigation of mobile robots in unknown environments , 2011 .

[50]  Antonio Bicchi,et al.  Safety for Physical Human-Robot Interaction , 2008, Springer Handbook of Robotics.

[51]  Mike Fraser,et al.  Building safer robots: Safety driven control , 2012, Int. J. Robotics Res..

[52]  Izabela Nielsen,et al.  A methodology for implementation of mobile robot in adaptive manufacturing environments , 2017, J. Intell. Manuf..

[53]  Ellen J. Bass,et al.  A Systematic Approach to Model Checking Human–Automation Interaction Using Task Analytic Models , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[54]  Pierre-Brice Wieber,et al.  Safe navigation strategies for a biped robot walking in a crowd , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[55]  Steffen Liebergeld,et al.  Taming the Robot , 2011 .

[56]  Jeha Ryu,et al.  Safe physical human robot interaction-past, present and future , 2008 .

[57]  Katherine M. Tsui,et al.  “I want that”: Human-in-the-loop control of a wheelchair-mounted robotic arm , 2011 .

[58]  Mohamad Bdiwi Integrated Sensors System for Human Safety during Cooperating with Industrial Robots for Handing-over and Assembling Tasks , 2014 .

[59]  Wolfram Burgard,et al.  Learning Motion Patterns of People for Compliant Robot Motion , 2005, Int. J. Robotics Res..

[60]  Peng Shi,et al.  Nonlinear Control for Tracking and Obstacle Avoidance of a Wheeled Mobile Robot With Nonholonomic Constraint , 2016, IEEE Transactions on Control Systems Technology.

[61]  Antonios Gasteratos,et al.  Recent trends in social aware robot navigation: A survey , 2017, Robotics Auton. Syst..

[62]  Andrea Maria Zanchettin,et al.  Safety Control of Industrial Robots Based on a Distributed Distance Sensor , 2014, IEEE Transactions on Control Systems Technology.

[63]  Dragoljub Surdilovic,et al.  Safe physical human-robot interaction with industrial dual-arm robots , 2013, 9th International Workshop on Robot Motion and Control.

[64]  Yangsheng Xu,et al.  Mechanical design and implementation of a soft inflatable robot arm for safe human-robot interaction , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[65]  Sotiris Makris,et al.  Human–robot interaction review and challenges on task planning and programming , 2016, Int. J. Comput. Integr. Manuf..

[66]  Rachid Alami,et al.  How may I serve you?: a robot companion approaching a seated person in a helping context , 2006, HRI '06.

[67]  Thomas Bak,et al.  Trajectory planning for robots in dynamic human environments , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[68]  Paul M. Salmon,et al.  Systems-based accident analysis methods: A comparison of Accimap, HFACS, and STAMP , 2012 .

[69]  Antonio Bicchi,et al.  Integration of active and passive compliance control for safe human-robot coexistence , 2009, 2009 IEEE International Conference on Robotics and Automation.

[70]  Rachid Alami,et al.  Physiological and subjective evaluation of a human-robot object hand-over task. , 2011, Applied ergonomics.

[71]  Matteo Saveriano,et al.  Distance based dynamical system modulation for reactive avoidance of moving obstacles , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[72]  Christian Laugier,et al.  Leader following: A study on classification and selection , 2016, Robotics Auton. Syst..

[73]  Alin Albu-Schäffer,et al.  Safe Physical Human-Robot Interaction: Measurements, Analysis and New Insights , 2007, ISRR.

[74]  Milad Geravand,et al.  Human-robot physical interaction and collaboration using an industrial robot with a closed control architecture , 2013, 2013 IEEE International Conference on Robotics and Automation.

[75]  Nikolaos G. Tsagarakis,et al.  Safe human robot interaction via energy regulation control , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[76]  Dana Kulic,et al.  Physiological and subjective responses to articulated robot motion , 2006, Robotica.

[77]  Kishor S. Trivedi,et al.  Coverage Modeling for Dependability Analysis of Fault-Tolerant Systems , 1989, IEEE Trans. Computers.

[78]  Ioannis Kostavelis,et al.  Towards Developing Dependable Systems Suitable for Crisis Management Applications , 2017, ISCRAM-med.

[79]  Georgios Ch. Sirakoulis,et al.  Robot Guided Crowd Evacuation , 2015, IEEE Transactions on Automation Science and Engineering.

[80]  G. Giralt,et al.  Safe and dependable physical human-robot interaction in anthropic domains: State of the art and challenges , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[81]  Lihui Wang,et al.  Active collision avoidance for human–robot collaboration driven by vision sensors , 2017, Int. J. Comput. Integr. Manuf..

[82]  O. Ogorodnikova Methodology of safety for a human robot interaction designing stage , 2008, 2008 Conference on Human System Interactions.

[83]  Ryan M. Eustice,et al.  Ford Campus vision and lidar data set , 2011, Int. J. Robotics Res..

[84]  Tadej Petric,et al.  Human-in-the-loop approach for teaching robot assembly tasks using impedance control interface , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[85]  Dimitrios Tzovaras,et al.  A POMDP Design Framework for Decision Making in Assistive Robots , 2017, HCI.

[86]  Carlos Morato,et al.  Toward Safe Human Robot Collaboration by Using Multiple Kinects Based Real-Time Human Tracking , 2014, J. Comput. Inf. Sci. Eng..

[87]  Jeff Fryman Updating the Industrial Robot Safety Standard , 2014, ISR 2014.

[88]  Yvonne Eriksson,et al.  Safety Assessment Strategy for Collaborative Robot Installations , 2017 .

[89]  Bilge Mutlu,et al.  Human-robot proxemics: Physical and psychological distancing in human-robot interaction , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[90]  Henrik I. Christensen,et al.  Embodied Social Interaction for Service Robots in Hallway Environments , 2005, FSR.

[91]  David John Atkinson,et al.  Autonomous Agents and Human Interpersonal Trust: Can We Engineer a Human-Machine Social Interface for Trust? , 2013, AAAI Spring Symposium: Trust and Autonomous Systems.

[92]  Lorenzo Molinari Tosatti,et al.  Dynamic safety in collaborative robot workspaces through a network of devices fulfilling functional safety requirements , 2014, ISR 2014.

[93]  José Barata,et al.  Human-Aware Navigation for Autonomous Mobile Robots for Intra-factory Logistics , 2017, Symbiotic.

[94]  Peter Christiansen,et al.  FieldSAFE: Dataset for Obstacle Detection in Agriculture , 2017, Sensors.

[95]  J. R. Llata,et al.  Working Together: A Review on Safe Human-Robot Collaboration in Industrial Environments , 2017, IEEE Access.

[96]  Oussama Khatib,et al.  A Depth Space Approach for Evaluating Distance to Objects , 2015, J. Intell. Robotic Syst..

[97]  Barbara Deml,et al.  Human-Aware Robotic Assistant for Collaborative Assembly: Integrating Human Motion Prediction With Planning in Time , 2018, IEEE Robotics and Automation Letters.

[98]  K. Madhava Krishna,et al.  Safe proactive plans and their execution , 2006, Robotics Auton. Syst..

[99]  Alessandro De Luca,et al.  Collision Detection and Safe Reaction with the DLR-III Lightweight Manipulator Arm , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[100]  Dmitry Berenson,et al.  Human-robot collaborative manipulation planning using early prediction of human motion , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[101]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[102]  Lihui Wang,et al.  Depth camera based collision avoidance via active robot control , 2014 .

[103]  André Crosnier,et al.  Multimodal control for human-robot cooperation , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[104]  Christian Osendorfer,et al.  Using tactile sensation for learning contact knowledge: Discriminate collision from physical interaction , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[105]  Antonios Gasteratos,et al.  Collision risk assessment for autonomous robots by offline traversability learning , 2012, Robotics Auton. Syst..

[106]  Jeremy A. Marvel,et al.  Performance Metrics of Speed and Separation Monitoring in Shared Workspaces , 2013, IEEE Transactions on Automation Science and Engineering.

[107]  Paolo Rocco,et al.  Towards safe human-robot interaction in robotic cells: An approach based on visual tracking and intention estimation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[108]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[109]  Thomas Reisinger,et al.  Example Application of ISO/TS 15066 to a Collaborative Assembly Scenario , 2016 .

[110]  Charles C. Kemp,et al.  Human-Robot Interaction for Cooperative Manipulation: Handing Objects to One Another , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[111]  Jeffrey Too Chuan Tan,et al.  Triple stereo vision system for safety monitoring of human-robot collaboration in cellular manufacturing , 2011, 2011 IEEE International Symposium on Assembly and Manufacturing (ISAM).

[112]  Yu Sun,et al.  Visual servoing control of a 9-DoF WMRA to perform ADL tasks , 2012, 2012 IEEE International Conference on Robotics and Automation.

[113]  Clifton A. Ericson,et al.  Hazard Analysis Techniques for System Safety , 2005 .

[114]  Alessandro De Luca,et al.  Integrated control for pHRI: Collision avoidance, detection, reaction and collaboration , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[115]  Rachid Alami,et al.  A step towards a sociable robot guide which monitors and adapts to the person's activities , 2009, 2009 International Conference on Advanced Robotics.

[116]  Stefano Stramigioli,et al.  The Safety of Domestic Robotics: A Survey of Various Safety-Related Publications , 2014, IEEE Robotics & Automation Magazine.

[117]  Claude Baron,et al.  UML based risk analysis - Application to a medical robot , 2004 .

[118]  Naoki Kawakami,et al.  Design, Control and Evaluation of a Whole-Sensitive Robot Arm for Physical Human-Robot Interaction , 2009, Int. J. Humanoid Robotics.

[119]  Marko Pfeifer,et al.  A new strategy for ensuring human safety during various levels of interaction with industrial robots , 2017 .

[120]  John Kenneth Salisbury,et al.  Playing it safe [human-friendly robots] , 2004, IEEE Robotics & Automation Magazine.

[121]  Alessandro De Luca,et al.  Real-Time Computation of Distance to Dynamic Obstacles With Multiple Depth Sensors , 2017, IEEE Robotics Autom. Lett..

[122]  Antonios Gasteratos,et al.  Thorough robot navigation based on SVM local planning , 2015, Robotics Auton. Syst..

[123]  Przemyslaw A. Lasota,et al.  Analyzing the Effects of Human-Aware Motion Planning on Close-Proximity Human–Robot Collaboration , 2015, Hum. Factors.

[124]  Paula Boddington EPSRC Principles of Robotics: commentary on safety, robots as products, and responsibility , 2017, Connect. Sci..

[125]  Jérémie Guiochet,et al.  Human-robot interactions: model-based risk analysis and safety case construction , 2012 .

[126]  Fanny Dufossé,et al.  Model-Checking and Game theory for Synthesis of Safety Rules , 2015, 2015 IEEE 16th International Symposium on High Assurance Systems Engineering.

[127]  Christoph Walter,et al.  Safeguarding and Supporting Future Human-robot Cooperative Manufacturing Processes by a Projection- and Camera-based Technology , 2017 .

[128]  Oussama Khatib,et al.  A depth space approach to human-robot collision avoidance , 2012, 2012 IEEE International Conference on Robotics and Automation.

[129]  Christopher G. Atkeson,et al.  Robots with inflatable links , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[130]  Fabio Paternò Formal Reasoning about Dialogue Properties with Automatic Support , 1997, Interact. Comput..

[131]  Christopher G. Atkeson,et al.  Physical human interaction for an inflatable manipulator , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[132]  Koji Ikuta,et al.  Safety Evaluation Method of Design and Control for Human-Care Robots , 2003, Int. J. Robotics Res..

[133]  John K. Tsotsos,et al.  Robot navigation via spatial and temporal coherent semantic maps , 2016, Eng. Appl. Artif. Intell..

[134]  H. Schneider Failure mode and effect analysis : FMEA from theory to execution , 1996 .

[135]  Leslie Pack Kaelbling,et al.  Provably safe robot navigation with obstacle uncertainty , 2017, Robotics: Science and Systems.

[136]  Panos E. Trahanias,et al.  Predictive autonomous robot navigation , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[137]  Katsu Yamane,et al.  3D printed soft skin for safe human-robot interaction , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[138]  Cristian Secchi,et al.  Safe navigation and experimental evaluation of a novel tire workshop assistant robot , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[139]  Ellen J. Bass,et al.  Generating phenotypical erroneous human behavior to evaluate human-automation interaction using model checking , 2012, Int. J. Hum. Comput. Stud..

[140]  Brian F. Goldiez,et al.  Human-aware robot motion planning with velocity constraints , 2008, 2008 International Symposium on Collaborative Technologies and Systems.

[141]  Masayuki Inaba,et al.  Multi-layered real-time controllers for humanoid's manipulation and locomotion tasks with emergency stop , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[142]  Yasushi Nakauchi,et al.  A Social Robot that Stands in Line , 2002, Auton. Robots.

[143]  Dimitrios Tzovaras,et al.  Robot's Workspace Enhancement with Dynamic Human Presence for Socially-Aware Navigation , 2017, ICVS.

[144]  Rachid Alami,et al.  Assessing the social criteria for human-robot collaborative navigation: A comparison of human-aware navigation planners , 2017, 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[145]  Gérard Poisson,et al.  An ISO10218-compliant adaptive damping controller for safe physical human-robot interaction , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[146]  Tadej Petric,et al.  Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach , 2014, Auton. Robots.

[147]  Dimitrios Tzovaras,et al.  Understanding of Human Behavior with a Robotic Agent Through Daily Activity Analysis , 2019, Int. J. Soc. Robotics.

[148]  Robin Passama,et al.  A unified multimodal control framework for human-robot interaction , 2015, Robotics Auton. Syst..

[149]  Ann Blandford,et al.  An approach to formal verification of human–computer interaction , 2007, Formal Aspects of Computing.

[150]  Xie Tong,et al.  A Systematic Analysis of Functional Safety Certification Practices in Industrial Robot Software Development , 2017 .

[151]  Dan Simon,et al.  Safe navigation in dynamic, unknown, continuous, and cluttered environments , 2017, 2017 IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR).

[152]  Florian Jentsch,et al.  Building Appropriate Trust in Human-Robot Teams , 2013, AAAI Spring Symposium: Trust and Autonomous Systems.

[153]  Javier Minguez,et al.  Sensor-based robot motion generation in unknown, dynamic and troublesome scenarios , 2005, Robotics Auton. Syst..

[154]  Alain Riwan,et al.  Numerical evaluation of a new robotic manipulator based on inflatable joints , 2012, 2012 IEEE International Conference on Automation Science and Engineering (CASE).

[155]  Antonios Gasteratos,et al.  Semantic maps from multiple visual cues , 2017, Expert Syst. Appl..

[156]  Yu Sun,et al.  Determining the benefit of human input in human-in-the-loop robotic systems , 2013, 2013 IEEE RO-MAN.

[157]  Dimitrios Tzovaras,et al.  Human Aware Robot Navigation in Semantically Annotated Domestic Environments , 2016, HCI.

[158]  Andrea Maria Zanchettin,et al.  Safety in human-robot collaborative manufacturing environments: Metrics and control , 2016, IEEE Transactions on Automation Science and Engineering.

[159]  Russell H. Taylor,et al.  Preliminary experiments in cooperative human/robot force control for robot assisted microsurgical manipulation , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[160]  Alessandro De Luca,et al.  Nonlinear decoupled motion-stiffness control and collision detection/reaction for the VSA-II variable stiffness device , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[161]  Christoph Walter,et al.  A projection-based sensor system for safe physical human-robot collaboration , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[162]  Antonio Bicchi,et al.  On making robots understand safety: Embedding injury knowledge into control , 2012, Int. J. Robotics Res..

[163]  Lihui Wang,et al.  Vision-guided active collision avoidance for human-robot collaborations , 2013 .

[164]  Lihui Wang,et al.  Human-robot collaborative assembly in cyber-physical production: Classification framework and implementation , 2017 .

[165]  Kerstin Johansen,et al.  Risk Assessment Process for Collaborative Assembly – A Job Safety Analysis Approach , 2016 .

[166]  Ioannis Kostavelis,et al.  Dependability Levels on Autonomous Systems: The Case Study of a Crisis Management Robot , 2017, Int. J. Inf. Syst. Crisis Response Manag..

[167]  Kai Oliver Arras,et al.  Inverse Reinforcement Learning algorithms and features for robot navigation in crowds: An experimental comparison , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[168]  Leila Takayama,et al.  Influences on proxemic behaviors in human-robot interaction , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[169]  Alexander Zelinsky,et al.  Quantitative Safety Guarantees for Physical Human-Robot Interaction , 2003, Int. J. Robotics Res..

[170]  Maja J. Mataric,et al.  People-aware navigation for goal-oriented behavior involving a human partner , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[171]  Robin R. Murphy,et al.  Survey of metrics for human-robot interaction , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[172]  Alin Albu-Schäffer,et al.  Human-Like Adaptation of Force and Impedance in Stable and Unstable Interactions , 2011, IEEE Transactions on Robotics.

[173]  Antonio Bicchi,et al.  An atlas of physical human-robot interaction , 2008 .

[174]  Illah R. Nourbakhsh,et al.  A survey of socially interactive robots , 2003, Robotics Auton. Syst..

[175]  Andrew Lewis,et al.  Systems-Theoretic Safety Assessment of Robotic Telesurgical Systems , 2015, SAFECOMP.

[176]  John Kenneth Salisbury,et al.  A New Actuation Approach for Human Friendly Robot Design , 2004, Int. J. Robotics Res..

[177]  Niklas Bergström,et al.  Modeling of natural human-robot encounters , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[178]  Jérémie Guiochet,et al.  Safety-critical advanced robots: A survey , 2017, Robotics Auton. Syst..

[179]  Daniel Campos,et al.  Localization and Navigation of an Omnidirectional Mobile Robot: The Robot@Factory Case Study , 2016, IEEE Revista Iberoamericana de Tecnologias del Aprendizaje.

[180]  Antonios Gasteratos,et al.  Robot navigation in large-scale social maps: An action recognition approach , 2016, Expert Syst. Appl..

[181]  Christian Laugier,et al.  From Proxemics Theory to Socially-Aware Navigation: A Survey , 2014, International Journal of Social Robotics.