A shared control method for obstacle avoidance with mobile robots and its interaction with communication delay

Teleoperation allows human operators to safely extend themselves to remote environments that are typically difficult or dangerous to access. The remote environments are often unstructured (i.e. not having clear roads or paths to follow) and only accessible by wireless communication (introducing factors such as degraded signals and communication delay). Teleoperated driving under these conditions can result in slow operation speeds and unintended collisions with obstacles. Automating portions of the teleoperation task can help mitigate some of the negative effects of wireless communication. Shared control is used to combine inputs from the human teleoperator and automation. This work presents a new model predictive control based shared control method. We introduce a new representation for obstacle free regions that works well with unstructured robot environments and allows for an model predictive control problem formulation that can be solved rapidly. The shared control method is implemented in a robot simulator and tested with human subjects. Two user studies involving a search task with a mobile robot evaluate the effectiveness of the shared control method and explore its interaction with factors such as communication delay and input interface style. Communication delay is found to have the largest magnitude effect on performance and safety measures. Results demonstrate that the shared control method can improve both performance and safety when delays are present.

[1]  Jess R. Kerlin,et al.  Towards the Principled Study of Variable Autonomy in Mobile Robots , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[2]  Jenay M. Beer,et al.  Toward a framework for levels of robot autonomy in human-robot interaction , 2014, Journal of human-robot interaction.

[3]  Leila Takayama,et al.  Assisted driving of a mobile remote presence system: System design and controlled user evaluation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[4]  Jessie Y. C. Chen,et al.  Human Performance Issues and User Interface Design for Teleoperated Robots , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[5]  Robin Deits,et al.  Efficient mixed-integer planning for UAVs in cluttered environments , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Sterling J. Anderson,et al.  An optimal-control-based framework for trajectory planning, threat assessment, and semi-autonomous control of passenger vehicles in hazard avoidance scenarios , 2010 .

[7]  Yang Xu,et al.  Video Telephony for End-Consumers: Measurement Study of Google+, iChat, and Skype , 2012, IEEE/ACM Transactions on Networking.

[8]  Hui-Min Huang,et al.  Autonomy levels for unmanned systems (ALFUS) framework: safety and application issues , 2007, PerMIS.

[9]  M R Endsley,et al.  Level of automation effects on performance, situation awareness and workload in a dynamic control task. , 1999, Ergonomics.

[10]  J. Gaspar,et al.  CONTROL OF UNICYCLE TYPE ROBOTS Tracking, Path Following and Point Stabilization , 2008 .

[11]  Patricia L. McDermott,et al.  An investigation of real world control of robotic assets under communication latency , 2006, HRI '06.

[12]  P. Brockhoff,et al.  lmerTest: Tests for random and fixed effects for linear mixed effect models (lmer objects of lme4 package) , 2014 .

[13]  Kaleb McDowell,et al.  The Effects of Time Lag on Driving Performance and a Possible Mitigation , 2010, IEEE Transactions on Robotics.

[14]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[15]  Jing-Sin Liu,et al.  Collision-free curvature-bounded smooth path planning using composite Bezier curve based on Voronoi diagram , 2009, 2009 IEEE International Symposium on Computational Intelligence in Robotics and Automation - (CIRA).

[16]  Mo-Yuen Chow,et al.  Control gain adaptation in virtual reality mediated human–telerobot interaction , 2005 .

[17]  Thomas B. Sheridan,et al.  Remote Manipulative Control with Transmission Delay , 1963 .

[18]  Robin Deits,et al.  Computing Large Convex Regions of Obstacle-Free Space Through Semidefinite Programming , 2014, WAFR.

[19]  Ilya Kolmanovsky,et al.  Model predictive control and Extended Command Governor for improving robustness of relative motion guidance and control , 2014 .

[20]  Douglas Guimarães Macharet,et al.  A collaborative control system for telepresence robots , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Siddhartha S. Srinivasa,et al.  A policy-blending formalism for shared control , 2013, Int. J. Robotics Res..

[22]  Taylor R. George,et al.  A Real-Time, Interactive Simulation Environment for Unmanned Ground Vehicles: The Autonomous Navigation Virtual Environment Laboratory (ANVEL) , 2012, 2012 Fifth International Conference on Information and Computing Science.

[23]  Michael A. Goodrich,et al.  Ecological Interfaces for Improving Mobile Robot Teleoperation , 2007, IEEE Transactions on Robotics.

[24]  O. Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[25]  Dawn M. Tilbury,et al.  Equating user performance among communication latency distributions and simulation fidelities for a teleoperated mobile robot , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Gregory D. Hager,et al.  Semi-autonomous telerobotic assembly over high-latency networks , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  Michael A. Goodrich,et al.  Characterizing efficiency of human robot interaction: a case study of shared-control teleoperation , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Sterling J. Anderson,et al.  Experimental Performance Analysis of a Homotopy-Based Shared Autonomy Framework , 2014, IEEE Transactions on Human-Machine Systems.

[29]  François Michaud,et al.  Egocentric and exocentric teleoperation interface using real-time, 3D video projection , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[30]  Wayne Book,et al.  Blended Shared Control of Zermelo's navigation problem , 2010, Proceedings of the 2010 American Control Conference.

[31]  Ruzena Bajcsy,et al.  Semiautonomous Vehicular Control Using Driver Modeling , 2014, IEEE Transactions on Intelligent Transportation Systems.

[32]  Birsen Donmez,et al.  The Effects of Predictive Displays on Performance in Driving Tasks with Multi-Second Latency , 2013 .

[33]  Robert Meyers,et al.  Real-time photorealistic virtualized reality interface for remote mobile robot control , 2010, ISRR.

[34]  Jason S. Metcalfe,et al.  Field testing of tele-operation versus shared and traded control for military assets: an evaluation involving real-time embedded simulation and soldier assessment , 2010, Defense + Commercial Sensing.

[35]  Steven Eric Vozar,et al.  A Framework for Improving the Speed and Performance of Teleoperated Mobile Manipulators. , 2013 .

[36]  Stephen P. Boyd,et al.  CVXGEN: a code generator for embedded convex optimization , 2011, Optimization and Engineering.

[37]  Kris K. Hauser,et al.  Recognition, prediction, and planning for assisted teleoperation of freeform tasks , 2012, Autonomous Robots.

[38]  Michael A. Goodrich,et al.  Experiments in adjustable autonomy , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[39]  Yoram Koren,et al.  The vector field histogram-fast obstacle avoidance for mobile robots , 1991, IEEE Trans. Robotics Autom..

[40]  Tulga Ersal,et al.  A Multi-Stage Optimization Formulation for MPC-Based Obstacle Avoidance in Autonomous Vehicles Using a LIDAR Sensor , 2014 .

[41]  Tulga Ersal,et al.  An MPC Algorithm With Combined Speed and Steering Control for Obstacle Avoidance in Autonomous Ground Vehicles , 2015 .

[42]  J. Christian Gerdes,et al.  Incorporating non-linear tire dynamics into a convex approach to shared steering control , 2014, 2014 American Control Conference.

[43]  Franz Aurenhammer,et al.  Voronoi diagrams—a survey of a fundamental geometric data structure , 1991, CSUR.

[44]  Pamela A. Savage-Knepshield,et al.  Designing Soldier Systems: Current Issues in Human Factors , 2013 .

[45]  Osamu Takahashi,et al.  Motion planning in a plane using generalized Voronoi diagrams , 1989, IEEE Trans. Robotics Autom..

[46]  Siddhartha S. Srinivasa,et al.  Shared Autonomy via Hindsight Optimization , 2015, Robotics: Science and Systems.

[47]  Magnus Egerstedt,et al.  Less Is More: Mixed-Initiative Model-Predictive Control With Human Inputs , 2013, IEEE Transactions on Robotics.