Dynamic human-computer collaboration in real-time unmanned vehicle scheduling

Abstract : Advances in autonomy have made it possible to invert the operator-to-vehicle ratio so that a single operator can control multiple heterogeneous Unmanned Vehicles (UVs). This autonomy will reduce the need for the operator to manually control each vehicle, enabling the operator to focus on higher-level goal setting and decision-making. Computer optimization algorithms that can be used in UV path-planning and task allocation usually have an a priori coded objective function that only takes into account pre-determined variables with set weightings. Due to the complex, time-critical, and dynamic nature of command and control missions, brittleness due to a static objective function could cause higher workload as the operator manages the automation. Increased workload during critical decision-making could lead to lower system performance which, in turn, could result in a mission or life-critical failure. This research proposes a method of collaborative multiple UV control that enables operators to dynamically modify the weightings within the objective function of an automated planner during a mission. After a review of function allocation literature, an appropriate taxonomy was used to evaluate the likely impact of human interaction with a dynamic objective function. This analysis revealed a potential reduction in the number of cognitive steps required to evaluate and select a plan, by aligning the objectives of the operator with the automated planner. A multiple UV simulation testbed was modified to provide two types of dynamic objective functions.

[1]  Judith Orasanu,et al.  Design of a cooperative problem-solving system for enroute flight planning: An empirical study of its use by airline dispatchers , 1993 .

[2]  Mary L. Cummings,et al.  Developing Operator Capacity Estimates for Supervisory Control of Autonomous Vehicles , 2007, Hum. Factors.

[3]  Mary L. Cummings,et al.  Collaborative Human-Automation Decision Making , 2009, Handbook of Automation.

[4]  M. Cummings,et al.  Human-Automation Collaboration in Complex Multivariate Resource Allocation Decision Support Systems , 2010 .

[5]  Jens Rasmussen,et al.  Outlines of a Hybrid Model of the Process Plant Operator , 1976 .

[6]  S S Stevens,et al.  HUMAN ENGINEERING FOR AN EFFECTIVE AIR-NAVIGATION AND TRAFFIC-CONTROL SYSTEM, AND APPENDIXES 1 THRU 3 , 1951 .

[7]  Kenneth Munson,et al.  Jane's unmanned aerial vehicles and targets , 2006 .

[8]  The influence of aircraft proximity data on the subjective mental workload of controllers in the air traffic control task. , 1999, Ergonomics.

[9]  Birsen Donmez,et al.  Effects of Time Pressure on the Use of an Automated Decision Support System for Strike Planning , 2009 .

[10]  Joe Marks,et al.  Human-Guided Simple Search , 2000, AAAI/IAAI.

[11]  J.K. Hedrick,et al.  Border patrol and surveillance missions using multiple unmanned air vehicles , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[12]  C. Wickens Engineering psychology and human performance, 2nd ed. , 1992 .

[13]  Jur P. van den Berg,et al.  Anytime path planning and replanning in dynamic environments , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[14]  Harold E. Price,et al.  The Allocation of Functions in Systems , 1985 .

[15]  Han-Lim Choi,et al.  Real-Time Multi-UAV Task Assignment in Dynamic and Uncertain Environments , 2009 .

[16]  William S. Havens,et al.  A Framework for Designing and Evaluating Mixed-Initiative Optimization Systems , 2005 .

[17]  Jonathan P. How,et al.  Increasing autonomy of UAVs , 2009, IEEE Robotics & Automation Magazine.

[18]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[19]  Daniel Noel Southern,et al.  Human-guided management of collaborating unmanned vehicles in degraded communication environments , 2010 .

[20]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[21]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[22]  Kenneth Y. Goldberg,et al.  Hydra: A framework and algorithms for mixed-initiative UAV-assisted search and rescue , 2008, 2008 IEEE International Conference on Automation Science and Engineering.

[23]  Shahriar Keshmiri,et al.  The Meridian UAV Flight Performance Analysis Using Analytical and Experimental Data , 2009 .

[24]  Sidney W. A. Dekker,et al.  MABA-MABA or Abracadabra? Progress on Human–Automation Co-ordination , 2002, Cognition, Technology & Work.

[25]  Liling Ren,et al.  Interaction of Automation and Time Pressure in a Route Replanning Task , 2002 .

[26]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[27]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[28]  Anthony Stentz,et al.  Optimal and efficient path planning for partially-known environments , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[29]  M. L. Cummings,et al.  Collaborative Human-Computer Decision Making in Network Centric Warfare , 2005 .

[30]  David B. Kaber,et al.  Design of Automation for Telerobots and the Effect on Performance, Operator Situation Awareness, and Subjective Workload , 2000 .

[31]  Philip J. Smith,et al.  Design of a Cooperative Problem-Solving System for En-Route Flight Planning: An Empirical Evaluation , 1994 .

[32]  Philip J. Smith,et al.  Brittleness in the design of cooperative problem-solving systems: the effects on user performance , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[33]  Emilie Roth,et al.  Developing Mixed-Initiative Interaction with Intelligent Systems: Lessons Learned from Supervising Multiple UAVs , 2004 .

[34]  Stephanie Guerlain Using the Critiquing Approach to Cope with Brittle Expert Systems , 1995 .

[35]  Timothy W. McLain,et al.  Cooperative forest fire surveillance using a team of small unmanned air vehicles , 2006, Int. J. Syst. Sci..

[36]  Victor A. Riley,et al.  A General Model of Mixed-Initiative Human-Machine Systems , 1989 .

[37]  J. Anderson,et al.  Fundamentals of Aerodynamics , 1984 .

[38]  Mehmet Karamanoglu,et al.  Investigating the mobility of unmanned ground vehicles , 2009 .

[39]  Jens Rasmussen,et al.  Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[40]  Mark W. Powell,et al.  Mars Exploration Rover Operations with the Science Activity Planner , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[41]  Mary L. Cummings,et al.  Assessing Operator Workload and Performance in Expeditionary Multiple Unmanned Vehicle Control , 2010 .

[42]  Barry G. Silverman Human-Computer Collaboration , 1992, Hum. Comput. Interact..

[43]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[44]  John D. Lee,et al.  Trust in Automation: Designing for Appropriate Reliance , 2004, Hum. Factors.

[45]  Joseph L Thomer,et al.  Trust-Based Design of Human-Guided Algorithms , 2007 .

[46]  Thomas B. Sheridan,et al.  Function allocation: algorithm, alchemy or apostasy? , 2000, Int. J. Hum. Comput. Stud..

[47]  Jonathan P. How,et al.  Robust Decentralized Task Assignment for Cooperative UAVs , 2006 .

[48]  Jeffrey D. Anderson,et al.  Managing autonomy in robot teams: Observations from four experiments , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[49]  Mary L. Cummings,et al.  Automation Architecture for Single Operator, Multiple UAV Command and Control, , 2007 .

[50]  Mary L. Cummings,et al.  The Role of Human-Automation Consensus in Multiple Unmanned Vehicle Scheduling , 2010, Hum. Factors.

[51]  L. Darrell Whitley,et al.  Mixed Initiative Scheduling for the Air Force Satellite Control Network , 1999 .

[52]  Mary L. Cummings,et al.  Automation Bias in Intelligent Time Critical Decision Support Systems , 2004 .

[53]  J. Márquez,et al.  Introducing a Human-Automation Collaboration Taxonomy (HACT) in Command and Control Decision-Support Systems , 2007 .

[54]  Laurence A. Kramer,et al.  Optimizing for Change : Mixed-Initiative Resource Allocation with the AMC Barrel Allocator , 2002 .

[55]  Thomas B. Sheridan,et al.  Human and Computer Control of Undersea Teleoperators , 1978 .

[56]  Ulrich Hoffrage,et al.  Inferences under time pressure: how opportunity costs affect strategy selection. , 2008, Acta psychologica.

[57]  J. Gibson The Ecological Approach to Visual Perception , 1979 .

[58]  Chris W. Clegg,et al.  A critical assessment of task allocation methods and their applicability , 1997 .

[59]  Drew McDermott,et al.  Issues in the Development of Human-Computer Mixed Initiative Planning , 1994 .

[60]  J M Hoc,et al.  From human – machine interaction to human – machine cooperation , 2000, Ergonomics.

[61]  Heath A. Ruff,et al.  Human Interaction with Levels of Automation and Decision-Aid Fidelity in the Supervisory Control of Multiple Simulated Unmanned Air Vehicles , 2002, Presence: Teleoperators & Virtual Environments.

[62]  李幼升,et al.  Ph , 1989 .

[63]  Stacey D. Scott,et al.  Investigating human-computer optimization , 2002, CHI.

[64]  Robert P. Goldman,et al.  The Playbook™ Approach to Adaptive Automation , 2005 .

[65]  Sundaram Narayanan,et al.  Human-centered time-pressured decision making in dynamic complex systems , 2006 .

[66]  Mary L. Cummings,et al.  Predicting Controller Capacity in Supervisory Control of Multiple UAVs , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[67]  Jessica J. Márquez,et al.  Human-automation collaboration : decision support for lunar and planetary exploration , 2006 .

[68]  V. Barnett,et al.  Applied Linear Statistical Models , 1975 .

[69]  R Parasuraman,et al.  Designing automation for human use: empirical studies and quantitative models , 2000, Ergonomics.

[70]  J. Westwood Global Prospects for AUVs , 2001 .

[71]  Timothy W. Simpson,et al.  Evaluating the Performance of Visual Steering Commands for User-Guided Pareto Frontier Sampling During Trade Space Exploration , 2008, DAC 2008.

[72]  A. Tversky,et al.  Judgment under Uncertainty , 1982 .