Operator Choice Modeling for Collaborative UAV Visual Search Tasks

Unmanned aerial vehicles (UAVs) provide unprecedented access to imagery of possible ground targets of interest in real time. The availability of this imagery is expected to increase with envisaged future missions of one operator controlling multiple UAVs. This research investigates decision models that can be used to develop assistive decision support for UAV operators involved in these complex search missions. Previous human-in-the-loop experiments have shown that operator detection probabilities may decay with increased search time. Providing the operators with the ability to requeue difficult images with the option of relooking at targets later was hypothesized to help operators improve their search accuracy. However, it was not well understood how mission performance could be impacted by operators performing requeues with multiple UAVs. This work extends a queuing model of the human operator by developing a retrial queue model (ReQM) that mathematically describes the use of relooks. We use ReQM to generate performance predictions through discrete event simulation. We validate these predictions through a human-in-the-loop experiment that evaluates the impact of requeuing on a simulated multiple-UAV mission. Our results suggest that, while requeuing can improve detection accuracy and decrease mean search times, operators may need additional decision support to use relooks effectively.

[1]  Bong D. Choi,et al.  The M/G/1 retrial queue with bernoulli schedule , 1990, Queueing Syst. Theory Appl..

[2]  Carl Nehme,et al.  Modeling human supervisory control in heterogeneous unmanned vehicle systems , 2009 .

[3]  Christof Koch,et al.  Visual attention and target detection in cluttered natural scenes , 2001 .

[4]  J. Wolfe,et al.  What Can 1 Million Trials Tell Us About Visual Search? , 1998 .

[5]  K. Preston White,et al.  Systems engineering models of human-machine interaction , 1981, Proceedings of the IEEE.

[6]  Wynn C. Stirling Satisficing Games and Decision Making: With Applications to Engineering and Computer Science , 2003 .

[7]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[8]  R. Herrnstein Rational Choice Theory Necessary but Not Sufficient , 1990 .

[9]  Jesús R. Artalejo,et al.  Retrial Queueing Systems , 2008 .

[10]  Colin G. Drury,et al.  Sensitivity and validity of visual search models for multiple targets , 2002 .

[11]  Mary L. Cummings,et al.  The Role of Human-Automation Consensus in Multiple Unmanned Vehicle Scheduling , 2010, Hum. Factors.

[12]  Diana Schmidt A Queuing Analysis of the Air Traffic Controller''s Workload , 1978 .

[13]  Mary L. Cummings,et al.  Predicting Controller Capacity in Supervisory Control of Multiple UAVs , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[14]  M. Potter,et al.  A two-stage model for multiple target detection in rapid serial visual presentation. , 1995, Journal of experimental psychology. Human perception and performance.

[15]  S. Darbha,et al.  Optimal Sequential Inspection , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[16]  Arthur F Kramer,et al.  Visual Skills in Airport-Security Screening , 2004, Psychological science.

[17]  Emilio Frazzoli,et al.  Human-in-the-loop vehicle routing policies for dynamic environments , 2008, 2008 47th IEEE Conference on Decision and Control.

[18]  G Sperling,et al.  The attention operating characteristic: examples from visual search. , 1978, Science.

[19]  Meir Pachter,et al.  Stochastic Dynamic rogramming for Uncertainty Handling in UAV Operations , 2007, 2007 American Control Conference.

[20]  M. Loeb,et al.  The Psychology of Vigilance , 1982 .

[21]  Adele Diederich,et al.  Survey of decision field theory , 2002, Math. Soc. Sci..

[22]  David E. Irwin,et al.  Visual Search has Memory , 2001, Psychological science.

[23]  Jeffrey P. Kharoufeh,et al.  An M/M/1 retrial queue with unreliable server , 2006, Oper. Res. Lett..

[24]  J. Townsend,et al.  Fundamental derivations from decision field theory , 1992 .

[25]  J. Wolfe,et al.  Guided Search 2.0 A revised model of visual search , 1994, Psychonomic bulletin & review.

[26]  Evan M. Palmer,et al.  Reaction time distributions constrain models of visual search , 2010, Vision Research.

[27]  P. Verghese Visual Search and Attention A Signal Detection Theory Approach , 2001, Neuron.

[28]  K. Vahala Handbook of stochastic methods for physics, chemistry and the natural sciences , 1986, IEEE Journal of Quantum Electronics.

[29]  C. Drury,et al.  The Effect of Area, Density, and Number of Background Characters on Visual Search , 1978, Human factors.

[30]  Richard G. Swensson,et al.  The elusive tradeoff: Speed vs accuracy in visual discrimination tasks , 1972 .

[31]  Mary L. Cummings,et al.  Predicting Controller Capacity in Remote Supervision of Multiple Unmanned Vehicles , 2008 .

[32]  Gregory Hobson,et al.  Visual detection with search: an empirical model , 1991, IEEE Trans. Syst. Man Cybern..

[33]  Mary L. Cummings,et al.  Developing Operator Capacity Estimates for Supervisory Control of Autonomous Vehicles , 2007, Hum. Factors.

[34]  Mustapha Mouloua,et al.  Automation and Human Performance : Theory and Applications , 1996 .

[35]  Jeremy M Wolfe,et al.  What are the shapes of response time distributions in visual search? , 2011, Journal of experimental psychology. Human perception and performance.

[36]  M. G. King,et al.  Visual Search Processes in Camouflage Detection , 1984 .

[37]  H. Pashler,et al.  Attention capacity and task difficulty in visual search , 2005, Cognition.

[38]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[39]  Adele Diederich,et al.  Simple matrix methods for analyzing diffusion models of choice probability, choice response time, and simple response time , 2003 .

[40]  Jeremy M. Wolfe,et al.  26.5 brief comms NEW , 2005 .

[41]  M L Cummings,et al.  Choice modeling of relook tasks for UAV search missions , 2010, Proceedings of the 2010 American Control Conference.

[42]  Diederich,et al.  Dynamic Stochastic Models for Decision Making under Time Constraints , 1997, Journal of mathematical psychology.

[43]  Todd S. Horowitz,et al.  Visual search has no memory , 1998, Nature.

[44]  John M. Irvine Assessing target search performance: the free-response operator characteristic model , 2004 .