Human–Agent Teaming for Multirobot Control: A Review of Human Factors Issues

The human factors literature on intelligent systems was reviewed in relation to the following: efficient human supervision of multiple robots, appropriate human trust in the automated systems, maintenance of human operator's situation awareness, individual differences in human-agent (H-A) interaction, and retention of human decision authority. A number of approaches-from flexible automation to autonomous agents-were reviewed, and their advantages and disadvantages were discussed. In addition, two key human performance issues (trust and situation awareness) related to H-A teaming for multirobot control and some promising user interface design solutions to address these issues were discussed. Some major individual differences factors (operator spatial ability, attentional control ability, and gaming experience) were identified that may impact H-A teaming in the context of robotics control.

[1]  Jinwen Ma,et al.  Rank sum method for related gene selection and its application to tumor diagnosis , 2004 .

[2]  Alberto Valero-Gomez,et al.  Impact of Two Adjustable-Autonomy Models on the Scalability of Single-Human/Multiple-Robot Teams for Exploration Missions , 2011, Hum. Factors.

[3]  Julie Shah,et al.  Human-Robot Teaming using Shared Mental Models , 2012 .

[4]  P W Singer.  War of the machines. , 2010, Scientific American.

[5]  Kezhi Mao,et al.  Feature subset selection for support vector machines through discriminative function pruning analysis , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  James H. Martin,et al.  Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition , 2000 .

[7]  David J. Musliner,et al.  Delegating to Automation , 2011 .

[8]  Cees J. H. Midden,et al.  The effects of errors on system trust, self-confidence, and the allocation of control in route planning , 2003, Int. J. Hum. Comput. Stud..

[9]  John Yen,et al.  Human–Agent Collaboration for Time-Stressed Multicontext Decision Making , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[10]  Jacob W. Crandall,et al.  Attention Allocation Efficiency in Human-UV Teams , 2007 .

[11]  Peter-Paul van Maanen,et al.  A framework for explaining reliance on decision aids , 2013, Int. J. Hum. Comput. Stud..

[12]  Jessie Y C Chen,et al.  UAV-guided navigation for ground robot tele-operation in a military reconnaissance environment , 2010, Ergonomics.

[13]  Raj M. Ratwani,et al.  Using Peripheral Processing and Spatial Memory to Facilitate Task Resumption , 2007 .

[14]  Mary L. Cummings,et al.  Design and Evaluation of Path Planning Decision Support for Planetary Surface Exploration , 2008, J. Aerosp. Comput. Inf. Commun..

[15]  Christian B. Carstens,et al.  Intuitive Speech-based Robotic Control , 2010 .

[16]  Philippe Besse,et al.  Sparse PLS discriminant analysis: biologically relevant feature selection and graphical displays for multiclass problems , 2011, BMC Bioinformatics.

[17]  Jessie Y. C. Chen,et al.  Human Performance Issues and User Interface Design for Teleoperated Robots , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[18]  Colin G. Drury,et al.  Foundations for an Empirically Determined Scale of Trust in Automated Systems , 2000 .

[19]  Nadine B. Sarter,et al.  Supporting Trust Calibration and the Effective Use of Decision Aids by Presenting Dynamic System Confidence Information , 2006, Hum. Factors.

[20]  Mark W. Wiggins,et al.  Polychronicity and Information Acquisition in Pilot Decision Making , 2008 .

[21]  Stacey D. Scott,et al.  Assisting Interruption Recovery in Supervisory Control of Multiple Uavs , 2006 .

[22]  Linda Onnasch,et al.  Human Performance Consequences of Automated Decision Aids , 2012 .

[23]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[24]  G. Wahba,et al.  Multicategory Support Vector Machines , Theory , and Application to the Classification of Microarray Data and Satellite Radiance Data , 2004 .

[25]  L. J. Wei,et al.  Asymptotic Conservativeness and Efficiency of Kruskal-Wallis Test for K Dependent Samples , 1981 .

[26]  Michael A. Goodrich,et al.  On using mixed-initiative control: A perspective for managing large-scale robotic teams , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  Stephanie Guerlain.  Using the Critiquing Approach to Cope with Brittle Expert Systems , 1995 .

[28]  Brett Browning,et al.  xBots: An approach to generating and executing optimal multi-robot plans with cross-schedule dependencies , 2012, 2012 IEEE International Conference on Robotics and Automation.

[29]  Gary Klein,et al.  A Naturalistic Study of Insight , 2011 .

[30]  Christopher D. Wickens,et al.  Factors that Mediate Flight Plan Monitoring and Errors in Plan Revision: Planning Under Automated and High Workload Conditions , 2003 .

[31]  S. Monsell.  Task switching , 2003, Trends in Cognitive Sciences.

[32]  D. Meyer,et al.  Executive control of cognitive processes in task switching. , 2001, Journal of experimental psychology. Human perception and performance.

[33]  Marian Stewart Bartlett,et al.  Face recognition by independent component analysis , 2002, IEEE Trans. Neural Networks.

[34]  Catherine Neubauer,et al.  Fatigue and Voluntary Utilization of Automation in Simulated Driving , 2012, Hum. Factors.

[35]  J. Gregory Trafton,et al.  ACT-R/E , 2013, HRI 2013.

[36]  Carryl L. Baldwin,et al.  Individual Differences in Route-Learning Strategy and Associated Working Memory Resources , 2009, Hum. Factors.

[37]  Michael C. Dorneich,et al.  Towards a Characterization of Adaptive Systems: a Framework for Researchers and System Designers , 2017 .

[38]  Christopher A. Miller,et al.  Trust and etiquette in high-criticality automated systems , 2004, CACM.

[39]  Joel S. Warm,et al.  Vigilance Requires Hard Mental Work and Is Stressful , 2008, Hum. Factors.

[40]  Anand S. Rao,et al.  BDI Agents: From Theory to Practice , 1995, ICMAS.

[41]  Guo-Zheng Li,et al.  Feature Selection for Partial Least Square Based Dimension Reduction , 2009, Foundations of Computational Intelligence.

[42]  Jessie Y.C. Chen,et al.  Effects of imperfect automation and individual differences on concurrent performance of military and robotics tasks in a simulated multitasking environment , 2009, Ergonomics.

[43]  Christopher D. Wickens,et al.  Pilots' Monitoring Strategies and Performance on Automated Flight Decks: An Empirical Study Combining Behavioral and Eye-Tracking Data , 2007, Hum. Factors.

[44]  Emilie Roth,et al.  Human in the Loop Evaluation of a Mixed-Initiative System for Planning and Control of Multiple UAV Teams , 2004 .

[45]  Hong Yan,et al.  Feature Extraction and Uncorrelated Discriminant Analysis for High-Dimensional Data , 2008, IEEE Transactions on Knowledge and Data Engineering.

[46]  D. Kahneman,et al.  Relation of a test of attention to road accidents. , 1973 .

[47]  David D. Woods,et al.  Envisioning human-robot coordination in future operations , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[48]  Raja Parasuraman,et al.  Effects of Imperfect Automation on Decision Making in a Simulated Command and Control Task , 2007, Hum. Factors.

[49]  L. F. Barrett,et al.  Individual differences in working memory capacity and dual-process theories of the mind. , 2004, Psychological bulletin.

[50]  Naotaka Fujii,et al.  Higher Order Partial Least Squares (HOPLS): A Generalized Multilinear Regression Method , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Linda G. Pierce,et al.  Automation Usage Decisions: Controlling Intent and Appraisal Errors in a Target Detection Task , 2007, Hum. Factors.

[52]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[53]  Raja Parasuraman,et al.  Adaptive Automation for Human-Robot Teaming in Future Command and Control Systems , 2007 .

[54]  Leslie S. Smith,et al.  Feature subset selection in large dimensionality domains , 2010, Pattern Recognit..

[55]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[56]  Santosh Mathan,et al.  Considering Etiquette in the Design of an Adaptive System , 2012 .

[57]  Matthias Scheutz,et al.  Affective Goal and Task Selection for Social Robots , 2009 .

[58]  Zijiang Yang,et al.  Using partial least squares and support vector machines for bankruptcy prediction , 2011, Expert Syst. Appl..

[59]  Joseph B. Lyons,et al.  Human–Human Reliance in the Context of Automation , 2012, Hum. Factors.

[60]  J. H. Davis,et al.  An Integrative Model Of Organizational Trust , 1995 .

[61]  R Parasuraman,et al.  Effects of automation and task load on task switching during human supervision of multiple semi-autonomous robots in a dynamic environment , 2010, Ergonomics.

[62]  Alexander I. Rudnicky,et al.  Exploring Spoken Dialog Interaction in Human-Robot Teams , 2009 .

[63]  Mark H. Draper,et al.  11. Multi-Sensory Interfaces for Remotely Operated Vehicles , 2006 .

[64]  Paul Wintz,et al.  Digital image processing (2nd ed.) , 1987 .

[65]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[66]  Gheorghe Tecuci,et al.  Seven Aspects of Mixed-Initiative Reasoning: An Introduction to this Special Issue on Mixed-Initiative Assistants , 2007, AI Mag..

[67]  Christopher D. Wickens,et al.  ANALYSIS OF PILOTS' MONITORING AND PERFORMANCE ON AN AUTOMATED FLIGHT DECK , 2001 .

[68]  Stephen Rice,et al.  System-Wide versus Component-Specific Trust Using Multiple Aids , 2009, The Journal of general psychology.

[69]  Jianqing Fan,et al.  High Dimensional Classification Using Features Annealed Independence Rules. , 2007, Annals of statistics.

[70]  F. Freeman,et al.  A Closed-Loop System for Examining Psychophysiological Measures for Adaptive Task Allocation , 2000, The International journal of aviation psychology.

[71]  Raja Parasuraman,et al.  Performance Consequences of Automation-Induced 'Complacency' , 1993 .

[72]  Eugene Santos,et al.  Intent-Driven Insider Threat Detection in Intelligence Analyses , 2008, 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[73]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  John D. Lee,et al.  Human-Automation Collaboration in Dynamic Mission Planning: A Challenge Requiring an Ecological Approach , 2006 .

[75]  Michael A. Goodrich,et al.  Task Switching and Multi-Robot Teams , 2005 .

[76]  Chaomei Chen,et al.  Individual differences in virtual environments-introduction and overview , 2000 .

[77]  Harvey S. Smallman,et al.  Staying Up to Speed: Four Design Principles for Maintaining and Recovering Situation Awareness , 2008 .

[78]  Caroline C. Hayes,et al.  Uncertainty Visualizations , 2012 .

[79]  Mark R. Wilson,et al.  Gaze training enhances laparoscopic technical skill acquisition and multi-tasking performance: a randomized, controlled study , 2011, Surgical Endoscopy.

[80]  Andrew S Clare,et al.  Modeling real-time human-automation collaborative scheduling of unmanned vehicles , 2013 .

[81]  Zijiang Yang,et al.  PLS-Based Gene Selection and Identification of Tumor-Specific Genes , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[82]  Raja Parasuraman,et al.  Adaptable and Adaptive Automation for Supervisory Control of Multiple Autonomous Vehicles , 2012 .

[83]  Maia B. Cook,et al.  Human Factors of the Confirmation Bias in Intelligence Analysis: Decision Support From Graphical Evidence Landscapes , 2008, Hum. Factors.

[84]  Raja Parasuraman,et al.  Adaptive Automation for Human Supervision of Multiple Uninhabited Vehicles: Effects on Change Detection, Situation Awareness, and Mental Workload , 2009 .

[85]  J Y C Chen,et al.  Effects of tactile cueing on concurrent performance of military and robotics tasks in a simulated multitasking environment , 2008, Ergonomics.

[86]  Anne-Laure Boulesteix,et al.  Partial least squares: a versatile tool for the analysis of high-dimensional genomic data , 2006, Briefings Bioinform..

[87]  Takayuki Kanda,et al.  Do people hold a humanoid robot morally accountable for the harm it causes? , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[88]  Jason Weston,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2002, Machine Learning.

[89]  Maya Cakmak,et al.  Designing robot learners that ask good questions , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[90]  Kristin P. Bennett,et al.  Multicategory Classification by Support Vector Machines , 1999, Comput. Optim. Appl..

[91]  Philip N Johnson-Laird,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:Mental models and human reasoning , 2010 .

[92]  Jessie Y C Chen,et al.  Supervisory control of multiple robots in dynamic tasking environments , 2012, Ergonomics.

[93]  Kim-Anh Lê Cao,et al.  Multiclass classification and gene selection with a stochastic algorithm , 2009, Comput. Stat. Data Anal..

[94]  Mary L. Cummings,et al.  The Need for Command and Control Instant Message Adaptive Interfaces: Lessons Learned from Tactical Tomahawk Human-in-the-Loop Simulations , 2004, Cyberpsychology Behav. Soc. Netw..

[95]  Arthur D. Fisk,et al.  Understanding the Effect of Workload on Automation Use for Younger and Older Adults , 2011, Hum. Factors.

[96]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[97]  Johnny E. Triplett.  The Effects of Commercial Video Game Playing: A Comparison of Skills and Abilities for the Predator UAV , 2012 .

[98]  Jessie Y. C. Chen,et al.  Supervisory Control of Multiple Robots , 2012, Hum. Factors.

[99]  Rosemarie E. Yagoda,et al.  Improvements in robot navigation through operator speech preferences , 2012, Paladyn J. Behav. Robotics.

[100]  Ron Shamir,et al.  SlimPLS: A Method for Feature Selection in Gene Expression-Based Disease Classification , 2009, PloS one.

[101]  Keith S. Jones,et al.  Human-Robot Interaction Toward Usable Personal Service Robots , 2011 .

[102]  Matthias Scheutz,et al.  Multi-modal Belief Updates in Multi-Robot Human-Robot Dialogue Interactions , 2012 .

[103]  Salim Hariri,et al.  A new dependency and correlation analysis for features , 2005, IEEE Transactions on Knowledge and Data Engineering.

[104]  Francisco Herrera,et al.  An overview of ensemble methods for binary classifiers in multi-class problems: Experimental study on one-vs-one and one-vs-all schemes , 2011, Pattern Recognit..

[105]  F. Tong,et al.  Training Improves Multitasking Performance by Increasing the Speed of Information Processing in Human Prefrontal Cortex , 2009, Neuron.

[106]  Matthew R. Walter,et al.  Approaching the Symbol Grounding Problem with Probabilistic Graphical Models , 2011, AI Mag..

[107]  Xin Zhou,et al.  MSVM-RFE: extensions of SVM-RFE for multiclass gene selection on DNA microarray data , 2007, Bioinform..

[108]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[109]  T. Inagaki,et al.  Smart collaboration between humans and machines based on mutual understanding , 2008, Annu. Rev. Control..

[110]  D. Derryberry,et al.  Anxiety-related attentional biases and their regulation by attentional control. , 2002, Journal of abnormal psychology.

[111]  Patrick Tan,et al.  Genetic algorithms applied to multi-class prediction for the analysis of gene expression data , 2003, Bioinform..

[112]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[113]  Paula J. Durlach,et al.  Human–Robot Interaction in the Context of Simulated Route Reconnaissance Missions , 2008 .

[114]  Gary Klein,et al.  Naturalistic Decision Making , 2008, Hum. Factors.

[115]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[116]  Prasanna Velagapudi,et al.  Choosing Autonomy Modes for Multirobot Search , 2010, Hum. Factors.

[117]  J. G. Holmes,et al.  Trust in close relationships. , 1985 .

[118]  Tyler H. Shaw,et al.  Individual differences in vigilance: Personality, ability and states of stress , 2010 .

[119]  Ping Xue,et al.  Controlled English for Effective Communication during Coalition Operations , 2013 .

[120]  S. D. Jong.  SIMPLS: an alternative approach to partial least squares regression , 1993 .

[121]  J. Gregory Trafton,et al.  Human control of multiple unmanned vehicles: effects of interface type on execution and task switching times , 2006, HRI '06.

[122]  James L. Szalma,et al.  Individual differences in human–technology interaction: incorporating variation in human characteristics into human factors and ergonomics research and design , 2009 .

[123]  Raja Parasuraman,et al.  Humans and Automation: Use, Misuse, Disuse, Abuse , 1997, Hum. Factors.

[124]  Paul Scerri,et al.  Transitioning multiagent technology to UAV applications , 2008, AAMAS.

[125]  Jennifer M. Glass,et al.  Virtually Perfect Time Sharing in Dual-Task Performance: Uncorking the Central Cognitive Bottleneck , 2001, Psychological science.

[126]  Greg A. Jamieson,et al.  The impact of context-related reliability on automation failure detection and scanning behaviour , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[127]  Mohamed A. Deriche,et al.  A new mutual information based measure for feature selection , 2003, Intell. Data Anal..

[128]  Jagath C. Rajapakse,et al.  One-Versus-One and One-Versus-All Multiclass SVM-RFE for Gene Selection in Cancer Classification , 2007, EvoBIO.

[129]  Huan Liu,et al.  Efficient Feature Selection via Analysis of Relevance and Redundancy , 2004, J. Mach. Learn. Res..

[130]  Jonathan P. How,et al.  Operator Object Function Guidance for a Real-Time Unmanned Vehicle Scheduling Algorithm , 2012, J. Aerosp. Comput. Inf. Commun..

[131]  Ann M. Bisantz,et al.  The impact of cognitive feedback on judgment performance and trust with decision aids , 2008 .

[132]  C. S. Green,et al.  Enumeration versus multiple object tracking: the case of action video game players , 2006, Cognition.

[133]  James F. Allen,et al.  Mixed-Initiative Systems for Collaborative Problem Solving , 2007, AI Mag..

[134]  Yang Jiang,et al.  Individual differences in cognition, affect, and performance: Behavioral, neuroimaging, and molecular genetic approaches , 2012, NeuroImage.

[135]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[136]  D. Wiegmann,et al.  Similarities and differences between human–human and human–automation trust: an integrative review , 2007 .

[137]  L. G. Weiss,et al.  Autonomous robots in the fog of war , 2011, IEEE Spectrum.

[138]  Dimitrios Gunopulos,et al.  Locally Adaptive Metric Nearest-Neighbor Classification , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[139]  Eugene Santos,et al.  Intelligence Analyses and the Insider Threat , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[140]  Thomas Fincannon,et al.  Beyond “spatial ability”: Examining the impact of multiple individual differences in a perception by Proxy framework , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[141]  Jaap Ham,et al.  Trust in Smart Systems , 2012, Hum. Factors.

[142]  Matthew R. Walter,et al.  Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation , 2011, AAAI.

[143]  S Lewandowsky,et al.  The dynamics of trust: comparing humans to automation. , 2000, Journal of experimental psychology. Applied.

[144]  Raj M. Ratwani,et al.  An Eye Movement Analysis of the Effect of Interruption Modality on Primary Task Resumption , 2010, Hum. Factors.

[145]  A. Boulesteix.  PLS Dimension Reduction for Classification with Microarray Data , 2004, Statistical applications in genetics and molecular biology.

[146]  J. Geoffrey Chase,et al.  Human-Robot Collaboration: A Literature Review and Augmented Reality Approach in Design , 2008 .

[147]  Jessie Y. C. Chen,et al.  A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction , 2011, Hum. Factors.

[148]  Mary L. Cummings,et al.  The Role of Human-Automation Consensus in Multiple Unmanned Vehicle Scheduling , 2010, Hum. Factors.

[149]  Jiawei Han,et al.  Cancer classification using gene expression data , 2003, Inf. Syst..

[150]  Nicholas Roy,et al.  Human-automated path planning optimization and decision support , 2012, Int. J. Hum. Comput. Stud..

[151]  Jean Scholtz,et al.  Theory and evaluation of human robot interactions , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[152]  Linda G. Pierce,et al.  Predicting Misuse and Disuse of Combat Identification Systems , 2001 .

[153]  Jessie Y. C. Chen,et al.  Supervisory Control of Multiple Robots: Human-Performance Issues and User-Interface Design , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[154]  Mary Czerwinski,et al.  Selected Human Factors Issues in Information Visualization , 2009 .

[155]  William S. Rayens,et al.  PLS and dimension reduction for classification , 2007, Comput. Stat..

[156]  Mark S. Nixon,et al.  Gait Feature Subset Selection by Mutual Information , 2007, 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems.

[157]  Anna S. Law,et al.  Multitasking: multiple, domain-specific cognitive functions in a virtual environment , 2011, Memory & cognition.

[158]  Justin G. Hollands,et al.  Beyond Identity: Incorporating System Reliability Information Into an Automated Combat Identification System , 2011, Hum. Factors.

[159]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[160]  Leo Gugerty,et al.  Effects of Electronic Map Displays and Individual Differences in Ability on Navigation Performance , 2012, Hum. Factors.

[161]  MengChu Zhou,et al.  Optimizing Operator–Agent Interaction in Intelligent Adaptive Interface Design: A Conceptual Framework , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[162]  Florian Jentsch,et al.  Human-Robot Interactions in Future Military Operations , 2010 .

[163]  M. Cummings,et al.  Human-Automation Collaboration in Complex Multivariate Resource Allocation Decision Support Systems , 2010 .

[164]  Alexander I. Rudnicky,et al.  Instruction Taking in the TeamTalk System , 2010, AAAI Fall Symposium: Dialog with Robots.

[165]  M. Barker,et al.  Partial least squares for discrimination , 2003 .

[166]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[167]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[168]  Grant S. Taylor,et al.  Individual differences in response to automation: the five factor model of personality. , 2011, Journal of experimental psychology. Applied.

[169]  Arthur D. Fisk,et al.  Age-Related Differences in Reliance Behavior Attributable to Costs Within a Human-Decision Aid System , 2008, Hum. Factors.

[170]  David B. Kaber,et al.  Situation awareness implications of adaptive automation for information processing in an air traffic control-related task , 2006 .

[171]  Lei Liu,et al.  Feature selection with dynamic mutual information , 2009, Pattern Recognit..

[172]  Maaike Harbers Delft.  Enhancing Team Performance through Effective Communication , 2012 .

[173]  Huiyang Li,et al.  Human Performance Consequences of Stages and Levels of Automation , 2014, Hum. Factors.

[174]  Tara A. Rench,et al.  Predictors of multitasking performance in a synthetic work paradigm , 2010 .

[175]  Tao Jiang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[176]  Bobbie D. Seppelt,et al.  Making adaptive cruise control (ACC) limits visible , 2007, Int. J. Hum. Comput. Stud..

[177]  Andrew S. Clare,et al.  Modeling the Impact of Operator Trust on Performance in Multiple Robot Control , 2013, AAAI Spring Symposium: Trust and Autonomous Systems.

[178]  Danh V. Nguyen,et al.  Multi-class cancer classification via partial least squares with gene expression profiles , 2002, Bioinform..

[179]  Jerry M Crutchfield,et al.  Individual differences in working memory capacity predict visual attention allocation , 2003, Psychonomic bulletin & review.

[180]  C. A. Murthy,et al.  Multiscale Classification Using Nearest Neighbor Density Estimates , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[181]  Anthony Stentz,et al.  Integrating Perception and Cognition for AGI , 2011, AGI.

[182]  R Andy McKinley,et al.  Operator selection for unmanned aerial systems: comparing video game players and pilots. , 2011, Aviation, space, and environmental medicine.

[183]  Nancy J. Cooke,et al.  The synthetic teammate project , 2009, Comput. Math. Organ. Theory.

[184]  Sebastian Clauss,et al.  Design and Evaluation of a UAS combining Cognitive Automation and Optimal Control , 2012, Infotech@Aerospace.

[185]  John D. Lee,et al.  Trust, self-confidence, and operators' adaptation to automation , 1994, Int. J. Hum. Comput. Stud..

[186]  John D. Lee,et al.  Review of a Pivotal Human Factors Article: “Humans and Automation: Use, Misuse, Disuse, Abuse” , 2008, Hum. Factors.

[187]  Shuicheng Yan,et al.  Correntropy based feature selection using binary projection , 2011, Pattern Recognit..

[188]  Daniel R. Ilgen,et al.  Not All Trust Is Created Equal: Dispositional and History-Based Trust in Human-Automation Interactions , 2008, Hum. Factors.

[189]  Stephanie M. Merritt.  Affective Processes in Human–Automation Interactions , 2011, Hum. Factors.

[190]  Jeffrey M. Bradshaw,et al.  Trust in Automation , 2013, IEEE Intelligent Systems.

[191]  Michael J. Barnes and A. William Evans.  Soldier-Robot Teams in Future Battlefields: An Overview , 2016 .

[192]  Richard Weber,et al.  A wrapper method for feature selection using Support Vector Machines , 2009, Inf. Sci..

[193]  Robert G. Abbott,et al.  Individual Differences in Multitasking Ability and Adaptability , 2012, Hum. Factors.

[194]  Thomas B. Sheridan,et al.  DRAFT COPY 1 CH XX : Human Supervisory Control Challenges in Network Centric Operations , 2010 .

[195]  G Salvendy,et al.  Information visualization; assisting low spatial individuals with information access tasks through the use of visual mediators. , 1995, Ergonomics.

[196]  Peter A. Hancock,et al.  Design Principles for Adaptive Automation and Aiding , 2009 .

[197]  Larry A. Rendell,et al.  The Feature Selection Problem: Traditional Methods and a New Algorithm , 1992, AAAI.

[198]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[199]  Paul H. Morris,et al.  Mixed-Initiative Planning in Space Mission Operations , 2007, AI Mag..

[200]  Raja Parasuraman,et al.  Designing for Flexible Interaction Between Humans and Automation: Delegation Interfaces for Supervisory Control , 2007, Hum. Factors.

[201]  Prasanna Velagapudi,et al.  How search and its subtasks scale in N robots , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[202]  Ronald C. Arkin,et al.  Overriding Ethical Constraints in Lethal Autonomous Systems , 2012 .

[203]  I. Spence,et al.  Video Games and Spatial Cognition , 2010 .

[204]  Ellen M Purdy.  The Increasing Role of Robots in National Security , 2008 .

[205]  David M. Rocke,et al.  Dimension Reduction for Classification with Gene Expression Microarray Data , 2006, Statistical applications in genetics and molecular biology.

[206]  Richard Pak,et al.  Decision support aids with anthropomorphic characteristics influence trust and performance in younger and older adults , 2012, Ergonomics.

[207]  Nadine B. Sarter,et al.  How in the World Did We Ever Get into That Mode? Mode Error and Awareness in Supervisory Control , 1995, Hum. Factors.

[208]  C. Nass,et al.  Machines and Mindlessness , 2000 .

[209]  Birsen Donmez,et al.  Supporting intelligent and trustworthy maritime path planning decisions , 2010, Int. J. Hum. Comput. Stud..

[210]  Andrea M Philipp,et al.  Control and interference in task switching--a review. , 2010, Psychological bulletin.

[211]  John E. Laird,et al.  Performance evaluation of declarative memory systems in Soar , 2011 .

[212]  Carla T. Joyner,et al.  Concurrent Performance of Gunner's and Robotics Operator's Tasks in a Multitasking Environment , 2009 .

[213]  Glenn F. Wilson,et al.  Operator Functional State Classification Using Multiple Psychophysiological Features in an Air Traffic Control Task , 2003, Hum. Factors.

[214]  Tyler H. Shaw,et al.  Predicting vigilance: A fresh look at an old problem , 2009, Ergonomics.

[215]  Teruko Mitamura,et al.  14. Controlled language for authoring and translation , 2003 .

[216]  Michael A. Goodrich,et al.  Validating human-robot interaction schemes in multitasking environments , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[217]  Corinna E. Lathan,et al.  The Effects of Operator Spatial Perception and Sensory Feedback on Human-Robot Teleoperation Performance , 2002, Presence: Teleoperators & Virtual Environments.

[218]  Dietrich Manzey,et al.  Human Performance Consequences of Automated Decision Aids in States of Sleep Loss , 2011, Hum. Factors.

[219]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[220]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[221]  S. Wold,et al.  PLS: Partial Least Squares Projections to Latent Structures , 1993 .

[222]  Raja Parasuraman,et al.  Complacency and Bias in Human Use of Automation: An Attentional Integration , 2010, Hum. Factors.

[223]  Cornelius J. König,et al.  Working Memory Dimensions as Differential Predictors of the Speed and Error Aspect of Multitasking Performance , 2006 .

[224]  Konstantinos Vougas,et al.  The Protein Profile of the Human Immature T-cell Line CCRF-CEM. , 2005, Cancer genomics & proteomics.

[225]  A.,et al.  Cognitive Engineering , 2008, Encyclopedia of GIS.

[226]  J. Downing,et al.  Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling. , 2002, Cancer cell.

[227]  Troy D. Kelley,et al.  The Effects of Communication Style on Robot Navigation Performance , 2009 .

[228]  Daphne Bavelier,et al.  Stretching the limits of visual attention: the case of action video games. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[229]  Robin R. Murphy,et al.  The Safe Human-Robot Ratio , 2016 .

[230]  Hiroshi Furukawa,et al.  Supporting System-Centered View of Operators Through Ecological Interface Design: Two Experiments on Human-Centered Automation , 2003 .

[231]  N A Stanton,et al.  What's skill got to do with it? Vehicle automation and driver mental workload , 2007, Ergonomics.

[232]  Raja Parasuraman,et al.  The World is not Enough: Trust in Cognitive Agents , 2012 .

[233]  Janice Langan-Fox,et al.  Human–automation teams and adaptable control for future air traffic management , 2009 .

[234]  D. Kahneman,et al.  Conditions for intuitive expertise: a failure to disagree. , 2009, The American psychologist.

[235]  Christian B. Carstens,et al.  Scalability of Robotic Controllers: An Evaluation of Controller Options , 2008 .

[236]  Zhihua Qu,et al.  RoboLeader for reconnaissance by a team of robotic vehicles , 2010, 2010 International Symposium on Collaborative Technologies and Systems.

[237]  Raja Parasuraman,et al.  Display Integration Enhances Information Sampling and Decision Making in Automated Fault Management in a Simulated Spaceflight Micro-World , 2002, Aviation, space, and environmental medicine.

[238]  Yiming Yang,et al.  Analysis of recursive gene selection approaches from microarray data , 2005, Bioinform..

[239]  Susmita Datta,et al.  Surrogate variable analysis using partial least squares (SVA-PLS) in gene expression studies , 2012, Bioinform..

[240]  Nancy J. Cooke,et al.  Measuring Team Knowledge , 2000, Hum. Factors.