A Cognitive Systems Engineering Evaluation of a Tool to Aid Imagery Analysts

A cognitive systems engineering evaluation of an imagery analysis system was conducted to capture baseline performance and workload and compare it to performance with advanced filtering capabilities. Experienced Imagery Analysts searched for and annotated targets of interest in full-motion video in Army- relevant scenarios. Measures of performance included percent of primary targets found, time to find primary target, total targets found, and interactions with the system (via mouse clicks). Performance metrics were augmented with continuous physiological and behavioral measurements in order to capture more accurate cognitive state fluctuations during human-system interaction. The findings suggest that in time-pressured situations, analysts were able to identify more targets with the advanced filter capabilities than in the baseline condition. The findings were used to suggest specific design changes to address workflow deficiencies. The study also developed and implemented a multi-aspect approach to estimate operator functional state during system evaluation.

[1]  Glenn F. Wilson,et al.  An Analysis of Mental Workload in Pilots During Flight Using Multiple Psychophysiological Measures , 2002 .

[2]  David F. Dinges,et al.  Microcomputer analyses of performance on a portable, simple visual RT task during sustained operations , 1985 .

[3]  Tzyy-Ping Jung,et al.  2010 Neuroscience Director's Strategic Initiative , 2011 .

[4]  Qiang Ji,et al.  Cross-subject workload classification with a hierarchical Bayes model , 2012, NeuroImage.

[5]  Glenn F. Wilson,et al.  Performance Enhancement in an Uninhabited Air Vehicle Task Using Psychophysiologically Determined Adaptive Aiding , 2007, Hum. Factors.

[6]  Chris Berka,et al.  Drowsiness/alertness algorithm development and validation using synchronized EEG and cognitive performance to individualize a generalized model , 2011, Biological Psychology.

[7]  Ulf Ahlstrom,et al.  Using eye movement activity as a correlate of cognitive workload , 2006 .

[8]  S. Makeig,et al.  Lapses in alertness: coherence of fluctuations in performance and EEG spectrum. , 1993, Electroencephalography and clinical neurophysiology.

[9]  D. Dinges,et al.  EVALUATION OF TECHNIQUES FOR OCULAR MEASUREMENT AS AN INDEX OF FATIGUE AND THE BASIS FOR ALERTNESS MANAGEMENT , 1998 .

[10]  Robert J. K. Jacob,et al.  Using fNIRS brain sensing to evaluate information visualization interfaces , 2013, CHI.

[11]  Chris Berka,et al.  Eeg-Derived Estimators of Present and Future Cognitive Performance , 2011, Front. Hum. Neurosci..

[12]  Michelle N. Lumicao,et al.  EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. , 2007, Aviation, space, and environmental medicine.