Enhancing Mitigation in Augmented Cognition

In augmented cognition (AugCog), mitigation strategies are used as real-time intervention techniques that are triggered by the outcome of cognitive state assessment and context to significantly improve human-systems performance. Yet, no common ground has been established regarding best practices and what aspects to consider during implementation. This paper discusses mitigation strategies currently used in AugCog systems and provides insights into their strengths and weaknesses. An event-based conceptual framework is introduced that aids real-time mitigation strategy selection by linking system events to real-time cognitive state indicators, which together determine when, what, and how to mitigate. Insights from the implementation of this framework in an AugCog system designed to optimize situation awareness are presented, which support the architecture of the framework and identify further challenges to mitigation. Future work should focus on further validating the proposed framework and leveraging techniques from other domains (e.g., film, theater) to create more effective mitigation concepts in AugCog systems.

[1]  Donald A. Norman,et al.  User Centered System Design: New Perspectives on Human-Computer Interaction , 1988 .

[2]  Mark W. Scerbo,et al.  Adaptive Automation , 2006, Neuroergonomics.

[3]  W. Karwowski.  International encyclopedia of ergonomics and human factors , 2001 .

[4]  D Gopher,et al.  Switching tasks and attention policies. , 2000, Journal of experimental psychology. General.

[5]  Michael Hildebrandt,et al.  Time Design , 2004 .

[6]  D. Kirsh.  A Few Thoughts on Cognitive Overload , 2000 .

[7]  Suzanne Watzman,et al.  Visual design principles for usable interfaces , 2002 .

[8]  Eduardo Salas,et al.  Making decisions under stress: Implications for individual and team training. , 1998 .

[9]  Anind K. Dey,et al.  Heuristic evaluation of ambient displays , 2003, CHI '03.

[10]  Donald P. Greenberg,et al.  Design and simulation of opera lighting and projection effects , 1991, SIGGRAPH.

[11]  Gavriel Salvendy,et al.  Review and reappraisal of adaptive interfaces: Toward biologically inspired paradigms , 2002 .

[12]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[13]  Jason Pascoe,et al.  Adding generic contextual capabilities to wearable computers , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[14]  D. Norman,et al.  Psychological Issues in Support of Multiple Activities , 1986 .

[15]  J. Jacko,et al.  The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications , 2002 .

[16]  M. Posner,et al.  Attention and the detection of signals. , 1980, Journal of experimental psychology.

[17]  H. Winter.  Intelligent Decision Aids for Human Operators. , 1997 .

[18]  Kay M. Stanney,et al.  Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations , 2004, IEEE Computer Graphics and Applications.

[19]  Gregory D. Abowd,et al.  Towards a Better Understanding of Context and Context-Awareness , 1999, HUC.

[20]  Daniel C. McFarlane,et al.  Comparison of Four Primary Methods for Coordinating the Interruption of People in Human-Computer Interaction , 2002, Hum. Comput. Interact..

[21]  K. Smith-Jentsch,et al.  Team dimensional training: A strategy for guided team self-correction. , 1998 .

[22]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[23]  James T. Townsend,et al.  A note on the identifiability of parallel and serial processes , 1971 .

[24]  Dylan M. Jones,et al.  Interactive multimedia for instruction: A cognitive analysis of the role of audition and vision , 1992, Int. J. Hum. Comput. Interact..

[25]  C D Wickens,et al.  Codes and Modalities in Multiple Resources: A Success and a Qualification , 1988, Human factors.

[26]  L. Barsalou,et al.  Verifying Different-Modality Properties for Concepts Produces Switching Costs , 2003, Psychological science.

[27]  Christopher D. Wickens,et al.  Attentional Filtering in the Design of Electronic Map Displays: A Comparison of Color Coding, Intensity Coding, and Decluttering Techniques , 2001, Hum. Factors.

[28]  Lena Mamykina,et al.  Time Aura: interfaces for pacing , 2001, CHI.

[29]  David Chandler,et al.  The''Grammar''of Television and Film , 1994 .

[30]  James D. Foley,et al.  Coupling a UI framework with automatic generation of context-sensitive animated help , 1990, UIST '90.

[31]  Erik M. Altmann,et al.  Tasks of a feather flock together: Similarity effects in task switching , 2003, Memory & cognition.

[32]  Raja Parasuraman,et al.  Three Experiments Examining the Use of Electroencephalogram, Event-Related Potentials, and Heart-Rate Variability for Real- Time Human-Centered Adaptive Automation Design , 2003 .

[33]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[34]  Aude Dufresne,et al.  Sound, space, and metaphor: Multimodal access to windows for blind users , 1996 .

[35]  Ernesto Arroyo,et al.  Interruptions as multimodal outputs: which are the less disruptive? , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[36]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[37]  M R Endsley,et al.  Level of automation effects on performance, situation awareness and workload in a dynamic control task. , 1999, Ergonomics.

[38]  David B. Kaber,et al.  Comparison of Performance Effects of Adaptive Automation Applied to Various Stages of Human-Machine System Information Processing , 2002 .

[39]  J Kawahara,et al.  The Effect of Informative and Uninformative Cueing of Attention on Feature Integration , 2001, The Journal of general psychology.

[40]  Carl Gutwin,et al.  Learning from Games: HCI Design Innovations in Entertainment Software , 2003, Graphics Interface.

[41]  P A Hancock,et al.  Pilot performance and preference for short cycles of automation in adaptive function allocation. , 1995, Applied ergonomics.

[42]  Christine D. Wilson,et al.  Grounding conceptual knowledge in modality-specific systems , 2003, Trends in Cognitive Sciences.

[43]  Christopher D. Wickens,et al.  Unreliable Automated Attention Cueing for Air-Ground Targeting and Traffic Maneuvering , 1999 .

[44]  Peter A. Hancock,et al.  Implementing Adaptive Function Allocation , 2001 .

[45]  Harvey S. Smallman,et al.  Heuristic Automation for Decluttering Tactical Displays , 2005, Hum. Factors.

[46]  Robert W. Lindeman,et al.  Effective Vibrotactile Cueing in a Visual Search Task , 2003, INTERACT.

[47]  Yili Liu,et al.  Queueing Network-Model Human Processor (QN-MHP): A computational architecture for multitask performance in human-machine systems , 2006, TCHI.

[48]  Johanna D. Moore,et al.  Proceedings of the Conference on Human Factors in Computing Systems , 1989 .

[49]  Marc M. Sebrechts,et al.  HANDBOOK OF VIRTUAL ENVIRONMENTS , 2014 .

[50]  David B. Kaber,et al.  On the Design of Adaptive Automation for Complex Systems , 2001 .

[51]  Gavriel Salvendy,et al.  Handbook of Human Factors and Ergonomics , 2005 .