Cyber Operations Stress Survey (COSS): Studying fatigue, frustration, and cognitive workload in cybersecurity operations

Operator stress is a common, persistent, and disabling effect of cyber operations and an important risk factor for performance, safety, and employee burnout. We designed the Cyber Operations Stress Survey (COSS) as a low-cost method for studying fatigue, frustration, and cognitive workload in real-time tactical cyber operations. The combination of preand post-operational measures with well validated factors from the NASA Task Load Index and additional contextual factors provide a quick, easy, and valuable assessment of cognitive stress. We report on our experiences developing and fielding the survey instrument, validation, and describe the use and results of the COSS in four studies of cyber operations across the National Security Agency.

[1]  M. A. Champion,et al.  Team-based cyber defense analysis , 2012, 2012 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support.

[2]  C. Vaught,et al.  Judgment and decision making under stress: an overview for emergency managers , 2003 .

[3]  Wen-Chin Li,et al.  The Investigation of Visual Attention and Workload by Experts and Novices in the Cockpit , 2013, HCI.

[4]  M. Csíkszentmihályi,et al.  Experience Sampling Method: Measuring the Quality of Everyday Life , 2006 .

[5]  A. Bakker,et al.  A multifaceted investigation of the link between mental fatigue and task disengagement. , 2015, Psychophysiology.

[6]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[7]  Rebecca A. Grier.  How High is High? A Meta-Analysis of NASA-TLX Global Workload Scores , 2015 .

[8]  Michael D. Coovert,et al.  The Looming Cybersecurity Crisis and What It Means for the Practice of Industrial and Organizational Psychology , 2018, Industrial and Organizational Psychology.

[9]  Paul E. Spector,et al.  Using the Job-Related Affective Well-Being Scale (JAWS) to investigate affective responses to work stressors. , 2000, Journal of occupational health psychology.

[10]  William Hayes,et al.  Sources of Occupational Stress and Prevalence of Burnout and Clinical Distress Among U.S. Air Force Cyber Warfare Operators , 2013 .

[11]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[12]  Paul Eling,et al.  Work stress and attentional difficulties: An initial study on burnout and cognitive failures , 2005 .

[13]  Joel S. Warm,et al.  Stress and Workload Profiles of Network Analysis: Not All Tasks Are Created Equal , 2016 .

[14]  Konstantin Beznosov,et al.  Towards understanding IT security professionals and their tools , 2007, SOUPS '07.

[15]  Sherwood W Samn,et al.  Estimating Aircrew Fatigue: A Technique with Application to Airlift Operations , 1982 .

[16]  Joel S. Warm,et al.  Cyber Vigilance , 2014 .

[17]  J. Dykstra,et al.  Understanding Operator Fatigue , Frustration , and Cognitive Workload in Tactical Cybersecurity Operations , 2022 .

[18]  Celeste Lyn Paul.  Human-Centered Study of a Network Operations Center: Experience Report and Lessons Learned , 2014, SIW '14.