Development and Validation of the Air Force Cyber Intruder Alert Testbed (CIAT)

Presently, cyber defense heavily relies on human network analysts who must detect and investigate potential suspicious activity, a demanding, fatiguing process that takes a heavy toll on human operators. Given the criticality of these operators to cyber defense, research is needed to investigate and mitigate the sources of those challenges. Currently, few cyber-focused synthetic task environments (STEs) exist, and those that do are not well suited to investigate the problems of network analysts. Therefore, a new cyber STE focused on network analysts called the Air Force Cyber Intruder Alert Testbed (CIAT) was developed. This STE was designed to emulate key functions of Enterprise-level cyber defense platforms. Specifically, CIAT simulates a network analyst environment, including an intrusion detection system, signature database, packet capture software, and network list. The purpose of this paper is to describe the development and validation of the CIAT STE.

[1]  Joel S. Warm,et al.  Stress and Workload Profiles of Network Analysis: Not All Tasks Are Created Equal , 2016 .

[2]  Theodore B. Aldrich,et al.  Task Analysis of the UH-60 Mission and Decision Rules for Developing a UH-60 Workload Prediction Model. Volume 1. Summary Report , 1989 .

[3]  Daniela M. Witten,et al.  An Introduction to Statistical Learning: with Applications in R , 2013 .

[4]  Mark T. Maybury.  Toward the Assured Cyberspace Advantage: Air Force Cyber Vision 2025 , 2015, IEEE Security & Privacy.

[5]  Karen A. Scarfone,et al.  Guide to Intrusion Detection and Prevention Systems (IDPS) , 2007 .

[6]  J. R. Comstock.  MAT - MULTI-ATTRIBUTE TASK BATTERY FOR HUMAN OPERATOR WORKLOAD AND STRATEGIC BEHAVIOR RESEARCH , 1994 .

[7]  Joel S. Warm,et al.  Cyber Vigilance , 2014 .

[8]  Charneta Samms.  Improved Performance Research Integration Tool (IMPRINT): Human Performance Modeling for Improved System Design , 2010 .

[9]  M. A. Champion,et al.  Team-based cyber defense analysis , 2012, 2012 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support.

[10]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[11]  Prashanth Rajivan.  A Synthetic Task Environment for Measuring Cyber Situation Awareness , 2011 .

[12]  M. McNeese,et al.  idsNETS: An experimental platform to study situation awareness for intrusion detection analysts , 2012, 2012 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support.

[13]  Eric T. Greenlee,et al.  Augmenting Cyber Defender Performance and Workload through Sonified Displays , 2015 .

[14]  Nancy J. Cooke,et al.  Designing a Synthetic Task Environment , 2017 .

[15]  Brett J. Borghetti,et al.  Coordinated Displays to Assist Cyber Defenders , 2016 .

[16]  J H McCracken,et al.  Analyses of Selected LHX Mission Functions: Implications for Operator Workload and System Automation Goals , 1984 .

[17]  Trevor Hastie,et al.  An Introduction to Statistical Learning , 2013, Springer Texts in Statistics.

[18]  William Hayes,et al.  Sources of Occupational Stress and Prevalence of Burnout and Clinical Distress Among U.S. Air Force Cyber Warfare Operators , 2013 .

[19]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .