Matched and Mismatched SOCs: A Qualitative Study on Security Operations Center Issues

Organizations, such as companies and governments, created Security Operations Centers (SOCs) to defend against computer security attacks. SOCs are central defense groups that focus on security incident management with capabilities such as monitoring, preventing, responding, and reporting. They are one of the most critical defense components of a modern organization's defense. Despite their critical importance to organizations, and the high frequency of reported security incidents, only a few research studies focus on problems specific to SOCs. In this study, to understand and identify the issues of SOCs, we conducted 18 semi-structured interviews with SOC analysts and managers who work for organizations from different industry sectors. Through our analysis of the interview data, we identified technical and non-technical issues that exist in SOC. Moreover, we found inherent disagreements between SOC managers and their analysts that, if not addressed, could entail a risk to SOC efficiency and effectiveness. We distill these issues into takeaways that apply both to future academic research and to SOC management. We believe that research should focus on improving the efficiency and effectiveness of SOCs.

[1]  John Yen,et al.  A cyber security data triage operation retrieval system , 2018, Comput. Secur..

[2]  Ken Allen,et al.  CyberCIEGE: Gaming for Information Assurance , 2005, IEEE Secur. Priv..

[3]  Katharina Krombholz,et al.  Investigating System Operators' Perspective on Security Misconfigurations , 2018, CCS.

[4]  Sadie Creese,et al.  Sonification in security operations centres: what do security practitioners think? , 2018, ArXiv.

[5]  Hossein Gharaee,et al.  Log management comprehensive architecture in Security Operation Center (SOC) , 2011, 2011 International Conference on Computational Aspects of Social Networks (CASoN).

[6]  Marcel Hoffmann,et al.  A Tale of Three Security Operation Centers , 2014, SIW '14.

[7]  Brian C. Nelson,et al.  Scientific Inquiry in Educational Multi-user Virtual Environments , 2007 .

[8]  John McHugh,et al.  An Anthropological Approach to Studying CSIRTs , 2014, IEEE Security & Privacy.

[9]  Eser Kandogan,et al.  Field studies of computer system administrators: analysis of system management tools and practices , 2004, CSCW.

[10]  Zhou Li,et al.  Acing the IOC Game: Toward Automatic Discovery and Analysis of Open-Source Cyber Threat Intelligence , 2016, CCS.

[11]  Shuhong Yuan,et al.  The security operations center based on correlation analysis , 2011, 2011 IEEE 3rd International Conference on Communication Software and Networks.

[12]  Peter Reiher,et al.  A taxonomy of DDoS attack and DDoS defense mechanisms , 2004, CCRV.

[13]  Eben M. Haber,et al.  Design guidelines for system administration tools developed through ethnographic field studies , 2007, CHIMIT '07.

[14]  William K. Robertson,et al.  Beehive: large-scale log analysis for detecting suspicious activity in enterprise networks , 2013, ACSAC.

[15]  Elissa M. Redmiles,et al.  The Battle for New York: A Case Study of Applied Digital Threat Modeling at the Enterprise Level , 2018, USENIX Security Symposium.

[16]  Paul C. van Oorschot,et al.  SoK: Science, Security and the Elusive Goal of Security as a Scientific Pursuit , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[17]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[18]  Brian C. Nelson,et al.  Design-based research strategies for studying situated learning in a multi-user virtual environment , 2004 .

[19]  Ding Li,et al.  NoDoze: Combatting Threat Alert Fatigue with Automated Provenance Triage , 2019, NDSS.

[20]  David Janos Feher,et al.  Security Concerns Towards Security Operations Centers , 2018, 2018 IEEE 12th International Symposium on Applied Computational Intelligence and Informatics (SACI).

[21]  Chris Maurer,et al.  Teaching Case: Security Breach at Target , 2018, J. Inf. Syst. Educ..

[22]  Lawrence R. Ness,et al.  Are We There Yet - Data Saturation in Qualitative Research (TQR Published).pdf , 2015 .

[23]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[24]  Kirstie Hawkey,et al.  Security practitioners in context: their activities and interactions , 2008, CHI Extended Abstracts.

[25]  Gianluca Stringhini,et al.  Hit 'em where it hurts: a live security exercise on cyber situational awareness , 2011, ACSAC '11.

[26]  Pratyusa K. Manadhata,et al.  The Operational Role of Security Information and Event Management Systems , 2014, IEEE Security & Privacy.

[27]  P. Biernacki,et al.  Snowball Sampling: Problems and Techniques of Chain Referral Sampling , 1981 .

[28]  Kaido Kikkas,et al.  A Live Virtual Simulator for Teaching Cybersecurity to Information Technology Students , 2016, HCI.

[29]  Cyril Onwubiko,et al.  Cyber security operations centre: Security monitoring for protecting business and supporting cyber defense strategy , 2015, 2015 International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA).

[30]  Robert A. Bridges,et al.  Situ: Identifying and Explaining Suspicious Behavior in Networks , 2019, IEEE Transactions on Visualization and Computer Graphics.

[31]  B. Berg Qualitative Research Methods for the Social Sciences , 1989 .

[32]  John McHugh,et al.  Humans are dynamic. Our tools should be too. Innovations from the Anthropological Study of Security Operations Centers. , 2017 .

[33]  Natalia Miloslavskaya,et al.  Analysis of SIEM Systems and Their Usage in Security Operations and Security Intelligence Centers , 2017, BICA 2017.

[34]  Daniel R. Tesone,et al.  Achieving Cyber Defense Situational Awareness: A Cognitive Task Analysis of Information Assurance Analysts , 2005 .

[35]  John McHugh,et al.  Turning Contradictions into Innovations or: How We Learned to Stop Whining and Improve Security Operations , 2016, SOUPS.

[36]  John McHugh,et al.  A Human Capital Model for Mitigating Security Analyst Burnout , 2015, SOUPS.

[37]  Kirstie Hawkey,et al.  An integrated view of human, organizational, and technological challenges of IT security management , 2009, Inf. Manag. Comput. Secur..

[38]  Kirstie Hawkey,et al.  Security practitioners in context: their activities and interactions , 2008, Int. J. Hum. Comput. Stud..

[39]  Konstantin Beznosov,et al.  Towards understanding IT security professionals and their tools , 2007, SOUPS '07.

[40]  H. Russell Bernard,et al.  Social Research Methods: Qualitative and Quantitative Approaches , 2000 .

[41]  Chi-Chang Chen,et al.  An online game approach for improving students' learning performance in web-based problem-solving activities , 2012, Comput. Educ..