A journey towards rigorous cybersecurity experiments: on the application of criminological theories

In this paper, we report some lessons learned on conducting empirical studies in cybersecurity. We first describe our attempts to obtain and process security data collected by other organizations. We then list the security data available for research at the University of Maryland and the research we have conducted using these data. Facing critiques - mainly on the lack of generalizability - we describe how we decided to collaborate with social scientists, especially criminologists, to apply social science theories to cybersecurity. Two ongoing research threads are then described to illustrate the feasibility of introducing some science to cybersecurity through the application of criminological theories.

[1]  R.K. Cunningham,et al.  Evaluating intrusion detection systems: the 1998 DARPA off-line intrusion detection evaluation , 2000, Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00.

[2]  Lawrence E. Cohen,et al.  Social Change and Crime Rate Trends: A Routine Activity Approach , 1979 .

[3]  Michael Craft,et al.  Social Deviance , 1965, Mental Health.

[4]  R. Paternoster,et al.  The deterrent effect of the perceived certainty and severity of punishment: A review of the evidence and issues , 1987 .

[5]  Thomas J. Holt,et al.  Malware Victimization: A Routine Activities Framework , 2011 .

[6]  Michel Cukier,et al.  Analysis of Computer Security Incident Data Using Time Series Models , 2008, 2008 19th International Symposium on Software Reliability Engineering (ISSRE).

[7]  Heith Copes,et al.  Understanding Identity Theft , 2009 .

[8]  Michael D. Reisig,et al.  Routine Online Activity and Internet Fraud Targeting: Extending the Generality of Routine Activity Theory , 2010 .

[9]  Michel Cukier,et al.  Finding Corrupted Computers Using Imperfect Intrusion Prevention System Event Data , 2008, SAFECOMP.

[10]  Michel Cukier,et al.  On the Use of Security Metrics Based on Intrusion Prevention System Event Data: An Empirical Analysis , 2008, 2008 11th IEEE High Assurance Systems Engineering Symposium.

[11]  T. Holt,et al.  Examining the Applicability of Lifestyle-Routine Activities Theory for Cybercrime Victimization , 2008 .

[12]  Robin Berthier,et al.  Characterizing Attackers and Attacks: An Empirical Study , 2011, 2011 IEEE 17th Pacific Rim International Symposium on Dependable Computing.

[13]  John McHugh,et al.  The 1998 Lincoln Laboratory IDS Evaluation , 2000, Recent Advances in Intrusion Detection.

[14]  Robin Berthier,et al.  Nfsight: netflow-based network awareness tool , 2010 .

[15]  H. Laurence Ross,et al.  Deterrence in criminology and social policy. , 1986 .

[16]  Robin Berthier,et al.  Analyzing the process of installing rogue software , 2009, 2009 IEEE/IFIP International Conference on Dependable Systems & Networks.

[17]  Tao He,et al.  Applying Software Reliability Models on Security Incidents , 2007, The 18th IEEE International Symposium on Software Reliability (ISSRE '07).

[18]  Kenneth C. Land,et al.  Behavioral and Social Science: Fifty Years of Discovery. , 1987 .