Risk-Based Vulnerability Management. Exploiting the economic nature of the attacker to build sound and measurable vulnerability mitigation strategies

Vulnerability bulletins and feeds report hundreds of vulnerabilities a month that a system administrator or a Chief Information Officer working for an organisation has to take care of. Because of the load of work, vulnerability prioritisation is a must in any complex-enough organisation. Currently, the industry employs the Common Vulnerability Scoring System (CVSS in short) as a metric to prioritise vulnerability risk. However, the CVSS base score is a technical measure of severity, not of risk. By using a severity measure to estimate risk, current practices assume that every vulnerability is characterised by the same exploitation likelihood, and that vulnerability risk can be assessed through a technical analysis of the vulnerability. In this Thesis we argue that this is not the case, and that the economic forces that drive the attacker are a key factor in understanding vulnerability risk. In particular, we argue that attacker's rationality and the economic infrastructure supporting cybercrime's activities play a major role in determining which vulnerabilities will the attackers massively exploit, and therefore which vulnerabilities will represent a (substantially higher than the rest) risk. Our ultimate goal is to show that `risk-based' vulnerability management policies, as opposed to currently employed `criticality-based' ones, are possible and can outperform current practices in terms of patching efficiency without losing in effectiveness (i.e. reduction of risk in the wild). To this aim we perform an extensive data-collection work on vulnerabilities, proof-of-concept exploits, exploits traded in the cybercrime markets, and exploits detected in the wild. We further collaborated with Symantec to collect actual records of attacks in the wild delivered against about 1M machines worldwide. A good part of our data-collection efforts has been also dedicated in infiltrating and analysing the cybercrime markets. We used this data collection to evaluate four `running hypotheses' underlying our main thesis: vulnerability risk is influenced by the attacker's rationality (1), and the underground markets are credible sources of risk that provide technically proficient attack tools (2), are a mature (3) and sound (4) from an economic perspective. We then put this in practice and evaluate the effectiveness of criticality-based and risk-based vulnerability management policies (based on the aforementioned findings) in mitigating real attacks in the wild. We compare the policies in terms of the `risk reduction' they entail, i.e. the gap between `risk' addressed by the policy and residual risk. Our results show that risk-based policies entail a significantly higher risk reduction than criticality-based ones, and thwart the majority of risk in the wild by addressing only a small fraction of the patching work prescribed by current practices.

[1]  Muhammad Zubair Shafiq,et al.  A large scale exploratory analysis of software vulnerability life cycles , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[2]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[3]  W. Tirenin,et al.  A concept for strategic cyber defense , 1999, MILCOM 1999. IEEE Military Communications. Conference Proceedings (Cat. No.99CH36341).

[4]  Leyla Bilge,et al.  The Attack of the Clones: A Study of the Impact of Shared Code on Vulnerability Patching , 2015, 2015 IEEE Symposium on Security and Privacy.

[5]  Cormac Herley When Does Targeting Make Sense for an Attacker? , 2013, IEEE Security & Privacy.

[6]  Paul A. Taylor,et al.  Hackers: Crime in the Digital Sublime , 1999 .

[7]  Fabio Massacci,et al.  MalwareLab: Experimentation with Cybercrime Attack Tools , 2013, CSET.

[8]  Lawrence A. Gordon,et al.  The economics of information security investment , 2002, TSEC.

[9]  J.D. McCalley,et al.  An overview of risk based security assessment , 1999, 1999 IEEE Power Engineering Society Summer Meeting. Conference Proceedings (Cat. No.99CH36364).

[10]  Orly Turgeman-Goldschmidt Hackers' Accounts , 2005 .

[11]  Sushil Jajodia,et al.  Pareto-Optimal Adversarial Defense of Enterprise Systems , 2015, TSEC.

[12]  A. Ozment,et al.  Bug Auctions: Vulnerability Markets Reconsidered , 2004 .

[13]  Sandy Clark,et al.  Familiarity breeds contempt: the honeymoon effect and the role of legacy code in zero-day vulnerabilities , 2010, ACSAC '10.

[14]  Fabio Massacci,et al.  An independent validation of vulnerability discovery models , 2012, ASIACCS '12.

[15]  R. Tibshirani,et al.  Generalized Additive Models , 1991 .

[16]  Tudor Dumitras,et al.  Toward a standard benchmark for computer security research: the worldwide intelligence network environment (WINE) , 2011, BADGERS '11.

[17]  Per Larsen,et al.  Profile-guided automated software diversity , 2013, Proceedings of the 2013 IEEE/ACM International Symposium on Code Generation and Optimization (CGO).

[18]  Jan Willemson,et al.  Rational Choice of Security Measures Via Multi-parameter Attack Trees , 2006, CRITIS.

[19]  Stefan Savage,et al.  An analysis of underground forums , 2011, IMC '11.

[20]  Bernhard Plattner,et al.  Modelling the Security Ecosystem- The Dynamics of (In)Security , 2009, WEIS.

[21]  Tudor Dumitras,et al.  Ask WINE: Are We Safer Today? Evaluating Operating System Security through Big Data Analysis , 2012, LEET.

[22]  Yashwant K. Malaiya,et al.  Assessing Vulnerabilities in Apache and IIS HTTP Servers , 2006, 2006 2nd IEEE International Symposium on Dependable, Autonomic and Secure Computing.

[23]  Cormac Herley,et al.  Nobody Sells Gold for the Price of Silver: Dishonesty, Uncertainty and the Underground Economy , 2009, WEIS.

[24]  Rahul Telang,et al.  An Empirical Analysis of the Impact of Software Vulnerability Announcements on Firm Stock Price , 2007, IEEE Transactions on Software Engineering.

[25]  Hamed Okhravi,et al.  Evaluation of Patch Management Strategies , 2022 .

[26]  A. Greif Contract Enforceability and Economic Institutions in Early Trade: the Maghribi Traders' Coalition , 1993 .

[27]  J. Tirole Cognition and Incomplete Contracts , 2009 .

[28]  Mehran Bozorgi,et al.  Beyond heuristics: learning to classify vulnerabilities and predict exploits , 2010, KDD.

[29]  Yashwant K. Malaiya,et al.  Modeling the vulnerability discovery process , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[30]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[31]  Leyla Bilge,et al.  Before we knew it: an empirical study of zero-day attacks in the real world , 2012, CCS.

[32]  Fabio Massacci,et al.  Anatomy of Exploit Kits - Preliminary Analysis of Exploit Kits as Software Artefacts , 2013, ESSoS.

[33]  Niels Provos,et al.  All Your iFRAMEs Point to Us , 2008, USENIX Security Symposium.

[34]  Niels Provos,et al.  The Ghost in the Browser: Analysis of Web-based Malware , 2007, HotBots.

[35]  Vyas Sekar,et al.  Measuring user confidence in smartphone security and privacy , 2012, SOUPS.

[36]  Karen A. Scarfone,et al.  SP 800-117. Guide to Adopting and Using the Security Content Automation Protocol (SCAP) Version 1.0 , 2010 .

[37]  Guido Schryen,et al.  A Comprehensive and Comparative Analysis of the Patching Behavior of Open Source and Closed Source Software Vendors , 2009, 2009 Fifth International Conference on IT Security Incident Management and IT Forensics.

[38]  Yashwant K. Malaiya,et al.  Application of Vulnerability Discovery Models to Major Operating Systems , 2008, IEEE Transactions on Reliability.

[39]  Brent Byunghoon Kang,et al.  Peer-to-Peer Botnets: Overview and Case Study , 2007, HotBots.

[40]  Charles Miller,et al.  The Legitimate vulnerability market: the secretive world of 0-day exploit sales , 2007, WEIS.

[41]  Tyler Moore,et al.  Measuring the Cost of Cybercrime , 2012, WEIS.

[42]  M. Albert E-Buyer Beware: Why Online Auction Fraud Should Be Regulated , 2002 .

[43]  H. White A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity , 1980 .

[44]  Wenke Lee,et al.  Modeling Botnet Propagation Using Time Zones , 2006, NDSS.

[45]  L Evans,et al.  The effectiveness of safety belts in preventing fatalities. , 1986, Accident; analysis and prevention.

[46]  Nigel Shadbolt,et al.  Why forums?: an empirical analysis into the facilitating factors of carding forums , 2013, WebSci.

[47]  William Luke,et al.  Smoking and carcinoma of the lung , 1954 .

[48]  Jun Zhang,et al.  Economics of Security Patch Management , 2006, WEIS.

[49]  Christopher Krügel,et al.  Analysis of a Botnet Takeover , 2011, IEEE Security & Privacy.

[50]  Mary M. Calkins,et al.  My Reputation Always Had More Fun Than Me: The Failure of eBay's Feedback Model to Effectively Prevent Online Auction Fraud , 2001 .

[51]  Hao Xu,et al.  Optimal Policy for Software Vulnerability Disclosure , 2008, Manag. Sci..

[52]  Ramayya Krishnan,et al.  Correlated Failures, Diversification, and Information Security Risk Management , 2011, MIS Q..

[53]  Tudor Dumitras,et al.  Some Vulnerabilities Are Different Than Others - Studying Vulnerabilities and Attack Surfaces in the Wild , 2014, RAID.

[54]  Julian Williams,et al.  Information security trade-offs and optimal patching policies , 2012, Eur. J. Oper. Res..

[55]  Sebastian Günther Folk Models of Home Computer Security , 2012 .

[56]  Cormac Herley,et al.  Why do Nigerian Scammers Say They are From Nigeria? , 2012, WEIS.

[57]  Dawn Xiaodong Song,et al.  Suspended accounts in retrospect: an analysis of twitter spam , 2011, IMC '11.

[58]  David Brumley,et al.  Q: Exploit Hardening Made Easy , 2011, USENIX Security Symposium.

[59]  Tony Bradley,et al.  PCI Compliance: Understand and Implement Effective PCI Data Security Standard Compliance , 2007 .

[60]  Andy Ozment,et al.  The Likelihood of Vulnerability Rediscovery and the Social Utility of Vulnerability Hunting , 2005, WEIS.

[61]  A. Arora,et al.  Impact of Vulnerability Disclosure and Patch Availability - An Empirical Analysis , 2004 .

[62]  Thomas J. Holt,et al.  Examining the social networks of malware writers and hackers , 2012 .

[63]  Chris Kanich,et al.  Spamalytics: an empirical analysis of spam marketing conversion , 2008, CCS.

[64]  Johannes M. Bauer,et al.  Cybersecurity: Stakeholder incentives, externalities, and policy options , 2009 .

[65]  Ahmad-Reza Sadeghi,et al.  Just-In-Time Code Reuse: On the Effectiveness of Fine-Grained Address Space Layout Randomization , 2013, 2013 IEEE Symposium on Security and Privacy.

[66]  Sushil Jajodia,et al.  An Attack Graph-Based Probabilistic Security Metric , 2008, DBSec.

[67]  K. Eisenhardt Agency Theory: An Assessment and Review , 1989 .

[68]  Bill Curtis,et al.  A field study of the software design process for large systems , 1988, CACM.

[69]  Chaim Kaufmann Threat Inflation and the Failure of the Marketplace of Ideas: The Selling of the Iraq War , 2004, International Security.

[70]  Stefan Savage,et al.  Manufacturing compromise: the emergence of exploit-as-a-service , 2012, CCS.

[71]  Salim Hariri,et al.  Impact Analysis of Faults and Attacks in Large-Scale Networks , 2003, IEEE Secur. Priv..

[72]  Robert A. Small,et al.  Reducing Internet-Based Intrusions: Effective Security Patch Management , 2003, IEEE Softw..

[73]  Karen A. Scarfone,et al.  An analysis of CVSS version 2 vulnerability scoring , 2009, ESEM 2009.

[74]  Johannes M. Bauer,et al.  OF MALWARE : SECURITY DECISIONS , INCENTIVES AND EXTERNALITIES , 2008 .

[75]  Herbert Bos,et al.  Prudent Practices for Designing Malware Experiments: Status Quo and Outlook , 2012, 2012 IEEE Symposium on Security and Privacy.

[76]  Hadi Asghari,et al.  Security Economics in the HTTPS Value Chain , 2013 .

[77]  D. Altman,et al.  Multiple significance tests: the Bonferroni method , 1995, BMJ.

[78]  Yanyan Zhuang,et al.  It's the psychology stupid: how heuristics explain software vulnerabilities and how priming can illuminate developer's blind spots , 2014, ACSAC.

[79]  Paul Resnick,et al.  Trust among strangers in internet transactions: Empirical analysis of eBay' s reputation system , 2002, The Economics of the Internet and E-commerce.

[80]  Danny Dolev,et al.  On the security of public key protocols , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[81]  Michael Howard,et al.  Measuring Relative Attack Surfaces , 2005 .

[82]  Karen A. Scarfone,et al.  A Complete Guide to the Common Vulnerability Scoring System Version 2.0 | NIST , 2007 .

[83]  M. Melnik,et al.  Does a Seller's Ecommerce Reputation Matter? Evidence from Ebay Auctions , 2003 .

[84]  David A. Wagner,et al.  An Empirical Study of Vulnerability Rewards Programs , 2013, USENIX Security Symposium.

[85]  Luca Allodi Attacker Economics for Internet-scale Vulnerability Risk Assessment , 2013, LEET.

[86]  George A. Akerlof The Market for “Lemons”: Quality Uncertainty and the Market Mechanism , 1970 .

[87]  Zinta S. Byrne,et al.  The Psychology of Security for the Home Computer User , 2012, 2012 IEEE Symposium on Security and Privacy.

[88]  Huseyin Cavusoglu,et al.  The critical elements of the patch management process , 2009, Commun. ACM.

[89]  Henrique Madeira,et al.  Security Benchmarks for Web Serving Systems , 2014, 2014 IEEE 25th International Symposium on Software Reliability Engineering.

[90]  H. Simon,et al.  Theories of Decision-Making in Economics and Behavioural Science , 1966 .

[91]  Stefan Savage,et al.  An inquiry into the nature and causes of the wealth of internet miscreants , 2007, CCS '07.

[92]  B. Kitchenham,et al.  Case Studies for Method and Tool Evaluation , 1995, IEEE Softw..

[93]  Ramayya Krishnan,et al.  An Empirical Analysis of Software Vendors' Patch Release Behavior: Impact of Vulnerability Disclosure , 2010, Inf. Syst. Res..

[94]  Jun Zhang,et al.  Security Patch Management: Share the Burden or Share the Damage? , 2008, Manag. Sci..