An Empirical Study of Vulnerability Rewards Programs

We perform an empirical study to better understand two well-known vulnerability rewards programs, or VRPs, which software vendors use to encourage community participation in finding and responsibly disclosing software vulnerabilities. The Chrome VRP has cost approximately $580,000 over 3 years and has resulted in 501 bounties paid for the identification of security vulnerabilities. The Firefox VRP has cost approximately $570,000 over the last 3 years and has yielded 190 bounties. 28% of Chrome's patched vulnerabilities appearing in security advisories over this period, and 24% of Firefox's, are the result of VRP contributions. Both programs appear economically efficient, comparing favorably to the cost of hiring full-time security researchers. The Chrome VRP features low expected payouts accompanied by high potential payouts, while the Firefox VRP features fixed payouts. Finding vulnerabilities for VRPs typically does not yield a salary comparable to a full-time job; the common case for recipients of rewards in either program is that they have received only one reward. Firefox has far more critical-severity vulnerabilities than Chrome, which we believe is attributable to an architectural difference between the two browsers.

[1]  Adam Barth,et al.  The Security Architecture of the Chromium Browser , 2009 .

[2]  B. Kahle THE INTERNET ARCHIVE , 2012 .

[3]  Bernhard Plattner,et al.  Software Security Economics: Theory, in Practice , 2012, WEIS.

[4]  Joseph Bonneau,et al.  What's in a Name? , 2020, Financial Cryptography.

[5]  V. Rich Personal communication , 1989, Nature.

[6]  Laurie A. Williams,et al.  One Technique is Not Enough: A Comparison of Vulnerability Discovery Techniques , 2011, 2011 International Symposium on Empirical Software Engineering and Measurement.

[7]  Banu Diri,et al.  A systematic review of software fault prediction studies , 2009, Expert Syst. Appl..

[8]  Thomas A. Garrett,et al.  Why People Choose Negative Expected Return Assets - an Empirical Examination of a Utility Theoretic Explanation , 2006 .

[9]  Matthew Finifter Exploring the Relationship Between Web Application Development Tools and Security , 2011, WebApps.

[10]  Les Hatton Predicting the total number of faults using parallel code inspections , 2005 .

[11]  Andreas Zeller,et al.  Predicting vulnerable software components , 2007, CCS '07.

[12]  David A. Wagner,et al.  An Empirical Study on the Effectiveness of Security Code Review , 2013, ESSoS.

[13]  Stuart E. Schechter,et al.  Milk or Wine: Does Software Security Improve with Age? , 2006, USENIX Security Symposium.

[14]  Eric S. Raymond,et al.  The Cathedral and the Bazaar , 2000 .

[15]  Charles Miller,et al.  The Legitimate vulnerability market: the secretive world of 0-day exploit sales , 2007, WEIS.

[16]  Eric Rescorla,et al.  Is finding security holes a good idea? , 2005, IEEE Security & Privacy.

[17]  Engin Kirda,et al.  Quo Vadis? A Study of the Evolution of Input Validation Vulnerabilities in Web Applications , 2011, Financial Cryptography.

[18]  Noopur Davis Secure Software Development Life Cycle Processes: A Technology Scouting Report , 2005 .