What Norwegian Developers Want and Need From Security-Directed Program Analysis Tools: A Survey

Code enforcing access control policies often has high inherent complexity, making it challenging to test using only classical review and testing techniques. To more thoroughly test such code, it is strategic to also use program analysis tools, which often can find subtle, critical bugs going unnoticed to humans. These powerful tools are however rarely used in software consultancy practice, due to factors such as bad usability or unsatisfactory non-functional characteristics. To encourage wider adoption of such tools, more must be learned about how to design them to the preferences of software consultants. Towards this goal, we conducted a survey of Norwegian software consultants. Among our findings is a positive relation between preference for soundness over completeness in tools and preference for annotation-based over automated tools. 51% of the developers surveyed prefer soundness over completeness when detecting access control vulnerabilities, while only 37.5% view completeness as the more important characteristic. Qualitative responses illuminate concerns regarding usability, soundness, completeness, and performance.

[1]  Jingyue Li,et al.  Evaluation of Open-Source IDE Plugins for Detecting Security Vulnerabilities , 2019, EASE.

[2]  Prasad Naldurg,et al.  MACE: Detecting Privilege Escalation Vulnerabilities in Web Applications , 2014, CCS.

[3]  B. J. Oates,et al.  Researching Information Systems and Computing , 2005 .

[4]  Yves Le Traon,et al.  Access control enforcement testing , 2013, 2013 8th International Workshop on Automation of Software Test (AST).

[5]  R. L. Herron,et al.  Use and Misuse of the Likert Item Responses and Other Ordinal Measures , 2015, International journal of exercise science.

[6]  Jun Zhu,et al.  Detecting Privilege Escalation Attacks through Instrumenting Web Application Source Code , 2016, SACMAT.

[7]  Onur Ozdemir,et al.  Automated Vulnerability Detection in Source Code Using Deep Representation Learning , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[8]  Marco Pistoia,et al.  ALETHEIA: Improving the Usability of Static Security Analysis , 2014, CCS.

[9]  Joseph P. Near,et al.  Finding Security Bugs in Web Applications Using a Catalog of Access Control Patterns , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[10]  David M. Eyers,et al.  FlowWatcher: Defending against Data Disclosure Vulnerabilities in Web Applications , 2015, CCS.

[11]  Humera Farooq,et al.  Predicting Web Vulnerabilities in Web Applications Based on Machine Learning , 2018 .

[12]  Mohammad Ghafari,et al.  JIT Feedback - What Experienced Developers Like about Static Analysis , 2018, 2018 IEEE/ACM 26th International Conference on Program Comprehension (ICPC).

[13]  Shari Lawrence Pfleeger,et al.  Personal Opinion Surveys , 2008, Guide to Advanced Empirical Software Engineering.

[14]  Tevfik Bultan,et al.  Finding access control bugs in web applications with CanCheck , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[15]  Arjen Hommersom,et al.  Discovering software vulnerabilities using data-flow analysis and machine learning , 2018, ARES.

[16]  Christoforos E. Kozyrakis,et al.  Nemesis: Preventing Authentication & Access Control Vulnerabilities in Web Applications , 2009, USENIX Security Symposium.

[17]  Jun Zhu,et al.  Mitigating Access Control Vulnerabilities through Interactive Static Analysis , 2015, SACMAT.

[18]  Zhejun Fang,et al.  Static Detection of Logic Vulnerabilities in Java Web Applications , 2012, 2012 IEEE 11th International Conference on Trust, Security and Privacy in Computing and Communications.

[19]  Christian Bird,et al.  What developers want and need from program analysis: An empirical study , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[20]  Zahid Anwar,et al.  Semantic security against web application attacks , 2014, Inf. Sci..

[21]  Emerson R. Murphy-Hill,et al.  A study of interactive code annotation for access control vulnerabilities , 2015, 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[22]  Hamid Reza Shahriari,et al.  Athena: A framework to automatically generate security test oracle via extracting policies from source code and intended software behaviour , 2019, Inf. Softw. Technol..

[23]  Vitaly Shmatikov,et al.  SAFERPHP: finding semantic vulnerabilities in PHP applications , 2011, PLAS '11.

[24]  G. Deepa,et al.  DetLogic: A black-box approach for detecting logic vulnerabilities in web applications , 2018, J. Netw. Comput. Appl..

[25]  Anders Møller,et al.  Automated detection of client-state manipulation vulnerabilities , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[26]  Zhendong Su,et al.  Static Detection of Access Control Vulnerabilities in Web Applications , 2011, USENIX Security Symposium.

[27]  Ciera Jaspan,et al.  Tricorder: Building a Program Analysis Ecosystem , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[28]  Leonard R. Sussman,et al.  Nominal, Ordinal, Interval, and Ratio Typologies are Misleading , 1993 .

[29]  R. Shepard.  Metric structures in ordinal data , 1966 .

[30]  Xiangyu Zhang,et al.  Path sensitive static analysis of web applications for remote code execution vulnerability detection , 2013, 2013 35th International Conference on Software Engineering (ICSE).