Integrating Manual and Automatic Risk Assessment for Risk-Based Testing

In this paper we define a model-based risk assessment procedure that integrates automatic risk assessment by static analysis, semi-automatic risk assessment and guided manual risk assessment. In this process probability and impact criteria are determined by metrics which are combined to estimate the risk of specific system development artifacts. The risk values are propagated to the assigned test cases providing a prioritization of test cases. This supports to optimize the allocation of limited testing time and budget in a risk-based testing methodology. Therefore, we embed our risk assessment process into a generic risk-based testing methodology. The calculation of probability and impact metrics is based on system and requirements artifacts which are formalized as model elements. Additional time metrics consider the temporal development of the system under test and take for instance the bug and version history of the system into account. The risk assessment procedure integrates several stakeholders and is explained by a running example.

[1]  Kalle Lyytinen,et al.  Components of Software Development Risk: How to Address Them? A Project Manager Survey , 2000, IEEE Trans. Software Eng..

[2]  Andreas Zeller,et al.  Mining metrics to predict component failures , 2006, ICSE.

[3]  C. H. Lie,et al.  Fault Tree Analysis, Methods, and Applications ߝ A Review , 1985, IEEE Transactions on Reliability.

[4]  Ming Zhao,et al.  A comparison between software design and code metrics for the prediction of software fault content , 1998, Inf. Softw. Technol..

[5]  Barbara Paech,et al.  Exploring the relationship of a file's history and its fault-proneness: An empirical method and its application to open source programs , 2010, Inf. Softw. Technol..

[6]  Paolo Giorgini,et al.  Modelling Risk and Identifying Countermeasure in Organizations , 2006, CRITIS.

[7]  Seungjoo Kim,et al.  Information Security and Cryptology - ICISC 2005 , 2005, Lecture Notes in Computer Science.

[8]  N. Nagappan,et al.  Static analysis tools as early indicators of pre-release defect density , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[9]  Karen Scarfone,et al.  Common Vulnerability Scoring System , 2006, IEEE Security & Privacy.

[10]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[11]  Jyrki Kontio,et al.  Risk management in software development: a technology overview and the riskit method , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[12]  Sjouke Mauw,et al.  Foundations of Attack Trees , 2005, ICISC.

[13]  Andreas Metzger,et al.  Employing Requirements Metrics for Automating Early Risk Assessment , 2007 .

[14]  Yue Jiang,et al.  Comparing design and code metrics for software quality prediction , 2008, PROMISE '08.

[15]  Suresh L. Konda,et al.  Taxonomy-Based Risk Identification , 1993 .

[16]  Shari Lawrence Pfleeger.  Risky business: what we have yet to learn about risk management , 2000, J. Syst. Softw..

[17]  Ian F. Alexander,et al.  Misuse Cases: Use Cases with Hostile Intent , 2003, IEEE Softw..

[18]  Software Engineering Risk Management: A Just-in-Time Approach , 1995 .

[19]  Klaus Pohl,et al.  An automated technique for risk-based test case generation and prioritization , 2008, AST '08.

[20]  Barry W. Boehm,et al.  A spiral model of software development and enhancement , 1986, Computer.

[21]  James Bach,et al.  Heuristic Risk-Based Testing , 1999 .

[22]  Shin Ta Liu,et al.  Risk Modeling, Assessment, and Management , 1999, Technometrics.

[23]  Walter F. Tichy,et al.  Proceedings 25th International Conference on Software Engineering , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[24]  Bernhard Plattner,et al.  Large-scale vulnerability analysis , 2006, LSAD '06.

[25]  Ståle Amland.  Risk-based testing: : Risk analysis fundamentals and metrics for software testing including a financial application case study , 2000, J. Syst. Softw..

[26]  Foreword,et al.  Table of Content , 2020, 2020 7th International Conference on Technical Education (ICTechEd7).

[27]  Y. Haimes.  Risk Modeling, Assessment, and Management: Haimes/Risk Modeling, Assessment 2e , 2005 .