Practical large scale what-if queries: case studies with software risk assessment

When a lack of data inhibits decision-making, large-scale what-if queries can be conducted over the uncertain parameter ranges. Such queries can generate an overwhelming amount of data. We describe a general method for understanding that data. Large-scale what-if queries can guide Monte Carlo simulations of a model. Machine learning can then be used to summarize the output. The summarization is an ensemble of decision trees. The TARZAN system [so-called because it swings through (or searches) the decision trees] can poll the ensemble looking for majority conclusions regarding what factors change the classifications of the data. TARZAN can succinctly present the results from very large what-if queries. For example, in one of the studies presented, we can view the significant features from 10/sup 9/ what-if queries on half a page.

[1]  Barry W. Boehm,et al.  Software Engineering Economics , 1993, IEEE Transactions on Software Engineering.

[2]  Chris F. Kemerer,et al.  An empirical validation of software cost estimation models , 1987, CACM.

[3]  Steve Easterbrook,et al.  Handling conflict between domain descriptions with computer-supported negotiation , 1991 .

[4]  Michael J. Prietula,et al.  Examining the Feasibility of a Case-Based Reasoning Model for Software Effort Estimation , 1992, MIS Q..

[5]  Peter Neumann,et al.  Safeware: System Safety and Computers , 1995, SOEN.

[6]  Nancy G. Leveson,et al.  Safeware: System Safety and Computers , 1995 .

[7]  Michael R. Lyu,et al.  Handbook of software reliability engineering , 1996 .

[8]  Raymond J. Madachy,et al.  Heuristic Risk Assessment Using Cost Factors , 1997, IEEE Softw..

[9]  Tim Menzies,et al.  Applications of abduction: hypothesis testing of neuroendocrinological qualitative compartmental models , 1997, Artif. Intell. Medicine.

[10]  Ellis Horowitz,et al.  Cocomo ii model definition manual , 1998 .

[11]  Bojan Cukic,et al.  On the sufficiency of limited testing for knowledge based systems , 1999, Proceedings 11th International Conference on Tools with Artificial Intelligence.

[12]  Barry W. Boehm,et al.  Bayesian Analysis of Empirical Software Engineering Cost Models , 1999, IEEE Trans. Software Eng..

[13]  Bashar Nuseibeh,et al.  An empirical investigation of multiple viewpoint reasoning in requirements engineering , 1999, Proceedings IEEE International Symposium on Requirements Engineering (Cat. No.PR00188).