A Methodology for Modeling VVT Risks and

The cost of large systems’ Verification, Validation, and Testing (VVT) is in the neighborhood of 40% of the total life cycle cost. The cost associated with systems’ failures is even more dramatic, often exceeding 10% of industrial organizations turnover. There is a great potential benefit in streamlining and optimizing the VVT process. The first step in accomplishing this aim is to define a VVT strategy and then to quantify the cost and risk associated with carrying it out. This paper provides an overview of the methodologies for risk and cost monitoring for VVT and proposes a novel approach for modeling VVT strategies as decision problems. A quantitative VVT process and risk model is proposed. Due to the nondeterministic nature of risk, simulation is used to generate distributions of possible costs, schedules, and risk outcomes. These distributions represent a probabilistic approach and are analyzed in relation to impact events. The model provides means to explore different VVT strategies for optimizing relevant decision parameters. To demonstrate the proposed procedure the paper describes a case study depicting a planned avionics suite upgrade program for a fighter aircraft. Some simplified partial quantitative results are also presented. © 2003 Wiley Periodicals, Inc. Syst Eng 6: 135–151, 2003

[1]  Paul R. Muessig,et al.  Optimizing the selection of VV&A activities: a risk/benefit approach , 1997, WSC '97.

[2]  Yacov Y. Haimes,et al.  Risk modeling, assessment, and management , 1998 .

[3]  Dan X. Houston,et al.  COST OF SOFTWARE QUALITY: A MEANS OF PROMOTING SOFTWARE PROCESS IMPROVEMENT , 1998 .

[4]  Edmund H. Conrow,et al.  Implementing Risk Management on Software Intensive Projects , 1997, IEEE Softw..

[5]  Elaine M. Hall,et al.  Managing Risk: Methods for Software Systems Development , 1998 .

[6]  Tyson R. Browning,et al.  Modeling and analyzing cost, schedule, and performance in complex system product development , 1998 .

[7]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[8]  Cliff Schexnayder,et al.  Defining a beta distribution function for construction simulation , 1999, WSC '99.

[9]  Christopher Ittner,et al.  Exploratory Evidence on the Behavior of Quality Costs , 1996, Oper. Res..

[10]  Yacov Y. Haimes,et al.  Assessing and managing risks to information assurance: A methodological approach , 2002 .

[11]  David E. Bell,et al.  Sequential Testing in Product Development , 2001, Manag. Sci..

[12]  Lars Sörqvist.  Poor quality costing , 1998 .

[13]  John R. Callahan,et al.  Verification and validation in software product line engineering , 1999 .

[14]  James N. Martin,et al.  Systems Engineering Guidebook: A Process for Developing Systems and Products , 1996 .

[15]  Yacov Y Haimes,et al.  Risk Filtering, Ranking, and Management Framework Using Hierarchical Holographic Modeling , 2002, Risk analysis : an official publication of the Society for Risk Analysis.

[16]  Kimberly M Thompson,et al.  Variability and Uncertainty Meet Risk Management and Risk Communication , 2002, Risk analysis : an official publication of the Society for Risk Analysis.

[17]  B Dean Edwin,et al.  Cost Risk Analysis Based on Perception of the Engineering Process , 1986 .

[18]  D. J. Smith.  Risk Simulation and the Appraisal of Investment Projects , 2000 .

[19]  Philippe Jorion.  Value at Risk , 2001 .

[20]  Gregg Rothermel,et al.  An empirical study of regression test application frequency , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.