Planning the verification, validation, and testing process: a case study demonstrating a decision support model

ABSTRACT System VVT (verification, validation, and testing) are essential activities within the domain of system engineering and need to be performed before delivering or marketing products; but to ensure the product complies with customer and engineering requirements in the best possible manner, all three activities should be implemented throughout the product project life cycle. The main objective of VVT is to ensure the customer receives a quality product and neither the customer nor the supplier is burdened with the cost of rework due to failures of the product after delivery. The literature includes very little research for effectively associating VVT methods to VVT activities and also taking into consideration project constraints such as cost, risk, and schedule. In our previous studies, we developed an analytical model that, given project constraints, produced an optimal VVT plan and provided a recovery procedure to manage unexpected events within the VVT process. The use of the model was demonstrated on a sample problem incorporating some unforeseen events. In this paper, we go one step further and use our model to analyse a real industrial project. As expected in a case study, the model has been further developed throughout the case implementation; specifically, a new quantitative model for assessing quality as a function of the test coverage of the VVT process was developed. The project manager found the model output valuable and informative to his work and thus enthusiastically collaborated on this case study research. Besides the report of the case study, we also conducted three focus group discussions to obtain insights, opinions, attitudes, and perceptions of the participants on the proposed analytical model as a decision support model.

[1]  Khadija Tahera,et al.  Integrating virtual and physical testing to accelerate the engineering product development process , 2014, Int. J. Inf. Technol. Manag..

[2]  Jacob Shabi,et al.  Developing an analytical model for planning systems verification, validation and testing processes , 2012, Adv. Eng. Informatics.

[3]  Daniel L. Moody,et al.  Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions , 2005, Data Knowl. Eng..

[4]  Yoram Reich,et al.  SOS – subjective objective system for generating optimal product concepts , 2005 .

[5]  J. Tinsley Oden,et al.  Verification and validation in computational engineering and science: basic concepts , 2004 .

[6]  Daniel L. Moody,et al.  Keynote Speech: Theoretical and Practical Issues in Evaluating the Quality of Conceptual Models , 2002, ER.

[7]  H. Harrington.  Performance improvement: a total poor‐quality cost system , 1999 .

[8]  Tyson R. Browning,et al.  3 Sources of Performance Risk in Complex System Development 1 , 1999 .

[9]  Jerome G. Lake,et al.  4 V & V in Plain English , 1999 .

[10]  Jack R. Meredith,et al.  Building operations management theory through case and field research , 1998 .

[11]  John Edward Gillett,et al.  Hazard Study and Risk Assessment in the Pharmaceutical Industry , 1997 .

[12]  J. Kitzinger,et al.  Qualitative Research: Introducing focus groups , 1995 .

[13]  V. Mitchell.  Organizational Risk Perception and Reduction: A Literature Review , 1995 .

[14]  J. Macintosh.  Focus groups in distance nursing education. , 1993, Journal of advanced nursing.

[15]  David M. McCutcheon,et al.  Conducting case study research in operations management , 1993 .

[16]  Kevin Forsberg,et al.  The Relationship of Systems Engineering to the Project Cycle , 1992 .

[17]  Richard E. Korf,et al.  An Average-Case Analysis of Branch-and-Bound with Applications: Summary of Results , 1992, AAAI.

[18]  Allen S. Lee.  A Scientific Methodology for MIS Case Studies , 1989, MIS Q..

[19]  John A. Zachman,et al.  A Framework for Information Systems Architecture , 1987, IBM Syst. J..

[20]  T. Bonoma.  Case Research in Marketing: Opportunities, Problems, and a Process , 1985 .

[21]  R. Yin.  Case Study Research: Design and Methods , 1984 .

[22]  R. Berk.  The Construction of Rating Instruments for Faculty Evaluation: A Review of Methodological Issues. , 1979 .

[23]  Jill Michaels.  POOR QUALITY , 2015 .

[24]  J. Ivey.  Focus groups. , 2011, Pediatric nursing.

[25]  R. Barbour.  Introducing Focus Groups , 2007 .

[26]  G. Tassey.  The economic impacts of inadequate infrastructure for software testing , 2002 .

[27]  James N. Martin,et al.  Systems Engineering Guidebook: A Process for Developing Systems and Products , 1996 .

[28]  J. Kitzinger.  The methodology of focus groups: the importance of interaction between research participants , 1994 .

[29]  Douglas R. Smith,et al.  Random Trees and the Analysis of Branch and Bound Procedures , 1984, JACM.

[30]  Douglas Robert Smith,et al.  On the computational complexity of branch and bound search strategies , 1979 .