Calibration and Validation of the Checkpoint Model to the Air Force Electronic Systems Center Software Database

Abstract : This research effort focused on the calibration and validation of CHECKPOINT Version 2.3.1, a computerized software cost estimating tool, to the USAF Electronic Systems Center (ESC) software database. This thesis is a direct follow-on to a 1996 CHECKPOINT study at the Air Force Institute of Technology, which successfully calibrated and validated CHECKPOINT to the SMC software database. While this research generally parallels the methodology in the aforementioned study, it offers advancements in the CHECKPOINT calibration and validation procedure, and it refines the data stratification process and the statistical analyses employed. After stratifying the ESC software database into ten usable data sets, the author calibrated and validated the CHECKPOINT model on each data set. Although the results of this study exhibited occasional improvements in estimating accuracy for both the calibration and validation subsets, the model generally failed to satisfy the accuracy criteria used to assess overall calibration success and estimating accuracy (MMRE 0.75). Thus, the CHECKPOINT model was not successfully calibrated or validated to the 1997 version of the ESC database. The results of this study illuminate the need for complete, accurate and homogeneous data as a requirement for a successful calibration and validation effort.