NRL invitational workshop on testing and proving: two approaches to assurance

Overview The Naval Research Laboratory sponsored this workshop to invigorate research in both program verification and program testing through cross-fertilization, to document the state of the art and practice in both areas, and to identify current assurance requirements and techniques for meeting them. Approximately 50 invited researchers and practitioners participated over a 3-day period. The workshop was held in conjunction with the COMPASS 86 conference, and the initial and final days of the workshop were open to COMPASS attendees. Tutorials characterizing the current state of testing and proving techniques and identifying industry and government assurance requirements occupied the first day of the workshop. These provided a common base for five discussion groups held during the second day. The discussion groups addressed (1) the role of specifications in testing and proving, (2) hybrid approaches of testing and proving, (3) levels of assurance, (4) interactions between testing/proving and software engineering, and (5) cost effectiveness. The leader of each of these groups summarized the discussions and conclusions on the final morning. Har-lan Mills of IBM then provided a critique of these results. Amrit Goel of Syra-cuse provided an impromptu closing talk on alternative statistical models for software testing The accompanying summaries were written by the discussion group leaders following the workshop as a means of documenting our results and circulating them to a wider audience. Although the leaders have tried to record faithfully the results of the discussions, these summaries (and this preface) have not been reviewed or approved by the other participants of the groups. In addition, Dr. Mills has provided a note on his thoughts about the workshop topics. Comments An interesting fact that cannot be gleaned from examining the group summaries is the popularity of each group as determined by the number of participants requesting to be in that group. For example, we had originally planned for a sixth group on domains of applicability, but could find no takers, and the group on cost effectiveness was not popular among those participants who ACM SIGSOFT SOFTWARE ENGINEERING NOTES vol II no 5 Oct Ig~8 Page: 84 expressed a group preference. Perhaps the two phenomena are related since a major factor in determining whether an approach is useful in a certain domain is its cost effectiveness in that domain. In any event, the concept of a domain of applicability for each approach arose often in the groups. To get a tighter grip …