Test reuse in the spreadsheet paradigm

Spreadsheet languages are widely used by a variety of end users to perform many important tasks. Despite their perceived simplicity, spreadsheets often contain faults. Furthermore, users modify their spreadsheets frequently, which can render previously correct spreadsheets faulty. To address this problem, we previously introduced a visual approach by which users can systematically test their spreadsheets, see where new tests are required after changes, and request automated generation of potentially useful test inputs. To date, however, this approach has not taken advantage of previously developed test cases, which means that users of the approach cannot benefit, when re-testing following changes, from prior testing efforts. We have therefore been investigating ways to add support for test re-use into our spreadsheet testing methodology. In this paper we present a test re-use strategy for spreadsheets, and the algorithms that implement it, and describe their integration into our spreadsheet testing methodology. We report results of a case study examining the application of this strategy.

[1]  Gregg Rothermel,et al.  WYSIWYT testing in the spreadsheet paradigm: an empirical evaluation , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[2]  Wei-Tek Tsai,et al.  Regression testing in an industrial environment , 1998, CACM.

[3]  Margaret M. Burnett,et al.  Forms/3: A first-order visual language to explore the boundaries of the spreadsheet paradigm , 2001, Journal of Functional Programming.

[4]  Elaine J. Weyuker,et al.  An Applicable Family of Data Flow Testing Criteria , 1988, IEEE Trans. Software Eng..

[5]  Alan F. Blackwell,et al.  See What You Need: Helping End-users to Build Abstractions , 2001, J. Vis. Lang. Comput..

[6]  B. VanVoorst,et al.  Toward visual programming languages for steering scientific computations , 1994, IEEE Computational Science and Engineering.

[7]  Raymond R. Panko,et al.  What we know about spreadsheet errors , 1998 .

[8]  Gregg Rothermel,et al.  Incorporating incremental validation and impact analysis into spreadsheet maintenance: an empirical study , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[9]  Gregg Rothermel,et al.  Testing Homogeneous Spreadsheet Grids with the "What You See Is What You Test" Methodology , 2002, IEEE Trans. Software Eng..

[10]  Allen L. Ambler,et al.  Visual representation and manipulation of matrices , 1992, J. Vis. Lang. Comput..

[11]  Gregg Rothermel,et al.  A methodology for testing spreadsheets , 2001, TSEM.

[12]  Philip T. Cox,et al.  Expanding the utility of spreadsheets through the integration of visual programming and user interface objects , 1996, AVI '96.

[13]  John D. Gould,et al.  An experimental study of people creating spreadsheets , 1987, TOIS.

[14]  Gregg Rothermel,et al.  Testing strategies for form-based visual programs , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[15]  Jonathan J. Cadiz,et al.  Does continuous visual feedback aid debugging in direct-manipulation programming systems? , 1997, CHI.

[16]  Brad A. Myers Graphical techniques in a spreadsheet for specifying user interfaces , 1991, CHI '91.

[17]  Bonnie A. Nardi,et al.  A Small Matter of Programming: Perspectives on End User Computing , 1993 .

[18]  Margaret M. Burnett,et al.  Graphical definitions: expanding spreadsheet languages through direct manipulation and gestures , 1998, TCHI.

[19]  David S. Rosenblum,et al.  TESTTUBE: a system for selective regression testing , 1994, Proceedings of 16th International Conference on Software Engineering.

[20]  Gregg Rothermel,et al.  What you see is what you test: a methodology for testing form-based visual programs , 1998, Proceedings of the 20th International Conference on Software Engineering.

[21]  Gregg Rothermel,et al.  Analyzing Regression Test Selection Techniques , 1996, IEEE Trans. Software Eng..

[22]  John Riedl,et al.  A spreadsheet approach to information visualization , 1997, Proceedings of VIZ '97: Visualization Conference, Information Visualization Symposium and Parallel Rendering Symposium.

[23]  Mary Lou Soffa,et al.  An incremental approach to unit testing during maintenance , 1988, Proceedings. Conference on Software Maintenance, 1988..

[24]  Gregg Rothermel,et al.  Scaling up a "What you see is what you test" methodology to spreadsheet grids , 1999, Proceedings 1999 IEEE Symposium on Visual Languages.

[25]  Phyllis G. Frankl,et al.  Pythia: a regression test selection tool based on textual differencing , 1997 .

[26]  Gregg Rothermel,et al.  Automated test case generation for spreadsheets , 2002, ICSE '02.

[27]  Bogdan Korel,et al.  The chaining approach for software test data generation , 1996, TSEM.

[28]  John Riedl,et al.  A spreadsheet approach to information visualization , 1997, INFOVIS.