Modeling Examples to Test and Understand Software

One of the oldest techniques to explain abstract concepts is to provide concrete examples. By explaining an abstract concept with a concrete example people make sure that the concept is understood and remembered. Examples in software can be used both to test the software and to illustrate its functionality. Object-oriented programs are built around the concepts of classes, methods and variables, where methods are the atoms of the functionality. But the meta-models of object-oriented languages do not allow developers to associate runnable and composable examples with these concepts and particularly not with methods. Unit tests on the other hand, assure the quality of the units under test and document them. Not being integrated into the language, unit tests are not linked explicitly to their units under test which makes it unnecessarily dif ficult to use them for documenting, typing and debugging software. In addition they are not composable making it hard to develop higher level test scenarios in parallel with higher level objects. In this thesis we analyze unit tests to learn about implicit dependencies among tests and from tests to the methods under test. We develop a technique to partially order unit tests in terms of their covered methods, which reveals possible redundancies due to the lack of composability. We show how partial orders can be used to debug and to comprehend software. We then develop a taxonomy based on several case studies revealing that a high fraction of unit tests already implicitly focuses on single methods. We show that the rest of the tests can be decomposed into commands focusing on single methods. We build a meta-model based on our findings of analyzing test interdependencies which establishes how tests can be explicitly linked to their method under test and how they can be composed to form higher-level test scenarios. We explain how the problems of missing links between tests and units under test are solved using our meta-model. Furthermore, we implemented the meta-model and a first user interface on top of it to give first evidence of how our model supports the developer.

[1]  Gregg Rothermel,et al.  Analyzing Regression Test Selection Techniques , 1996, IEEE Trans. Software Eng..

[2]  Henry Lieberman,et al.  A Session with Tinker: Interleaving Program Testing with Program Writing , 1980, LISP Conference.

[3]  Joseph Robert Horgan,et al.  A study of effective regression testing in practice , 1997, Proceedings The Eighth International Symposium on Software Reliability Engineering.

[4]  Andreas Zeller,et al.  Why Programs Fail: A Guide to Systematic Debugging , 2005 .

[5]  John Mason,et al.  Getting Students to Create Boundary Examples , 2001 .

[6]  Stéphane Ducasse,et al.  Object-oriented legacy system trace-based logic testing , 2006, Conference on Software Maintenance and Reengineering (CSMR'06).

[7]  Stéphane Ducasse,et al.  Runtime bytecode transformation for Smalltalk , 2006, Comput. Lang. Syst. Struct..

[8]  Michele Lanza,et al.  CodeCrawler-lessons learned in building a software visualization tool , 2003, Seventh European Conference onSoftware Maintenance and Reengineering, 2003. Proceedings..

[9]  Jonathan Edwards,et al.  Example centric programming , 2004, SIGP.

[10]  Raymond R. Panko,et al.  Spreadsheets on trial: a survey of research on spreadsheet risks , 1996, Proceedings of HICSS-29: 29th Hawaii International Conference on System Sciences.

[11]  Jürg Nievergelt,et al.  Die Aussagekraft von Beispielen , 2006, Informatik-Spektrum.

[12]  Marti A. Hearst,et al.  The state of the art in automating usability evaluation of user interfaces , 2001, CSUR.

[13]  Kent Beck,et al.  Test-infected: programmers love writing tests , 2000 .

[14]  Oscar Nierstrasz,et al.  Classboxes: controlling visibility of class extensions , 2005, Comput. Lang. Syst. Struct..

[15]  John Maloney,et al.  Back to the Future The Story of Squeak, A Practical Smalltalk Written in Itself , 1997 .

[16]  Alan R. Hevner,et al.  System Test Planning of Software: An Optimization Approach , 2006, IEEE Transactions on Software Engineering.

[17]  Lukas Renggli.  Smallwiki: Collaborative content management , 2003 .

[18]  Volume 28 , 2003, Intensive Care Medicine.

[19]  Linda M. Wills,et al.  Reverse Engineering , 1996, Springer US.

[20]  Timothy M. O'Brien,et al.  Maven - a developer's notebook , 2005 .

[21]  Arie van Deursen,et al.  Refactoring test code , 2001 .

[22]  Oscar Nierstrasz,et al.  Composing Unit Tests , 2005 .

[23]  Philippe Marschall.  Detecting the Methods under Test in Java , 2005 .

[24]  Steve Freeman,et al.  Endo-testing: unit testing with mock objects , 2001 .

[25]  John R. Anderson,et al.  Learning to Program in LISP , 1984, Cogn. Sci..

[26]  Andreas Holzinger,et al.  Usability engineering methods for software developers , 2005, CACM.

[27]  Michael D. Ernst,et al.  Reducing wasted development time via continuous testing , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[28]  Ivar Jacobson,et al.  Object Design: Roles, Responsibilities, and Collaborations , 2002 .

[29]  A. Favero,et al.  Italy , 1996, The Lancet.

[30]  Arie van Deursen,et al.  Predicting class testability using object-oriented metrics , 2004, Source Code Analysis and Manipulation, Fourth IEEE International Workshop on.

[31]  Stéphane Ducasse,et al.  Yesterday's Weather: guiding early reverse engineering efforts by summarizing the evolution of changes , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[32]  Rick Mugridge,et al.  Agile Test Composition , 2005, XP.

[33]  Evans,et al.  Domain-driven design , 2003 .

[34]  Rainer Koschke,et al.  Journal of Software Maintenance and Evolution: Research and Practice Software Visualization in Software Maintenance, Reverse Engineering, and Re-engineering: a Research Survey , 2022 .

[35]  Phillip D. Summers,et al.  A Methodology for LISP Program Construction from Examples , 1977, J. ACM.

[36]  Gregg Rothermel,et al.  Prioritizing test cases for regression testing , 2000, ISSTA '00.

[37]  Stéphane Ducasse,et al.  Characterizing the evolution of class hierarchies , 2005, Ninth European Conference on Software Maintenance and Reengineering.

[38]  William G. Griswold,et al.  An Overview of AspectJ , 2001, ECOOP.

[39]  Gregg Rothermel,et al.  The impact of test suite granularity on the cost-effectiveness of regression testing , 2002, ICSE '02.

[40]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[41]  Oscar Nierstrasz,et al.  Ordering broken unit tests for focused debugging , 2004, 20th IEEE International Conference on Software Maintenance, 2004. Proceedings..

[42]  W. Buxton.  Human-Computer Interaction , 1988, Springer Berlin Heidelberg.

[43]  Mark N. Wegman,et al.  Execution Patterns in Object-Oriented Visualization , 1998, COOTS.

[44]  Ralph E. Johnson,et al.  Wrappers to the Rescue , 1998, ECOOP.

[45]  Arie van Deursen,et al.  The Video Store Revisited – Thoughts on Refactoring and Testing , 2002 .

[46]  Stéphane Ducasse,et al.  Analyzing software evolution through feature views , 2006, J. Softw. Maintenance Res. Pract..

[47]  Oscar Nierstrasz,et al.  Software Evolution as the Key to Productivity , 2002, RISSEF.

[48]  Rick Mugridge,et al.  Fit for Developing Software: Framework for Integrated Tests (Robert C. Martin) , 2005 .

[49]  Rothermel Karen,et al.  A Comparative Study of Coarse-and Fine-Grained Safe Regression Test Selection , 1999 .

[50]  Ivan Moore.  Jester - a JUnit test tester , 2001 .

[51]  Henry Lieberman,et al.  A session with Tinker: Interleaving program testing with program design , 1980, LISP Conference.

[52]  Angela Greiling Keane,et al.  NO SILVER BULLET , 2003 .

[53]  Alan C. Kay.  The early history of Smalltalk , 1993, HOPL-II.

[54]  William M. Smith,et al.  A Study of Thinking , 1956 .

[55]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[56]  Atif M. Memon,et al.  GUI ripping: reverse engineering of graphical user interfaces for testing , 2003, 10th Working Conference on Reverse Engineering, 2003. WCRE 2003. Proceedings..

[57]  Standard Glossary of Software Engineering Terminology , 1990 .

[58]  Oscar Nierstrasz,et al.  Towards a Taxonomy of SUnit Tests , 2005, ESUG.

[59]  Richard Pawson,et al.  Naked Objects , 2002, OOPSLA '02.

[60]  Markus Gaelli.  PhD-Symposium: Correlating Unit Tests and Methods under Test , 2004 .

[61]  Jean-Marc Jézéquel.  Object-Oriented Software Engineering with Eiffel , 1996, Addison-Wesley Eiffel in practice series.

[62]  Steve Freeman,et al.  Mock Roles, not Objects , 2004 .

[63]  Mary Lou Soffa,et al.  Hierarchical GUI Test Case Generation Using Automated Planning , 2001, IEEE Trans. Software Eng..

[64]  Stéphane Ducasse,et al.  Evaluating Message Passing Control Techniques in Smalltalk , 1999 .

[65]  Gregg Rothermel,et al.  Test case prioritization: an empirical study , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[66]  David E. Kieras,et al.  The GOMS Family of Analysis Techniques: Tools for Design and Evaluation , 1994 .

[67]  Jonathan Edwards.  Example centric programming , 2004, OOPSLA 2004.

[68]  Stéphane Ducasse,et al.  The Moose Reengineering Environment , 2001 .

[69]  Gregg Rothermel,et al.  A methodology for testing spreadsheets , 2001, TSEM.

[70]  Brandon Dixon,et al.  Extreme Unit Testing : Ordering Test Cases to Maximize Early Testing , 2001 .

[71]  Mike Cohn,et al.  User Stories Applied: For Agile Software Development , 2004 .

[72]  Andreas Zeller,et al.  Simplifying and Isolating Failure-Inducing Input , 2002, IEEE Trans. Software Eng..

[73]  Robert Hirschfeld,et al.  AspectS - Aspect-Oriented Programming with Squeak , 2002, NetObjectDays.

[74]  Bertrand Meyer,et al.  Applying 'design by contract' , 1992, Computer.

[75]  Andreas Zeller,et al.  Finding Failure Causes through Automated Testing , 2000, AADEBUG.

[76]  J. Davenport.  Editor , 1960 .

[77]  A. Zaidman.  Universiteit Antwerpen Faculteit Wetenschappen Departement Wiskunde – Informatica 2005 – 2006 Coevolution of Software and Tests An Initial Assessment , 2006 .

[78]  Anne Watson,et al.  Student-Generated Examples in the Learning of Mathematics , 2002 .

[79]  Mordechai Ben-Menachem,et al.  Writing effective use cases , 2001, SOEN.

[80]  Robert V. Binder,et al.  Testing Object-Oriented Systems: Models, Patterns, and Tools , 1999 .

[81]  Oscar Nierstrasz,et al.  One-Method Commands: Linking Methods and Their Tests , 2004 .

[82]  Peter Schuh,et al.  Easing Test Object Creation in XP , 2001 .

[83]  Alexander Spoon.  Package Universes: Which Components Are Real Candidates? , 2006 .