An architectural model for software testing lesson learned systems

Context: Software testing is a key aspect of software reliability and quality assurance in a context where software development constantly has to overcome mammoth challenges in a continuously changing environment. One of the characteristics of software testing is that it has a large intellectual capital component and can thus benefit from the use of the experience gained from past projects. Software testing can, then, potentially benefit from solutions provided by the knowledge management discipline. There are in fact a number of proposals concerning effective knowledge management related to several software engineering processes. Objective: We defend the use of a lesson learned system for software testing. The reason is that such a system is an effective knowledge management resource enabling testers and managers to take advantage of the experience locked away in the brains of the testers. To do this, the experience has to be gathered, disseminated and reused. Method: After analyzing the proposals for managing software testing experience, significant weaknesses have been detected in the current systems of this type. The architectural model proposed here for lesson learned systems is designed to try to avoid these weaknesses. This model (i) defines the structure of the software testing lessons learned; (ii) sets up procedures for lesson learned management; and (iii) supports the design of software tools to manage the lessons learned. Results: A different approach, based on the management of the lessons learned that software testing engineers gather from everyday experience, with two basic goals: usefulness and applicability. Conclusion: The architectural model proposed here lays the groundwork to overcome the obstacles to sharing and reusing experience gained in the software testing and test management. As such, it provides guidance for developing software testing lesson learned systems.

[1]  Farhad Daneshgar,et al.  Investigating Knowledge Management practices in software development organisations - An Australian experience , 2008, Inf. Softw. Technol..

[2]  Steven C. Wheelwright,et al.  Revolutionizing New Product Development: Quantum Leaps in Speed, Efficiency, and Quality (Революция в разработке продуктов: количественный скачок в скорости, эффективности и качестве) , 1992 .

[3]  Jörg Rech,et al.  Using Wikis to Tackle Reuse in Software Projects , 2007, IEEE Software.

[4]  E. Carayannis,et al.  Post-project reviews as a key project management competence , 2008 .

[5]  Katja Karhu,et al.  Outsourcing and Knowledge Management in Software Testing , 2007, EASE.

[6]  Cem Kaner,et al.  Lessons Learned in Software Testing , 2001 .

[7]  Natalia Juristo Juzgado,et al.  Guest Editors' Introduction: Software Testing Practices in Industry , 2006, IEEE Software.

[8]  James P. O'Connor,et al.  Reuse in Command-and-Control Systems , 1994, IEEE Software.

[9]  Michael Uschold,et al.  Ontologies: principles, methods and applications , 1996, The Knowledge Engineering Review.

[10]  Rosina O. Weber,et al.  Categorizing Intelligent Lessons Learned Systems , 2003 .

[11]  Antonia Bertolino,et al.  Software Testing Research: Achievements, Challenges, Dreams , 2007, Future of Software Engineering (FOSE '07).

[12]  Frank Houdek,et al.  Quality patterns—an approach to packaging software engineering experience , 1997, SSR '97.

[13]  Sira Vegas,et al.  A Characterisation Schema for Software Testing Techniques , 2005, Empirical Software Engineering.

[14]  Rosina O. Weber,et al.  Intelligent lessons learned systems , 2001, Expert Syst. Appl..

[15]  Giuseppe A. Di Lucca,et al.  Testing Web-based applications: The state of the art and future trends , 2006, Inf. Softw. Technol..

[16]  Natalia Juristo Juzgado,et al.  Towards building a solid empirical body of knowledge in testing techniques , 2004, SOEN.

[17]  Javier Andrade Garda,et al.  Guidelines for the development of e-learning systems by means of proactive questions , 2008, Comput. Educ..

[18]  Natalia Juristo Juzgado,et al.  Packaging experiences for improving testing technique selection , 2006, J. Syst. Softw..

[19]  Ralph L. Keeney,et al.  Book Reviews : Scientific Opportunities and Public Needs: Improv ing Priority Setting and Public Input at the National Institutes of Health. Institute of Medicine. Washington, DC: National Academy Press, 1998, 136 pages, $26.00 , 1998 .

[20]  Alfonso Rodríguez-Patón,et al.  Towards a lessons learned system for critical software , 2007, Reliab. Eng. Syst. Saf..

[21]  Morten T. Hansen,et al.  What's your strategy for managing knowledge? , 1999, Harvard business review.

[22]  A. Bertolino,et al.  A Brief Essay on Software Testing , 2004 .

[23]  Javier Andrade Garda,et al.  Lessons learned for the knowledge management systems development , 2003, Proceedings Fifth IEEE Workshop on Mobile Computing Systems and Applications.

[24]  Lionel C. Briand On the many ways software engineering can benefit from knowledge engineering , 2002, SEKE '02.

[25]  K. Wiig Knowledge management: Where did it come from and where will it go? , 1997 .

[26]  Natalia Juristo Juzgado,et al.  Identifying relevant information for testing technique selection - an instatiated characterization schema , 2003, International series in software engineering.

[27]  Gertjan van Heijst,et al.  Corporate memories as a tool for knowledge management , 1997 .

[28]  Cem Kaner,et al.  Testing Computer Software , 1988 .

[29]  Sandra M. Richardson,et al.  Knowledge Partitioning and Knowledge Transfer Mechanisms in Software Testing : An Empirical Investigation , 2007 .

[30]  Olivier Corby,et al.  Methods and tools for corporate knowledge management , 1999, Int. J. Hum. Comput. Stud..

[31]  A. Tversky Features of Similarity , 1977 .

[32]  Edward Lorenz,et al.  Why all this fuss about codified and tacit knowledge , 2002 .

[33]  Asunción Gómez-Pérez,et al.  Ontological Engineering: With Examples from the Areas of Knowledge Management, e-Commerce and the Semantic Web , 2004, Advanced Information and Knowledge Processing.

[34]  Carsten Tautz,et al.  Improving Organizational Memories Through User Feedback , 2000 .

[35]  Javier Andrade Garda,et al.  Formal conceptualisation as a basis for a more procedural knowledge management , 2008, Decis. Support Syst..

[36]  Edward V. Berard Bringing Testing Into the Fold , 1996, IEEE Softw..

[37]  Cem Kaner,et al.  Lessons learned in software testing ; a context - driven approach , 2002 .

[38]  Jay Liebowitz,et al.  Knowledge Management Handbook , 1999 .

[39]  Robert L. Glass,et al.  Software Testing and Industry Needs , 2006, IEEE Softw..

[40]  Milan Zeleny,et al.  Multiple Criteria Decision Making (MCDM) , 2004 .

[41]  Kalyan Moy Gupta,et al.  Bridging the Lesson Distribution Gap , 2001, IJCAI.

[42]  M. Uschold,et al.  Methods and applications , 1953 .

[43]  Thomas H. Davenport,et al.  Book review:Working knowledge: How organizations manage what they know. Thomas H. Davenport and Laurence Prusak. Harvard Business School Press, 1998. $29.95US. ISBN 0‐87584‐655‐6 , 1998 .

[44]  I. Nonaka,et al.  The Knowledge Creating Company , 2008 .

[45]  Duen-Ren Liu,et al.  Collaborative relevance assessment for task-based knowledge support , 2008, Decis. Support Syst..

[46]  Ben Shneiderman,et al.  Dynamic queries for visual information seeking , 1994, IEEE Software.

[47]  I. Nonaka,et al.  How Japanese Companies Create the Dynamics of Innovation , 1995 .

[48]  Ong Kein Wei,et al.  Knowledge management approach in mobile software system testing , 2007, 2007 IEEE International Conference on Industrial Engineering and Engineering Management.

[49]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[50]  Goro Amamiya,et al.  Learning from bugs (software quality control) , 1993, IEEE Software.

[51]  K. D. Joshi,et al.  A formal knowledge management ontology: Conduct, activities, resources, and influences: Research Articles , 2004 .

[52]  Andreas Abecker,et al.  Towards a Well-Founded Technology For Organizational Memories , 1997 .

[53]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[54]  Lionel C. Briand Novel Applications of Machine Learning in Software Testing , 2008, 2008 The Eighth International Conference on Quality Software.

[55]  Guochang Gu,et al.  Research and Application of Knowledge Management Model Oriented Software Testing Process , 2008 .

[56]  Javier Andrade Garda,et al.  A Reference Model for Knowledge Management in Software Engineering , 2006, Eng. Lett..

[57]  Jacob Nørbjerg,et al.  Strategies for Organizational Learning in SPI , 2001 .

[58]  David B. Martin,et al.  'Good' Organisational Reasons for 'Bad' Software Testing: An Ethnographic Study of Testing in a Small Software Company , 2007, 29th International Conference on Software Engineering (ICSE'07).

[59]  Sira Vegas,et al.  Identifying Relevant Information for Testing Technique Selection , 2003, The Springer International Series in Software Engineering.

[60]  Cem Kaner,et al.  Testing Computer Software, Second Edition , 1993 .

[61]  Andreas Abecker,et al.  Corporate Memories for Knowledge Management in Industrial Practice: Prospects and Challenges , 1997, J. Univers. Comput. Sci..

[62]  Rudolf Ramler,et al.  The Role of Experience in Software Testing Practice , 2008, 2008 34th Euromicro Conference Software Engineering and Advanced Applications.

[63]  R. A. Falbo,et al.  Using knowledge management to improve software process performance in a CMM level 3 organization , 2004 .

[64]  Steven C. Wheelwright,et al.  Revolutionizing Product Development: Quantum Leaps in Speed, Efficiency and Quality , 1992 .

[65]  I. K. Mak,et al.  Adaptive Random Testing , 2004, ASIAN.

[66]  Guilherme H. Travassos,et al.  Integrating Verification and Validation Techniques Knowledge into Software Engineering Environments , 2004 .

[67]  Lisa Crispin,et al.  Agile Testing: A Practical Guide for Testers and Agile Teams , 2008 .

[68]  David W. Aha,et al.  Active Delivery for Lessons Learned Systems , 2000, EWCBR.

[69]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[70]  Warren Harrison A software engineering lessons learned repository , 2002, 27th Annual NASA Goddard/IEEE Software Engineering Workshop, 2002. Proceedings..