The Role of Experience in Software Testing Practice

Practitioners report that experience plays an important role in effective software testing. We investigate the role of experience in a multiple case study about three successful projects conducted at Siemens Austria and document the state of practice in testing software systems. The studied projects were employed from the domains telecommunications, insurance and banking, as well as safety-critical railway systems. The study shows that test design is to a considerable extent based on experience in all three projects and that experience-based testing is an important supplementary approach to requirements-based testing. The study further analyzes the different sources of experience, the perceived value of experience for testing, and the measures taken to manage and evolve this experience.

[1]  Katharina Probst,et al.  Considering users and their opinions in knowledge management systems , 2008, JCDL '08.

[2]  Natalia Juristo Juzgado,et al.  Packaging experiences for improving testing technique selection , 2006, J. Syst. Softw..

[3]  Adam A. Porter,et al.  Empirical studies of software engineering: a roadmap , 2000, ICSE '00.

[4]  Matthias Heindl,et al.  Issues in Testing Dependable Event-Based Systems at a Systems Integration Company , 2007, The Second International Conference on Availability, Reliability and Security (ARES'07).

[5]  Alain Abran,et al.  The Guide to the Software Engineering Body of Knowledge , 1999, IEEE Softw..

[6]  Meg Lewis,et al.  Building Blocks for Success , 2008 .

[7]  Rudolf Ramler,et al.  Testing High-Reliability Software for Continuous Casting Steel Plants - Experiences and Lessons Learned from Siemens VAI , 2007, 33rd EUROMICRO Conference on Software Engineering and Advanced Applications (EUROMICRO 2007).

[8]  P. Muller.  33rd Euromicro Conference on Software Engineering and Advanced Applications - Seaa 2007 , 2008 .

[9]  Per Runeson,et al.  A survey of unit testing practices , 2006, IEEE Software.

[10]  Meliha Handzic,et al.  Managing Software Engineering Knowledge , 2010, Springer Berlin Heidelberg.

[11]  Juha Itkonen,et al.  Exploratory testing: a multiple case study , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[12]  Phillip G. Armour.  The unconscious art of software testing , 2005, CACM.

[13]  Barry Boehm,et al.  An Empirical Study on MBASE and LeanMBASE , 2007, ESEM 2007.

[14]  David B. Martin,et al.  'Good' Organisational Reasons for 'Bad' Software Testing: An Ethnographic Study of Testing in a Small Software Company , 2007, 29th International Conference on Software Engineering (ICSE'07).

[15]  M. Lindvall,et al.  Knowledge management in software engineering , 2002, IEEE Software.

[16]  Katja Karhu,et al.  Observing Software Testing Practice from the Viewpoint of Organizations and Knowledge Management , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[17]  Gilbert Probst,et al.  Managing Knowledge: Building Blocks for Success , 1999 .

[18]  van E.P.W.M. Veenendaal,et al.  The Testing Practitioner , 2007 .

[19]  Cem Kaner,et al.  Lessons learned in software testing ; a context - driven approach , 2002 .