Empirical studies in reverse engineering: state of the art and future trends

Starting with the aim of modernizing legacy systems, often written in old programming languages, reverse engineering has extended its applicability to virtually every kind of software system. Moreover, the methods originally designed to recover a diagrammatic, high-level view of the target system have been extended to address several other problems faced by programmers when they need to understand and modify existing software. The authors’ position is that the next stage of development for this discipline will necessarily be based on empirical evaluation of methods. In fact, this evaluation is required to gain knowledge about the actual effects of applying a given approach, as well as to convince the end users of the positive cost–benefit trade offs. The contribution of this paper to the state of the art is a roadmap for the future research in the field, which includes: clarifying the scope of investigation, defining a reference taxonomy, and adopting a common framework for the execution of the experiments.

[1]  T. Lethbridge,et al.  Understanding Software Maintenance Tools: Some Empirical Research , 1997 .

[2]  Jussi Koskinen,et al.  Software Maintenance Cost Estimation and Modernization Support 6 Approaches for Software Modernization and Its Support..................... 37 , 2003 .

[3]  Anneliese Amschler Andrews,et al.  Comprehension processes during large scale maintenance , 1994, Proceedings of 16th International Conference on Software Engineering.

[4]  Elliot Soloway,et al.  Mental models and software maintenance , 1986, J. Syst. Softw..

[5]  Fred D. Davis Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[6]  Hausi A. Müller,et al.  Cognitive design elements to support the construction of a mental model during software visualization , 1997, Proceedings Fifth International Workshop on Program Comprehension. IWPC'97.

[7]  Alexander E. Quilici,et al.  Proceedings on the 5th working conference on reverse engineering , 1998 .

[8]  Kathryn A. Ingle,et al.  Reverse Engineering , 1996, Springer US.

[9]  Jeffrey C. Carver,et al.  Knowledge-Sharing Issues in Experimental Software Engineering , 2004, Empirical Software Engineering.

[10]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[11]  Fred D. Davis,et al.  A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies , 2000, Management Science.

[12]  L. Erlikh,et al.  Leveraging legacy system dollars for e-business , 2000 .

[13]  Gregg Rothermel,et al.  Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact , 2005, Empirical Software Engineering.

[14]  Shari Lawrence Pfleeger,et al.  Principles of survey research part 2: designing a survey , 2002, SOEN.

[15]  Marvin V. Zelkowitz,et al.  Experimental validation in software engineering , 1997, Inf. Softw. Technol..

[16]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[17]  Jeffrey C. Carver,et al.  Replicating software engineering experiments: addressing the tacit knowledge problem , 2002, Proceedings International Symposium on Empirical Software Engineering.

[18]  N. Schneidewind,et al.  Towards an Ontology of software maintenance , 1999 .

[19]  Nancy Pennington,et al.  Comprehension strategies in programming , 1987 .

[20]  Marco Torchiano,et al.  Developing Services for the Wireless Internet , 2006 .

[21]  Ruven E. Brooks Towards a theory of the cognitive processes in computer programming , 1999, Int. J. Hum. Comput. Stud..

[22]  Kate Ehrlich,et al.  Empirical Studies of Programming Knowledge , 1984, IEEE Transactions on Software Engineering.

[23]  James H. Cross,et al.  Reverse engineering and design recovery: a taxonomy , 1990, IEEE Software.

[24]  Stanley Letovsky,et al.  Cognitive processes in program comprehension , 1986, J. Syst. Softw..

[25]  P. Tonella Reverse engineering of object oriented code , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[26]  Andrew Walenstein,et al.  COGNITIVE SUPPORT IN SOFTWARE ENGINEERING TOOLS: A DISTRIBUTED COGNITION FRAMEWORK , 2002 .

[27]  Victor R. Basili,et al.  The role of experimentation in software engineering: past, current, and future , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[28]  Barbara A. Kitchenham,et al.  Combining empirical results in software engineering , 1998, Inf. Softw. Technol..

[29]  Susan Elliott Sim Case Studies for Reverse Engineers , 2006, 2006 13th Working Conference on Reverse Engineering.

[30]  Amela Karahasanovic,et al.  A survey of controlled experiments in software engineering , 2005, IEEE Transactions on Software Engineering.

[31]  Tim Menzies,et al.  The \{PROMISE\} Repository of Software Engineering Databases. , 2005 .

[32]  Shari Lawrence Pfleeger,et al.  Preliminary Guidelines for Empirical Research in Software Engineering , 2002, IEEE Trans. Software Eng..

[33]  Marco Torchiano,et al.  Software Development - Case Studies in Java , 2005 .

[34]  Wang Zhi-jian Using Benchmarking to Advance Research:A Challenge to Software Engineering , 2005 .

[35]  James Miller,et al.  Applying meta-analytical procedures to software engineering experiments , 2000, J. Syst. Softw..

[36]  I K SjobergDag,et al.  A Survey of Controlled Experiments in Software Engineering , 2005 .

[37]  T. Capers Jones,et al.  Estimating software costs , 1998 .

[38]  Reidar Conradi,et al.  Empirical Methods and Studies in Software Engineering: Experiences from Esernet , 2003 .

[39]  Walter F. Tichy,et al.  Proceedings 25th International Conference on Software Engineering , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[40]  Mary Shaw,et al.  What makes good research in software engineering? , 2002, International Journal on Software Tools for Technology Transfer.

[41]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[42]  Bonnie E. John,et al.  Tracking the effectiveness of usability evaluation methods , 1997, Behav. Inf. Technol..

[43]  Hausi A. Müller,et al.  Reverse engineering: a roadmap , 2000, ICSE '00.

[44]  Paul Lukowicz,et al.  Experimental evaluation in computer science: A quantitative study , 1995, J. Syst. Softw..

[45]  Janice Singer Using the American Psychological Association (APA) Style Guidelines to Report Experimental Results , 2007 .

[46]  Richard C. Waters,et al.  Reverse engineering: progress along many dimensions , 1994, CACM.

[47]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[48]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[49]  Shari Lawrence Pfleeger,et al.  Principles of survey research: part 1: turning lemons into lemonade , 2001, SOEN.

[50]  Harry M. Sneed Encapsulating legacy software for use in client/server systems , 1996, Proceedings of WCRE '96: 4rd Working Conference on Reverse Engineering.

[51]  Susan Elliott Sim,et al.  A structured demonstration of five program comprehension tools: lessons learnt , 2000, Proceedings Seventh Working Conference on Reverse Engineering.

[52]  Susan Elliott Sim,et al.  A structured demonstration of program comprehension tools , 2000, Proceedings Seventh Working Conference on Reverse Engineering.

[53]  Iyad Zayour,et al.  Adoption of reverse engineering tools: a cognitive perspective and methodology , 2001, Proceedings 9th International Workshop on Program Comprehension. IWPC 2001.

[54]  Marvin V. Zelkowitz,et al.  Experimental Models for Validating Technology , 1998, Computer.