A systematic review of Web engineering research

This paper uses a systematic literature review as means of investigating the rigor of claims arising from Web engineering research. Rigor is measured using criteria combined from software engineering research. We reviewed 173 papers and results have shown that only 5% would be considered rigorous methodologically. In addition to presenting our results, we also provide suggestions for improvement of Web engineering research based on lessons learnt by the software engineering community.

[1]  Barbara A. Kitchenham,et al.  Modeling Software Measurement Data , 2001, IEEE Trans. Software Eng..

[2]  Jeffrey C. Carver,et al.  Issues in using students in empirical studies in software engineering education , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[3]  Pearl Brereton,et al.  An investigation of software engineering curricula , 2005, J. Syst. Softw..

[4]  Mary Shaw,et al.  Prospects for an engineering discipline of software , 1990, IEEE Software.

[5]  Collins Dictionaries.  Collins English Dictionary , 1991 .

[6]  Adam A. Porter,et al.  Comparing Detection Methods for Software Requirements Inspections: A Replicated Experiment , 1995, IEEE Trans. Software Eng..

[7]  Martin Goldstein,et al.  How We Know: An Exploration of the Scientific Process. , 1980 .

[8]  San Murugesan,et al.  Web Engineering : Managing Diversity and Complexity of Web Application Development , 2001 .

[9]  Mario Piattini,et al.  A Web Metrics Survey Using WQM , 2004, ICWE.

[10]  Victor R. Basili,et al.  The role of experimentation in software engineering: past, current, and future , 1996, Proceedings of IEEE 18th International Conference on Software Engineering.

[11]  Briony J. Oates,et al.  Empirical Methodologies for Web Engineering , 2004, ICWE.

[12]  Barbara A. Kitchenham,et al.  Combining empirical results in software engineering , 1998, Inf. Softw. Technol..

[13]  Oscar Pastor,et al.  Towards the Design of a Metrics Cataloging System by Exploiting Conceptual and Semantic Web Approaches , 2003, ICWE.

[14]  Tore Dybå,et al.  Evidence-based software engineering , 2004, Proceedings. 26th International Conference on Software Engineering.

[15]  Sandro Morasca,et al.  An Operational Process for Goal-Driven Definition of Measures , 2002, IEEE Trans. Software Eng..

[16]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[17]  Will Hayes,et al.  Research synthesis in software engineering: a case for meta-analysis , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[18]  Venkataraman Ramesh,et al.  Research in computer science: an empirical study , 2004, J. Syst. Softw..

[19]  Venkataraman Ramesh,et al.  Research in software engineering: an analysis of the literature , 2002, Inf. Softw. Technol..

[20]  Mauro Pezzè,et al.  The Road to Maturity: Navigating Between Craft and Science , 1997, IEEE Softw..

[21]  Shari Lawrence Pfleeger,et al.  Status Report on Software Measurement , 1997, IEEE Softw..

[22]  Claes Wohlin,et al.  How much information is needed for usage-based reading? A series of experiments , 2002, Proceedings International Symposium on Empirical Software Engineering.

[23]  Tore Dybå,et al.  Conducting realistic experiments in software engineering , 2002, Proceedings International Symposium on Empirical Software Engineering.

[24]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[25]  Walter F. Tichy,et al.  A Controlled Experiment in Maintenance Comparing Design Patterns to Simpler Solutions , 2001, IEEE Trans. Software Eng..

[26]  Shari Lawrence Pfleeger,et al.  Preliminary Guidelines for Empirical Research in Software Engineering , 2002, IEEE Trans. Software Eng..

[27]  H. Dieter Rombach.  Fraunhofer: the German model for applied research and technology transfer , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[28]  Shari Lawrence Pfleeger,et al.  Albert Einstein and Empirical Software Engineering , 1999, Computer.

[29]  Martin Gaedke,et al.  WebComposition: An Object-Oriented Support System for the Web Engineering Lifecycle , 1997, Comput. Networks.

[30]  James Miller.  Can results from software engineering experiments be safely combined? , 1999, Proceedings Sixth International Software Metrics Symposium (Cat. No.PR00403).

[31]  Reidar Conradi,et al.  An empirical study of software reuse vs. defect-density and stability , 2004, Proceedings. 26th International Conference on Software Engineering.

[32]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[33]  E. James Whitehead.  A Proposed Curriculum for a Masters in Web Engineering , 2002, J. Web Eng..

[34]  B. Kitchenham,et al.  Case Studies for Method and Tool Evaluation , 1995, IEEE Softw..

[35]  Robert L. Glass,et al.  Science and substance: a challenge to software engineers , 1994, IEEE Software.

[36]  Jeffrey C. Carver,et al.  Replicating software engineering experiments: addressing the tacit knowledge problem , 2002, Proceedings International Symposium on Empirical Software Engineering.

[37]  Adam A. Porter,et al.  Empirical studies of software engineering: a roadmap , 2000, ICSE '00.

[38]  Stuart R. Faulk.  Achieving industrial relevance with academic excellence: lessons from the Oregon master of software engineering , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[39]  Günther Ruhe.  Methodological contributions to professional education and training , 2000, Proceedings 24th Annual International Computer Software and Applications Conference. COMPSAC2000.

[40]  John M Colford,et al.  Systematic reviews and meta-analyses: an illustrated, step-by-step guide. , 2004, The National medical journal of India.

[41]  William Marsh,et al.  Making resource decisions for software projects , 2004, Proceedings. 26th International Conference on Software Engineering.