A mapping study to investigate component-based software system metrics

A component-based software system (CBSS) is a software system that is developed by integrating components that have been deployed independently. In the last few years, many researchers have proposed metrics to evaluate CBSS attributes. However, the practical use of these metrics can be difficult. For example, some of the metrics have concepts that either overlap or are not well defined, which could hinder their implementation. The aim of this study is to understand, classify and analyze existing research in component-based metrics, focusing on approaches and elements that are used to evaluate the quality of CBSS and its components from a component consumer's point of view. This paper presents a systematic mapping study of several metrics that were proposed to measure the quality of CBSS and its components. We found 17 proposals that could be applied to evaluate CBSSs, while 14 proposals could be applied to evaluate individual components in isolation. Various elements of the software components that were measured are reviewed and discussed. Only a few of the proposed metrics are soundly defined. The quality assessment of the primary studies detected many limitations and suggested guidelines for possibilities for improving and increasing the acceptance of metrics. However, it remains a challenge to characterize and evaluate a CBSS and its components quantitatively. For this reason, much effort must be made to achieve a better evaluation approach in the future.

[1]  J. Voas,et al.  Software Testability: The New Verification , 1995, IEEE Softw..

[2]  Nasib S. Gill,et al.  Few important considerations for deriving interface complexity metric for component-based systems , 2004, SOEN.

[3]  Lianping Chen,et al.  A systematic review of evaluation of variability management approaches in software product lines , 2011, Inf. Softw. Technol..

[4]  N. Schneidewind,et al.  Towards an Ontology of software maintenance , 1999 .

[5]  Soo Dong Kim,et al.  Component metrics to measure component quality , 2001, Proceedings Eighth Asia-Pacific Software Engineering Conference.

[6]  Nasib S. Gill,et al.  Software size prediction before coding , 2004, SOEN.

[7]  Jeffrey C. Carver,et al.  Characterizing software architecture changes: A systematic review , 2010, Inf. Softw. Technol..

[8]  José Ambrosio Toval Álvarez,et al.  A systematic review of UML model consistency management , 2009, Inf. Softw. Technol..

[9]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[10]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007 .

[11]  Nael Salman,et al.  Complexity Metrics AS Predictors of Maintainability and Integrability of Software components , 2006 .

[12]  Sajjad Mahmood,et al.  A survey of component based system quality assurance and assessment , 2005, Inf. Softw. Technol..

[13]  Parvinder S. Sandhu,et al.  A Complexity Measure for Java Bean based Software Components , 2008 .

[14]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[15]  Martin Loomes,et al.  Applying software metrics to formal specifications: a cognitive approach , 1998, Proceedings Fifth International Software Metrics Symposium. Metrics (Cat. No.98TB100262).

[16]  Magne Jørgensen,et al.  A Systematic Review of Software Development Cost Estimation Studies , 2007, IEEE Transactions on Software Engineering.

[17]  Kristian Kirsch Microsoft Visual C 2008 Step By Step , 2008 .

[18]  Miguel Goulão,et al.  Cross-Validation of a Component Metrics Suite , 2004 .

[19]  Rengaramanujam Srinivasan,et al.  A retrospective on software component quality models , 2008, SOEN.

[20]  Ted W. Faison,et al.  Component Based Development With Visual C , 2002 .

[21]  Sandeep Purao,et al.  Product metrics for object-oriented systems , 2003, CSUR.

[22]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[23]  Antonio Vallecillo,et al.  A Survey on the Quality Information Provided by Software Component Vendors , 2003 .

[24]  Stéphane Ducasse,et al.  Object-Oriented Metrics in Practice , 2005 .

[25]  Clemens A. Szyperski,et al.  Component software - beyond object-oriented programming, 2nd Edition , 2002, Addison-Wesley component software series.

[26]  Guo Wei,et al.  Metrics of Graph Abstraction for Component-Based Software Architecture , 2009, 2009 WRI World Congress on Computer Science and Information Engineering.

[27]  Mike Jackson,et al.  Software measurement methods: recipes for success? , 1994, Inf. Softw. Technol..

[28]  Narsingh Deo,et al.  Graph Theory with Applications to Engineering and Computer Science , 1975, Networks.

[29]  N.R. Malik,et al.  Graph theory with applications to engineering and computer science , 1975, Proceedings of the IEEE.

[30]  Sajjad Mahmood,et al.  A complexity measure for UML component-based system specification , 2008 .

[31]  Jeffrey C. Carver,et al.  Characterizing Software Architecture Changes: An Initial Study , 2007, ESEM 2007.

[32]  Elisa Yumi Nakagawa,et al.  Using systematic mapping to explore software architecture knowledge , 2010, SHARK '10.

[33]  Manik Lal Das,et al.  Evaluation of a Suite of Metrics for Component Based Software Engineering (CBSE) , 2009 .

[34]  Horia F. Pop,et al.  A conceptual framework for component-based system metrics definition , 2010, 9th RoEduNet IEEE International Conference.

[35]  Kai Qian,et al.  Component-Oriented Programming , 2005 .

[36]  Anil Kumar Tripathi,et al.  Testing Component-Based Software: What It has to do with Design and Component Selection , 2011, J. Softw. Eng. Appl..

[37]  Helen D. Karatza Cache affinity and resequencing in a shared-memory multiprocessing system , 2000, J. Syst. Softw..

[38]  Petr Hnětynka Component Model for Unified Deployment of Distributed Component-based Software , 2004 .

[39]  Pearl Brereton,et al.  Using mapping studies as the basis for further research - A participant-observer case study , 2011, Inf. Softw. Technol..

[40]  Richard Lai,et al.  Component Point: A system-level size measure for Component-Based Software Systems , 2010, J. Syst. Softw..

[41]  Mark Lycett,et al.  Development Of Component-based Information Systems (Advances in Management Information Systems) , 2005 .

[42]  V. Lakshmi Narasimhan,et al.  Some theoretical considerations for a suite of metrics for the integration of software components , 2007, Inf. Sci..

[43]  S.M. Alhazbi Measuring the complexity of component-based system architecture , 2004, Proceedings. 2004 International Conference on Information and Communication Technologies: From Theory to Applications, 2004..

[44]  Antonio Vallecillo,et al.  Quality Attributes for COTS Components , 2002 .

[45]  Jan Bosch,et al.  WCOP '98 Summary of the Third International Workshop on Component-Oriented Programming , 1998, ECOOP Workshops.

[46]  Jeffrey M. Voas,et al.  Dependability certification of software components , 2000, J. Syst. Softw..

[47]  Mario Piattini,et al.  A systematic review measurement in software engineering: State-of-the-art in Measures , 2006, ICSOFT.

[48]  George Valença,et al.  Accepted Manuscript Requirements Engineering for Software Product Lines: a Systematic Literature Review Accepted Manuscript Requirements Engineering for Software Product Lines: a Systematic Literature Review Accepted Manuscript , 2022 .

[49]  Renata Pontin de Mattos Fortes,et al.  A systematic review of domain analysis tools , 2010, Inf. Softw. Technol..

[50]  Banu Diri,et al.  A systematic review of software fault prediction studies , 2009, Expert Syst. Appl..

[51]  Miguel Goulão,et al.  Composition assessment metrics for CBSE , 2005 .

[52]  Shari Lawrence Pfleeger,et al.  Towards a Framework for Software Measurement Validation , 1995, IEEE Trans. Software Eng..

[53]  V. Prasanna Venkatesan,et al.  A Metrics Suite for Measuring Software Components , 2009, J. Convergence Inf. Technol..

[54]  Nenad Medvidovic,et al.  Using service utilization metrics to assess the structure of product line architectures , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[55]  Jun Han A comprehensive interface definition framework for software components , 1998, Proceedings 1998 Asia Pacific Software Engineering Conference (Cat. No.98EX240).

[56]  Wan M.N. Wan-Kadir,et al.  A review on the component evaluation approaches to support software reuse , 2008, 2008 International Symposium on Information Technology.

[57]  Linda M. Laird,et al.  Software Measurement and Estimation: A Practical Approach , 2006 .

[58]  Nasib S. Gill,et al.  Dependency and interaction oriented complexity metrics of component-based systems , 2008, SOEN.

[59]  H. Conrad Cunningham,et al.  Mapping component specifications to Enterprise JavaBeans implementations , 2004, ACM-SE 42.

[60]  Jianguo Chen,et al.  A Review of Component Coupling Metrics for Component-Based Development , 2009, 2009 WRI World Congress on Software Engineering.

[61]  Pearl Brereton,et al.  The value of mapping studies - A participant-observer case study , 2010, EASE.

[62]  Michele Lanza,et al.  Object-Oriented Metrics in Practice - Using Software Metrics to Characterize, Evaluate, and Improve the Design of Object-Oriented Systems , 2006 .

[63]  Mario Piattini,et al.  Towards a consistent terminology for software measurement , 2006, Inf. Softw. Technol..

[64]  Saeed Araban,et al.  Interface metrics for reusability analysis of components , 2004, 2004 Australian Software Engineering Conference. Proceedings..

[65]  Arvinder Kaur,et al.  Component Based Software Engineering , 2010 .

[66]  Silvio Romero de Lemos Meira,et al.  Quality Attributes for a Component Quality Model , 2005 .

[67]  Pearl Brereton,et al.  Lessons from applying the systematic literature review process within the software engineering domain , 2007, J. Syst. Softw..

[68]  Hironori Washizaki,et al.  A metrics suite for measuring reusability of software components , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[69]  Helen Sharp,et al.  Motivation in Software Engineering: A systematic literature review , 2008, Inf. Softw. Technol..

[70]  Antonio Vallecillo,et al.  Measuring the usability of software components , 2006, J. Syst. Softw..

[71]  Tony Gorschek,et al.  A systematic review of domain analysis solutions for product lines , 2009, J. Syst. Softw..

[72]  Paula Gomes Mian,et al.  Systematic Review in Software Engineering , 2005 .

[73]  Pearl Brereton,et al.  Using Mapping Studies in Software Engineering , 2008, PPIG.

[74]  P. S. Grover,et al.  Empirical Evaluation and Validation of Interface Complexity Metrics for Software Components , 2008, Int. J. Softw. Eng. Knowl. Eng..

[75]  Barbara A. Kitchenham,et al.  Modeling Software Measurement Data , 2001, IEEE Trans. Software Eng..

[76]  Pearl Brereton,et al.  Systematic literature reviews in software engineering - A systematic literature review , 2009, Inf. Softw. Technol..

[77]  Miguel Goulão,et al.  Software Components Evaluation: an Overview , 2004 .

[78]  Octavian Paul Rotaru,et al.  Reusability metrics for software components , 2005, The 3rd ACS/IEEE International Conference onComputer Systems and Applications, 2005..

[79]  GhafoorArif,et al.  Software Engineering Metrics for COTS-Based Systems , 2001 .

[80]  Latika Kharb,et al.  Complexity metrics for component-oriented software systems , 2008, SOEN.

[81]  Anil Tripathi Dependence Analysis of Component Based Software through Assumptions , 2011 .

[82]  Tore Dybå,et al.  Empirical studies of agile software development: A systematic review , 2008, Inf. Softw. Technol..

[83]  Michael Halvorson,et al.  Microsoft Visual Basic 2005 step by step , 1995 .

[84]  Clemens A. Szyperski,et al.  Component software - beyond object-oriented programming , 2002 .

[85]  Giuseppe De Pietro,et al.  Automatic implementation of constraints in component based applications , 2005, Inf. Softw. Technol..

[86]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[87]  Mark Staples,et al.  Experiences using systematic review guidelines , 2006, J. Syst. Softw..

[88]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[89]  Jon Hopkins,et al.  Component primer , 2000, CACM.

[90]  Mark Lycett,et al.  The Development of Component-based Information Systems , 2016 .

[91]  Javier Tuya,et al.  Testing in Service Oriented Architectures with dynamic binding: A mapping study , 2011, Inf. Softw. Technol..

[92]  P. S. Grover,et al.  Dependency analysis for component-based software systems , 2009, SOEN.

[93]  John Sharp,et al.  Microsoft Visual C♯ 2005 step by step , 2005 .

[94]  Debra J. Richardson,et al.  Describing Dependencies in Component Access Points , 2001 .

[95]  Michael Grottke,et al.  A systematic literature review of software quality cost research , 2011, J. Syst. Softw..

[96]  Ivica Crnkovic,et al.  Building Reliable Component-Based Software Systems , 2002 .

[97]  Paula Kotzé,et al.  Assessment of Coupling and Cohesion for Component-Based Software by using Shannon Languages , 2004, Trans. SDPS.

[98]  Barbara Kitchenham,et al.  What's up with software metrics? - A preliminary mapping study , 2010, J. Syst. Softw..

[99]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[100]  Shari Lawrence Pfleeger,et al.  Preliminary Guidelines for Empirical Research in Software Engineering , 2002, IEEE Trans. Software Eng..

[101]  Elaine J. Weyuker,et al.  Evaluating Software Complexity Measures , 2010, IEEE Trans. Software Eng..

[102]  Nasib S. Gill,et al.  Component-based measurement: few useful guidelines , 2003, SOEN.

[103]  Jung-Yeop Kim,et al.  Component-based metrics applying the strength of dependency between classes , 2009, SAC '09.

[104]  George T. Heineman,et al.  Building instead of buying: a rebuttal , 2001 .

[105]  Cem Kaner,et al.  Software Engineering Metrics: What Do They Measure and How Do We Know? , 2004 .