Systematic Metric Systems Engineering: Reference Architecture and Process Model

In the recent past, the research community contributed a considerable amount of work to extend the understanding of the theoretical foundations of metric systems. However, a dedicated approach for engineering of metric systems is still missing. As a result, they are often developed chaotically. This thesis introduces MeDIC – a dedicated metric systems engineering approach, which fills this gap. MeDIC supports flexible conception, design, construction, and operation of metric systems. The approach is based on two pillars: the MeDIC process model and the MeDIC reference architecture. They integrate software engineering best practices, emerging concepts, and well-established metric-related standards and techniques. The MeDIC reference architecture provides technical guides with a layered architecture blue-print of loosely interconnected micro-services. The MeDIC process model provides ready-to-use process elements and artifacts (fragments), which drastically ease the setup of a specific engineering process. The reference architecture and process model are based on formal foundations, which provide additional benefits for conceptual analysis of metrics systems. Various field studies, in cooperation with multiple industry partners, were used to evaluate the approach. This thesis provides insight into three selected field studies, which utilize various aspects of MeDIC in industrial environments. The evaluation shows the practical application, usefulness, and efficiency of MeDIC. Challenges associated with the development and operation of metric systems in industrial environments can thus be overcome by MeDIC. As a result, the engineering of better, more reliable, and sustainable metric systems is possible.

[1]  Oscar Pastor,et al.  Towards the Design of a Metrics Cataloging System by Exploiting Conceptual and Semantic Web Approaches , 2003, ICWE.

[2]  Tim Bray,et al.  Internet Engineering Task Force (ietf) the Javascript Object Notation (json) Data Interchange Format , 2022 .

[3]  Hans van Vliet,et al.  Measurement program success factors revisited , 2001, Inf. Softw. Technol..

[4]  Evan Asfoura,et al.  A General Model for Measurement Improvement , 2009, IWSM/Mensura.

[5]  Erhard Rahm,et al.  Generic schema matching, ten years later , 2011, Proc. VLDB Endow..

[6]  Colin Kirsopp Measurement and the software development process , 2001 .

[7]  T. Wright,et al.  Organizational Benchmarking Using the ISBSG Data Repository , 2001, IEEE Softw..

[8]  Luigi Lavazza Providing Automated Support for the GQM Measurement Process , 2000, IEEE Softw..

[9]  Richard W. Selby,et al.  Measurement-driven dashboards enable leading indicators for requirements and design of large-scale systems , 2005, 11th IEEE International Software Metrics Symposium (METRICS'05).

[10]  Vieri Del Bianco,et al.  Model-based functional size measurement , 2008, ESEM '08.

[11]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[12]  Surajit Chaudhuri,et al.  An overview of data warehousing and OLAP technology , 1997, SGMD.

[13]  Farhad Arbab,et al.  A Logical Interface Description Language for Components , 2000, COORDINATION.

[14]  Mario Piattini,et al.  Managing software process measurement: A metamodel-based approach , 2007, Inf. Sci..

[15]  Félix García,et al.  Software Measurement Programs in SMEs - Defining Software Indicators: A Methodological Framework , 2007, PROFES.

[16]  Nicolai M. Josuttis,et al.  Soa In Practice The Art Of Distributed System Design , 2007 .

[17]  Stanley M. Sutton,et al.  The problem of private information in large software organizations , 2011, ICSSP '11.

[18]  R. Kaplan,et al.  The balanced scorecard--measures that drive performance. , 2015, Harvard business review.

[19]  Scott Lewicki,et al.  Bootstrapping Process Improvement Metrics: CMMI Level 4 Process Improvement Metrics in a Level 3 World , 2011, 2011 44th Hawaii International Conference on System Sciences.

[20]  Philip Johnson You can’t even ask them to push a button: Toward ubiquitous, developer-centric, empirical software engineering , 2001 .

[21]  Roberto Chinnici,et al.  Web Services Description Language (WSDL) Version 2.0 Part 1: Core Language , 2007 .

[22]  Wayne W. Eckerson Performance Dashboards: Measuring, Monitoring, and Managing Your Business , 2005 .

[23]  Steven A. Melnyk,et al.  Metrics and performance measurement in operations management: dealing with the metrics maze , 2004 .

[24]  V. Basili Software modeling and measurement: the Goal/Question/Metric paradigm , 1992 .

[25]  Martin Kunz Framework for a service-oriented measurement infrastructure , 2009 .

[26]  Faris Nizamic,et al.  Service-Oriented Computing , 2010, Lecture Notes in Computer Science.

[27]  Peter Liggesmeyer,et al.  Visualization of Software and Systems as Support Mechanism for Integrated Software Project Control , 2009, HCI.

[28]  Tore Dybå,et al.  An empirical investigation of the key factors for success in software process improvement , 2005, IEEE Transactions on Software Engineering.

[29]  Oscar Pastor,et al.  Towards a Reusable Repository for Web Metrics , 2002, J. Web Eng..

[31]  Jouni Markkula,et al.  How Does a Measurement Programme Evolve in Software Organizations? , 2008, PROFES.

[32]  Sandro Morasca,et al.  Towards a theoretical framework for measuring software attributes , 1997, Proceedings Fourth International Software Metrics Symposium.

[33]  F.T. Anbari,et al.  Earned Value Project Management Method and Extensions , 2003, IEEE Engineering Management Review.

[34]  Horst Lichter,et al.  Experience on a Microservice-Based Reference Architecture for Measurement Systems , 2014, 2014 21st Asia-Pacific Software Engineering Conference.

[35]  Philippe Kruchten,et al.  The 4+1 View Model of Architecture , 1995, IEEE Softw..

[36]  Juan Antonio de la Puente,et al.  A Software Architecture Evaluation Model , 1998, ESPRIT ARES Workshop.

[37]  Steve Vinoski,et al.  CORBA: integrating diverse applications within distributed heterogeneous environments , 1997, IEEE Commun. Mag..

[38]  Horst Lichter,et al.  Software Engineering - Grundlagen, Menschen, Prozesse, Techniken , 2007 .

[39]  Chris Phillips,et al.  Logging and Log Management: The Authoritative Guide to Understanding the Concepts Surrounding Logging and Log Management , 2012 .

[40]  Horst Lichter,et al.  Towards a Maintainable Federalist Enterprise Measurement Infrastructure , 2013, 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 8th International Conference on Software Process and Product Measurement.

[41]  Geert Poels,et al.  Metamodeling the Quality of the Web Development Process' Intermediate Artifacts , 2007, ICWE.

[42]  H. Dieter Rombach Integrated Software Process and Product Lines , 2013, Perspectives on the Future of Software Engineering.

[43]  Jürgen Münch,et al.  Evaluating Software Project Control Centers in Industrial Environments , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[44]  Prakash M. Nadkarni,et al.  Guidelines for the effective use of entity-attribute-value modeling for biomedical databases , 2007, Int. J. Medical Informatics.

[45]  Reinhold Plösch,et al.  The Quamoco product quality modelling and assessment approach , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[46]  Barbara A. Kitchenham,et al.  The use and usefulness of the ISO/IEC 9126 quality standard , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[47]  Erich Schikuta,et al.  Quo vadis interface definition languages? Towards a interface definition language for RESTful services , 2009, 2009 IEEE International Conference on Service-Oriented Computing and Applications (SOCA).

[48]  Alon Y. Halevy,et al.  Enterprise information integration: successes, challenges and controversies , 2005, SIGMOD '05.

[49]  Sandro Morasca,et al.  On the application of measurement theory in software engineering , 2004, Empirical Software Engineering.

[50]  Luis Olsina,et al.  Towards an ontology for software metrics and indicators as the foundation for a cataloging Web system , 2003, Proceedings of the IEEE/LEOS 3rd International Conference on Numerical Simulation of Semiconductor Optoelectronic Devices (IEEE Cat. No.03EX726).

[51]  Reiner R. Dumke,et al.  Agent Measurement and Self-Measuring Agent-Based Systems , 2001 .

[52]  Luis Olsina,et al.  Ontology for Software Metrics and Indicators: Building Process and Decisions Taken , 2004, ICWE.

[53]  Luis Olsina,et al.  Towards the Support of Contextual Information to a Measurement and Evaluation Framework , 2007, 6th International Conference on the Quality of Information and Communications Technology (QUATIC 2007).

[54]  Keith H. Bennett,et al.  Software maintenance and evolution: a roadmap , 2000, ICSE '00.

[55]  Reiner R. Dumke,et al.  Ein Portal zur funktionalen Größenmessung von Software , 2005, Softwaretechnik-Trends.

[56]  Marcus Ciolkowski,et al.  Empirical results from using custom-made software project control centers in industrial environments , 2008, ESEM '08.

[57]  Mark Lycett,et al.  The precursor to an industrial software metrics program , 2008, ITI 2008 - 30th International Conference on Information Technology Interfaces.

[58]  Jean Bacon,et al.  Using events to build distributed applications , 1995, Second International Workshop on Services in Distributed and Networked Environments.

[59]  Victor R. Basili,et al.  The TAME Project: Towards Improvement-Oriented Software Environments , 1988, IEEE Trans. Software Eng..

[60]  Carolyn B. Seaman,et al.  Why do programmers avoid metrics? , 2008, ESEM '08.

[61]  Jürgen Münch,et al.  Goal-Oriented Setup and Usage of Custom-Tailored Software Cockpits , 2008, PROFES.

[62]  Charles W. Krueger,et al.  Variation Management for Software Production Lines , 2002, SPLC.

[63]  Dieter Rombach,et al.  Software-Messung und -Bewertung , 2002 .

[64]  Victor R. Basili,et al.  Support for comprehensive reuse , 1991, Softw. Eng. J..

[65]  Tobias Rötschke,et al.  Applying Test Case Metrics in a Tool Supported Iterative Architecture and Code Improvement Process , 2009, IWSM/Mensura.

[66]  L. Lavazza,et al.  Automated support for process-aware definition and execution of measurement plans , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[67]  Ralf Steinmetz,et al.  Management of Service-oriented Architecture (SoA)-based Application Systems , 2005, Enterp. Model. Inf. Syst. Archit. Int. J. Concept. Model..

[68]  Viviane Torres da Silva,et al.  Model Metrication in MOVA : A Metamodel-Based Approach using OCL , 2007 .

[69]  Victor R. Basili,et al.  Validation on an Approach for Improving Existing Measurement Frameworks , 2000, IEEE Trans. Software Eng..

[70]  Martin Fowler,et al.  Patterns of Enterprise Application Architecture , 2002 .

[71]  Jeffrey K. Liker,et al.  The Toyota way : 14 management principles from the world's greatest manufacturer , 2004 .

[72]  Norman E. Fenton,et al.  Implementing Effective Software Metrics Programs , 1997, IEEE Softw..

[73]  Mario Piattini,et al.  MIS-PyME Software Measurement Maturity Model-Supporting the Definition of Software Measurement Programs , 2008, PROFES.

[74]  Alberto Sillitti,et al.  A case-study on using an Automated In-process Software Engineering Measurement and Analysis system in an industrial environment , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[75]  Gaël Thomas,et al.  Mediation and Enterprise Service Bus A position paper , 2005 .

[76]  Erhard Rahm,et al.  A survey of approaches to automatic schema matching , 2001, The VLDB Journal.

[77]  Frank Bomarius,et al.  Knowledge Management for Building Learning Software Organizations , 2000, Inf. Syst. Frontiers.

[78]  Michael Wooldridge,et al.  Intelligent Agents: The Key Concepts , 2001, Multi-Agent-Systems and Applications.

[79]  Rakesh Agrawal,et al.  A framework for expressing and combining preferences , 2000, SIGMOD '00.

[80]  R. Fielding,et al.  Architectural Styles and the Design of Network-based Software Architectures (CHAPTER 5) , 2000 .

[81]  Jürgen Münch,et al.  Practical Guidelines for Introducing Software Cockpits in Industry , 2008 .

[82]  Ulrich Remus,et al.  Critical success factors for implementing enterprise portals: A comparison with ERP implementations , 2007, Bus. Process. Manag. J..

[83]  James F. Power,et al.  Towards the re-usability of software metricdefinitions at the meta level , 2006 .

[84]  Rajiv D. Banker,et al.  Software complexity and maintenance costs , 1993, CACM.

[85]  Michael Kaib,et al.  Enterprise Application Integration , 2002 .

[86]  Gregor Hohpe,et al.  Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions , 2003 .

[87]  Félix García,et al.  MIS-PyME software measurement capability maturity model - Supporting the definition of software measurement programs and capability determination , 2010, Adv. Eng. Softw..

[88]  Reinhold Plösch,et al.  Operationalised product quality models and assessment: The Quamoco approach , 2014, Inf. Softw. Technol..

[89]  Norman E. Fenton,et al.  Measurement : A Necessary Scientific Basis , 2004 .

[90]  Scott J. Ambler,et al.  Refactoring Databases: Evolutionary Database Design , 2006 .

[91]  David Thomas,et al.  The Pragmatic Programmer: From Journeyman to Master , 1999 .

[92]  Tobias Rötschke,et al.  Werkzeugunterstützung für iterative Modernisierungsprozesse , 2009, Softwaretechnik-Trends.

[93]  Miroslaw Staron,et al.  A framework for developing measurement systems and its industrial evaluation , 2009, Inf. Softw. Technol..

[94]  Mario Piattini,et al.  Model-Driven Software Measurement Framework: A Case Study , 2009, 2009 Ninth International Conference on Quality Software.

[95]  Rajiv Kishore,et al.  Enterprise integration using the agent paradigm: foundations of multi-agent-based integrative business information systems , 2006, Decis. Support Syst..

[96]  Robert Neumann,et al.  Service-Oriented Measurement Infrastructure , 2010, 2010 Eighth ACIS International Conference on Software Engineering Research, Management and Applications.

[97]  A. Schmietendorf,et al.  Resource Metrics for Service-Oriented Infrastructures , 2007 .

[98]  Bernhard Rumpe,et al.  SSELab: A plug-in-based framework for web-based project portals , 2012, 2012 Second International Workshop on Developing Tools as Plug-Ins (TOPI).

[99]  Martin Kunz,et al.  Towards a service-oriented measurement infrastructure , 2006 .

[100]  Mario Piattini,et al.  Implementing Software Measurement Programs in Non Mature Small Settings , 2007, IWSM/Mensura.

[101]  Horst Lichter,et al.  Lessons Learned on Systematic Metric System Development at a Large IT Service Provider , 2014, 2014 21st Asia-Pacific Software Engineering Conference.

[102]  Horst Zuse,et al.  Formal Descriptions of Software Measurement and Evaluation A Short Overview and Evaluation , 2005 .

[103]  Horst Lichter,et al.  Towards Systematic Reuse of Metric Specifications , 2012 .

[104]  Shari Lawrence Pfleeger,et al.  Lessons learned in building a corporate metrics program , 1993, IEEE Software.

[105]  Paolo Traverso,et al.  Service-Oriented Computing: a Research Roadmap , 2008, Int. J. Cooperative Inf. Syst..

[106]  Jürgen Münch,et al.  Software project control centers: concepts and approaches , 2004, J. Syst. Softw..

[107]  Horst Lichter,et al.  History and Lessons Learnt from a Metrics Program at a CMMI Level 3 Company , 2013, 2013 20th Asia-Pacific Software Engineering Conference (APSEC).

[108]  Stephanie Boehm,et al.  Information Dashboard Design The Effective Visual Communication Of Data , 2016 .

[109]  Reiner R. Dumke,et al.  Conception of a Web-Based SPE Development Infrastructure , 2001, Performance Engineering.

[110]  Jean-Marc Jézéquel,et al.  A Model-Driven Measurement Approach , 2008, MoDELS.

[111]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[112]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .