Dashboards for Continuous Monitoring of Quality for Software Product under Development

This chapter contributes with a systematic overview of good examples on how dashboards are used to monitor quality of software products under development – both using multiple measures and a single indicator which combines quality and development progress. In this chapter we extract recommendations for building such dashboards for practitioners by exploring how three companies use dashboards for monitoring and controlling external and internal quality of large software products under development. The dashboards presented by each company contain a number of indicators each, and have different premises due to the domain of the product, its purpose and the organization. We describe a number of common principles behind a set of measures, which address the challenge of quantifying readiness to deliver of software products to their end customers. The experiences presented in this chapter come from multiple case studies at Ericsson, two studies at Volvo Car Corporation (VCC) and one at Saab Electronic Defense Systems in Sweden. All companies have a long experience with software development and have undergone a transition into Agile and Lean software development; however the experience with these new paradigms differs from two to five years depending on the company. The difference in the experience provide a possibility to observe that companies with longer experience tend to focus on using measures to support self-organized teams whereas companies with shorter experience tend to focus on using measures to communicate the status from teams to management.

[1]  Robert Phaal,et al.  Technology roadmapping—A planning framework for evolution and revolution , 2004 .

[2]  Miroslaw Staron,et al.  Using Models to Develop Measurement Systems: A Method and Its Industrial Use , 2009, IWSM/Mensura.

[3]  Miroslaw Staron,et al.  Ensuring Reliability of Information Provided by Measurement Systems , 2009, IWSM/Mensura.

[4]  Miroslaw Staron,et al.  A framework for developing measurement systems and its industrial evaluation , 2009, Inf. Softw. Technol..

[5]  Walt Scacchi,et al.  Experience with software process simulation and modeling , 1999, J. Syst. Softw..

[6]  Pierfrancesco Bellini,et al.  Comparing fault-proneness estimation models , 2005, 10th IEEE International Conference on Engineering of Complex Computer Systems (ICECCS'05).

[7]  Barbara A. Kitchenham,et al.  An empirical validation of the relationship between the magnitude of relative error and project size , 2002, Proceedings Eighth IEEE Symposium on Software Metrics.

[8]  Thomas Zimmermann,et al.  Information needs for software development analytics , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[9]  Jan Bosch,et al.  A reference architecture template for software-intensive embedded systems , 2012, WICSA/ECSA '12.

[10]  Günther Ruhe,et al.  The art and science of software release planning , 2005, IEEE Software.

[11]  Cei International vocabulary of basic and general terms in metrology = vocabulaire international des termes fondamentaux et généraux de métrologie , 1993 .

[12]  Vineet Sinha,et al.  Using dependency models to manage complex software architecture , 2005, OOPSLA '05.

[13]  Patrik Berander,et al.  From Traditional to Streamline Development — opportunities and challenges , 2008 .

[14]  Tsun Chow,et al.  A survey study of critical success factors in agile software projects , 2008, J. Syst. Softw..

[15]  Miroslaw Staron,et al.  Developing measurement systems: an industrial case study , 2011, J. Softw. Maintenance Res. Pract..

[16]  Diane M. Strong,et al.  AIMQ: a methodology for information quality assessment , 2002, Inf. Manag..

[17]  Miroslaw Staron,et al.  Presenting Software Metrics Indicators: A Case Study , 2010 .

[18]  Miroslaw Staron,et al.  Identifying Implicit Architectural Dependencies Using Measures of Source Code Change Waves , 2013, 2013 39th Euromicro Conference on Software Engineering and Advanced Applications.

[19]  Claes Wohlin,et al.  Measuring the flow in lean software development , 2011, Softw. Pract. Exp..

[20]  Ryan McGee,et al.  Stakeholder identification and quality attribute prioritization for a global Vehicle Control System , 2010, ECSA '10.

[21]  Håkan Gustavsson,et al.  Architecting Automotive Product Lines: Industrial Practice , 2010, SPLC.

[22]  David Raffo,et al.  Empirical analysis in software process simulation modeling , 2000, J. Syst. Softw..

[23]  Günther Ruhe,et al.  Software Engineering Decision Support ? A New Paradigm for Learning Software Organizations , 2002, LSO.

[24]  Miroslaw Staron,et al.  A method for forecasting defect backlog in large streamline software development projects and its industrial evaluation , 2010, Inf. Softw. Technol..

[25]  Miroslaw Staron,et al.  Monitoring Bottlenecks in Agile and Lean Software Development Projects - A Method and Its Industrial Use , 2011, PROFES.

[26]  Miroslaw Staron,et al.  An Industrial Case Study on Visualization of Dependencies between Software Measurements , 2007 .

[27]  Yuming Zhou,et al.  Empirical Analysis of Object-Oriented Design Metrics for Predicting High and Low Severity Faults , 2006, IEEE Transactions on Software Engineering.

[28]  Miroslaw Staron,et al.  Release Readiness Indicator for Mature Agile and Lean Software Development Projects , 2012, XP.

[29]  Helen Sharp,et al.  Models of motivation in software engineering , 2009, Inf. Softw. Technol..

[30]  H. D. Rombach,et al.  The Goal Question Metric Approach , 1994 .