Assessing the use and effectiveness of metrics in information systems: a case study

The Information Systems (1S) function has been under increasing pressure to improve productivity and quality in software development and maintenance. Essential to improvement of the software process is measurement (or “metrics”). Although software metrics programs are becoming popular in practice, evidence suggests that more than 80% do not survive beyond the first year. The objective of this research is to gain an understanding of why it is difficult to implement and sustain an IS metrics program. The study presents and empirically validates a conceptual framework for analyzing the use and effectiveness of software metrics in IS. The framework draws from theories of goal setting, motivation and controllcybernetics. From these perspectives, metrics are viewed as performance feedback, and objectives for metrics implementation as goals. Characteristics of goals and metrics influence the individual’s adoption of metrics. An in-depth field study of a software metrics program that has been in place for several years in industry has been undertaken to validate the conceptual framework. The study involves interviews, observation, and a questionnaire. Results from the study can provide insights useful for IS practice as well as suggesting avenues for fiulher research in IS performance.

[1]  V. Vroom.  Work and motivation , 1964 .

[2]  J. Elashoff,et al.  Multiple Regression in Behavioral Research. , 1975 .

[3]  R. Sitgreaves.  Psychometric theory (2nd ed.). , 1979 .

[4]  M. B. Packer.  Productivity Analysis Using Subjective Output Measures: A Perceptual Mapping Approach for “Knowledge Work” Organizations , 1985 .

[5]  E. Lawler,et al.  Information and control in organizations , 1978 .

[6]  P. Hanges,et al.  A control system model of organizational motivation: Theoretical development and applied implications , 1987 .

[7]  Robert B. Grady,et al.  Software Metrics: Establishing a Company-Wide Program , 1987 .

[8]  Capers Jones,et al.  Applied software measurement: assuring productivity and quality , 1991 .

[9]  D. Potier,et al.  Experiments with computer software complexity and reliability , 1982, ICSE '82.

[10]  F KemererChris,et al.  Cyclomatic Complexity Density and Software Maintenance Productivity , 1991 .

[11]  Paul E. Spector.  Summated rating scale construction , 1991 .

[12]  D. Garvin.  Building a learning organization. , 1993, Harvard business review.

[13]  Barry W. Boehm,et al.  Improving Software Productivity , 1987, Computer.

[14]  D. Scott Sink,et al.  Productivity management : planning, measurement and evaluation, control, and improvement , 1985 .

[15]  Norman E. Fenton,et al.  Software Metrics: A Rigorous Approach , 1991 .

[16]  M. Taylor,et al.  Consequences of individual feedback on behavior in organizations. , 1979 .

[17]  Henry L. Tosi.  A Theory of Goal Setting and Task Performance , 1991 .

[18]  Kevin J. Dooley,et al.  Productivity as a Performance Measure , 1992 .

[19]  Bill Hetzel,et al.  Making Software Measurement Work: Building an Effective Measurement Program , 1993 .

[20]  K Hurst,et al.  Measure for measure. , 1988, Nursing times.

[21]  L. Cronbach.  A Research Worker's Treasure Chest. , 1984, Multivariate behavioral research.

[22]  Bill Curtis,et al.  Measuring the Psychological Complexity of Software Maintenance Tasks with the Halstead and McCabe Metrics , 1979, IEEE Transactions on Software Engineering.

[23]  David A. Specht,et al.  On the evaluation of causal models , 1975 .

[24]  Shari Lawrence Pfleeger,et al.  Maturity, models, and goals: How to build a metrics plan , 1995, J. Syst. Softw..

[25]  Shari Lawrence Pfleeger,et al.  Lessons learned in building a corporate metrics program , 1993, IEEE Software.

[26]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[27]  Chris F. Kemerer,et al.  Reliability of function points measurement: a field experiment , 2015, CACM.

[28]  William C. Hetzel.  The sorry state of software practice measurement and evaluation , 1995, J. Syst. Softw..