Measuring IT service management performance: a model development

Prompted by the realisation that IT is now seen as a service, with a customer focus and process orientation, the authors propose a model to measure IT service management (ITSM) performance. Measuring ITSM performance will enable organisations to demonstrate the benefit from their investment. The model is based on a systematic literature review that progressed from considering the general areas of organisation performance measurement to examining commonly used performance metrics. Although there are a number of studies on ITSM implementation, only a few considered the performance measurement of ITSM. A structured method for the design of the model was adopted through a three-level analysis. A comparison of existing performance measurement frameworks was first made to identify those that are suitable for ITSM and that would facilitate communication between the business and IT function. This was done using appropriate dimensions from past work of various performance measurement researchers. The frameworks were then classified along these dimensions to identify their completeness, eliminate unnecessary dimensions, and identify the natural dimensions for ITSM.

[1]  Aileen Cater-Steel,et al.  itSMF Australia 2007 Conference: summary of ITSM standards and frameworks survey responses , 2008 .

[2]  Kenneth L. Kraemer,et al.  Information technology and economic performance , 2003, ACM Comput. Surv..

[3]  Robert M. Davison,et al.  The balanced scorecard: a foundation for the strategic management of information systems , 1999, Decis. Support Syst..

[4]  Christopher Ward,et al.  An integration model for organizing IT service management , 2007 .

[5]  Andy Neely,et al.  The Performance Prism: The Scorecard for Measuring and Managing Business Success , 2002 .

[6]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[7]  Andy Neely,et al.  Performance measurement system design , 1995 .

[8]  Jacques Philippe Sauvé,et al.  A quantitative approach to IT investment allocation to improve business results , 2006, Seventh IEEE International Workshop on Policies for Distributed Systems and Networks (POLICY'06).

[9]  Vernon J. Richardson,et al.  Returns on Investments in Information Technology: A Research Synthesis , 2002, J. Inf. Syst..

[10]  Richard H. Hall,et al.  Effectiveness Theory and Organizational Effectiveness , 1980 .

[11]  Dzenana Donko,et al.  Measurement of service effectiveness and establishment of baselines , 2009 .

[12]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[13]  Pierre J. Richard,et al.  Measuring Organizational Performance: Towards Methodological Best Practice , 2009 .

[14]  A. Parasuraman,et al.  A Conceptual Model of Service Quality and Its Implications for Future Research , 1985 .

[15]  Stefan Tangen,et al.  Demystifying productivity and performance , 2005 .

[16]  D. Larcker,et al.  Coming up short on nonfinancial performance measurement. , 2003, Harvard business review.

[17]  Yixin Diao,et al.  Estimating business value of IT services through process complexity analysis , 2008, NOMS 2008 - 2008 IEEE Network Operations and Management Symposium.

[18]  Cristina Masella,et al.  Design of performance measures for time based companies , 1991 .

[19]  A. Barua,et al.  The Information Technology Productivity Paradox Revisited: A Theoretical and Empirical Investigation in the Manufacturing Sector , 1997 .

[20]  Aileen Cater-Steel,et al.  itSMF Australia 2005 Conference: Summary of ITIL Adoption Survey Responses , 2005 .

[21]  Sue Conger,et al.  An overview of IT service management , 2009, CACM.

[22]  Lorin M. Hitt,et al.  Beyond the productivity paradox , 1998, CACM.

[23]  Richard T. Watson,et al.  Service Quality: A Measure of Information System Effectiveness , 1995, MIS Q..

[24]  A. Neely,et al.  The performance prism in practice , 2001 .

[25]  Asbjørn Rolstadås,et al.  Enterprise performance measurement , 1998 .

[26]  Alain Abran,et al.  A multidimensional performance model for consolidating balanced scorecards , 2003 .

[27]  Peter Weill,et al.  The Relationship Between Investment in Information Technology and Firm Performance: A Study of the Valve Manufacturing Sector , 1992, Inf. Syst. Res..

[28]  Claus-Peter Präg,et al.  IT-Service Cachet - Managing IT-Service Performance and IT-Service Quality , 2006, HICSS.

[29]  Sue Conger,et al.  Confusion in the Ranks: IT Service Management Practice and Terminology , 2009, Inf. Syst. Manag..

[30]  W. Brenner,et al.  Evaluation of service-oriented IT management in practice , 2005, Proceedings of ICSSSM '05. 2005 International Conference on Services Systems and Services Management, 2005..

[31]  William R. King,et al.  Measuring the Performance of Information Systems: A Functional Scorecard , 2005, J. Manag. Inf. Syst..

[32]  R. Kaplan,et al.  The balanced scorecard--measures that drive performance. , 2015, Harvard business review.

[33]  J. T. M. van Der Zee,et al.  Alignment Is Not Enough: Integrating Business and Information Technology Management with the Balanced Business Scorecard , 1999, J. Manag. Inf. Syst..

[34]  Alan J. Keel,et al.  From a technology-oriented to a service-oriented approach to IT management , 2007, IBM Syst. J..

[35]  Antti Lahtela,et al.  Implementing an ITIL-Based IT Service Management Measurement System , 2010, 2010 Fourth International Conference on Digital Society.

[36]  Aileen Cater-Steel,et al.  IT Service Departments Struggle to Adopt a Service-Oriented Philosophy , 2009, Int. J. Inf. Syst. Serv. Sect..

[37]  Christian Kay,et al.  The Oxford English Dictionary Online , 2004, Lit. Linguistic Comput..

[38]  Stefan Tangen,et al.  Performance measurement: from philosophy to practice , 2004 .

[39]  L. Edvinsson.  Developing intellectual capital at Skandia , 1997 .

[40]  Rüdiger Zarnekow,et al.  ITIL as common practice reference model for IT service management: formal assessment and implications for practice , 2005, 2005 IEEE International Conference on e-Technology, e-Commerce and e-Service.

[41]  Louis Raymond,et al.  Researching performance measurement systems , 2008 .

[42]  Judy E. Scott.  The measurement of information systems effectiveness: evaluating a measuring instrument , 1995, DATB.

[43]  Derek Steeple,et al.  A FRAMEWORK FOR AUDITING AND ENHANCING PERFORMANCE MEASUREMENT SYSTEMS , 2000 .

[44]  Pradeep Kumar Ray,et al.  Designing an evaluation framework for IT service management , 2010, Inf. Manag..

[45]  W. V. Grembergen,et al.  Using COBIT and the balanced scorecard as instruments for service level management , 2003 .

[46]  Jerry N. Luftman,et al.  Key Issues for IT Executives 2009: Difficult Economy’s Impact on IT , 2010, MIS Q. Executive.

[47]  C. Grönroos.  From Scientific Management to Service Management , 1994 .

[48]  Aileen Cater-Steel,et al.  Implementation of IT infrastructure library (ITIL) in Australia: progress and success factors , 2005 .

[49]  Béatrix Barafort,et al.  Benefits Resulting from the Combined Use of ISO/IEC 15504 with the Information Technology Infrastructure Library (ITIL) , 2002, PROFES.

[50]  Aileen Cater-Steel,et al.  An International Analysis of IT Service Management Benefits and Performance Measurement , 2010 .

[51]  Aileen Cater-Steel,et al.  Transforming IT service management - the ITIL impact , 2006 .

[52]  Rodney McAdam,et al.  A critique of benchmarking and performance measurement: Lead or lag? , 2004 .

[53]  Linda Tracy,et al.  IT Key Metrics Data: 2008 Table of Contents , 2008 .

[54]  Joan Ballantine,et al.  Performance measurement in service businesses revisited , 1996 .

[55]  Jean-François Henri,et al.  Performance Measurement and Organizational Effectiveness: Bridging the Gap , 2004 .

[56]  Sue Conger,et al.  Service Management in Operations , 2008, AMCIS.

[57]  Harry Katzan,et al.  Service Science: Concepts, Technology, Management , 2008 .

[58]  Carol Stoak Saunders,et al.  Measuring Performance of the Information Systems Function , 1992, J. Manag. Inf. Syst..

[59]  Stefano Tonchia,et al.  Performance measurement systems - Models, characteristics and measures , 2001 .

[60]  Walter Brenner,et al.  European Conference on Information Systems ( ECIS ) 2005 Service Oriented IT Management : Benefit , Cost and Success Factors , 2017 .

[61]  Stephen L. Vargo,et al.  Evolving to a New Dominant Logic for Marketing , 2004 .

[62]  Ananthanarayanan Parasuraman,et al.  Assessing and improving service performancefor maximum impact: insights from a two‐decade‐long research journey , 2004 .

[63]  Leslie P. Willcocks,et al.  Measuring organizational IS effectiveness: an overview and update of senior management perspectives , 2002, DATB.

[64]  Aileen Cater-Steel,et al.  Justifications, Strategies, and Critical Success Factors in Successful ITIL Implementations in U.S. and Australian Companies: An Exploratory Study , 2009, Inf. Syst. Manag..