Towards an integrated measurement of IS project performance: The project performance scorecard

Determining the contribution of information system (IS) projects is a difficult endeavour. This research presents a framework for evaluating and measuring IS project performance that seek to address this problem while moving away from the reliance on the traditional assessment method (time, cost, specifications). When IS projects fail or succeed is difficult to determine what conditions enabled the outcome. However, if we are to develop a higher level of IS project management competence we need frameworks that can help in assessing IS project outcomes while critically reflecting and learning from IS project management practices. The Project Performance Scorecard (PPS) recommended here is one framework that can help to advance our understanding IS project management and evaluation approaches. This framework integrates concepts from existing models of IS success, the Balanced Scorecard and project management practices. A case study approach is used to illustrate the use of the PPS, and offers insights into how it can be useful in assessing the performance of IS projects.

[1]  James C. Wetherbe,et al.  Key Issues in Information Systems Management: 1994-95 SIM Delphi Results , 1996, MIS Q..

[2]  Kalle Lyytinen,et al.  Expectation failure concept and systems analysts' view of information system failures: Results of an exploratory study , 1988, Inf. Manag..

[3]  Adam Collins,et al.  PROJECT SUCCESS — A SURVEY , 2004 .

[4]  R. Kaplan,et al.  The Balanced Scorecard: Translating Strategy into Action , 1996 .

[5]  Arun Rai,et al.  Assessing the Validity of IS Success Models: An Empirical Test and Theoretical Analysis , 2002, Inf. Syst. Res..

[6]  Aaron J. Shenhar,et al.  Refining the search for project success factors: a multivariate, typological approach , 2002 .

[7]  Jeffrey K. Pinto,et al.  Critical factors in successful project implementation , 1987, IEEE Transactions on Engineering Management.

[8]  J. Wilkes,et al.  Corporate performance management , 2003 .

[9]  Madely du Preez.  Project Management for Modern Information Systems , 2007 .

[10]  Bernard Marr.  Corporate Performance Measurement , 2005 .

[11]  Robert L. Glass.  IT Failure Rates--70% or 10-15%? , 2005, IEEE Softw..

[12]  Allen S. Lee.  A Scientific Methodology for MIS Case Studies , 1989, MIS Q..

[13]  Erik Brynjolfsson,et al.  The productivity paradox of information technology , 1993, CACM.

[14]  Dorothy Leonard-Barton,et al.  A Dual Methodology for Case Studies: Synergistic Use of a Longitudinal Single Site with Replicated Multiple Sites , 1990 .

[15]  Jie Lu.  A Model for Evaluating E-Commerce Based on Cost/Benefit and Customer Satisfaction , 2003, Inf. Syst. Frontiers.

[16]  Elias G. Carayannis,et al.  The story of managing projects : an interdisciplinary approach , 2003 .

[17]  Anton de Wit,et al.  Measurement of project success , 1988 .

[18]  Leonard-BartonDorothy.  A Dual Methodology for Case Studies , 1990 .

[19]  D. Sandy Staples,et al.  Dimensions of Information Systems Success , 1999, Commun. Assoc. Inf. Syst..

[20]  Shailendra C. Palvia,et al.  A socio-technical framework for quality assessment of computer information systems , 2001, Ind. Manag. Data Syst..

[21]  John Wateridge,et al.  How can IS/IT projects be measured for success? , 1998 .

[22]  D. Bryde.  Methods for Managing Different Perspectives of Project Success , 2005 .

[23]  John J. Marciniak,et al.  Encyclopedia of Software Engineering , 1994, Encyclopedia of Software Engineering.

[24]  Kam Jugdev,et al.  A Retrospective look at our Evolving Understanding of Project Success , 2005, IEEE Engineering Management Review.

[25]  David Bryde,et al.  Modelling project management performance , 2003 .

[26]  Urvashi Rathod,et al.  Defining ‘success’ for software projects: An exploratory revelation , 2006 .

[27]  Virginia Franke Kleist,et al.  An Approach to Evaluating E-Business Information Systems Projects , 2003, Inf. Syst. Frontiers.

[28]  Graham Pervan,et al.  A review of IS/IT investment evaluation and benefits management issues, problems, and processes , 2001 .

[29]  L GlassRobert.  The Standish report , 2006 .

[30]  Robert M. Davison,et al.  The balanced scorecard: a foundation for the strategic management of information systems , 1999, Decis. Support Syst..

[31]  T. Saaty.  How to Make a Decision: The Analytic Hierarchy Process , 1990 .

[32]  Guy Fitzgerald,et al.  Evaluating information systems projects: a multidimensional approach , 1998, J. Inf. Technol..

[33]  David Baccarini,et al.  The Logical Framework Method for Defining Project Success , 1999 .

[34]  Mark Winter,et al.  Directions for future research in project management: The main findings of a UK government-funded research network , 2006 .

[35]  H. D. Rombach,et al.  The Goal Question Metric Approach , 1994 .

[36]  Kaj Grønbæk,et al.  CSCW challenges: cooperative design in engineering projects , 1993, CACM.

[37]  D. Garvin.  Competing on the Eight Dimensions of Quality , 1987 .

[38]  R. Kaplan,et al.  The balanced scorecard--measures that drive performance. , 2015, Harvard business review.

[39]  T. Saaty.  Risk-Its Priority and Probability: The Analytic Hierarchy Process , 1987 .

[40]  Peter W. G. Morris,et al.  The Anatomy of Major Projects: A Study of the Reality of Project Management , 1988 .

[41]  Robert L. Glass,et al.  Evolving a new theory of project success , 1999, Commun. ACM.

[42]  C. S Lim,et al.  Criteria of project success: an exploratory re-examination , 1999 .

[43]  Evan W. Duggan,et al.  Measuring Information Systems Delivery Quality , 2006 .

[44]  Ephraim R. McLean,et al.  The DeLone and McLean Model of Information Systems Success: A Ten-Year Update , 2003, J. Manag. Inf. Syst..

[45]  Peter B. Seddon.  A Respecification and Extension of the DeLone and McLean Model of IS Success , 1997, Inf. Syst. Res..

[46]  M. Lynne Markus,et al.  Rituals in Information System Design , 1984, MIS Q..

[47]  Beth McNeil.  The Project Manager's MBA: How to Translate Project Decisions into Business Success , 2001 .

[48]  Thomas L. Saaty,et al.  How to Make a Decision: The Analytic Hierarchy Process , 1990 .

[49]  Robert L. Glass,et al.  The Standish report: does it really describe a software crisis? , 2006, CACM.

[50]  Anand Teltumbde,et al.  A framework for evaluating ERP projects , 2000 .

[51]  Keng Siau,et al.  Strategic implications of mobile technology: A case study using Value-Focused Thinking , 2005, J. Strateg. Inf. Syst..

[52]  Ephraim R. McLean,et al.  Information Systems Success: The Quest for the Dependent Variable , 1992, Inf. Syst. Res..

[53]  J P O'Keefe,et al.  If we build it, they will come. , 1998, Journal.

[54]  Vijay K. Vaishnavi,et al.  Predicting Maintenance Performance Using Object-Oriented Design Complexity Metrics , 2003, IEEE Trans. Software Eng..

[55]  Zbigniew H. Przasnyski,et al.  On Information Systems Project Abandonment: An Exploratory Study of Organizational Practices , 1991, MIS Q..

[56]  R. Ryan Nelson,et al.  Project Retrospectives: Evaluating Project Success, Failure, and Everything in Between , 2005, MIS Q. Executive.

[57]  James C. Wetherbe,et al.  Key issues in information systems management , 1987 .

[58]  R. Kaplan,et al.  Linking the Balanced Scorecard to Strategy , 1996 .

[59]  Suzanne D. Pawlowski,et al.  The Delphi method as a research tool: an example, design considerations and applications , 2004, Inf. Manag..

[60]  Terry Cooke-Davies,et al.  The “real” success factors on projects , 2002 .

[61]  Tarek Zayed,et al.  Assessing risk and uncertainty inherent in Chinese highway projects using AHP , 2008 .

[62]  Roger Atkinson,et al.  Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria , 1999 .

[63]  Joyce Fortune,et al.  Current practice in project management — an empirical study , 2002 .

[64]  Aaron J. Shenhar,et al.  What is really important for project success? A refined, multivariate, comprehensive analysis , 2003 .

[65]  Thiagarajan Ravichandran,et al.  The dimensions and correlates of systems development quality , 1994, SIGCPR '94.

[66]  A. Shenhar,et al.  Project Success: A Multidimensional Strategic Concept , 2001 .