Project performance development framework: An approach for developing performance criteria & measures for information systems (IS) projects

Performance criteria typically reflect the values of the project stakeholders in how they will be viewing their projects. A critical analysis of current research shows that several key challenges being faced in information systems (IS) projects include the lack of clearly defined objectives, mismatched stakeholders' expectations and lack of sufficient or formal methods to aid practitioners in developing relevant performance criteria. This is accentuated by growing calls to rethink the traditional measurement paradigm. Moreover, most of the known solutions in practice are without structured or easily implementable procedures to elicit and develop performance criteria and measures. Against this background, the paper proposes a formal method to develop a comprehensive set of performance criteria or objectives grounded in the views of the project stakeholders and associated measures aligned to these objectives. The project performance framework (PPDF) primarily relies on the principles and advantages of the value focused thinking (VFT) and goal question metric (GQM) to elicit and develop these performance criteria and associated measures based on the values of the stakeholders in the project. Three project cases are used to illustrate and assess the model.

[1]  Corlane Barclay,et al.  Towards an integrated measurement of IS project performance: The project performance scorecard , 2008, Inf. Syst. Frontiers.

[2]  Graeme G. Shanks,et al.  Successfully completing case study research: combining rigour, relevance and pragmatism , 1998, Inf. Syst. J..

[3]  R. Ryan Nelson,et al.  Project Retrospectives: Evaluating Project Success, Failure, and Everything in Between , 2005, MIS Q. Executive.

[4]  G. Huber.  Organizational Learning: The Contributing Processes and the Literatures , 1991 .

[5]  A. Shenhar,et al.  Project Success: A Multidimensional Strategic Concept , 2001 .

[6]  Derek H.T. Walker,et al.  Visualizing Stakeholder Influence — Two Australian Examples , 2006 .

[7]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[8]  Adam Collins,et al.  PROJECT SUCCESS — A SURVEY , 2004 .

[9]  N. Bhushan,et al.  Strategic Decision Making: Applying the Analytic Hierarchy Process , 2004 .

[10]  Walid Belassi,et al.  A new framework for determining critical success/failure factors in projects , 1996 .

[11]  M. Engwall.  No project is an island: linking projects to history and context , 2003 .

[12]  Aaron J. Shenhar,et al.  Refining the search for project success factors: a multivariate, typological approach , 2002 .

[13]  Jon R. Katzenbach,et al.  The Wisdom of Teams: Creating the High-Performance Organization , 1992 .

[14]  Gurpreet Dhillon,et al.  Interpreting Dimensions of Consumer Trust in E-Commerce , 2003, Inf. Technol. Manag..

[15]  Harvey Maylor,et al.  From Projectification to Programmification , 2006 .

[16]  Ralph L. Keeney,et al.  Value-Focused Thinking: A Path to Creative Decisionmaking , 1992 .

[17]  Guy Fitzgerald,et al.  Evaluating information systems projects: a multidimensional approach , 1998, J. Inf. Technol..

[18]  Peter W. G. Morris,et al.  Research updating the APM Body of Knowledge 4th edition , 2006 .

[19]  Keng Siau,et al.  Value of Mobile Commerce to Customers , 2004, AMCIS.

[20]  Leonard-BartonDorothy.  A Dual Methodology for Case Studies , 1990 .

[21]  John J. Marciniak,et al.  Encyclopedia of Software Engineering , 1994, Encyclopedia of Software Engineering.

[22]  Francis T. Hartman,et al.  Project Management in the Information Systems and Information Technologies Industries , 2002 .

[23]  Gurpreet Dhillon,et al.  Value‐focused assessment of information system security in organizations , 2006, Inf. Syst. J..

[24]  R. Yin.  Case Study Research: Design and Methods , 1984 .

[25]  Kaj Grønbæk,et al.  CSCW challenges: cooperative design in engineering projects , 1993, CACM.

[26]  T. Saaty.  How to Make a Decision: The Analytic Hierarchy Process , 1990 .

[27]  N. Denzin,et al.  Handbook of Qualitative Research , 1994 .

[28]  E. Salas,et al.  Shared mental models in expert team decision making. , 1993 .

[29]  Robert K. Yin,et al.  Applications of case study research , 1993 .

[30]  José Esteves,et al.  A GOAL/QUESTION/METRIC RESEARCH PROPOSAL TO MONITOR USER INVOLVEMENT AND PARTICIPATION IN ERP IMPLEMENTATION PROJECTS , 2003 .

[31]  Noel Bryson.  Supporting consensus formation in Group Support Systems using the Qualitative Discriminant Process , 1997 .

[32]  Robert M. Davison,et al.  The balanced scorecard: a foundation for the strategic management of information systems , 1999, Decis. Support Syst..

[33]  Ieee Std,et al.  IEEE Guide Adoption of PMI Standard A Guide to the Project Management Body of Knowledge , 2004 .

[34]  A. Pouloudi,et al.  Stakeholder identification in inter-organizational systems: gaining insights for drug use management systems , 1997 .

[35]  Timo Ala-Risku,et al.  Material delivery problems in construction projects: A possible solution , 2006 .

[36]  Mark Winter,et al.  Directions for future research in project management: The main findings of a UK government-funded research network , 2006 .

[37]  R. Kaplan,et al.  The balanced scorecard--measures that drive performance. , 2015, Harvard business review.

[38]  Jim Browne,et al.  A review of performance measurement: Towards performance management , 2005, Comput. Ind..

[39]  Ralph L. Keeney,et al.  Value-Focused Thinking , 1996 .

[40]  Tutu Wegelius-Lehtonen,et al.  Performance measurement in construction logistics , 2001 .

[41]  Galal H. Galal-Edeen,et al.  Stakeholder identification in the requirements engineering process , 1999, Proceedings. Tenth International Workshop on Database and Expert Systems Applications. DEXA 99.

[42]  Urvashi Rathod,et al.  Defining ‘success’ for software projects: An exploratory revelation , 2006 .

[43]  Shlomo Globerson,et al.  Issues in developing a performance criteria system for an organization , 1985 .

[44]  Erik Brynjolfsson,et al.  The productivity paradox of information technology , 1993, CACM.

[45]  Robert L. Glass.  IT Failure Rates--70% or 10-15%? , 2005, IEEE Softw..

[46]  Joyce Fortune,et al.  Framing of project critical success factors by a systems model , 2006 .

[47]  Kalle Lyytinen,et al.  Information systems failures—a survey and classification of the empirical literature , 1988 .

[48]  David Baccarini,et al.  The Logical Framework Method for Defining Project Success , 1999 .

[49]  H. D. Rombach,et al.  The Goal Question Metric Approach , 1994 .

[50]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[51]  Mark Winter,et al.  The importance of ‘process’ in Rethinking Project Management: The story of a UK Government-funded research network , 2006 .

[52]  J. Creswell,et al.  Determining Validity in Qualitative Inquiry , 2000 .

[53]  John Wateridge,et al.  How can IS/IT projects be measured for success? , 1998 .

[54]  Kenneth L. Kraemer,et al.  Review: Information Technology and Organizational Performance: An Integrative Model of IT Business Value , 2004, MIS Q..

[55]  Lynda Bourne,et al.  USING A VISUALISING TOOL TO STUDY STAKEHOLDER INFLUENCE - TWO AUSTRALIAN EXAMPLES - , 2006 .

[56]  Elias G. Carayannis,et al.  The story of managing projects : an interdisciplinary approach , 2003 .

[57]  D. Bryde,et al.  Client versus contractor perspectives on project success criteria , 2005 .

[58]  Victor R. Basili,et al.  A Methodology for Collecting Valid Software Engineering Data , 1984, IEEE Transactions on Software Engineering.

[59]  Anasis Majumdar,et al.  Capability Maturity Model Integration (CMMI) , 2011 .

[60]  Roger Atkinson,et al.  Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria , 1999 .

[61]  Kweku Ewusi-Mensah,et al.  Critical issues in abandoned information systems development projects , 1997, CACM.

[62]  Dorothy Leonard-Barton,et al.  A Dual Methodology for Case Studies: Synergistic Use of a Longitudinal Single Site with Replicated Multiple Sites , 1990 .

[63]  Rodney Anthony Stewart,et al.  A framework for the life cycle management of information technology projects: ProjectIT , 2008 .

[64]  Terry Cooke-Davies,et al.  The “real” success factors on projects , 2002 .

[65]  Line Dubé,et al.  Rigor in Information Systems Positivist Case Research: Current Practices , 2003, MIS Q..

[66]  M. Gibbert,et al.  What passes as a rigorous case study , 2008 .

[67]  P. I. Zorkoczy,et al.  Oxford Surveys in Information Technology , 1985 .

[68]  Hennie A. Kruger,et al.  Value-focused assessment of ICT security awareness in an academic environment , 2007, Comput. Secur..

[69]  Lynn Crawford,et al.  Fundamental uncertainties in projects and the scope of project management , 2006 .

[70]  Peter W. G. Morris,et al.  The Anatomy of Major Projects: A Study of the Reality of Project Management , 1988 .

[71]  C. S Lim,et al.  Criteria of project success: an exploratory re-examination , 1999 .

[72]  Robert J. Graham,et al.  The Project Manager's MBA: How to Translate Project Decisions into Business Success , 2000 .

[73]  K. Milis,et al.  Success factors regarding the implementation of ICT investment projects , 2002 .

[74]  Joyce Fortune,et al.  Current practice in project management — an empirical study , 2002 .

[75]  Arshinder,et al.  Supply chain coordination: Perspectives, empirical studies and research directions , 2008 .

[76]  David Eager,et al.  Improving cross‐company project management performance with a collaborative project scorecard , 2008 .

[77]  Leonard Steinborn.  International Organization for Standardization ISO 9001:2000 Quality Management Systems — Requirements , 2004 .

[78]  Cristina Masella,et al.  Design of performance measures for time based companies , 1991 .

[79]  Thomas A. Schwandt.  Qualitative Inquiry: A Dictionary of Terms , 1997 .