A novel methodology for performance evaluation of IT projects in a fuzzy environment: a case study

Most of the information technology (IT) projects are canceled or failed due to different reasons, while some of them are performed poorly in terms of cost, time, scope, customer satisfaction, etc. Therefore, it is important to identify the most important criteria affecting the performance of IT projects, measure their performance, identify the issues, and continually improve the performance of both the projects and the organizations. Although IT organizations place value on the importance of the project performance to release on time, with low cost, and in accordance with the customer expectations, they lack ways of determining the most suitable performance criteria for continually measuring, improving, and controlling them. This study proposes a new methodology to evaluate the performance of IT projects in a fuzzy environment. For this aim, firstly, the most suitable criteria are identified based on the balanced scorecard method. In the second step, the relative priorities of the criteria are determined with the help of expert judgments and hesitant fuzzy weights with hesitant multiplicative geometric operator. Then, the priorities of the criteria are used to evaluate the performance of IT projects in a Turkish company by using real data.

[1]  Eric Overby,et al.  Enterprise agility and the enabling role of information technology , 2006, Eur. J. Inf. Syst..

[2]  Ashly Pinnington,et al.  Exploring the Value of Project Management: Linking Project Management Performance and Project Success , 2014 .

[3]  Mary A. Malina,et al.  Relations among Measures, Climate of Control, and Performance Measurement Models* , 2007 .

[4]  John Wateridge,et al.  How can IS/IT projects be measured for success? , 1998 .

[5]  Shu Lin,et al.  Project Performance and the Enabling Role of Information Technology: An Exploratory Study on the Role of Alignment , 2007, Manuf. Serv. Oper. Manag..

[6]  Carolyn Mizell,et al.  A project management approach to using simulation for cost estimation on large, complex software development projects , 2009, IEEE Engineering Management Review.

[7]  Soleyman Iranzadeh,et al.  Project Duration Performance Measurement By Fuzzy Approach under Uncertainty , 2017 .

[8]  Leslie P. Willcocks,et al.  Measuring organizational IS effectiveness: an overview and update of senior management perspectives , 2002, DATB.

[9]  James Eastham,et al.  PLM software selection model for project management using hierarchical decision modeling with criteria from PMBOK knowledge areas , 2013, 2013 Proceedings of PICMET '13: Technology Management in the IT-Driven Services (PICMET).

[10]  Homayoun Khamooshi,et al.  EDM: Earned Duration Management, a new approach to schedule performance management and measurement , 2014 .

[11]  Zeshui Xu,et al.  Analytic hierarchy process-hesitant group decision making , 2014, Eur. J. Oper. Res..

[12]  Mark Graham Brown,et al.  Keeping Score: Using the Right Metrics to Drive World-Class Performance , 1996 .

[13]  Ali Jaafari.  Time and priority allocation scheduling technique for projects , 1996 .

[14]  Graham Bradley,et al.  Measuring the performance of IT services: An assessment of SERVQUAL , 2002, Int. J. Account. Inf. Syst..

[15]  The Role of Information Technology in the Business Sector , 2014 .

[16]  Kriengsak Panuwatwanich,et al.  Linking project health to project performance indicators: Multiple case studies of construction projects in Saudi Arabia , 2012 .

[17]  L. M. Naeni,et al.  A fuzzy approach for the earned value management , 2011 .

[18]  Robert M. Davison,et al.  The balanced scorecard: a foundation for the strategic management of information systems , 1999, Decis. Support Syst..

[19]  Søren Munch Lindhard,et al.  Identifying the key process factors affecting project performance , 2016 .

[20]  H. Ramesh,et al.  Review of Information Technology Effect on Competitive Advantage- Strategic Perspective , 2010 .

[21]  B. N. Baker,et al.  Factors Affecting Project Success , 2008 .

[22]  Andy Neely,et al.  Performance measurement system design: developing and testing a process‐based approach , 2000 .

[23]  G. Vaidyanathan,et al.  When is a balanced scorecard a balanced scorecard , 2011 .

[24]  Evelina Ericsson,et al.  Quantifying Success Factors for IT Projects—An Expert-Based Bayesian Model , 2014, Inf. Syst. Manag..

[25]  Qing Hu,et al.  The Use of Formal IT Investment Evaluation Methods in Organizations: A Survey of European Countries , 2006, AMCIS.

[26]  Víctor Yepes,et al.  Complete fuzzy scheduling and fuzzy earned value management in construction projects , 2012 .

[27]  P. Gallo,et al.  NON-FINANCIAL INDICATORS AND THEIR IMPORTANCE I , 2018, Journal of Competitiveness.

[28]  C. S Lim,et al.  Criteria of project success: an exploratory re-examination , 1999 .

[29]  Johnny Lind,et al.  The impact of the corporate balanced scorecard on corporate control—A research note , 2010 .

[30]  Ayfer Başar,et al.  Hesitant fuzzy pairwise comparison for software cost estimation: a case study in Turkey , 2017, Turkish J. Electr. Eng. Comput. Sci..

[31]  F. Ling,et al.  Key project management practices affecting Singaporean firms’ project performance in China , 2009 .

[32]  Bon-Gang Hwang,et al.  Capital project performance measurement and benchmarking in Singapore , 2013 .

[33]  Albert P.C. Chan,et al.  Development of a partnering performance index (PPI) for construction projects in Hong Kong: a Delphi study , 2007 .

[34]  Frank Bannister,et al.  The effective measurement and management of ICT costs and benefits. 3rd edition. , 2007 .

[35]  Fernando Tobal Berssaneti,et al.  Identification of variables that impact project success in Brazilian companies , 2015 .

[36]  Gareth R. Jones,et al.  The role of information technology in the organization: a review, model, and assessment , 2001 .

[37]  Van Truong Luu,et al.  Improving project management performance of large contractors using benchmarking approach , 2008 .

[38]  Efraim Turban,et al.  Information Technology for Management: Making Connections for Strategic Advantage , 2000 .

[39]  T. Saaty.  How to Make a Decision: The Analytic Hierarchy Process , 1990 .

[40]  W. Duncan.  A GUIDE TO THE PROJECT MANAGEMENT BODY OF KNOWLEDGE , 1996 .

[41]  Özgür Kabak,et al.  A Decision Support Methodology for Locating Bank Branches: A Case Study in Turkey , 2017, Int. J. Inf. Technol. Decis. Mak..

[42]  Terry Cooke-Davies,et al.  The “real” success factors on projects , 2002 .

[43]  Cristiano Busco,et al.  The causal relationships between performance drivers and outcomes , 2012 .

[44]  Saumyendu Ghosh,et al.  Determination of Key Performance Indicators with Enterprise Resource Planning Systems in Engineering Construction Firms , 2009 .

[45]  Özgür Kabak,et al.  A Tabu Search Algorithm for Multi-Period Bank Branch Location Problem: A Case Study in a Turkish Bank , 2018 .

[46]  Navdeep Kaur,et al.  Multi criteria decision making approach for selecting effort estimation model , 2012, ArXiv.

[47]  Mario Vanhoucke,et al.  Introduction to earned duration , 2015 .

[48]  D. Robey,et al.  Going Global: Using information technology to advance the competitiveness of the virtual transnational organization , 1998 .

[49]  David R. Riley,et al.  Project Delivery Metrics for Sustainable, High-Performance Buildings , 2011 .

[50]  A. K. Munns,et al.  The role of project management in achieving project success , 1996 .

[51]  Michael R. Duffey,et al.  A model for effective implementation of Earned Value Management methodology , 2003 .

[52]  Efraim Turban,et al.  Information Technology for Management: Making Connections for Strategic Advantage,3rd Edition , 2001 .

[53]  Anastasios K. Boukanos.  THE CRITERIA OF PROJECT SUCCESS , 2007 .

[54]  John Wateridge,et al.  IT projects: a basis for success , 1995 .

[55]  Devika Kannan,et al.  Green vendor evaluation and selection using AHP and Taguchi loss functions in production outsourcing in mining industry , 2015 .

[56]  V. Torra,et al.  A framework for linguistic logic programming , 2010 .

[57]  Mostafa Salari,et al.  Estimation of Project Performance Using Earned Value Management and Fuzzy Regression , 2014 .

[58]  V. Anantatmula,et al.  Role of Organizational Project Management Maturity Factors on Project Success , 2018, Engineering Management Journal.

[59]  David Baccarini,et al.  The Logical Framework Method for Defining Project Success , 1999 .

[60]  Birgul Kutlu,et al.  Project Management Software Selection Using Analytic Hierarchy Process Method , 2014 .

[61]  Antoni Gelonch,et al.  An IT Balance Scorecard Design under Service Management Philosophy , 2012, 2012 45th Hawaii International Conference on System Sciences.

[62]  Parviz Rad.  Project success attributes , 2003 .

[63]  Aminah Robinson Fayek,et al.  Initial metrics and pilot program results for measuring the performance of the Canadian construction industry , 2008 .

[64]  R. Johnston,et al.  Performance Measurement in Service Businesses , 1992 .

[65]  Robert N. Lussier,et al.  Entrepreneurs Use a Balanced Scorecard to Translate Strategy into Performance Measures , 2006 .

[66]  A. Shenhar,et al.  Project Success: A Multidimensional Strategic Concept , 2001 .

[67]  Raja R. A. Issa,et al.  Management’s Perception of Key Performance Indicators for Construction , 2003 .

[68]  Catherine M. Beise,et al.  Do project managers practice what they preach, and does it matter to project success? , 2010 .

[69]  Vivian W. Y Tam,et al.  Impacts of multi-layer chain subcontracting on project management performance , 2011 .

[70]  Albert P.C. Chan,et al.  Developing a Benchmarking Model for Construction Projects in Hong Kong , 2013 .