Performance measurement of research and development activities

Purpose – This paper aims to study the problem of designing a Performance Measurement System (PMS) for R&D. In particular, it aims at investigating the influence exerted by the type of activity being measured (i.e. Basic and Applied Research or New Product Development) on the design of the PMS constitutive elements. Design/methodology/approach – First, a literature review made it possible to build a theoretical model that identifies the constitutive elements of the PMS for R&D. Second, a survey involving 129 Italian firms (with a response rate of 33 per cent) was performed to unearth a number of similarities and differences between PMSs used in Research and New Product Development settings. Finally, a follow-up multiple case study investigation made it possible to understand the reason underlying the dissimilarities which emerged from the survey. Findings – The analysis suggests that a specialisation of the performance measurement practices in research and development is pursued by the Italian firms in the sample, especially when a number of conditions are in place (e.g. high level of uncertainty in R&D, availability of resources, organisational separation between the research and development functions). Research limitations/implications – The empirical analyses reported here focus on R&D-intensive firms, because they invest more heavily in R&D and hence are more interested in measuring its performance. The generalisability of these results to other empirical settings is discussed and represents a promising avenue for future research. Practical implications – The paper provides R&D managers with a number of criteria they should employ to design more effective PMSs for the R&D activities for which they are responsible. Moreover, it suggests that designing and using two different PMSs for research and new product development can be a valuable alternative but only under specific circumstances. Originality/value – The paper is one of the first contributions that empirically assess the differences in the approaches employed to measure performance of research and new product development activities.

[1]  John H. Waterhouse,et al.  A contingency framework for management accounting systems research , 1978 .

[2]  Warren B. Brown,et al.  Observations on the measurement of R&D productivity: a case study , 1992 .

[3]  Federico Frattini,et al.  TOWARDS A SYSTEM OF PERFORMANCE MEASURES FOR RESEARCH ACTIVITIES: NIKEM RESEARCH CASE STUDY , 2006 .

[4]  Rene Cordero,et al.  The measurement of innovation performance in the firm: An overview , 1990 .

[5]  David L. Deeds.  The Role of R&D Intensity, Technical Development and Absorptive Capacity in Creating Entrepreneurial Wealth in High Technology Start-Ups , 2001 .

[6]  W. Bruns,et al.  BUDGETARY CONTROL AND ORGANIZATION STRUCTURE , 1975 .

[7]  Robert A. Simons.  The role of management control systems in creating competitive advantage: New perspectives☆ , 1990 .

[8]  Christoph H. Loch,et al.  Measuring development performance in the electronics industry , 1996 .

[9]  Bill Nixon,et al.  Research and development performance measurement: A case study , 1998 .

[10]  Umit Bititci,et al.  Integrated performance measurement systems: a development guide , 1997 .

[11]  T. Kumpe,et al.  Toward the innovative firm - challenge for R&D management , 1994 .

[12]  Kenneth A. Merchant,et al.  Modern Management Control Systems: Text and Cases , 1997 .

[13]  Helena M. M. Lastres,et al.  Emerging Patterns of Innovation: Sources of Japan's Technological Edge , 1994 .

[14]  George A. Morgan,et al.  SPSS for Windows: An introduction To Use and interpretation in Research , 2000 .

[15]  Donald H. Liles,et al.  R&D Validation Planning: A Methodology to Link Technical Validations to Benefits Measurement , 2000 .

[16]  G. Hedlund.  The hypermodern MNC—A heterarchy? , 1986 .

[17]  William E. Souder,et al.  Measuring R&D Performance-State of the Art , 1997 .

[18]  R. Kaplan,et al.  The balanced scorecard--measures that drive performance. , 2015, Harvard business review.

[19]  Andrew Cooke,et al.  Design principles for the development of measurement systems for research and development processes , 1997 .

[20]  Marc Wouters,et al.  Designing a performance measurement system: A case study , 2004, Eur. J. Oper. Res..

[21]  Robert A. Simons,et al.  Levers of Control: How Managers Use Innovative Control Systems to Drive Strategic Renewal , 1994 .

[22]  Ashraf M. Ahmed,et al.  Virtual integrated performance measurement , 2002 .

[23]  Chang-Yang Lee,et al.  Industry R&D intensity distributions: regularities and underlying determinants , 2002 .

[24]  William G. Egelhoff.  Strategy and Structure in Multinational Corporations: An Information- Processing Approach , 1982 .

[25]  John R. Hauser,et al.  Research, development, and engineering metrics , 1998 .

[26]  B. W. Ang,et al.  A comparative analysis of R&D project evaluation methods , 2001 .

[27]  Vittorio Chiesa,et al.  Designing a performance measurement system for the research activities: A reference framework and an empirical study , 2008 .

[28]  Michel Lebas,et al.  Performance measurement and performance management , 1995 .

[29]  D. Otley,et al.  Accounting for Management Control , 1990 .

[30]  Raynold A. Svenson,et al.  Measuring R&D Productivity , 1988 .

[31]  K. Srinivasa Rao,et al.  Performance monitoring in R&D projects , 1996 .

[32]  J. West.  Institutions, information processing, and organization structure in research and development: evidence from the semiconductor industry , 2000 .

[33]  Kirsty Hughes,et al.  The interpretation and measurement of R&D intensity — A note☆ , 1988 .

[34]  D. Mcclelland,et al.  The Achievement Motive , 1954 .

[35]  Rod Coombs,et al.  Core competencies and the strategic management of R&D , 1996 .

[36]  Andy Neely,et al.  Performance measurement system design: Should process based approaches be adopted? , 1996 .

[37]  Michael L. Tushman,et al.  Information Processing as an Integrating Concept in Organizational Design. , 1978 .

[38]  R. Eccles.  The performance measurement manifesto. , 1991, Harvard business review.

[39]  Noah P. Barsky,et al.  Utilizing the balanced scorecard for R&D performance measurement , 2004 .

[40]  Paul A. Schumann,et al.  Measuring R&D Performance , 1995 .

[41]  Christoph H. Loch,et al.  Implementing a strategy-driven performance measurement system for an applied research group , 2002 .

[42]  Andy Neely,et al.  Performance measurement system design , 1995 .

[43]  I. C. Kerssens-van Drongelen,et al.  R&D as a business - what are the implications for performance measurement , 2000 .

[44]  John R. Hauser,et al.  Metrics to evaluate R,D&E , 1997 .

[45]  Antonio Davila,et al.  Performance Measurement and Control Systems for Implementing Strategy: Text and Cases , 1999 .

[46]  Vittorio Chiesa,et al.  How do measurement objectives influence the R&D performance measurement system design?: Evidence from a multiple case study , 2007 .

[47]  E. Kohler.  Accounting for Management , 1965 .

[48]  Bill Nixon,et al.  Productivity and performance measurement in R&D , 1997 .

[49]  Ching-Chow Yang,et al.  The establishment of project-oriented and cost-based NPD performance evaluation , 2006 .

[50]  Jay R. Galbraith.  Designing Complex Organizations , 1973 .

[51]  Roland Ortt,et al.  Innovation management: different approaches to cope with the same trends , 2006, Int. J. Technol. Manag..

[52]  Mike Gregory,et al.  Integrated performance measurement - a review of current practice and emerging trends , 1993 .

[53]  R. Yin.  Case Study Research: Design and Methods , 1984 .

[54]  Paul Rouse,et al.  Integrated Performance Measurement Design: Insights from an Application in Aircraft Maintenance , 2002 .

[55]  Donald S. Remer,et al.  Measuring R&D Productivity , 1985 .

[56]  Raffaele Oriani,et al.  Does the Market Value R&D Investment by European Firms? Evidence from a Panel of Manufacturing Firms in France, Germany, and Italy , 2004 .

[57]  Richard L. Daft,et al.  Organizational information requirements, media richness and structural design , 1986 .

[58]  Jaana Sandström,et al.  The problem of managing product development engineers: Can the balanced scorecard be an answer? , 2002 .

[59]  Vittorio Chiesa,et al.  Searching for an effective measure of R&D performance , 1996 .

[60]  Kulwant S. Pawar,et al.  Performance measurement for product design and development in a manufacturing environment , 1999 .

[61]  A. Fusfeld,et al.  Assessing the Value of Your Technology , 1995 .

[62]  Klas Eric Soderquist,et al.  Use and impact of performance measurement results in R&D and NPD: an exploratory study , 2004 .

[63]  Anil Menon,et al.  Making Innovation Happen in Organizations: Individual Creativity Mechanisms, Organizational Creativity Mechanisms or Both? , 2000 .

[64]  R. Keller,et al.  Technology-Information Processing Fit and the Performance of R&D Project Groups: A Test of Contingency Theory , 1994 .

[65]  Robert Szakonyi,et al.  Measuring R&D Effectiveness—I , 1994 .

[66]  Wesley M. Cohen,et al.  The Anatomy of Industry R&D Intensity Distributions , 1992 .

[67]  Ville Ojanen,et al.  Coping with the multiple dimensions of R&D performance analysis , 2006, Int. J. Technol. Manag..

[68]  Kulwant S. Pawar,et al.  Measuring product development performance in manufacturing organisations , 2000 .

[69]  Jan Bilderbeek,et al.  R&D Performance Measurement: More than Choosing a Set of Metrics , 1997 .

[70]  Thomas Triscari,et al.  Research versus development: Differences and similarities , 1987, IEEE Transactions on Engineering Management.

[71]  Andy Neely,et al.  Designing, implementing and updating performance measurement systems , 2000 .

[72]  Vittorio Chiesa,et al.  Exploring the Differences in Performance Measurement between Research and Development: Evidence from a Multiple Case Study , 2007 .

[73]  Vijay Govindarajan,et al.  Strategic Cost Management: The New Tool for Competitive Advantage , 1993 .