Influencing factors on hydrogen energy R&D projects: An ex-post performance evaluation

To enhance its energy security, Korea made a significant R&D investment in developing hydrogen energy technology over a period of 10 years. The Hydrogen Energy R&D Center was established in 2003 with a 100 billion Korean won (KRW) government investment that would fund the initiative through 2012. This paper is a meaningful attempt to evaluate the ex-post performance of completed hydrogen energy R&D projects using data envelopment analysis (DEA). Through empirical analysis, we focus mainly on measuring ex-post performance and uncovering the influencing factors on that performance. The results of our empirical analysis reveal several interesting findings. First, overall, the ex-post performance of completed projects is relatively low. However, there are major performance gaps among projects. Second, R&D type, technology type, and private sector investment are factors influencing project performance. More specifically, projects identified as basic research, storage technology, or without private sector investment had the highest performance, with an acceptable level of statistical significance. Third, DEA can provide unique information, such as results of returns to scale and slack analysis, which cannot be gleaned from parametric methods. As time went on, the decreasing returns to scale (DRS) ratio increased, and the increasing returns to scale (IRS) ratio decreased. This means that the R&D budget was used less and less effectively as time passed. Moreover, among the three-year project period, the second year’s R&D budget was used most efficiently in each R&D project. This study can provide meaningful information to policy makers to enhance the performance of future R&D projects and to promote hydrogen R&D projects as well as advance the hydrogen economy.

[1]  Jiancheng Guan,et al.  Measuring the innovation production process: A cross-region empirical study of China’s high-tech innovations , 2010 .

[2]  E. Wang R&D efficiency and economic performance: A cross-country analysis using the stochastic frontier approach , 2007 .

[3]  Alan W. Pearson,et al.  Application of option pricing theory to R&D , 1994 .

[4]  Roberto Rendeiro Martín-Cejas,et al.  An approximation to the productive efficiency of the Spanish airports network through a deterministic cost frontier , 2002 .

[5]  E. Melachrinoudis,et al.  The prioritization of technologies in a research laboratory , 1991 .

[6]  Abraham Charnes,et al.  Measuring the efficiency of decision making units , 1978 .

[7]  Joe Zhu,et al.  Rank order data in DEA: A general framework , 2006, Eur. J. Oper. Res..

[8]  Duu-Jong Lee,et al.  An economy-wide analysis of hydrogen economy in Taiwan , 2009 .

[9]  A. Banerjee,et al.  A global survey of hydrogen energy research, development and policy , 2006 .

[10]  Joe Zhu Setting scale efficient targets in DEA via returns to scale estimation method , 2000, J. Oper. Res. Soc..

[11]  Duu-Jong Lee,et al.  Hydrogen economy in Taiwan and biohydrogen , 2008 .

[12]  E. Wang,et al.  Relative efficiency of R&D activities: A cross-country study accounting for environmental factors in the DEA approach , 2007 .

[13]  Avraham Shtub,et al.  R&D project evaluation: An integrated DEA and balanced scorecard approach ☆ , 2008 .

[14]  Barry G. Silverman,et al.  Project Appraisal Methodology : A Multidimensional R&D Benefit/Cost Assessment Tool : Management Science , 1981 .

[15]  Kamal Golabi,et al.  Selecting a group of dissimilar projects for funding , 1987, IEEE Transactions on Engineering Management.

[16]  M. Farrell The Measurement of Productive Efficiency , 1957 .

[17]  Paul Friley,et al.  A hydrogen economy: opportunities and challenges , 2005 .

[18]  J. Linton,et al.  Analysis, ranking and selection of R&D projects in a portfolio , 2002 .

[19]  Geoff Lockett,et al.  Modelling a Research Portfolio Using AHP: A Group Decision Process , 1986 .

[20]  E. Seymour,et al.  Indicators of European public research in hydrogen and fuel cells—An input–output analysis , 2007 .

[21]  Lawrence M. Seiford,et al.  Data envelopment analysis (DEA) - Thirty years on , 2009, Eur. J. Oper. Res..

[22]  Duk Hee Lee,et al.  Impact of Green Innovation on Labor Productivity and its Determinants: an Analysis of the Korean Manufacturing Industry , 2014 .

[23]  Gento Mogi,et al.  Energy technology roadmap for the next 10 years : The case of Korea , 2009 .

[24]  Ari P. J. Vepsalainen,et al.  Assessing technology portfolios for contract competition: An analytic hierarchy process approach , 1986 .

[25]  Jiyong Kim,et al.  The role of hydrogen in the road transportation sector for a sustainable energy system: A case study of Korea , 2008 .

[26]  Hakyeon Lee,et al.  Comparative evaluation of performance of national R&D programs with heterogeneous objectives: A DEA approach , 2009, Eur. J. Oper. Res..

[27]  Nickolaos G. Tzeremes,et al.  Analyzing the Greek renewable energy sector: A Data Envelopment Analysis approach , 2012 .

[28]  James R. Freeland,et al.  Recent Advances in R&D Benefit Measurement and Project Selection Methods : Management Science , 1975 .

[29]  A. Hashimoto,et al.  Measuring the change in R&D efficiency of the Japanese pharmaceutical industry , 2008 .

[30]  Chen-Fu Chien,et al.  Using DEA to Evaluate R&D Performance of the Computers and Peripherals Firms in Taiwan , 2004 .

[31]  R. Färe,et al.  PRODUCTIVITY AND INTERMEDIATE PRODUCTS: A FRONTIER APPROACH , 1995 .

[32]  Yanghon Chung,et al.  The role of hydrogen energy development in the Korean economy: An input–output analysis , 2014 .

[33]  Ning Zhang,et al.  A note on the evolution of directional distance function and its development in energy and environmental studies 1997–2013 , 2014 .

[34]  R. Ramanathan An Introduction to Data Envelopment Analysis: A Tool for Performance Measurement , 2003 .

[35]  Moutaz Khouja,et al.  The use of data envelopment analysis for technology selection , 1995 .

[36]  J. Martino Research and Development project selection , 1995 .

[37]  Ram Rachamadugu,et al.  A closer look at the use of data envelopment analysis for technology selection , 1997 .

[38]  D. L. Hall,et al.  An interactive approach for selecting IR&D projects , 1990 .

[39]  Duu-Jong Lee,et al.  Biohydrogen development in United States and in China: An input–output model study , 2011 .

[40]  Jong Wook Kim,et al.  Prioritizing the weights of hydrogen energy technologies in the sector of the hydrogen economy by using a fuzzy AHP approach , 2011 .

[41]  Hugo Hollanders,et al.  Measuring innovation efficiency , 2007 .

[42]  Chang-Bong Chung Energy info. Korea , 2003 .

[43]  Yong-Tae Park,et al.  An international comparison of R&D efficiency: DEA approach , 2005 .

[44]  Yanghon Chung,et al.  Labor Union Effects on Innovation and Commercialization Productivity: An Integrated Propensity Score Matching and Two-Stage Data Envelopment Analysis , 2015 .

[45]  Gento Mogi,et al.  The competitiveness of Korea as a developer of hydrogen energy technology: The AHP approach , 2008 .

[46]  Santiago Herrera,et al.  Efficiency of Public Spending in Developing Countries: An Efficiency Frontier Approach , 2005 .

[47]  A. Charnes,et al.  Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis , 1984 .

[48]  Jonathan F. Bard Using Multicriteria Methods in the Early Stages of New Product Development , 1990 .