The accuracy of benefit-cost analyses (BCAs) in transportation: An ex-post evaluation of road projects

Abstract Ex-post evaluations of benefit-cost analyses (BCAs) of transportation projects are scarce in the literature. If conducted frequently, they could reveal the extent to which objectives are achieved and may give inputs that can improve the quality of ex-ante BCAs. We first explain the usefulness of ex-post BCA evaluations of transportation projects, depending on which planning phase a project is in. We then perform ex-post BCAs on 27 Norwegian road projects that have been in service for at least 5 years and compare the results with the ex-ante BCAs that were presented to the decision-makers. We use two different measures of aggregating the magnitudes of accuracy; the Mean Percentage Error (MPE) and Mean Absolute Percentage (MAPE). Using MPE, we find that: (1) ex-ante BCAs underestimate the actual net present value (NPV) by 50% on average and, the NPV per dollar invested by a mere 0.14% points on average; (2) the traffic level and traffic growth rates are often underestimated ex-ante, leading to an underestimation of the benefits and; (3) construction costs are underestimated by 5% on average, which is too small to offset the observed underestimation of benefits. By default, MAPE shows higher values of inaccuracies implying that one should not be indifferent as to the measure used. Overall, the Norwegian ex-ante BCAs perform fairly well. We urge that the authorities concerned must improve their traffic forecasts and construction cost estimates to assure that the ex-ante objectives are met. The upkeep of ex-ante data is essential to enable ex-post evaluation which, ceteris paribus, will enhance the transparency and credibility of BCA as an appropriate decision-making tool.

[1]  Rune Elvik,et al.  The importance of confounding in observational before-and-after studies of road safety measures. , 2002, Accident; analysis and prevention.

[2]  Maria Börjesson,et al.  Does Benefit-Cost Efficiency Influence Transport Investment Decisions? , 2015 .

[3]  James Odeck,et al.  Cost escalations in the front-end of projects – empirical evidence from Norwegian road projects , 2017 .

[4]  James Odeck What Determines Decision‐Makers’ Preferences for Road Investments? Evidence from the Norwegian Road Sector , 2010 .

[5]  James Odeck Do reforms reduce the magnitudes of cost overruns in road projects? Statistical evidence from Norway , 2014 .

[6]  James Odeck,et al.  The Impact of External Quality Assurance of Costs Estimates on Cost Overruns: Empirical Evidence from the Norwegian Road Sector , 2015, European Journal of Transport and Infrastructure Research.

[7]  A. Boardman,et al.  Cost-Benefit Analysis: Concepts and Practice , 1996 .

[8]  Søren L. Buhl,et al.  What Causes Cost Overrun in Transport Infrastructure Projects? , 2004, 1304.4476.

[9]  Israel Herraiz,et al.  Explaining light vehicle demand evolution in interurban toll roads: a dynamic panel data analysis in Spain , 2016 .

[10]  Aidan R. Vining,et al.  Learning from ex ante/ex post cost-benefit comparisons: the coquihalla highway example , 1994 .

[11]  S. Bråthen DO FIXED LINKS AFFECT LOCAL INDUSTRY?: A NORWEGIAN CASE STUDY , 2001 .

[12]  Maria Börjesson,et al.  An ex-post CBA for the Stockholm Metro , 2014 .

[13]  Morten Skou Nicolaisen,et al.  Ex-Post Evaluations of Demand Forecast Accuracy: A Literature Review , 2014 .

[14]  Ginés de Rus,et al.  Introduction to Cost-Benefit Analysis: Looking for Reasonable Shortcuts , 2010 .

[15]  Project selection with sets of mutually exclusive alternatives , 2016 .

[16]  Peter E.D. Love,et al.  Overruns in transportation infrastructure projects , 2014 .

[17]  Bent Flyvbjerg,et al.  Cost Overruns in Large-Scale Transportation Infrastructure Projects: Explanations and Their Theoretical Embeddedness , 2010, European Journal of Transport and Infrastructure Research.

[18]  Matti Siemiatycki,et al.  Academics and Auditors , 2009 .

[19]  M. Muñoz,et al.  Traffic uncertainty in toll motorway concessions in Spain , 2012 .

[20]  James Odeck How accurate are national road traffic growth-rate forecasts?--The case of Norway , 2013 .

[21]  Ricard Anguera,et al.  The Channel Tunnel – An Ex-post Economic Evaluation , 2006 .

[22]  J. Odeck,et al.  The accuracy of toll road traffic forecasts: An econometric evaluation , 2017 .

[23]  Bent Flyvbjerg,et al.  Policy and Planning for Large-Infrastructure Projects: Problems, Causes, Cures , 2005, 1303.7400.

[24]  Steven C. Wheelwright,et al.  Forecasting methods and applications. , 1979 .

[25]  James Odeck,et al.  Cost overruns in road construction--what are their sizes and determinants? , 2004 .

[26]  John Nellthorp,et al.  Ex Post Appraisal: What Lessons Can Be Learnt from EU Cohesion Funded Transport Projects , 2012 .

[27]  James Odeck,et al.  Do Planners Get it Right? : The Accuracy of Travel Demand Forecasting in Norway , 2011 .