The inventory performance of forecasting methods: Evidence from the M3 competition data

Abstract Forecasting competitions have been a major driver not only of improvements in forecasting methods’ performances, but also of the development of new forecasting approaches. However, despite the tremendous value and impact of these competitions, they do suffer from the limitation that performances are measured only in terms of the forecast accuracy and bias, ignoring utility metrics. Using the monthly industry series of the M3 competition, we empirically explore the inventory performances of various widely used forecasting techniques, including exponential smoothing, ARIMA models, the Theta method, and approaches based on multiple temporal aggregation. We employ a rolling simulation approach and analyse the results for the order-up-to policy under various lead times. We find that the methods that are based on combinations result in superior inventory performances, while the Naive, Holt, and Holt-Winters methods perform poorly.

[1]  Taylor Randall,et al.  In Search of the Bullwhip Effect , 2007, Manuf. Serv. Oper. Manag..

[2]  Stephen Michael Disney,et al.  Fill rate in a periodic review order-up-to policy under auto-correlated normally distributed, possibly negative, demand , 2015 .

[3]  Frank Y. Chen,et al.  Quantifying the Bullwhip Effect in a Simple Supply Chain: The Impact of Forecasting, Lead Times, and Information.: The Impact of Forecasting, Lead Times, and Information. , 2000 .

[4]  W. Van de Velde,et al.  Variance amplification and the golden ratio in production and inventory control , 2004 .

[5]  John E. Boylan,et al.  On the interaction between forecasting and stock control: The case of non-stationary demand , 2011 .

[6]  A. Syntetos,et al.  Forecast errors and inventory performance under forecast information sharing , 2012 .

[7]  R. Fildes,et al.  The effects of integrating management judgement into intermittent demand forecasts , 2009 .

[8]  F. Petropoulos,et al.  Improving forecasting by estimating time series structural components across multiple frequencies , 2014 .

[9]  Fotios Petropoulos,et al.  Another look at estimators for intermittent demand , 2016 .

[10]  M. Z. Babai,et al.  Impact of temporal aggregation on stock control performance of intermittent demand estimators: Empirical analysis , 2012 .

[11]  Philip Hans Franses,et al.  The M3 competition: Statistical tests of the results , 2005 .

[12]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[13]  Vineet Padmanabhan,et al.  Comments on "Information Distortion in a Supply Chain: The Bullwhip Effect" , 1997, Manag. Sci..

[14]  Xun Wang,et al.  The bullwhip effect: Progress, trends and directions , 2016, Eur. J. Oper. Res..

[15]  Nikolaos Kourentzes,et al.  On intermittent demand model optimisation and selection , 2014 .

[16]  Stephen Michael Disney,et al.  A delayed demand supply chain: Incentives for upstream players , 2012 .

[17]  S. Disney,et al.  On variance amplification in a three-echelon supply chain with minimum mean square error forecasting , 2022 .

[18]  Nikolaos Kourentzes,et al.  Distributions of forecasting errors of forecast combinations: Implications for inventory management , 2016 .

[19]  Pei-Chann Chang,et al.  Impacts of forecast, inventory policy, and lead time on supply chain inventory--A numerical study , 2010 .

[20]  Stephen M. Disney,et al.  Inventory performance under staggered deliveries and autocorrelated demand , 2016, Eur. J. Oper. Res..

[21]  H. H. Ippolite.  Inventory control. , 1960, Hospital management.

[22]  Brian G. Kingsman,et al.  Selecting the best periodic inventory control and demand forecasting methods for low demand items , 1997 .

[23]  Robert L. Winkler,et al.  The accuracy of extrapolation (time series) methods: Results of a forecasting competition , 1982 .

[24]  Edward A. Silver,et al.  Some insights regarding the optimal reorder period in periodic review inventory systems , 2008 .

[25]  Jr. Everette S. Gardner,et al.  Evaluating forecast performance in an inventory control system , 1990 .

[26]  Everette S. Gardner,et al.  Forecasting method selection in a global supply chain , 2012 .

[27]  Rob J Hyndman,et al.  Unmasking the Theta Method , 2003 .

[28]  Rob J Hyndman,et al.  Automatic Time Series Forecasting: The forecast Package for R , 2008 .

[29]  Aris A. Syntetos,et al.  Classification for forecasting and stock control: a case study , 2008, J. Oper. Res. Soc..

[30]  J. Boylan,et al.  The accuracy of intermittent demand estimates , 2005 .

[31]  A. Koehler,et al.  Models for optimising the theta method and their relationship to state space models , 2016 .

[32]  Louis J. Maccini,et al.  Taking Stock: A Critical Assessment of Recent Research on Inventories , 1991 .

[33]  Spyros Makridakis,et al.  The M3-Competition: results, conclusions and implications , 2000 .

[34]  S. Hewitt,et al.  Reproducibility , 2019, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[35]  J. Boylan,et al.  On the stock control performance of intermittent demand estimators , 2006 .

[36]  K. Nikolopoulos,et al.  The theta model: a decomposition approach to forecasting , 2000 .

[37]  Ruud H. Teunter,et al.  On the empirical performance of (T, s, S) heuristics , 2010, Eur. J. Oper. Res..

[38]  Brian G. Kingsman,et al.  Forecasting for the ordering and stock-holding of spare parts , 2004, J. Oper. Res. Soc..

[39]  John E. Boylan,et al.  Judging the judges through accuracy-implication metrics: The case of inventory forecasting , 2010 .

[40]  S. Disney,et al.  Damped trend forecasting and the order-up-to replenishment policy , 2014 .

[41]  M. Z. Babai,et al.  Forecasting intermittent inventory demands: simple parametric methods vs. bootstrapping ☆ , 2015 .

[42]  Aris A. Syntetos,et al.  On the categorization of demand patterns , 2005, J. Oper. Res. Soc..

[43]  Fotios Petropoulos,et al.  An aggregate–disaggregate intermittent demand approach (ADIDA) to forecasting: an empirical proposition and analysis , 2011, J. Oper. Res. Soc..

[44]  Nikolaos Kourentzes,et al.  Intermittent demand forecasts with neural networks , 2013 .

[45]  Fotios Petropoulos,et al.  To select or to combine? The inventory performance of model and expert forecasts , 2016 .

[46]  L. Duncan,et al.  Forecasting intermittent demand: a comparative study , 2009, J. Oper. Res. Soc..

[47]  Leonard J. Tashman,et al.  Out-of-sample tests of forecasting accuracy: an analysis and review , 2000 .

[48]  J. D. Croston.  Forecasting and Stock Control for Intermittent Demands , 1972 .

[49]  John E. Boylan,et al.  Reproducibility in forecasting research , 2015 .

[50]  Anne B. Koehler,et al.  Forecasting for Inventory Control with Exponential Smoothing , 2002 .