Forecasting model selection through out-of-sample rolling horizon weighted errors

Demand forecasting is an essential process for any firm whether it is a supplier, manufacturer or retailer. A large number of research works about time series forecast techniques exists in the literature, and there are many time series forecasting tools. In many cases, however, selecting the best time series forecasting model for each time series to be dealt with is still a complex problem. In this paper, a new automatic selection procedure of time series forecasting models is proposed. The selection criterion has been tested using the set of monthly time series of the M3 Competition and two basic forecasting models obtaining interesting results. This selection criterion has been implemented in a forecasting expert system and applied to a real case, a firm that produces steel products for construction, which automatically performs monthly forecasts on tens of thousands of time series. As result, the firm has increased the level of success in its demand forecasts.

[1]  Alan J. Miller.  Subset Selection in Regression , 1992 .

[2]  Cengiz Kahraman,et al.  A decision support system for demand forecasting with artificial neural networks and neuro-fuzzy models: A comparative analysis , 2009, Expert Syst. Appl..

[3]  Ching-Chin Chern,et al.  Designing a decision-support system for new product sales forecasting , 2010, Expert Syst. Appl..

[4]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[5]  E. Kim,et al.  A survey of decision support system applications (1995–2001) , 2006, J. Oper. Res. Soc..

[6]  Peter R. Winters,et al.  Forecasting Sales by Exponentially Weighted Moving Averages , 1960 .

[7]  Lutz Kilian,et al.  On the Selection of Forecasting Models , 2003, SSRN Electronic Journal.

[8]  T. A. Spedding,et al.  Forecasting demand and inventory management using Bayesian time series , 2000 .

[9]  A. A. Weiss,et al.  Estimating Time Series Models using the Relevant Forecast Evaluation Criterion , 1984 .

[10]  Spyros Makridakis,et al.  Sliding simulation: a new approach to time series forecasting , 1990 .

[11]  Bay Arinze,et al.  Selecting appropriate forecasting models using rule induction , 1994 .

[12]  Fred Collopy,et al.  Rule-Based Forecasting: Development and Validation of an Expert Systems Approach to Combining Time Series Extrapolations , 1992 .

[13]  Leonard J. Tashman,et al.  Out-of-sample tests of forecasting accuracy: an analysis and review , 2000 .

[14]  Fred Collopy,et al.  Toward computer-aided forecasting systems: gathering, coding, and validating the knowledge , 2008 .

[15]  Douglas C. Montgomery,et al.  Forecasting and time series analysis , 1976 .

[16]  J. D. Croston.  Forecasting and Stock Control for Intermittent Demands , 1972 .

[17]  C. Chatfield,et al.  The M2-competition: A real-time judgmentally based forecasting study , 1993 .

[18]  Sang M. Lee,et al.  A Survey of Decision Support System Applications (1971–April 1988) , 1990 .

[19]  Michèle Hibon,et al.  Accuracy of Forecasting: An Empirical Investigation , 1979 .

[20]  C. C. Holt,et al.  Planning Production, Inventories, and Work Force. , 1962 .

[21]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[22]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[23]  Mehdi Khashei,et al.  An artificial neural network (p, d, q) model for timeseries forecasting , 2010, Expert Syst. Appl..

[24]  J. Scott Armstrong,et al.  Potential Diffusion of Expert Systems in Forecasting , 2001 .

[25]  Chin-Tsai Lin,et al.  Artificial intelligence diagnosis algorithm for expanding a precision expert forecasting system , 2009, Expert Syst. Appl..

[26]  Konstantinos Nikolopoulos,et al.  Theta intelligent forecasting information system , 2003, Ind. Manag. Data Syst..

[27]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[28]  J. Boylan,et al.  The accuracy of intermittent demand estimates , 2005 .

[29]  R. Brown.  Statistical forecasting for inventory control , 1960 .

[30]  Scott P. Simkins.  Forecasting with vector autoregressive (VAR) models subject to business cycle restrictions , 1995 .

[31]  Fred Collopy,et al.  Rule-Based Forecasting: Using Judgment in Time-Series Extrapolation , 2001 .

[32]  Spyros Makridakis,et al.  The M3-Competition: results, conclusions and implications , 2000 .

[33]  Hossam A. Gabbar,et al.  A hybrid statistical genetic-based demand forecasting expert system , 2009, Expert Syst. Appl..

[34]  Paul R. Kleindorfer,et al.  Optimizer: IBM's Multi-Echelon Inventory System for Managing Service Logistics , 1990 .

[35]  Shiva Raj Singh,et al.  A computational method of forecasting based on high-order fuzzy time series , 2009, Expert Syst. Appl..

[36]  Alan J. Miller,et al.  Subset Selection in Regression , 1991 .

[37]  Ping-Feng Pai,et al.  Forecasting output of integrated circuit industry by support vector regression models with marriage honey-bees optimization algorithms , 2009, Expert Syst. Appl..

[38]  Shu-Hsien Liao,et al.  Expert system methodologies and applications - a decade review from 1995 to 2004 , 2005, Expert Syst. Appl..

[39]  Benito E. Flores,et al.  The use of an expert system in the M3 competition , 2000 .

[40]  Keith J. Burnham,et al.  Fuzzy decision support system for demand forecasting with a learning mechanism , 2006, Fuzzy Sets Syst..

[41]  Sean B. Eom,et al.  A survey of decision support system applications (1988–1994) , 1998, J. Oper. Res. Soc..

[42]  Hao-Tien Liu,et al.  An integrated fuzzy time series forecasting system , 2009, Expert Syst. Appl..

[43]  J. Z. Szymarkiewicz.  Forecasting and Time Series Analysis , 1978 .

[44]  Francisco de A. T. de Carvalho,et al.  A Modal Symbolic Classifier for selecting time series models , 2004, Pattern Recognit. Lett..

[45]  Bo K. Wong,et al.  Utilization and Benefits of Expert Systems in Manufacturing , 1994 .

[46]  Jan Fransoo,et al.  SKU demand forecasting in the presence of promotions , 2009, Expert Syst. Appl..

[47]  R. Fildes,et al.  The Impact of Empirical Accuracy Studies On Time Series Analysis and Forecasting , 1995 .

[48]  C. Granger,et al.  Experience with Forecasting Univariate Time Series and the Combination of Forecasts , 1974 .

[49]  Fred L. Collopy,et al.  Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons , 1992 .

[50]  K. Nikolopoulos,et al.  The theta model: a decomposition approach to forecasting , 2000 .

[51]  Seyyed M. T. Fatemi Ghomi,et al.  Development of a PSO-SA hybrid metaheuristic for a new comprehensive regression model to time-series forecasting , 2010, Expert Syst. Appl..

[52]  J. Scott Armstrong,et al.  Principles of forecasting , 2001 .

[53]  Víctor Gómez,et al.  Seasonal Adjustment and Signal Extraction in Economic Time Series , 1998 .

[54]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[55]  Robert L. Winkler,et al.  The accuracy of extrapolation (time series) methods: Results of a forecasting competition , 1982 .

[56]  Robert Fildes,et al.  Evaluation of Aggregate and Individual Forecast Method Selection Rules , 1989 .

[57]  Steven C. Wheelwright,et al.  Forecasting methods and applications. , 1979 .

[58]  R. J. Kuo,et al.  A decision support system for sales forecasting through fuzzy neural networks with asymmetric fuzzy weights , 1998, Decis. Support Syst..

[59]  Philip Hans Franses,et al.  A model selection strategy for time series with increasing seasonal variation , 1998 .

[60]  Rob J Hyndman,et al.  25 Years of Iif Time Series Forecasting: A Selective Review , 2005 .