Replications of Forecasting Research

We have examined the frequency of replications published in the two leading forecasting journals, the International Journal of Forecasting (IJF) and the Journal of Forecasting (JoF). Replications in the IJF and JoF between 1996 and 2008 comprised 8.4% of the empirical papers. Various other areas of management science have values ranging from 2.2% in the Journal of Marketing Research to 18.1% in the American Economic Review. We also found that 35.3% of the replications in forecasting journals provided full support for the findings of the initial study, 45.1% provided partial support, and 19.6% provided no support. Given the importance of replications, we recommend various steps to encourage replications, such as requiring a full disclosure of the methods and data used for all published papers, and inviting researchers to replicate specific important papers.

[1]  Bruce D. McCullough,et al.  Is it safe to assume that software is accurate , 2000 .

[2]  Robert Fildes,et al.  Generalising about univariate forecasting methods: further empirical , 1998 .

[3]  John E. Hunter,et al.  The Desperate Need for Replications , 2001 .

[4]  J. Ioannidis.  Contradicted and initially stronger effects in highly cited clinical research. , 2005, JAMA.

[5]  Jerry G. Thursby,et al.  Replication in Empirical Economics: The Journal of Money, Credit and Banking Project , 1986 .

[6]  J. Scott Armstrong,et al.  Are null results becoming an endangered species in marketing? , 1992 .

[7]  Raymond Hubbard,et al.  An empirical comparison of published replication research in accounting, economics, finance, management, and marketing , 1996 .

[8]  Vicki G. Morwitz,et al.  Sales Forecasts for Existing Consumer Products and Services: Do Purchase Intentions Contribute to Accuracy? , 2000 .

[9]  J. Scott Armstrong,et al.  Escalation bias: Does it extend to marketing? , 1993 .

[10]  J. Armstrong,et al.  Replications and Extensions in Marketing - Rarely Published But Quite Contrary , 1994 .

[11]  J. Scott Armstrong,et al.  Replication research's disturbing trend , 2007 .

[12]  B. D. McCullough,et al.  Got Replicability? The _Journal of Money, Credit and Banking_ Archive , 2007 .

[13]  Robert L. Winkler,et al.  The accuracy of extrapolation (time series) methods: Results of a forecasting competition , 1982 .

[14]  Everette S. Gardner,et al.  Exponential smoothing in the telecommunications data , 2008 .