Neural Network Models for Conditional Distribution Under Bayesian Analysis

We use neural networks (NN) as a tool for a nonlinear autoregression to predict the second moment of the conditional density of return series. The NN models are compared to the popular econometric GARCH(1,1) model. We estimate the models in a Bayesian framework using Markov chain Monte Carlo posterior simulations. The interlinked aspects of the proposed Bayesian methodology are identification of NN hidden units and treatment of NN complexity based on model evidence. The empirical study includes the application of the designed strategy to market data, where we found a strong support for a nonlinear multilayer perceptron model with two hidden units.

[1]  C. C. Homes,et al.  Bayesian Radial Basis Functions of Variable Dimension , 1998, Neural Computation.

[2]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[3]  Raquel Montes Diez,et al.  Bayesian Analysis of Nonlinear Autoregression Models Based on Neural Networks , 2005, Neural Computation.

[4]  Alan D. Marrs.  An Application of Reversible-Jump MCMC to Multivariate Spherical Gaussian Mixtures , 1997, NIPS.

[5]  Jouko Lampinen,et al.  Bayesian approach for neural networks--review and case studies , 2001, Neural Networks.

[6]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[7]  Joseph G. Ibrahim,et al.  Monte Carlo Methods in Bayesian Computation , 2000 .

[8]  A. Rukhin.  Bayes and Empirical Bayes Methods for Data Analysis , 1997 .

[9]  S. Chib,et al.  Marginal Likelihood From the Metropolis–Hastings Output , 2001 .

[10]  S. Frühwirth-Schnatter.  Markov chain Monte Carlo Estimation of Classical and Dynamic Switching and Mixture Models , 2001 .

[11]  Hermann Locarek-Junge,et al.  Estimating Value-at-Risk Using Neural Networks , 1998 .

[12]  John Geweke,et al.  Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments , 1991 .

[13]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[14]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[15]  M. Stephens.  Dealing with label switching in mixture models , 2000 .

[16]  Teruo Nakatsuma,et al.  Bayesian analysis of ARMA–GARCH models: A Markov chain sampling approach , 2000 .

[17]  D. Mackay,et al.  A Practical Bayesian Framework for Backprop Networks , 1991 .

[18]  A. Raftery,et al.  How Many Iterations in the Gibbs Sampler , 1991 .

[19]  Peter Müller,et al.  Issues in Bayesian Analysis of Neural Network Models , 1998, Neural Computation.

[20]  Siddhartha Chib,et al.  Markov Chain Monte Carlo Simulation Methods in Econometrics , 1996, Econometric Theory.

[21]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[22]  T. Bollerslev,et al.  Generalized autoregressive conditional heteroskedasticity , 1986 .

[23]  R. Donaldson,et al.  An artificial neural network-GARCH model for international stock return volatility , 1997 .

[24]  Xiao-Li Meng,et al.  SIMULATING RATIOS OF NORMALIZING CONSTANTS VIA A SIMPLE IDENTITY: A THEORETICAL EXPLORATION , 1996 .

[25]  Christophe Andrieu,et al.  Robust Full Bayesian Methods for Neural Networks , 1999, NIPS.

[26]  S. Frühwirth-Schnatter.  Estimating Marginal Likelihoods for Mixture and Markov Switching Models Using Bridge Sampling Techniques , 2004 .

[27]  Christian L. Dunis,et al.  Forecasting and Trading Currency Volatility: An Application of Recurrent Neural Regression and Model Combination , 2002 .

[28]  M. Newton.  Approximate Bayesian-inference With the Weighted Likelihood Bootstrap , 1994 .

[29]  S. Frühwirth-Schnatter.  Bayesian Model Discrimination and Bayes Factors for Linear Gaussian State Space Models , 1995 .

[30]  Sylvia Frühwirth-Schnatter.  MCMC Estimation of Classical and Dynamic Switching and Mixture Models , 1998 .

[31]  Sylvia Kaufmann,et al.  Bayesian analysis of switching ARCH models , 2002 .

[32]  Herbert K. H. Lee,et al.  Model selection and model averaging for neural networks , 1998 .

[33]  Jouko Lampinen,et al.  Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities , 2002, Neural Computation.

[34]  Jingtao Yao,et al.  Guidelines for Financial Forecasting with Neural Networks , 2001 .

[35]  E. Dockner,et al.  Forecasting Time-dependent Conditional Densities: A Semi-non- parametric Neural Network Approach , 2000 .

[36]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[37]  John Geweke,et al.  Federal Reserve Bank of Minneapolis Research Department Staff Report 249 Using Simulation Methods for Bayesian Econometric Models: Inference, Development, and Communication , 2022 .

[38]  P. Krishna Rao,et al.  Sea Surface Temperature , 1990 .

[39]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[40]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[41]  Georg Dorffner,et al.  A comparison of Bayesian model selection based on MCMC with an application to GARCH-type models , 2006 .

[42]  J. Geweke,et al.  Exact predictive densities for linear models with arch disturbances , 1989 .

[43]  A. F. Darrat,et al.  On Testing the Random Walk Hypothesis: A Model-Comparison Approach , 2000 .