Using Neural Networks to Model Conditional Multivariate Densities

Neural network outputs are interpreted as parameters of statistical distributions. This allows us to fit conditional distributions in which the parameters depend on the inputs to the network. We exploit this in modeling multivariate data, including the univariate case, in which there may be input-dependent (e.g., time-dependent) correlations between output components. This provides a novel way of modeling conditional correlation that extends existing techniques for determining input-dependent (local) error bars.

[1]  T. W. Anderson,et al.  An Introduction to Multivariate Statistical Analysis , 1959 .

[2]  M. Kupperman.  Linear Statistical Inference and Its Applications 2nd Edition (C. Radhakrishna Rao) , 1975 .

[3]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[4]  John Scott Bridle,et al.  Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition , 1989, NATO Neurocomputing.

[5]  Geoffrey E. Hinton,et al.  Adaptive Soft Weight Tying using Gaussian Mixtures , 1991, NIPS.

[6]  Wray L. Buntine,et al.  Bayesian Back-Propagation , 1991, Complex Syst..

[7]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[8]  William H. Press,et al.  Numerical Recipes in C, 2nd Edition , 1992 .

[9]  William H. Press,et al.  Numerical recipes in C (2nd ed.): the art of scientific computing , 1992 .

[10]  Geoffrey E. Hinton,et al.  Simplifying Neural Networks by Soft Weight-Sharing , 1992, Neural Computation.

[11]  Radford M. Neal.  Bayesian training of backpropagation networks by the hybrid Monte-Carlo method , 1992 .

[12]  Christopher M. Bishop,et al.  Curvature-driven smoothing: a learning algorithm for feedforward networks , 1993, IEEE Trans. Neural Networks.

[13]  Michael I. Jordan,et al.  Supervised learning from incomplete data via an EM approach , 1993, NIPS.

[14]  David A. Nix,et al.  Learning Local Error Bars for Nonlinear Regression , 1994, NIPS.

[15]  A. Weigend,et al.  Predictions with Confidence Intervals ( Local Error Bars ) , 1994 .

[16]  C. Bishop.  Mixture density networks , 1994 .

[17]  Peter M. Williams,et al.  Bayesian Regularization and Pruning Using a Laplace Prior , 1995, Neural Computation.

[18]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .