A Bayesian Neural Network Model with Extensions a Bayesian Neural Network Model with Extensions

This report deals with a Bayesian neural network in a classiier context. In our network model, the units represent stochastic events, and the state of the units are related to the probability of these events. The basic Bayesian model is a one-layer neural network, which calculates the posterior probabilities of events, given some observed, independent events. The formulas underlying this network are examined, and generalized in order to make the network handle graded input, n:ary attributes, and continuous valued attributes. The one-layer model is then extended to a multi-layer architecture, to handle dependencies between input attributes. A few variations of this multi-layer Bayesian neural network are discussed. The nal result is a fairly general multi-layer Bayesian neural network, capable of handling discrete as well as continuous valued attributes.

[1]  Lawrence D. Jackel,et al.  Large Automatic Learning, Rule Extraction, and Generalization , 1987, Complex Syst..

[2]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[3]  Padhraic Smyth,et al.  Rule-Based Neural Networks for Classification and Probability Estimation , 1992, Neural Computation.

[4]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[5]  David T. Brown,et al.  A Note on Approximations to Discrete Probability Distributions , 1959, Inf. Control..

[6]  Demetri Psaltis,et al.  Higher order associative memories and their optical implementations , 1988, Neural Networks.

[7]  Akira Namatame,et al.  Structural Connectionist Learning with Complementary Coding , 1992, Int. J. Neural Syst..

[8]  Patrik Floréen.  Computational Complexity Problems in Neural Associative Memories , 1992 .

[9]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Ronald L. Rivest,et al.  Training a 3-node neural network is NP-complete , 1988, COLT '88.

[11]  Jürgen Schmidhuber,et al.  Learning Factorial Codes by Predictability Minimization , 1992, Neural Computation.

[12]  Dag Wedelin.  Efficient Algorithms for Probabilistic Inference, Combinatorial Optimization and the Discovery of Causal Structure from Data , 1993 .

[13]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[14]  Joydeep Ghosh,et al.  Efficient Higher-Order Neural Networks for Classification and Function Approximation , 1992, Int. J. Neural Syst..

[15]  Philip M. Lewis,et al.  Approximating Probability Distributions to Reduce Storage Requirements , 1959, Information and Control.

[16]  A. Norman Redlich,et al.  Redundancy Reduction as a Strategy for Unsupervised Learning , 1993, Neural Computation.

[17]  L. Stein,et al.  Probability and the Weighing of Evidence , 1950 .

[18]  Anders Lansner,et al.  Document Retrieval and Protein Sequence Matching using a Neural Network , 1993 .