A Higher order Bayesian Neural Network with Spiking Units

We treat a Bayesian confidence propagation neural network, primarily in a classifier context. The one-layer version of the network implements a naive Bayesian classifier, which requires the input attributes to be independent. This limitation is overcome by a higher order network. The higher order Bayesian neural network is evaluated on a real world task of diagnosing a telephone exchange computer. By introducing stochastic spiking units, and soft interval coding, it is also possible to handle uncertain as well as continuous valued inputs.

[1]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[2]  D. Hubel,et al.  Ferrier lecture - Functional architecture of macaque monkey visual cortex , 1977, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[3]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[4]  Dag Wedelin.  Efficient Algorithms for Probabilistic Inference, Combinatorial Optimization and the Discovery of Causal Structure from Data , 1993 .

[5]  Apostolos P. Georgopoulos,et al.  A Neural Network for Coding of Trajectories by Time Series of Neuronal Population Vectors , 1994, Neural Computation.

[6]  Joydeep Ghosh,et al.  Efficient Higher-Order Neural Networks for Classification and Function Approximation , 1992, Int. J. Neural Syst..

[7]  Christopher M. Bishop,et al.  Bayesian Neural Networks , 1997, J. Braz. Comput. Soc..

[8]  A. Uttley,et al.  Properties of plastic networks. , 1962, Biophysical journal.

[9]  Ronald L. Rivest,et al.  Training a 3-node neural network is NP-complete , 1988, COLT '88.

[10]  J. Urgen Schmidhuber,et al.  Learning Factorial Codes by Predictability Minimization , 1992, Neural Computation.

[11]  Padhraic Smyth,et al.  Rule-Based Neural Networks for Classification and Probability Estimation , 1992, Neural Computation.

[12]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[13]  D. Purves,et al.  Iterated patterns of brain circuitry (or how the cortex gets its spots) , 1992, Trends in Neurosciences.

[14]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[15]  A. Norman Redlich,et al.  Redundancy Reduction as a Strategy for Unsupervised Learning , 1993, Neural Computation.

[16]  A. Lansner,et al.  Modelling Hebbian cell assemblies comprised of cortical neurons , 1992 .

[17]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[18]  Akira Namatame,et al.  Structural Connectionist Learning with Complementary Coding , 1992, Int. J. Neural Syst..

[19]  Örjan Ekeberg,et al.  A One-Layer Feedback Artificial Neural Network with a Bayesian Learning Rule , 1989, Int. J. Neural Syst..

[20]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[21]  Peter Földiák,et al.  Adaptation and decorrelation in the cortex , 1989 .

[22]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[23]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[24]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .

[25]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[26]  D. Winter,et al.  Models of recruitment and rate coding organization in motor-unit pools. , 1993, Journal of neurophysiology.

[27]  J.G. Daugman,et al.  Entropy reduction and decorrelation in visual coding by oriented neural receptive fields , 1989, IEEE Transactions on Biomedical Engineering.

[28]  Néstor Parga,et al.  Information processing by a perceptron in an unsupervised learning task , 1993 .

[29]  Anders Lansner,et al.  Improving the Realism of Attractor Models By using Cortical Columns as Functional Units , 1995 .

[30]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[31]  C. Malsburg,et al.  How patterned neural connections can be set up by self-organization , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[32]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[33]  R. Richter,et al.  Principles of Neural Science, 3rd edition , 1993 .

[34]  Naonori Ueda,et al.  A new competitive learning approach based on an equidistortion principle for designing optimal vector quantizers , 1994, Neural Networks.

[35]  Philip M. Lewis,et al.  Approximating Probability Distributions to Reduce Storage Requirements , 1959, Information and Control.

[36]  Hans G. C. Tråvén,et al.  A neural network approach to statistical pattern classification by 'semiparametric' estimation of probability density functions , 1991, IEEE Trans. Neural Networks.

[37]  Demetri Psaltis,et al.  Higher order associative memories and their optical implementations , 1988, Neural Networks.