Performance of the Bayesian Online Algorithm for the Perceptron

In this letter, we derive continuum equations for the generalization error of the Bayesian online algorithm (BOnA) for the one-layer perceptron with a spherical covariance matrix using the Rosenblatt potential and show, by numerical calculations, that the asymptotic performance of the algorithm is the same as the one for the optimal algorithm found by means of variational methods with the added advantage that the BOnA does not use any inaccessible information during learning

[1]  R. Urbanczik,et al.  SELF-AVERAGING AND ON-LINE LEARNING , 1998, cond-mat/9805339.

[2]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[3]  Manfred Opper,et al.  A Bayesian Approach to Online Learning , 2006 .

[4]  N. Caticha,et al.  Gradient descent learning in and out of equilibrium. , 2000, Physical review. E, Statistical, nonlinear, and soft matter physics.

[5]  Motoaki Kawanabe,et al.  On-line learning in changing environments with applications in supervised and unsupervised learning , 2002, Neural Networks.

[6]  O. Kinouchi,et al.  Optimal generalization in perceptions , 1992 .

[7]  O. Kinouchi,et al.  Learning algorithm that gives the Bayes generalization limit for perceptrons. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[8]  Esther Levin,et al.  A statistical approach to learning and generalization in layered neural networks , 1989, Proc. IEEE.

[9]  E. A. de Oliveira.  The Rosenblatt Bayesian Algorithm Learning in a Nonstationary Environment , 2007, IEEE Transactions on Neural Networks.

[10]  Christian Van den Broeck,et al.  Statistical Mechanics of Learning , 2001 .