Maximizing the robustness of a linear threshold classifier with discrete weights

Quantization of the parameters of a perceptron is a central problem in hardware implementation of neural networks using a numerical technology. An interesting property of neural networks used as classifiers is their ability to provide some robustness on input noise. This paper presents efficient learning algorithms for the maximization of the robustness of a perceptron and especially designed to tackle the combinatorial problem arising from the discrete weights.

[1]  Pablo Moscato,et al.  An introduction to population approaches for optimization and hierarchical objective functions: A discussion on the role of tabu search , 1993, Ann. Oper. Res..

[2]  Eddy Mayoraz,et al.  Benchmark of Some Learning Algorithms for Single-Layer and Hopfield Networks , 1990, Complex Syst..

[3]  Marcus R. Frean,et al.  A "Thermal" Perceptron Learning Rule , 1992, Neural Computation.

[4]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[5]  H M Kohler,et al.  Adaptive genetic algorithm for the binary perceptron problem , 1990 .

[6]  H. Gutfreund,et al.  Capacity of neural networks with discrete synaptic couplings , 1990 .

[7]  E. Mayoraz.  Maximizing the stability of a majority perceptron using Tabu search , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[8]  Edoardo Amaldi,et al.  Stability-Capacity Diagram of a Neural Network with Ising Bonds , 1989 .

[9]  Eddy Mayoraz,et al.  On the Power of Networks of Majority Functions , 1991, IWANN.

[10]  W. Krauth,et al.  Learning algorithms with optimal stability in neural networks , 1987 .

[11]  Stephen I. Gallant,et al.  Perceptron-based learning algorithms , 1990, IEEE Trans. Neural Networks.

[12]  A. Komoda,et al.  Storage capacity of a diluted neural network with Ising couplings , 1990 .

[13]  Mohamad H. Hassoun,et al.  Adaptive Ho-Kashyap rules for perceptron training , 1992, IEEE Trans. Neural Networks.

[14]  Jordi Carrabina,et al.  Study of a learning algorithm for neural networks with discrete synaptic couplings , 1992 .

[15]  Santosh S. Venkatesh,et al.  Directed Drift: A New Linear Threshold Algorithm for Learning Binary Weights On-Line , 1993, J. Comput. Syst. Sci..

[16]  M. Sinclair.  Tabu search annals of operations research, vol. 41 (1993): J.C. Baltzer AG, Science Publishers, Basel. 490 pages, ISSN 0254 5330 , 1994 .

[17]  W. Krauth,et al.  Storage capacity of memory networks with binary couplings , 1989 .

[18]  Werner Krauth,et al.  Critical storage capacity of the J = ± 1 neural network , 1989 .

[19]  H. Horner.  Dynamics of learning for the binary perceptron problem , 1992 .

[20]  Fred W. Glover,et al.  A user's guide to tabu search , 1993, Ann. Oper. Res..

[21]  Michel Verleysen,et al.  A high-storage capacity content-addressable memory and its learning algorithm , 1989 .

[22]  Edoardo Amaldi,et al.  A Review of Combinatorial Problems Arising in Feedforward Neural Network Design , 1994, Discret. Appl. Math..

[23]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[24]  Daniel Costa,et al.  An Evolutionary Tabu Search Algorithm And The NHL Scheduling Problem , 1995 .

[25]  William J. Cook,et al.  Sensitivity theorems in integer linear programming , 1986, Math. Program..

[26]  Eddy Mayoraz,et al.  Constructive Training Methods for feedforward Neural Networks with Binary weights , 1995, Int. J. Neural Syst..

[27]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[28]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .