Theory of interacting neural networks

In this contribution we give an overview over recent work on the theory of interacting neural networks. The model is defined in Section 2. The typical teacher/student scenario is considered in Section 3. A static teacher network is presenting training examples for an adaptive student network. In the case of multilayer networks, the student shows a transition from a symmetric state to specialisation. Neural networks can also generate a time series. Training on time series and predicting it are studied in Section 4. When a network is trained on its own output, it is interacting with itself. Such a scenario has implications on the theory of prediction algorithms, as discussed in Section 5. When a system of networks is trained on its minority decisions, it may be considered as a model for competition in closed markets, see Section 6. In Section 7 we consider two mutually interacting networks. A novel phenomenon is observed: synchronisation by mutual learning. In Section 8 it is shown, how this phenomenon can be applied to cryptography: Generation of a secret key over a public channel.

[1]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[2]  Michael Biehl,et al.  On-Line Learning with a Perceptron , 1994 .

[3]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[4]  Riegler,et al.  Learning from noisy data: An exactly solvable model. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[5]  Douglas R. Stinson,et al.  Cryptography: Theory and Practice , 1995 .

[6]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[7]  Kanter,et al.  Generation and prediction of time series by a neural network. , 1995, Physical review letters.

[8]  Kanter,et al.  Analytical study of time series generation by feed-forward networks. , 1995, Physical review letters.

[9]  W. Kinzel,et al.  Limit cycles of a perceptron , 1997, cond-mat/9710010.

[10]  I Kanter,et al.  Interacting neural networks. , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[11]  I. Kanter,et al.  Robust chaos generation by a perceptron , 2000 .

[12]  Zecchina,et al.  Statistical mechanics of systems with heterogeneous agents: minority games , 1999, Physical review letters.

[13]  Christian Van den Broeck,et al.  Statistical Mechanics of Learning , 2001 .

[14]  I Kanter,et al.  Generation of unpredictable time series by a neural network. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.