Theory of interacting neural networks

In this contribution we give an overview over recent work on the theory of interacting neural networks. The model is defined in Section 2. The typical teacher/student scenario is considered in Section 3. A static teacher network is presenting training examples for an adaptive student network. In the case of multilayer networks, the student shows a transition from a symmetric state to specialisation. Neural networks can also generate a time series. Training on time series and predicting it are studied in Section 4. When a network is trained on its own output, it is interacting with itself. Such a scenario has implications on the theory of prediction algorithms, as discussed in Section 5. When a system of networks is trained on its minority decisions, it may be considered as a model for competition in closed markets, see Section 6. In Section 7 we consider two mutually interacting networks. A novel phenomenon is observed: synchronisation by mutual learning. In Section 8 it is shown, how this phenomenon can be applied to cryptography: Generation of a secret key over a public channel.

[1]  W. Kinzel,et al.  Secure exchange of information by synchronization of neural networks , 2002, cond-mat/0202112.

[2]  Christian Van den Broeck,et al.  Statistical Mechanics of Learning , 2001 .

[3]  W. Kinzel,et al.  Generation of unpredictable time series by a neural network. , 2000, Physical review. E, Statistical, nonlinear, and soft matter physics.

[4]  I. Kanter,et al.  Robust chaos generation by a perceptron , 2000, cond-mat/0007074.

[5]  W. Kinzel,et al.  Interacting neural networks. , 2000, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[6]  Zecchina,et al.  Statistical mechanics of systems with heterogeneous agents: minority games , 1999, Physical review letters.

[7]  L. Ein-Dor,et al.  Time series generation by multilayer networks , 1998 .

[8]  W. Kinzel,et al.  Limit cycles of a perceptron , 1997, cond-mat/9710010.

[9]  Riegler,et al.  Learning from noisy data: An exactly solvable model. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[10]  Saad,et al.  On-line learning in soft committee machines. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[11]  Kanter,et al.  Analytical study of time series generation by feed-forward networks. , 1995, Physical review letters.

[12]  Kanter,et al.  Generation and prediction of time series by a neural network. , 1995, Physical review letters.

[13]  Michael Biehl,et al.  On-Line Learning with a Perceptron , 1994 .

[14]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[15]  A. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[16]  Michael Biehl,et al.  Statistical Mechanics of On{line Learning and Generalization the Handbook of Brain Theory and Neural Networks , 2003 .

[17]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[18]  Douglas R. Stinson,et al.  Cryptography: Theory and Practice , 1995 .