Learning to control a complex multistable system.

In this paper the control of a periodically kicked mechanical rotor without gravity in the presence of noise is investigated. In recent work it was demonstrated that this system possesses many competing attracting states and thus shows the characteristics of a complex multistable system. We demonstrate that it is possible to stabilize the system at a desired attracting state even in the presence of high noise level. The control method is based on a recently developed algorithm [S. Gadaleta and G. Dangelmayr, Chaos 9, 775 (1999)] for the control of chaotic systems and applies reinforcement learning to find a global optimal control policy directing the system from any initial state towards the desired state in a minimum number of iterations. Being data-based, the method does not require any information about governing dynamical equations.

[1]  Chin-Teng Lin,et al.  Controlling chaos by GA-based reinforcement learning neural network , 1999, IEEE Trans. Neural Networks.

[2]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[3]  Foss,et al.  Multistability and delayed recurrent loops. , 1996, Physical review letters.

[4]  J. Hervagault,et al.  Multiple steady states and dissipative structures in a circular and linear array of three cells : numerical and experimental approaches , 1991 .

[5]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[6]  F. Arecchi,et al.  Experimental evidence of subharmonic bifurcations, multistability, and turbulence in a Q-switched gas laser , 1982 .

[7]  Peter Dayan,et al.  Bee foraging in uncertain environments using predictive hebbian learning , 1995, Nature.

[8]  Gerald Sommer,et al.  Dynamic Cell Structure Learns Perfectly Topology Preserving Map , 1995, Neural Computation.

[9]  Thomas Martinetz,et al.  Topology representing networks , 1994, Neural Networks.

[10]  In,et al.  Experimental maintenance of chaos. , 1995, Physical Review Letters.

[11]  B. Chirikov.  A universal instability of many-dimensional oscillator systems , 1979 .

[12]  J. Milton,et al.  NOISE, MULTISTABILITY, AND DELAYED RECURRENT LOOPS , 1996 .

[13]  C. Grebogi,et al.  Multistability and the control of complexity. , 1997, Chaos.

[14]  Wacker,et al.  Simple model for multistability and domain formation in semiconductor superlattices. , 1994, Physical review. B, Condensed matter.

[15]  L. Tsimring,et al.  The analysis of observed chaotic data in physical systems , 1993 .

[16]  On riddling and weak attractors , 2000 .

[17]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[18]  Sabino Gadaleta,et al.  Optimal chaos control through reinforcement learning. , 1999, Chaos.

[19]  Brambilla,et al.  Transverse laser patterns. II. Variational principle for pattern selection, spatial multistability, and laser hydrodynamics. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[20]  C Grebogi,et al.  Preference of attractors in noisy multistable systems. , 1999, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[21]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[22]  R. Bellman.  Dynamic programming. , 1957, Science.

[23]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[24]  Grebogi,et al.  Map with more than 100 coexisting low-period periodic attractors. , 1996, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[25]  Brambilla,et al.  Transverse laser patterns. I. Phase singularity crystals. , 1991, Physical review. A, Atomic, molecular, and optical physics.