Motion Control of a nonlinear Spring by Reinforcement Learning

Recent research in vehicle platform stabilization has developed solutions that do not meet the stringent "low-energy" demands required for widespread general application of stabilizing active suspension. The energy requirement for active platform stabilization can reach up to 40% of the total output of the engine. This power consumption may lead directly to reductions in vehicle performance and fuel economy. The need for low energy consumption is thereby a fundamental requirement for successful active suspension system. Cameron's investigation [1] involves a mechanical system with a positive stiffness spring in series with a "negative stiffness spring" to provide for reducing the power consumption. We aim to control the motion of a horizontally moving vehicle mass on a rail, when it is subject to friction, under the influence of a controlled compressed spring. The compressed spring behaves as if it has a negative spring constant over a range of its displacements. The system has three separate stick regions for some values of the parameters, centred on its critical points. Our objective is to control the motion by keeping its displacements in a robust nonstick region that lies between two predetermined neighboring stick zones. We use reinforcement learning control that applies proper control forces to this nonlinear spring system in an attempt to keep the motion of the body within the targeted nonstick region. We describe a numerical scheme for the model and its learning control algorithm and present a number of computer simulations, with different initial conditions and driving force amplitudes, and frequencies.

[1]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .

[2]  Richard S. Sutton,et al.  Learning and Sequential Decision Making , 1989 .

[3]  Richard S. Sutton,et al.  Associative search network: A reinforcement learning associative memory , 1981, Biological Cybernetics.

[4]  A. H. Klopf,et al.  Brain Function and Adaptive Systems: A Heterostatic Theory , 1972 .

[5]  R. Sutton,et al.  Simulation of anticipatory responses in classical conditioning by a neuron-like adaptive element , 1982, Behavioural Brain Research.

[6]  Kai-Tai Song,et al.  Reinforcement learning and its application to force control of an industrial robot , 1998 .

[7]  P. Anandan,et al.  Pattern-recognizing stochastic learning automata , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  Hung-Yuan Chung,et al.  A self-learning fuzzy logic controller using genetic algorithms with reinforcements , 1997, IEEE Trans. Fuzzy Syst..

[9]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[10]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.

[11]  V. Gullapalli,et al.  Acquiring robot skills via reinforcement learning , 1994, IEEE Control Systems.

[12]  Vijaykumar Gullapalli,et al.  A stochastic reinforcement learning algorithm for learning real-valued functions , 1990, Neural Networks.

[13]  Meir Shillor,et al.  Frictional contact of a nonlinear spring , 2000 .

[14]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[15]  Chin-Teng Lin,et al.  Adaptive fuzzy command acquisition with reinforcement learning , 1998, IEEE Trans. Fuzzy Syst..