Vibration control of a nonlinear quarter-car active suspension system by reinforcement learning

This article presents the investigation of performance of a nonlinear quarter-car active suspension system with a stochastic real-valued reinforcement learning control strategy. As an example, a model of a quarter car with a nonlinear suspension spring subjected to excitation from a road profile is considered. The excitation is realised by the roughness of the road. The quarter-car model to be considered here can be approximately described as a nonlinear two degrees of freedom system. The experimental results indicate that the proposed active suspension system suppresses the vibrations greatly. A simulation of a nonlinear quarter-car active suspension system is presented to demonstrate the effectiveness and examine the performance of the learning control algorithm.

[1]  Toshio Yoshimura,et al.  Construction of an active suspension system of a quarter car model using fuzzy reasoning based on single input rule modules , 2000 .

[2]  Vijaykumar Gullapalli,et al.  A stochastic reinforcement learning algorithm for learning real-valued functions , 1990, Neural Networks.

[3]  Zhonghua Liu,et al.  Simulation and Experimental Study of Human Riding Comfort in Dynamic Man-Automobile System , 2008, ICIRA.

[4]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .

[5]  V. Gullapalli,et al.  Acquiring robot skills via reinforcement learning , 1994, IEEE Control Systems.

[6]  Toshio Yoshimura,et al.  Pneumatic active suspension system for a one-wheel car model using fuzzy reasoning and a disturbance observer , 2004, Journal of Zhejiang University. Science.

[7]  Chuan-Kai Lin,et al.  Adaptive critic autopilot design of Bank-to-turn missiles using fuzzy basis function networks , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  A. H. Klopf,et al.  Brain Function and Adaptive Systems: A Heterostatic Theory , 1972 .

[9]  P. Smagt van der,et al.  Neural Systems for Robotics , 2012 .

[10]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  Davorin David Hrovat.  Applications of Optimal Control to Advanced Automotive Suspension Design , 1993 .

[12]  B. Karlik,et al.  Active suspension control for vehicles and numerical calculations by using artificial neural networks , 2001 .

[13]  Antonio Moran,et al.  Optimal Active Control of Nonlinear Vehicle Suspensions Using Neural Networks , 1994 .

[14]  R. Sutton,et al.  Simulation of anticipatory responses in classical conditioning by a neuron-like adaptive element , 1982, Behavioural Brain Research.

[15]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[16]  Aleksander Hać.  Stochastic Optimal Control of Vehicles With Elastic Body and Active Suspension , 1986 .

[17]  Shaocheng Tong,et al.  Fuzzy adaptive control of multivariable nonlinear systems1 , 2000, Fuzzy Sets Syst..

[18]  P. Anandan,et al.  Pattern-recognizing stochastic learning automata , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[19]  Toshio Yoshimura,et al.  ACTIVE SUSPENSION OF PASSENGER CARS USING SLIDING MODE CONTROLLERS (BASED ON REDUCED MODELS) , 2014 .

[20]  Toshio Yoshimura,et al.  An active suspension for a vehicle travelling on flexible beams with an irregular surface , 1990 .

[21]  P. E. Wellstead,et al.  Active suspensions: some background , 1995 .

[22]  Edge C. Yeh,et al.  A fuzzy preview control scheme of active suspension for rough road , 1994 .

[23]  Toshio Yoshimura,et al.  CONSTRUCTION OF AN ACTIVE SUSPENSION SYSTEM OF A QUARTER CAR MODEL USING THE CONCEPT OF SLIDING MODE CONTROL , 2001 .

[24]  Ö. Gündoğdu.  Optimal seat and suspension design for a quarter car with driver model using genetic algorithms , 2007 .

[25]  Toshio Yoshimura,et al.  ACTIVE SUSPENSION SYSTEM OF A ONE-WHEEL CAR MODEL USING FUZZY REASONING AND COMPENSATORS. , 2003 .

[26]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[27]  Lixin Guo,et al.  Fuzzy Logic Control for Vehicle Suspension Systems , 2008, ICIRA.

[28]  Chin-Teng Lin,et al.  Adaptive fuzzy command acquisition with reinforcement learning , 1998, IEEE Trans. Fuzzy Syst..

[29]  A. Titli,et al.  Design of Active and Semi-Active Automotive Suspension Using Fuzzy Logic , 1993 .

[30]  D. Hrovat,et al.  Survey of Advanced Suspension Developments and Related Optimal Control Applications, , 1997, Autom..

[31]  Meir Shillor,et al.  Motion Control of a nonlinear Spring by Reinforcement Learning , 2008, Control. Intell. Syst..

[32]  Jacob Rosen,et al.  Modeling the human body/seat system in a vibration environment. , 2003, Journal of biomechanical engineering.

[33]  Kai-Tai Song,et al.  Reinforcement learning and its application to force control of an industrial robot , 1998 .

[34]  Hung-Yuan Chung,et al.  A self-learning fuzzy logic controller using genetic algorithms with reinforcements , 1997, IEEE Trans. Fuzzy Syst..

[35]  A. Barto,et al.  Learning and Sequential Decision Making , 1989 .

[36]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[37]  Richard S. Sutton,et al.  Associative search network: A reinforcement learning associative memory , 1981, Biological Cybernetics.

[38]  Rong-Jong Wai,et al.  Recurrent-fuzzy-neural-network sliding-mode controlled motor-toggle servomechanism , 2001 .

[39]  Igor Maciejewski,et al.  The vibration damping effectiveness of an active seat suspension system and its robustness to varying mass loading , 2010 .

[40]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.