Batch Reinforcement Learning - An Application to a Controllable Semi-active Suspension System

The design problem of optimal comfort-oriented semi-active suspension has been addressed with different standard techniques which failed to come out with an optimal strategy because the system is hard non-linear and the solution is too complex to be found analytically. In this work, we aimed at solving such complex problem by applying Batch Reinforcement Learning (BRL), that is an artificial intelligence technique that approximates the solution of optimal control problems without knowing the system dynamics. Recently, a quasi optimal strategy for semi-active suspension has been designed and proposed: the Mixed SH-ADD algorithm, which the strategy designed in this paper is compared to. We show that an accurately tuned BRL provides a policy able to guarantee the overall best performance.

[1]  Xubin Song,et al.  System Non-Linearities Induced by Skyhook Dampers , 2001 .

[2]  Cristiano Spelta,et al.  Mixed Sky-Hook and ADD: Approaching the Filtering Limits of a Semi-Active Suspension , 2007 .

[3]  Olivier Sename,et al.  Skyhook and H8 Control of Semi-active Suspensions: Some Practical Aspects , 2003 .

[4]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[5]  Csaba Szepesvári,et al.  Fitted Q-iteration in continuous action-space MDPs , 2007, NIPS.

[6]  Sergio M. Savaresi,et al.  A Single-Sensor Control Strategy for Semi-Active Suspensions , 2009, IEEE Transactions on Control Systems Technology.

[7]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[8]  Sergio M. Savaresi,et al.  Approximate linearization via feedback - an overview , 2001, Autom..

[9]  Pierre Geurts,et al.  Tree-Based Batch Mode Reinforcement Learning , 2005, J. Mach. Learn. Res..

[10]  R. A. Williams.  Automotive active suspensions Part 1: Basic principles , 1997 .

[11]  Sergio M. Savaresi,et al.  Acceleration-Driven-Damper (ADD): An Optimal Control Algorithm For Comfort-Oriented Semiactive Suspensions , 2005 .

[12]  Martin A. Riedmiller.  Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method , 2005, ECML.

[13]  Michael Valášek,et al.  Development of semi-active road-friendly truck suspensions , 1998 .

[14]  Sergio M. Savaresi,et al.  The Concept of Performance-Oriented Yaw-Control Systems: Vehicle Model and Analysis , 2002 .

[15]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[16]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[17]  D. Hrovat,et al.  Survey of Advanced Suspension Developments and Related Optimal Control Applications, , 1997, Autom..