Online discovery of AUV control policies to overcome thruster failures

We investigate methods to improve fault-tolerance of Autonomous Underwater Vehicles (AUVs) to increase their reliability and persistent autonomy. We propose a learning-based approach that is able to discover new control policies to overcome thruster failures as they happen. The proposed approach is a model-based direct policy search that learns on an on-board simulated model of the AUV. The model is adapted to a new condition when a fault is detected and isolated. Since the approach generates an optimal trajectory, the learned fault-tolerant policy is able to navigate the AUV towards a specified target with minimum cost. Finally, the learned policy is executed on the real robot in a closed-loop using the state feedback of the AUV. Unlike most existing methods which rely on the redundancy of thrusters, our approach is also applicable when the AUV becomes under-actuated in the presence of a fault. To validate the feasibility and efficiency of the presented approach, we evaluate it with three learning algorithms and three policy representations with increasing complexity. The proposed method is tested on a real AUV, Girona500.

[1]  R. Bono,et al.  EXPERIENCES ON ACTUATOR FAULT DETECTION , DIAGNOSIS AND ACCOMMODATION FOR ROVS , .

[2]  Jon Rigelsford,et al.  Underwater Robots: Motion and Force Control of Vehicle-Manipulator Systems , 2004 .

[3]  George Konidaris,et al.  Value Function Approximation in Reinforcement Learning Using the Fourier Basis , 2011, AAAI.

[4]  Nilanjan Sarkar,et al.  Fault-tolerant control of an autonomous underwater vehicle under thruster redundancy , 2001, Robotics Auton. Syst..

[5]  Matteo Leonetti,et al.  On-line learning to recover from thruster failures on Autonomous Underwater Vehicles , 2013, 2013 OCEANS - San Diego.

[6]  Gianni Di Pillo,et al.  A New Version of the Price's Algorithm for Global Optimization , 1997, J. Glob. Optim..

[7]  Thor I. Fossen,et al.  Guidance and control of ocean vehicles , 1994 .

[8]  Darwin G. Caldwell,et al.  On-line identification of autonomous underwater vehicles through global derivative-free optimization , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[10]  Sergio Grammatico,et al.  Geometric control for autonomous underwater vehicles: Overcoming a thruster failure , 2010, 49th IEEE Conference on Decision and Control (CDC).

[11]  B. Suman,et al.  A survey of simulated annealing as a tool for single and multiobjective optimization , 2006, J. Oper. Res. Soc..

[12]  Angelo Alessandri,et al.  A model-based approach to fault diagnosis in unmanned underwater vehicles , 1998, IEEE Oceanic Engineering Society. OCEANS'98. Conference Proceedings (Cat. No.98CH36259).

[13]  Tomoyuki Hiroyasu,et al.  Simulated annealing with advanced adaptive neighborhood , 2002 .

[14]  Konstantinos Kyriakopoulos,et al.  Persistent Autonomy: the Challenges of the PANDORA Project , 2012 .

[15]  Marc Carreras,et al.  Girona 500 AUV: From Survey to Intervention , 2012, IEEE/ASME Transactions on Mechatronics.

[16]  David M. Lane,et al.  Fault diagnosis on autonomous robotic vehicles with RECOVERY: an integrated heterogeneous-knowledge approach , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[17]  Nilanjan Sarkar,et al.  Fault tolerant control of an autonomous underwater vehicle under thruster redundancy: simulations and experiments , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[18]  Matteo Leonetti,et al.  Combining Local and Global Direct Derivative-Free Optimization for Reinforcement Learning , 2012 .

[19]  Mae L. Seto,et al.  An agent to optimally re-distribute control in an underactuated AUV , 2011, Int. J. Intell. Def. Support Syst..

[20]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[21]  Gianluca Antonelli A Survey of Fault Detection/Tolerance Strategies for AUVs and ROVs , 2003 .

[22]  Meyer Nahon,et al.  Fault-tolerant control of an autonomous underwater vehicle , 1998, IEEE Oceanic Engineering Society. OCEANS'98. Conference Proceedings (Cat. No.98CH36259).

[23]  Hayato Kondo,et al.  On fault-tolerant control of a hovering AUV with four horizontal and two vertical thrusters , 2010, OCEANS'10 IEEE SYDNEY.

[24]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..