A Scalable Model-Based Learning Algorithm With Application to UAVs

Unmanned aerial vehicles (UAVs) play essential roles in many areas including search and rescue, monitoring, and exploration. As such, fast and accurate trajectory tracking is crucial for UAVs especially in emergency situations or cluttered environment. This letter proposes a novel learning algorithm that improves UAVs’ tracking performance through learning without human intervention. This learning algorithm, while possessing self-learning capacity, falls into the model-based learning methodology and therefore inherits the advantages of control techniques. This algorithm is particularly pursued and developed for scenarios that the reference trajectory is either too aggressive to follow or not compatible with system dynamics. Numerical study is conducted to validate the effectiveness and efficiency of the proposed learning algorithm and demonstrate the enhanced tracking and learning performance.

[1]  J. Freudenberg,et al.  Right half plane poles and zeros and design tradeoffs in feedback systems , 1985 .

[2]  Kyunghyun Cho,et al.  Query-Efficient Imitation Learning for End-to-End Simulated Driving , 2017, AAAI.

[3]  C. Mohtadi.  Bode's integral theorem for discrete-time systems , 1990 .

[4]  Wenlong Zhang,et al.  A disturbance observer approach with online Q-filter tuning for position control of quadcopters , 2017, 2017 American Control Conference (ACC).

[5]  Masayoshi Tomizuka,et al.  Zero Phase Error Tracking Algorithm for Digital Control , 1987 .

[6]  S. Wich,et al.  Dawn of Drone Ecology: Low-Cost Autonomous Aerial Vehicles for Conservation , 2012 .

[7]  Vijay Kumar,et al.  Trajectory Generation and Control for Precise Aggressive Maneuvers with Quadrotors , 2010, ISER.

[8]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[9]  Andrew Zulu,et al.  A Review of Control Algorithms for Autonomous Quadrotors , 2014, ArXiv.

[10]  Heikki Saari,et al.  Processing and Assessment of Spectrometric, Stereoscopic Imagery Collected Using a Lightweight UAV Spectral Camera for Precision Agriculture , 2013, Remote. Sens..

[11]  Konstantinos Kanistras,et al.  A survey of unmanned aerial vehicles (UAVs) for traffic monitoring , 2013, 2013 International Conference on Unmanned Aircraft Systems (ICUAS).

[12]  Cong Wang,et al.  Robust Iterative Learning Control for Vibration Suppression of Industrial Robot Manipulators , 2018 .

[13]  Sebastian Magierowski,et al.  Vehicle Routing Problems for Drone Delivery , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[14]  Vijay Kumar,et al.  Minimum snap trajectory generation and control for quadrotors , 2011, 2011 IEEE International Conference on Robotics and Automation.

[15]  Lydia Tapia,et al.  Automated aerial suspended cargo delivery through reinforcement learning , 2017, Artif. Intell..

[16]  Sergey Levine,et al.  Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Steven Lake Waslander,et al.  Multi-agent quadrotor testbed control design: integral sliding mode vs. reinforcement learning , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  K. Mosalam,et al.  SIMULATION-BASED DATA-DRIVEN DAMAGE DETECTION FOR HIGHWAY BRIDGE SYSTEMS , 2017 .

[19]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[20]  Holger Voos,et al.  Controller design for quadrotor UAVs using reinforcement learning , 2010, 2010 IEEE International Conference on Control Applications.

[21]  Vijay Kumar,et al.  Cooperative Control of UAVs for Search and Coverage , 2006 .

[22]  Sergey Levine,et al.  Learning deep control policies for autonomous aerial vehicles with MPC-guided policy search , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Seungwon Choi,et al.  Inverse reinforcement learning control for trajectory tracking of a multirotor UAV , 2017 .

[24]  Cong Wang,et al.  Design of arbitrary-order robust iterative learning control based on robust control theory , 2017 .

[25]  H. Jin Kim,et al.  Autonomous Flight of the Rotorcraft-Based UAV Using RISE Feedback and NN Feedforward Terms , 2012, IEEE Transactions on Control Systems Technology.

[26]  Andreas Rauh,et al.  Feedforward control design for a four-rotor UAV using direct and indirect methods , 2012, 2012 17th International Conference on Methods & Models in Automation & Robotics (MMAR).

[27]  Alexey Dosovitskiy,et al.  End-to-End Driving Via Conditional Imitation Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Claire J. Tomlin,et al.  Quadrotor Helicopter Trajectory Tracking Control , 2008 .

[29]  Taeyoung Lee,et al.  Geometric tracking control of the attitude dynamics of a rigid body on SO(3) , 2010, Proceedings of the 2011 American Control Conference.

[30]  Martial Hebert,et al.  Learning monocular reactive UAV control in cluttered natural environments , 2012, 2013 IEEE International Conference on Robotics and Automation.

[31]  Minghui Zheng,et al.  $H_{\infty}$ Based Disturbance Observer Design for Non-minimum Phase Systems with Application to UAV Attitude Control , 2018, 2018 Annual American Control Conference (ACC).

[32]  Markus Hehn,et al.  An iterative learning scheme for high performance, periodic quadrocopter trajectories , 2013, 2013 European Control Conference (ECC).

[33]  Mohamed Medhat Gaber,et al.  Imitation Learning , 2017, ACM Comput. Surv..

[34]  Raffaello D'Andrea,et al.  Optimization-based iterative learning for precise quadrocopter trajectory tracking , 2012, Autonomous Robots.

[35]  Raffaello D'Andrea,et al.  A frequency domain iterative feed-forward learning scheme for high performance periodic quadrocopter maneuvers , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.