Temporal Convolutions for Multi-Step Quadrotor Motion Prediction

Model-based control methods for robotic systems such as quadrotors, autonomous driving vehicles and flexible manipulators require motion models that generate accurate predictions of complex nonlinear system dynamics over long periods of time. Temporal Convolutional Networks (TCNs) can be adapted to this challenge by formulating multi-step prediction as a sequence-to-sequence modeling problem. We present End2End-TCN: a fully convolutional architecture that integrates future control inputs to compute multi-step motion predictions in one forward pass. We demonstrate the approach with a thorough analysis of TCN performance for the quadrotor modeling task, which includes an investigation of scaling effects and ablation studies. Ultimately, End2End-TCN provides 55% error reduction over the state of the art in multi-step prediction on an aggressive indoor quadrotor flight dataset. The model yields accurate predictions across 90 timestep horizons over a 900 ms interval.

[1]  Gregory D. Hager,et al.  Temporal Convolutional Networks for Action Segmentation and Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Nima Mohajerin,et al.  Modeling Dynamic Systems for Multi-Step Prediction with Recurrent Neural Networks , 2017 .

[3]  Albert Y. Zomaya,et al.  Temporal Convolutional Networks for the Advance Prediction of ENSO , 2020, Scientific Reports.

[4]  Valentin Flunkert,et al.  DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks , 2017, International Journal of Forecasting.

[5]  Zhijie Liu,et al.  Dynamic modeling and vibration control for a nonlinear 3‐dimensional flexible manipulator , 2018 .

[6]  Anthony Tzes,et al.  Model predictive quadrotor control: attitude, altitude and position experimental studies , 2012 .

[7]  Roland Siegwart,et al.  A framework for maximum likelihood parameter identification applied on MAVs , 2018, J. Field Robotics.

[8]  Vladlen Koltun,et al.  An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling , 2018, ArXiv.

[9]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[10]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[11]  Davide Scaramuzza,et al.  NeuroBEM: Hybrid Aerodynamic Quadrotor Model , 2021, Robotics: Science and Systems.

[12]  Pieter Abbeel,et al.  Deep learning helicopter dynamics models , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Robert Mahony,et al.  Modelling and control of a quad-rotor robot , 2006 .

[14]  Giuseppe Loianno,et al.  Online Estimation of Geometric and Inertia Parameters for Multirotor Aerial Vehicles , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[15]  Steven Lake Waslander,et al.  Multistep Prediction of Dynamic Systems With Recurrent Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Claire J. Tomlin,et al.  Learning quadrotor dynamics using neural network for flight control , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[17]  Barack Wamkaya Wanjawa,et al.  ANN Model to Predict Stock Prices at Stock Exchange Markets , 2014, ArXiv.

[18]  Mohamed Abdel-Nasser,et al.  Accurate photovoltaic power forecasting models using deep LSTM-RNN , 2017, Neural Computing and Applications.

[19]  Huajun Chen,et al.  Knowledge-Driven Stock Trend Prediction and Explanation via Temporal Convolutional Network , 2019, WWW.

[20]  Sorin Grigorescu,et al.  A Survey of Deep Learning Techniques for Autonomous Driving , 2020, J. Field Robotics.

[21]  Sander Bohte,et al.  Conditional Time Series Forecasting with Convolutional Neural Networks , 2017, 1703.04691.

[22]  Goetz Bramesfeld,et al.  Performance Prediction of Multirotor Vehicles Using A Higher Order Potential Flow Method , 2018 .

[23]  Francesco Borrelli,et al.  Kinematic and dynamic vehicle models for autonomous driving control design , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[24]  Stephan Weiss,et al.  Filter-Based Online System-Parameter Estimation for Multicopter UAVs , 2021, Robotics: Science and Systems.

[25]  Roland Siegwart,et al.  Full control of a quadrotor , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Peter I. Corke,et al.  Multirotor Aerial Vehicles: Modeling, Estimation, and Control of Quadrotor , 2012, IEEE Robotics & Automation Magazine.

[27]  Ran Li,et al.  Deep Learning for Household Load Forecasting—A Novel Pooling Deep RNN , 2018, IEEE Transactions on Smart Grid.

[28]  Claire J. Tomlin,et al.  Learning-based model predictive control on a quadrotor: Onboard implementation and experimental results , 2012, 2012 IEEE International Conference on Robotics and Automation.

[29]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[30]  Steven L. Waslander,et al.  The Stanford testbed of autonomous rotorcraft for multi agent control (STARMAC) , 2004, The 23rd Digital Avionics Systems Conference (IEEE Cat. No.04CH37576).

[31]  Angela P. Schoellig,et al.  Flatness-Based Model Predictive Control for Quadrotor Trajectory Tracking , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[32]  Sen Wang,et al.  DeepVO: Towards end-to-end visual odometry with deep Recurrent Convolutional Neural Networks , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[33]  Zizhuo Wang,et al.  Probabilistic Forecasting with Temporal Convolutional Neural Network , 2019, Neurocomputing.

[34]  Yoshua Bengio,et al.  Architectural Complexity Measures of Recurrent Neural Networks , 2016, NIPS.

[35]  Vladlen Koltun,et al.  Deep Drone Acrobatics , 2020, Robotics: Science and Systems.

[36]  Claire J. Tomlin,et al.  Precision flight control for a multi-vehicle quadrotor helicopter testbed , 2011 .

[37]  Ruonan Rao,et al.  Time-Weighted LSTM Model with Redefined Labeling for Stock Trend Prediction , 2017, 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI).

[38]  Alec Radford,et al.  Scaling Laws for Neural Language Models , 2020, ArXiv.