Deep-Learning-Based Neural Network Training for State Estimation Enhancement: Application to Attitude Estimation

Achieving precise state estimation is needed for the unmanned aerial vehicle to perform a successful flight with a high degree of stability. Nonetheless, obtaining accurate state estimation is considered challenging due to the inaccuracies associated with the measurements of the onboard commercial-off-the-shelf inertial measurement unit. The immense vibration of the vehicle’s rotors makes these measurements suffer from issues like large drifts, biases, and immense unpredictable noise sequences. These issues cannot be significantly tackled using classical estimators, and an accurate sensor fusion technique needs to be developed. In this paper, a deep learning (DL) framework is developed to enhance the performance of the state estimator. A deep neural network (DNN) is trained using a deep-learning-based technique to identify the associated measurement noise models and filter them out. The dropout technique is adopted for training DNN to avoid overfitting and reduce the complexity of nets computations. Compared to the classical estimation results, the proposed DL technique demonstrates capabilities in identifying the measurement’s noise characteristics. As an example, an enhancement in estimating the attitude states at near hover is proven using this approach. Furthermore, an actual hover flight was performed to validate the proposed estimation enhancement method.

[1]  Raghad Al-Husari,et al.  Precision landing using an adaptive fuzzy multi-sensor data fusion architecture , 2018, Appl. Soft Comput..

[2]  Bo Hu,et al.  A deep learning based handover mechanism for UAV networks , 2017, 2017 20th International Symposium on Wireless Personal Multimedia Communications (WPMC).

[3]  Naif Alajlan,et al.  Deep Learning Approach for Car Detection in UAV Imagery , 2017, Remote. Sens..

[4]  Puneet Singla,et al.  Robust Attitude Estimation from Uncertain Observations of Inertial Sensors Using Covariance Inflated Multiplicative Extended Kalman Filter , 2018, IEEE Transactions on Instrumentation and Measurement.

[5]  S. Yoo,et al.  A transfer learning approach to parking lot classification in aerial imagery , 2017, 2017 New York Scientific Data Summit (NYSDS).

[6]  F. Caballero,et al.  Bioinspired vision-only UAV attitude rate estimation using machine learning , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[7]  H. Jin Kim,et al.  Adaptive Range Estimation in Perspective Vision System Using Neural Networks , 2018, IEEE/ASME Transactions on Mechatronics.

[8]  Inna Sharf,et al.  Attitude estimation for normal flight and collision recovery of a quadrotor UAV , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[9]  Chih-Lung Lin,et al.  Position Estimation and Smooth Tracking With a Fuzzy-Logic-Based Adaptive Strong Tracking Kalman Filter for Capacitive Touch Panels , 2015, IEEE Transactions on Industrial Electronics.

[10]  Bernardino Castillo-Toledo,et al.  Power line inspection via an unmanned aerial system based on the quadrotor helicopter , 2014, MELECON 2014 - 2014 17th IEEE Mediterranean Electrotechnical Conference.

[11]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[12]  Erdinç Altug,et al.  EKF Based Attitude Estimation and Stabilization of a Quadrotor UAV Using Vanishing Points in Catadioptric Images , 2011, J. Intell. Robotic Syst..

[13]  Maoguo Gong,et al.  Automatic Tobacco Plant Detection in UAV Images via Deep Neural Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[14]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[15]  Rita Cunha,et al.  Landing of a Quadrotor on a Moving Target Using Dynamic Image-Based Visual Servo Control , 2016, IEEE Transactions on Robotics.

[16]  Mohammad K. Al-Sharman,et al.  Attitude and Flapping Angles Estimation for a Small-Scale Flybarless Helicopter Using a Kalman Filter , 2015, IEEE Sensors Journal.

[17]  Mohammad R. Jahanshahi,et al.  NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion , 2018, IEEE Transactions on Industrial Electronics.

[18]  Juan Manuel Ramírez-Cortés,et al.  Attitude estimation using a Neuro-Fuzzy tuning based adaptive Kalman filter , 2015, J. Intell. Fuzzy Syst..

[19]  Steven Reece,et al.  An introduction to Gaussian processes for the Kalman filter expert , 2010, 2010 13th International Conference on Information Fusion.

[20]  Pascual Campoy Cervera,et al.  A Review of Deep Learning Methods and Applications for Unmanned Aerial Vehicles , 2017, J. Sensors.

[21]  Baochang Zhang,et al.  Realtime Human-UAV Interaction Using Deep Learning , 2017, CCBR.

[22]  Yong Wang,et al.  Planning and Tracking in Image Space for Image-Based Visual Servoing of a Quadrotor , 2018, IEEE Transactions on Industrial Electronics.

[23]  Yangquan Chen,et al.  A comparative evaluation of low-cost IMUs for unmanned autonomous systems , 2010, 2010 IEEE Conference on Multisensor Fusion and Integration.

[24]  Peter I. Corke,et al.  Multirotor Aerial Vehicles: Modeling, Estimation, and Control of Quadrotor , 2012, IEEE Robotics & Automation Magazine.

[25]  James E. Gentle,et al.  Matrix Algebra: Theory, Computations, and Applications in Statistics , 2007 .

[26]  Uri Shalit,et al.  Deep Kalman Filters , 2015, ArXiv.

[27]  K. Cheon,et al.  On Replacing PID Controller with Deep Learning Controller for DC Motor System , 2015 .

[28]  Mohammad K. Al-Sharman,et al.  Intelligent attitude and flapping angles estimation of flybarless helicopters under near-hover conditions , 2018, J. Frankl. Inst..

[29]  Wolfram Burgard,et al.  A Fully Autonomous Indoor Quadrotor , 2012, IEEE Transactions on Robotics.

[30]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[31]  Jing Lei,et al.  Deep Learning-Based Inversion Method for Imaging Problems in Electrical Capacitance Tomography , 2018, IEEE Transactions on Instrumentation and Measurement.

[32]  Jiangtao Wen,et al.  Intelligent Bearing Fault Diagnosis Method Combining Compressed Data Acquisition and Deep Learning , 2018, IEEE Transactions on Instrumentation and Measurement.

[33]  Mohammad K. Al-Sharman,et al.  State Estimation for a Small Scale Flybar-less Helicopter , 2014 .

[34]  Raymond Kristiansen,et al.  Quadrotor attitude estimation using adaptive fading multiplicative EKF , 2017, 2017 American Control Conference (ACC).

[35]  Lubin Chang,et al.  Initial Alignment by Attitude Estimation for Strapdown Inertial Navigation Systems , 2015, IEEE Transactions on Instrumentation and Measurement.

[36]  Jianbin Qiu,et al.  State Estimation in Nonlinear System Using Sequential Evolutionary Filter , 2016, IEEE Transactions on Industrial Electronics.

[37]  Farid Melgani,et al.  A Deep Learning Approach to UAV Image Multilabeling , 2017, IEEE Geoscience and Remote Sensing Letters.

[38]  Jie Cao,et al.  Application of deep learning and unmanned aerial vehicle technology in traffic flow monitoring , 2017, 2017 International Conference on Machine Learning and Cybernetics (ICMLC).

[39]  Vijay Kumar,et al.  Estimation, Control, and Planning for Aggressive Flight With a Small Quadrotor With a Single Camera and IMU , 2017, IEEE Robotics and Automation Letters.

[40]  Mamoun F. Abdel-Hafez,et al.  Enhanced, Delay Dependent, Intelligent Fusion for INS/GPS Navigation System , 2014, IEEE Sensors Journal.

[41]  Giuseppe Loianno,et al.  Aggressive Flight With Quadrotors for Perching on Inclined Surfaces , 2016 .

[42]  Patricia Scully,et al.  Deep Neural Networks for Learning Spatio-Temporal Features From Tomography Sensors , 2018, IEEE Transactions on Industrial Electronics.

[43]  Xiaoli Meng,et al.  Quaternion-Based Kalman Filter With Vector Selection for Accurate Orientation Tracking , 2012, IEEE Transactions on Instrumentation and Measurement.

[44]  Maryam Rahnemoonfar,et al.  Real-time scene understanding for UAV imagery based on deep convolutional neural networks , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[45]  Tara N. Sainath,et al.  Improving deep neural networks for LVCSR using rectified linear units and dropout , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[46]  Robert Jenssen,et al.  Automatic autonomous vision-based power line inspection: A review of current status and the potential role of deep learning , 2018, International Journal of Electrical Power & Energy Systems.

[47]  Yingwei Zhao,et al.  Cubature + Extended Hybrid Kalman Filtering Method and Its Application in PPP/IMU Tightly Coupled Navigation Systems , 2015, IEEE Sensors Journal.

[48]  Jie Ma,et al.  Deep-learning-based moving target detection for unmanned air vehicles , 2017, 2017 36th Chinese Control Conference (CCC).

[49]  David Hyunchul Shim,et al.  Perception, Guidance, and Navigation for Indoor Autonomous Drone Racing Using Deep Learning , 2018, IEEE Robotics and Automation Letters.

[50]  Mamoun F. Abdel-Hafez,et al.  Non-Linear Autoregressive Delay-Dependent INS/GPS Navigation System Using Neural Networks , 2017, IEEE Sensors Journal.

[51]  Panos Marantos,et al.  UAV State Estimation Using Adaptive Complementary Filters , 2016, IEEE Transactions on Control Systems Technology.

[52]  Ricardo Sanz,et al.  Improving attitude estimation using inertial sensors for quadrotor control systems , 2014, 2014 International Conference on Unmanned Aircraft Systems (ICUAS).