Static and Dynamic Algorithms for Terrain Classification in UAV Aerial Imagery

The ability to precisely classify different types of terrain is extremely important for Unmanned Aerial Vehicles (UAVs). There are multiple situations in which terrain classification is fundamental for achieving a UAV’s mission success, such as emergency landing, aerial mapping, decision making, and cooperation between UAVs in autonomous navigation. Previous research works describe different terrain classification approaches mainly using static features from RGB images taken onboard UAVs. In these works, the terrain is classified from each image taken as a whole, not divided into blocks; this approach has an obvious drawback when applied to images with multiple terrain types. This paper proposes a robust computer vision system to classify terrain types using three main algorithms, which extract features from UAV’s downwash effect: Static texturesGray-Level Co-Occurrence Matrix (GLCM), Gray-Level Run Length Matrix (GLRLM) and Dynamic texturesOptical Flow method. This system has been fully implemented using the OpenCV library, and the GLCM algorithm has also been partially specified in a Hardware Description Language (VHDL) and implemented in a Field Programmable Gate Array (FPGA)-based platform. In addition to these feature extraction algorithms, a neural network was designed with the aim of classifying the terrain into one of four classes. Lastly, in order to store and access all the classified terrain information, a dynamic map, with this information was generated. The system was validated using videos acquired onboard a UAV with an RGB camera.

[1]  Na Liu,et al.  On-Board Ortho-Rectification for Images Based on an FPGA , 2017, Remote. Sens..

[2]  Tobias Tiemerding,et al.  Comparison of different design methodologies of hardware-based image processing for automation in microrobotics , 2013, 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[3]  Liang Kim Meng,et al.  Implementing image processing algorithms using ‘Hardware in the loop’ approach for Xilinx FPGA , 2008, 2008 International Conference on Electronic Design.

[4]  Wai Yeung Yan,et al.  Urban land cover classification using airborne LiDAR data: A review , 2015 .

[5]  Bin Luo,et al.  Hierarchical Terrain Classification Based on Multilayer Bayesian Network and Conditional Random Field , 2017, Remote. Sens..

[6]  Rita Almeida Ribeiro,et al.  Land Cover Classification from Multispectral Data Using Computational Intelligence Tools: A Comparative Study , 2017, Inf..

[7]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[8]  Zahid Ullah,et al.  Fast Pattern Recognition Through an LBP Driven CAM on FPGA , 2018, IEEE Access.

[9]  Ahmad Shawahna,et al.  FPGA-Based Accelerators of Deep Learning Networks for Learning and Classification: A Review , 2019, IEEE Access.

[10]  Ana Beatriz de Tróia Salvado.  Aerial Semantic Mapping for Precision Agriculture using Multispectral Imagery , 2018 .

[11]  Michal Strzelecki,et al.  Texture Analysis Methods - A Review , 1998 .

[12]  Ricardo Mendonça,et al.  UAV Downwash-Based Terrain Classification Using Wiener-Khinchin and EMD Filters , 2019, DoCEIS.

[13]  Marko Beko,et al.  Algorithms for Estimating the Location of Remote Nodes Using Smartphones , 2019, IEEE Access.

[14]  Jianhua Gong,et al.  UAV Remote Sensing for Urban Vegetation Mapping Using Random Forest and Texture Analysis , 2015, Remote. Sens..

[15]  R. Harinarayan,et al.  Feature extraction of Digital Aerial Images by FPGA based implementation of edge detection algorithms , 2011, 2011 International Conference on Emerging Trends in Electrical and Computer Technology.

[16]  Jose Barata,et al.  Water detection from downwash-induced optical flow for a multirotor UAV , 2015, OCEANS 2015 - MTS/IEEE Washington.

[17]  Trong-Thuc Hoang,et al.  An FPGA-Based Hardware Accelerator for Energy-Efficient Bitmap Index Creation , 2018, IEEE Access.

[18]  Wayne Luk,et al.  A Real-Time Tree Crown Detection Approach for Large-Scale Remote Sensing Images on FPGAs , 2019, Remote. Sens..

[19]  L. Wallace,et al.  Assessment of Forest Structure Using Two UAV Techniques: A Comparison of Airborne Laser Scanning and Structure from Motion (SfM) Point Clouds , 2016 .

[20]  J. Andrew Bagnell,et al.  Terrain Classification from Aerial Data to Support Ground Vehicle Navigation , 2006 .

[21]  Marko Beko,et al.  Localization of Static Remote Devices Using Smartphones , 2018, 2018 IEEE 87th Vehicular Technology Conference (VTC Spring).

[22]  Prince Waqas Khan,et al.  UAV’s Agricultural Image Segmentation Predicated by Clifford Geometric Algebra , 2019, IEEE Access.

[23]  Jürgen Schmidhuber,et al.  A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots , 2016, IEEE Robotics and Automation Letters.

[24]  Matti Pietikäinen,et al.  Computer Vision Using Local Binary Patterns , 2011, Computational Imaging and Vision.

[25]  P. Ćwiąkała,et al.  Comparison of low-altitude UAV photogrammetry with terrestrial laser scanning as data-source methods for terrain covered in low vegetation , 2017 .

[26]  José Manuel Fonseca,et al.  Terrain Classification Using W-K Filter and 3D Navigation with Static Collision Avoidance , 2019, IntelliSys.

[27]  Lentin Joseph.  Mastering ROS for robotics programming : design, build, and simulate complex robots using robot operating system and master its out-of-the-box functionalities , 2015 .

[28]  Chih-Wei Lin,et al.  Fourier Dense Network to Conduct Plant Classification Using UAV-Based Optical Images , 2019, IEEE Access.

[29]  Guoqing Zhou,et al.  On-Board Detection and Matching of Feature Points , 2017, Remote. Sens..

[30]  Fatemeh Ebadi,et al.  Road Terrain detection and Classification algorithm based on the Color Feature extraction , 2017, 2017 Artificial Intelligence and Robotics (IRANOPEN).

[31]  Mary M. Galloway,et al.  Texture analysis using gray level run lengths , 1974 .

[32]  Jin Zhang,et al.  An overview and comparison of machine-learning techniques for classification purposes in digital soil mapping , 2016 .

[33]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[34]  R. D. Daruwala,et al.  Design of Sobel operator based image edge detection algorithm on FPGA , 2014, 2014 International Conference on Communication and Signal Processing.

[35]  Weiwei Sun,et al.  Spatial-Spectral Squeeze-and-Excitation Residual Network for Hyperspectral Image Classification , 2019, Remote. Sens..

[36]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[37]  José Manuel Fonseca,et al.  UAV Downwash Dynamic Texture Features for Terrain Classification on Autonomous Navigation , 2018, 2018 Federated Conference on Computer Science and Information Systems (FedCSIS).

[38]  Xiang Zhou,et al.  On-Board Georeferencing Using FPGA-Based Optimized Second-Order Polynomial Equation , 2019, Remote. Sens..

[39]  Anna Linderhed,et al.  Image Empirical Mode Decomposition: a New Tool for Image Processing , 2009, Adv. Data Sci. Adapt. Anal..

[40]  Andreas Zell,et al.  Grid-based visual terrain classification for outdoor robots using local features , 2011, 2011 IEEE Symposium on Computational Intelligence in Vehicles and Transportation Systems (CIVTS) Proceedings.

[41]  Yasmina Bestaoui Sebbane.  Intelligent Autonomy of UAVs: Advanced Missions and Future Use , 2018 .

[42]  José Manuel Fonseca,et al.  Use of Particle Swarm Optimization in Terrain Classification based on UAV Downwash , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[43]  Michael F Spigelmire,et al.  Unmanned Aircraft Systems and the Next War , 2013 .

[44]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[45]  Anil Vohra,et al.  A novel real-time resource efficient implementation of Sobel operator-based edge detection on FPGA , 2014 .