Tree extraction from multi-scale UAV images using Mask R-CNN with FPN

ABSTRACT Tree detection and counting have been performed using conventional methods or high costly remote sensing data. In the past few years, deep learning techniques have gained significant progress in the remote sensing area. Namely, convolutional neural networks (CNNs) have been recognized as one of the most successful and widely used deep learning approaches and they have been used for object detection. In this paper, we employed a Mask R-CNN model and feature pyramid network (FPN) for tree extraction from high-resolution RGB unmanned aerial vehicle (UAV) data. The main aim of this paper is to explore the employed method in images with different scales and tree contents. For this purpose, UAV images from two different areas were acquired and three big-scale test images were created for experimental analysis and accuracy assessment. According to the accuracy analyses, despite the scale and the content changes, the proposed model maintains its detection accuracy to a large extent. To our knowledge, this is the first time a Mask R-CNN model with FPN has been used with UAV data for tree extraction.

[1]  Bingliang Hu,et al.  Attention Mask R-CNN for Ship Detection and Segmentation From Remote Sensing Images , 2020, IEEE Access.

[2]  Tristan Perez,et al.  DeepFruits: A Fruit Detection System Using Deep Neural Networks , 2016, Sensors.

[3]  YangQuan Chen,et al.  Comparing U-Net convolutional network with mask R-CNN in the performances of pomegranate tree canopy segmentation , 2018, Asia-Pacific Remote Sensing.

[4]  Yiwen Hu,et al.  Building Extraction Using Mask Scoring R-CNN Network , 2019, CSAE 2019.

[5]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[6]  Vijay Kumar,et al.  Robust Fruit Counting: Combining Deep Learning, Tracking, and Structure from Motion , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Hossein Arefi,et al.  CNN-based estimation of pre- and post-earthquake height models from single optical images for identification of collapsed buildings , 2019, Remote Sensing Letters.

[8]  Brian C. Lovell,et al.  To What Extent Does Downsampling, Compression, and Data Scarcity Impact Renal Image Analysis? , 2019, 2019 Digital Image Computing: Techniques and Applications (DICTA).

[9]  Helmi Zulhaidi Mohd Shafri,et al.  Young and mature oil palm tree detection and counting using convolutional neural network deep learning method , 2019, International Journal of Remote Sensing.

[10]  Jingbo Wei,et al.  Supervised learning with convolutional neural networks for hyperspectral visualization , 2020 .

[11]  Yong Dou,et al.  Ship Detection Using Transfer Learned Single Shot Multi Box Detector , 2017 .

[12]  Raul Queiroz Feitosa,et al.  Assessment of CNN-Based Methods for Individual Tree Detection on Images Captured by RGB Cameras Attached to UAVs , 2019, Sensors.

[13]  Xuehua Zhao,et al.  Multiscale Mask R-CNN–Based Lung Tumor Detection Using PET Imaging , 2019, Molecular imaging.

[14]  Than D. Le,et al.  Efficient Human-Robot Interaction using Deep Learning with Mask R-CNN: Detection, Recognition, Tracking and Segmentation , 2018, 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV).

[15]  Zhiguo Jiang,et al.  Inshore Ship Detection Based on Mask R-CNN , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[16]  Vijay Kumar,et al.  Counting Apples and Oranges With Deep Learning: A Data-Driven Approach , 2017, IEEE Robotics and Automation Letters.

[17]  S. A. Shukor,et al.  A Review on Remote Sensing-based Method for Tree Detection and Delineation , 2019, IOP Conference Series: Materials Science and Engineering.

[18]  P. Hostert,et al.  Brightness gradient-corrected hyperspectral image mosaics for fractional vegetation cover mapping in northern California , 2020, Remote Sensing Letters.

[19]  James Patrick Underwood,et al.  Machine vision assessment of mango orchard flowering , 2018, Comput. Electron. Agric..

[20]  Edward J. Delp,et al.  Counting plants using deep learning , 2017, 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[21]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[22]  Weijia Li,et al.  Deep Learning Based Oil Palm Tree Detection and Counting for High-Resolution Remote Sensing Images , 2016, Remote. Sens..

[23]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[24]  Maggi Kelly,et al.  Identification of Citrus Trees from Unmanned Aerial Vehicle Imagery Using Convolutional Neural Networks , 2018, Drones.

[25]  Yu Qian,et al.  Mask R-CNN for Object Detection in Multitemporal SAR Images , 2019, 2019 10th International Workshop on the Analysis of Multitemporal Remote Sensing Images (MultiTemp).

[26]  Johan Holmgren,et al.  Tree crown segmentation based on a tree crown density model derived from Airborne Laser Scanning , 2019 .

[27]  Sergio Marconi,et al.  Individual Tree-Crown Detection in RGB Imagery Using Semi-Supervised Deep Learning Neural Networks , 2019, Remote. Sens..