Superpixel-Based Convolutional Neural Network for Georeferencing the Drone Images

Information extracted from aerial photographs has been used for many practical applications such as urban planning, forest management, disaster relief, and climate modeling. In many cases, labeling of information in the photo is still performed by human experts, making the process slow, costly, and error-prone. This article shows how a convolutional neural network can be used to determine the location of ground control points (GCPs) in aerial photos, which significantly reduces the amount of human labor in identifying GCP locations. Two convolutional neural network (CNN) methods, sliding-window CNN with superpixel-level majority voting, and superpixel-based CNN are evaluated and analyzed. The results of the classification and segmentation show that both of these methods can quickly extract the locations of objects from aerial photographs, but only superpixel-based CNN can unambiguously locate the GCPs.

[1]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[3]  Geoffrey E. Hinton,et al.  Learning to Detect Roads in High-Resolution Aerial Images , 2010, ECCV.

[4]  Luisa Verdoliva,et al.  Land Use Classification in Remote Sensing Images by Convolutional Neural Networks , 2015, ArXiv.

[5]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[6]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Convolutional Neural Network for Satellite Imagery , 2018 .

[8]  Marta C. González,et al.  Using Convolutional Networks and Satellite Imagery to Identify Patterns in Urban Environments at a Large Scale , 2017, KDD.

[9]  Geoffrey E. Hinton,et al.  Learning to Label Aerial Images from Noisy Data , 2012, ICML.

[10]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[11]  Baojun Xu,et al.  Profiles of phenolics, carotenoids and antioxidative capacities of thermal processed white, yellow, orange and purple sweet potatoes grown in Guilin, China , 2015 .

[12]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Quoc V. Le,et al.  High-accuracy 3D sensing for mobile manipulation: Improving object detection and door opening , 2009, 2009 IEEE International Conference on Robotics and Automation.

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  Yuting Zhang,et al.  Improving object detection with deep convolutional networks via Bayesian optimization and structured prediction , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Thomas Mauthner,et al.  Semantic Classification in Aerial Imagery by Integrating Appearance and Height Information , 2009, ACCV.

[18]  Brendt Wohlberg,et al.  Physically realistic training data construction for data-driven full-waveform inversion and traveltime tomography , 2020 .

[19]  Gerard T. Schuster,et al.  Separation of multi‐mode surface waves by supervised machine learning methods , 2020, Geophysical Prospecting.

[20]  Amy Loutfi,et al.  Classification and Segmentation of Satellite Orthoimagery Using Convolutional Neural Networks , 2016, Remote. Sens..

[21]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Kai Lu,et al.  Auto-windowed Super-virtual Interferometry via Machine Learning: A Strategy of First-arrival Traveltime Automatic Picking for Noisy Seismic Data , 2018 .