Object Detection in Aerial Images Using Feature Fusion Deep Networks

Object detection acts as an essential part in a wide range of measurement systems in traffic management, urban planning, defense, agriculture, and so on. Convolutional Neural Networks-based researches reach a great improvement on detection tasks in natural scene images enjoying from the strong ability of feature representations. However, because of the high density, the small size of objects, and the intricate background, the current methods achieve relatively low precision in aerial images. The intention of this work is to obtain better detection performance in aerial images by designing a novel deep neural network framework called Feature Fusion Deep Networks (FFDN). The novel architecture combines a designed structural learning layer based on a graphical model. As a result, the network not only provides powerful hierarchical representation but also strengthens the spatial relationship between the high-density objects. We demonstrate the great improvement of the proposed FFDN on the UAV123 data set and another novel challenging data set called UAVDT benchmark. The objects which appear with small size, partial occlusion and out of view, as well as in the dark background can be detected accurately.

[1]  Farid Melgani,et al.  Detecting Cars in UAV Images With a Catalog-Based Approach , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[2]  J. Reitberger,et al.  Automatic car detection in high resolution urban scenes based on an adaptive 3D-model , 2003, 2003 2nd GRSS/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas.

[3]  Mohammad Norouzi,et al.  Stacks of convolutional Restricted Boltzmann Machines for shift-invariant feature learning , 2009, CVPR.

[4]  Junwei Han,et al.  Human Motion Tracking by Multiple RGBD Cameras , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Nanning Zheng,et al.  Learning to Detect a Salient Object , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Shiming Xiang,et al.  Vehicle Detection in Satellite Images by Hybrid Deep Convolutional Neural Networks , 2014, IEEE Geoscience and Remote Sensing Letters.

[8]  Azriel Rosenfeld,et al.  Performance analysis of a simple vehicle detection algorithm , 2002, Image Vis. Comput..

[9]  Chi-Man Vong,et al.  Capturing High-Discriminative Fault Features for Electronics-Rich Analog System via Deep Learning , 2017, IEEE Transactions on Industrial Informatics.

[10]  Quoc V. Le,et al.  Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis , 2011, CVPR 2011.

[11]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[12]  Liujuan Cao,et al.  Vehicle Detection in High-Resolution Aerial Images via Sparse Representation and Superpixels , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Junwei Han,et al.  Local deep feature learning framework for 3D shape , 2015, Comput. Graph..

[17]  Junwei Han,et al.  Scene parsing using inference Embedded Deep Networks , 2016, Pattern Recognit..

[18]  Liang Xiao,et al.  CRF based road detection with multi-sensor fusion , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[19]  Sheng Wang Vehicle Detection on Aerial Images by Extracting Corner Features for Rotational Invariant Shape Matching , 2011, 2011 IEEE 11th International Conference on Computer and Information Technology.

[20]  Farid Melgani,et al.  Automatic Car Counting Method for Unmanned Aerial Vehicle Images , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Gellért Máttyus,et al.  Fast Multiclass Vehicle Detection on Aerial Images , 2015, IEEE Geoscience and Remote Sensing Letters.

[22]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[23]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[24]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.

[25]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[26]  Farid Melgani,et al.  Car speed estimation method for UAV images , 2014, 2014 IEEE Geoscience and Remote Sensing Symposium.

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  Larry S. Davis,et al.  Vehicle Detection Using Partial Least Squares , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[31]  Ling Shao,et al.  Learning Computational Models of Video Memorability from fMRI Brain Imaging , 2015, IEEE Transactions on Cybernetics.

[32]  Xuelong Li,et al.  Two-Stage Learning to Predict Human Eye Fixations via SDAEs , 2016, IEEE Transactions on Cybernetics.

[33]  Liujuan Cao,et al.  Vehicle Detection in High-Resolution Aerial Images Based on Fast Sparse Representation Classification and Multiorder Feature , 2016, IEEE Transactions on Intelligent Transportation Systems.

[34]  Bernard Ghanem,et al.  A Benchmark and Simulator for UAV Tracking , 2016, ECCV.

[35]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[36]  Wen Liu,et al.  Vehicle Extraction and Speed Detection from Digital Aerial Images , 2008, IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium.

[37]  Mohan M. Trivedi,et al.  Pushing the “Speed Limit”: High-Accuracy US Traffic Sign Recognition With Convolutional Neural Networks , 2016, IEEE Transactions on Intelligent Vehicles.

[38]  Yuexian Zou,et al.  Multi-Scale Object Detection with Feature Fusion and Region Objectness Network , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[39]  Huanxin Zou,et al.  Toward Fast and Accurate Vehicle Detection in Aerial Images Using Coupled Region-Based Convolutional Neural Networks , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[40]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Rongrong Ji,et al.  Learning High-Level Feature by Deep Belief Networks for 3-D Model Retrieval and Recognition , 2014, IEEE Transactions on Multimedia.

[42]  Qi Tian,et al.  The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking , 2018, ECCV.

[43]  Jian Yang,et al.  Object detection via feature fusion based single network , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[44]  Junwei Han,et al.  Unsupervised 3D Local Feature Learning by Circle Convolutional Restricted Boltzmann Machine. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[45]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Hao Jiang,et al.  Pedestrian Detection Based on Multi-Scale Fusion Features , 2018, 2018 International Conference on Network Infrastructure and Digital Content (IC-NIDC).

[47]  Fei Wang,et al.  Overtaking vehicle detection using a spatio-temporal CRF , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.