Semantic regions segmentation using a spatio-temporal model from an UAV image sequence with an optimal configuration for data acquisition*

ABSTRACT Unmanned aerial vehicles (UAV) are used to conduct a variety of recognition as well as specific missions such as target tracking and safe landing. The transmitted image sequences to be interpreted at ground station usually face limited requirements of the data transmission. In this paper, on the one hand, we handle a surveillance mission with segmenting an UAV video's content into semantic regions. A multi-class image segmentation approach is proposed considering UAV video-specific characteristics. After post-processing steps on the segmentation results, a support vector-machine classifier is used to recognize regions. A Markov model is introduced to combine the results from the previous frames in order to improve the accuracy. On the other hand, this study also assesses the influences of data reduction techniques on the proposed techniques. The comparisons between the untreated configuration and control conditions under manipulations of the frame rate, spatial resolution, and compression ratio, demonstrate how these data reduction techniques adversely influence the algorithm's performance. The experiments also point out the optimal configuration in order to obtain a trade-off between the target performance and requirements of the data transmission.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Nick G. Kingsbury,et al.  Semantic discriminant mapping for classification and browsing of remote sensing textures and objects , 2005, IEEE International Conference on Image Processing 2005.

[3]  Jon Atli Benediktsson,et al.  Classification and feature extraction for remote sensing images from urban areas based on morphological transformations , 2003, IEEE Trans. Geosci. Remote. Sens..

[4]  Qihao Weng,et al.  A survey of image classification methods and techniques for improving classification performance , 2007 .

[5]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[6]  Dorin Comaniciu,et al.  Mean shift analysis and applications , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[7]  Stephen Gould,et al.  Region-based Segmentation and Object Detection , 2009, NIPS.

[8]  C. Clark,et al.  A glacier respires: Quantifying the distribution and respiration CO2 flux of cryoconite across an entire Arctic supraglacial ecosystem , 2007 .

[9]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  David D. Lewis,et al.  Naive (Bayes) at Forty: The Independence Assumption in Information Retrieval , 1998, ECML.

[11]  Stephen Gould,et al.  Decomposing a scene into geometric and semantically consistent regions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[12]  Antonio Criminisi,et al.  TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation , 2006, ECCV.

[13]  D. Zaum,et al.  ROBUST BUILDING DETECTION IN AERIAL IMAGES , 2005 .

[14]  Emanuele Frontoni,et al.  Hybrid object-based approach for land use/land cover mapping using high spatial resolution imagery , 2011, Int. J. Geogr. Inf. Sci..

[15]  S. Ali Etemad,et al.  Robust Horizon Detection Using Segmentation for UAV Applications , 2012, 2012 Ninth Conference on Computer and Robot Vision.

[16]  M. Eichner,et al.  Autonomous Real-time Vehicle Detection from a Medium-Level UAV , 2009 .

[17]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[18]  H. Maas,et al.  Quantitative Measurement of Soil Erosion from Tls and Uav Data , 2013 .

[19]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[21]  Stefano Soatto,et al.  Class segmentation and object localization with superpixel neighborhoods , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[22]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[23]  P. Rudol,et al.  Human Body Detection and Geolocalization for UAV Search and Rescue Missions Using Color and Thermal Imagery , 2008, 2008 IEEE Aerospace Conference.

[24]  Guoqing Zhou,et al.  DETECTING AND COUNTING VEHICLES FROM SMALL LOW-COST UAV IMAGES , 2009 .

[25]  Gérard Dreyfus,et al.  Single-layer learning revisited: a stepwise procedure for building and training a neural network , 1989, NATO Neurocomputing.

[26]  Primo Zingaretti,et al.  Complete classification of raw LIDAR data and 3D reconstruction of buildings , 2006, Pattern Analysis and Applications.

[27]  Thi-Lan Le,et al.  Selections of Suitable UAV Imagery's Configurations for Regions Classification , 2016, ACIIDS.

[28]  Paul Gerin Fahlstrom,et al.  Introduction to UAV Systems , 2012 .

[29]  J. Serra,et al.  An overview of morphological filtering , 1992 .

[30]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[32]  Ramakant Nevatia,et al.  Building Detection and Description from a Single Intensity Image , 1998, Comput. Vis. Image Underst..

[33]  Uwe Stilla,et al.  Car detection in aerial thermal images by local and global evidence accumulation , 2006, Pattern Recognit. Lett..

[34]  Hui Cheng,et al.  Vehicle and Person Tracking in Aerial Videos , 2007, CLEAR.

[35]  Horst Bischof,et al.  On-line boosting-based car detection from aerial images , 2008 .

[36]  Alexei A. Efros,et al.  Putting Objects in Perspective , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).