Towards vegetation species discrimination by using data-driven descriptors

Abstract—In this paper, we analyse the use of Convolutional Neural Networks (CNNs or ConvNets) to discriminate vegetation species with few labelled samples. To the best of our knowledge, this is the first work dedicated to the investigation of the use of deep features in such task. The experimental evaluation demonstrate that deep features significantly outperform wellknown feature extraction techniques. The achieved results also show that it is possible to learn and classify vegetation patterns even with few samples. This makes the use of our approach feasible for real-world mapping applications, where it is often difficult to obtain large training sets.

[1]  Sylvie Philipp-Foliguet,et al.  Efficient and Effective Hierarchical Feature Propagation , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Jefersson Alex dos Santos,et al.  Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Jurandy Almeida,et al.  Phenological visual rhythms: Compact representations for fine-grained plant species identification , 2016, Pattern Recognit. Lett..

[4]  Victor S. Lempitsky,et al.  Neural Codes for Image Retrieval , 2014, ECCV.

[5]  Ramin Zabih,et al.  Comparing images using color coherence vectors , 1997, MULTIMEDIA '96.

[6]  Matthieu Cord,et al.  Pooling in image representation: The visual codeword point of view , 2013, Comput. Vis. Image Underst..

[7]  Hideki Kobayashi,et al.  Usability of time-lapse digital camera images to detect characteristics of tree phenology in a tropical rainforest , 2016, Ecol. Informatics.

[8]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[9]  Jurandy Almeida,et al.  Using phenological cameras to track the green up in a cerrado savanna and its on-the-ground validation , 2014, Ecol. Informatics.

[10]  Kenlo Nishida Nasahara,et al.  Uncertainties involved in leaf fall phenology detected by digital camera , 2015, Ecol. Informatics.

[11]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  Jurandy Almeida,et al.  Fusion of time series representations for plant recognition in phenology studies , 2016, Pattern Recognit. Lett..

[13]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[14]  Mario A. Nascimento,et al.  A compact and efficient image retrieval approach based on border/interior pixel classification , 2002, CIKM '02.

[15]  Ricardo da Silva Torres,et al.  Comparative study of global color and texture descriptors for web image retrieval , 2012, J. Vis. Commun. Image Represent..

[16]  Cor J. Veenman,et al.  Visual Word Ambiguity , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[18]  J. E. Bendz,et al.  Visually Improved Understanding of Three-Dimensionally Propagating Electromagnetic Fields in Wireless Networks , 2007 .

[19]  M. D. Schwartz.  Phenology: An Integrative Environmental Science , 2003, Tasks for Vegetation Science.

[20]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Jefersson Alex dos Santos,et al.  Evaluating the Potential of Texture and Color Descriptors for Remote Sensing Image Retrieval and Classification , 2010, VISAPP.

[22]  Jurandy Almeida,et al.  Applying machine learning based on multiscale classifiers to detect remote phenology patterns in Cerrado savanna trees , 2014, Ecol. Informatics.

[23]  Shawn D. Newsam,et al.  Geographic Image Retrieval Using Local Invariant Features , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[25]  Shawn D. Newsam,et al.  Comparing SIFT descriptors and gabor texture features for classification of remote sensed imagery , 2008, 2008 15th IEEE International Conference on Image Processing.

[26]  Peter I. Corke,et al.  Fine-grained bird species recognition via hierarchical subset learning , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[27]  Hideki Kobayashi,et al.  Utilization of ground-based digital photography for the evaluation of seasonal changes in the aboveground green biomass and foliage phenology in a grassland ecosystem , 2015, Ecol. Informatics.

[28]  Michael Unser,et al.  Sum and Difference Histograms for Texture Classification , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Tristan Perez,et al.  Fine-Grained Plant Classification Using Convolutional Neural Networks for Feature Extraction , 2014, CLEF.

[30]  Ricardo da Silva Torres,et al.  Visual word spatial arrangement for image retrieval and classification , 2014, Pattern Recognit..

[31]  Bo Tao,et al.  Texture Recognition and Image Retrieval Using Gradient Indexing , 2000, J. Vis. Commun. Image Represent..

[32]  Neucimar J. Leite,et al.  Rotation-Invariant and Scale-Invariant Steerable Pyramid Decomposition for Texture Image Retrieval , 2007 .

[33]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[34]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[35]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  Jurandy Almeida,et al.  Deriving vegetation indices for phenology analysis using genetic programming , 2015, Ecol. Informatics.

[37]  Jefersson Alex dos Santos,et al.  Towards better exploiting convolutional neural networks for remote sensing scene classification , 2016, Pattern Recognit..

[38]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[39]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Robinson Piramuthu,et al.  Fashion apparel detection: The role of deep convolutional neural network and pose-dependent priors , 2014, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[41]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[42]  Jurandy Almeida,et al.  Time series-based classifier fusion for fine-grained plant species recognition , 2016, Pattern Recognit. Lett..

[43]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.