A Guided Multi-Scale Categorization of Plant Species in Natural Images

Automatic categorization of plant species in natural images is an important computer vision problem with numerous applications in agriculture and botany. The problem is particularly challenging due to the large number of plant species, the inter-species similarity, the large scale variations in natural images, and the lack of annotated data. In this paper, we present a guided multi-scale approach that segments the regions of interest (containing a plant) from a complex background of the natural image and systematically extracts scale-representative patches based on those regions. These multi-scale patches are used to train state-of-the-art Convolutional Neural Network (CNN) models that analyze a given plant image and determine its species. Focusing specifically on the identification of plant species in natural images, we show that the proposed approach is a very effective way of making deep learning models more robust to scale variations. We perform a comprehensive experimental evaluation of our proposed method over several CNN models. Our best result on the Inception-ResNet-v2 model achieves a top-1 classification accuracy of 89.21% for 100 plant species which represents a 5.4% increase over using random cropping to generate training data.

[1]  Tony P. Pridmore,et al.  Deep Learning for Multi-task Plant Phenotyping , 2017, bioRxiv.

[2]  Pietro Perona,et al.  Bird Species Categorization Using Pose Normalized Deep Convolutional Nets , 2014, ArXiv.

[3]  Songfan Yang,et al.  Multi-scale Recognition with DAG-CNNs , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Donald Geman,et al.  Vantage Feature Frames for Fine-Grained Categorization , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Paolo Remagnino,et al.  How deep learning extracts and learns leaf features for plant classification , 2017, Pattern Recognit..

[6]  Pietro Perona,et al.  Caltech-UCSD Birds 200 , 2010 .

[7]  W. John Kress,et al.  Leafsnap: A Computer Vision System for Automatic Plant Species Identification , 2012, ECCV.

[8]  Bolei Zhou,et al.  Scene Parsing through ADE20K Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Lipyeow Lim,et al.  What's That Plant? WTPlant is a Deep Learning System to Identify Plants in Natural Images , 2018, BMVC.

[11]  Qingbo Wu,et al.  Segmentation quality evaluation based on multi-scale convolutional neural networks , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[12]  Xiao Liu,et al.  Fully Convolutional Attention Localization Networks: Efficient Attention Localization for Fine-Grained Recognition , 2016, ArXiv.

[13]  Patrick Mäder,et al.  Plant Species Identification Using Computer Vision Techniques: A Systematic Literature Review , 2017, Archives of Computational Methods in Engineering.

[14]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Lipyeow Lim,et al.  WTPlant (What's That Plant?): A Deep Learning System for Identifying Plants in Natural Images , 2018, ICMR.

[16]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Eibe Frank,et al.  Large-Scale Automatic Species Identification , 2017, Australasian Conference on Artificial Intelligence.

[18]  Shenghuo Zhu,et al.  Efficient Object Detection and Segmentation for Fine-Grained Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Paolo Remagnino,et al.  Deep-plant: Plant identification with convolutional neural networks , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[21]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[22]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[23]  Jing Hu,et al.  A Multiscale Fusion Convolutional Neural Network for Plant Leaf Recognition , 2018, IEEE Signal Processing Letters.

[24]  Yang Song,et al.  Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Peggy L. Peissig,et al.  Diagnostic Classification of Lung CT Images Using Deep 3D Multi-Scale Convolutional Neural Network , 2018, 2018 IEEE International Conference on Healthcare Informatics (ICHI).

[26]  Jonathan Krause,et al.  Fine-grained recognition without part annotations , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Yu Sun,et al.  Deep Learning for Plant Identification in Natural Environment , 2017, Comput. Intell. Neurosci..

[28]  Feng Zhou,et al.  Fine-Grained Categorization and Dataset Bootstrapping Using Deep Metric Learning with Humans in the Loop , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Pietro Perona,et al.  Multiclass recognition and part localization with humans in the loop , 2011, 2011 International Conference on Computer Vision.