Generating artificial images of plant seedlings using generative adversarial networks

Plants seedlings are a part of a domain with low inter-class and relatively high intra-class variance with respect to visual appearance. This paper presents an approach for generating artificial image samples of plant seedlings using generative adversarial networks (GAN) to alleviate for the lack of training data for deep learning systems in this domain. We show that it is possible to use GAN to produce samples that are visually distinct across nine different plants species and maintain a high amount variance within each species. The generated samples resemble the intended species with an average recognition accuracy of 58.9 ± 9.2 % , evaluated using a state-of-the-art classification model. The observed errors are related to samples representing species which are relatively anonymous at the dicotyledonous growth stage and to the model's incapability to reproduce small shape details. The artificial plant samples are also used for pretraining a classification model, which is finetuned using real data. The pretrained model achieves 62.0 ± 5.3 % accuracy on classifying real plant seedlings prior to any finetuning, thus providing a strong basis for further training. However, finetuning the pretrained models show no performance increase compared to models trained without finetuning, as both approaches are capable of achieving near perfect classification on the dataset applied in this work.

[1]  P. Prusinkiewicz,et al.  Quantitative Modeling of Arabidopsis Development1[w] , 2005, Plant Physiology.

[2]  Hanno Scharr,et al.  Machine Learning for Plant Phenotyping Needs Image Processing. , 2016, Trends in plant science.

[3]  H. Søgaard Weed Classification by Active Shape Models , 2005 .

[4]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[5]  Hanno Scharr,et al.  Leaf segmentation in plant phenotyping: a collation study , 2016, Machine Vision and Applications.

[6]  Peter Norvig,et al.  The Unreasonable Effectiveness of Data , 2009, IEEE Intelligent Systems.

[7]  S. Tsaftaris,et al.  Phenotiki: an open software and hardware platform for affordable and easy image‐based phenotyping of rosette‐shaped plants , 2017, The Plant journal : for cell and molecular biology.

[8]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[9]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[10]  Hanno Scharr,et al.  ARIGAN: Synthetic Arabidopsis Plants using Generative Adversarial Network , 2017, bioRxiv.

[11]  Yi Yang,et al.  Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Leonidas J. Guibas,et al.  The Earth Mover's Distance as a Metric for Image Retrieval , 2000, International Journal of Computer Vision.

[14]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[15]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[17]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[18]  Dimitris N. Metaxas,et al.  StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[20]  Rasmus Nyholm Jørgensen,et al.  Evaluation of intra variability between annotators of weed species in color images , 2016 .

[21]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[22]  Stéphane Robin,et al.  Nonparametric density estimation by exact leave-p-out cross-validation , 2008, Comput. Stat. Data Anal..

[23]  Przemyslaw Prusinkiewicz,et al.  The use of plant models in deep learning: an application to leaf counting in rosette plants , 2018, Plant Methods.

[24]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[25]  Zengchang Qin,et al.  Emotion Classification with Data Augmentation Using Generative Adversarial Networks , 2018, PAKDD.

[26]  Hayit Greenspan,et al.  Synthetic data augmentation using GAN for improved liver lesion classification , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[27]  Mads Dyrmann,et al.  Automated Classification of Seedlings Using Computer Vision: Pattern Recognition of Seedlings Combining Features of Plants and Leaves for Improved Discrimination , 2014 .

[28]  Chen Sun,et al.  Revisiting Unreasonable Effectiveness of Data in Deep Learning Era , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).