From leaf to label: A robust automated workflow for stomata detection

Abstract Plant leaf stomata are the gatekeepers of the atmosphere–plant interface and are essential building blocks of land surface models as they control transpiration and photosynthesis. Although more stomatal trait data are needed to significantly reduce the error in these model predictions, recording these traits is time‐consuming, and no standardized protocol is currently available. Some attempts were made to automate stomatal detection from photomicrographs; however, these approaches have the disadvantage of using classic image processing or targeting a narrow taxonomic entity which makes these technologies less robust and generalizable to other plant species. We propose an easy‐to‐use and adaptable workflow from leaf to label. A methodology for automatic stomata detection was developed using deep neural networks according to the state of the art and its applicability demonstrated across the phylogeny of the angiosperms. We used a patch‐based approach for training/tuning three different deep learning architectures. For training, we used 431 micrographs taken from leaf prints made according to the nail polish method from herbarium specimens of 19 species. The best‐performing architecture was tested on 595 images of 16 additional species spread across the angiosperm phylogeny. The nail polish method was successfully applied in 78% of the species sampled here. The VGG19 architecture slightly outperformed the basic shallow and deep architectures, with a confidence threshold equal to 0.7 resulting in an optimal trade‐off between precision and recall. Applying this threshold, the VGG19 architecture obtained an average F‐score of 0.87, 0.89, and 0.67 on the training, validation, and unseen test set, respectively. The average accuracy was very high (94%) for computed stomatal counts on unseen images of species used for training. The leaf‐to‐label pipeline is an easy‐to‐use workflow for researchers of different areas of expertise interested in detecting stomata more efficiently. The described methodology was based on multiple species and well‐established methods so that it can serve as a reference for future work.

[1]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[2]  Oliver Brendel,et al.  Automatic measurement of stomatal density from microphotographs , 2014, Trees.

[3]  Eric Hervet,et al.  Applications for deep learning in ecology , 2018, bioRxiv.

[4]  J. Berry,et al.  Stomata: key players in the earth system, past and present. , 2010, Current opinion in plant biology.

[5]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Matti Pietikäinen,et al.  Deep Learning for Generic Object Detection: A Survey , 2018, International Journal of Computer Vision.

[7]  T. Ando,et al.  Existence of two stomatal shapes in the genus Dendrobium (Orchidaceae) and its systematic significance , 1992 .

[8]  Fabio A. González,et al.  Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks , 2014, Medical Imaging.

[9]  A. Pitman,et al.  Impact of the representation of stomatal conductance on model projections of heatwave intensity , 2016, Scientific Reports.

[10]  Nadejda A. Soudzilovskaia,et al.  Mapping local and global variability in plant trait distributions , 2017, Proceedings of the National Academy of Sciences.

[11]  J. Gray,et al.  Impact of Stomatal Density and Morphology on Water-Use Efficiency in a Changing World , 2019, Front. Plant Sci..

[12]  Lynne Boddy,et al.  Handbook for the measurement of macrofungal functional traits: A start with basidiomycete wood fungi , 2018, Functional Ecology.

[13]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[15]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[16]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[17]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[18]  I. Hara-Nishimura,et al.  Enhancement of leaf photosynthetic capacity through increased stomatal density in Arabidopsis. , 2013, The New phytologist.

[19]  Joel H. Saltz,et al.  Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  D. Beerling,et al.  Fossil Plants and Global Warming at the Triassic-Jurassic Boundary. , 1999, Science.

[21]  Peter F. Stevens,et al.  The Linear Angiosperm Phylogeny Group (LAPG) III: A linear sequence of the families in APG III , 2009 .

[22]  Ming Xu,et al.  Effects of experimental warming on stomatal traits in leaves of maize (Zea may L.) , 2013, Ecology and evolution.

[23]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[24]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Sven Eberhardt,et al.  StomataCounter: a deep learning method applied to automatic stomatal identification and counting , 2018, bioRxiv.

[28]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[29]  Tony P. Pridmore,et al.  Deep machine learning provides state-of-the-art performance in image-based plant phenotyping , 2016, bioRxiv.

[30]  Guirui Yu,et al.  Latitudinal variation of leaf stomatal traits from species to community level in forests: linkage with ecosystem productivity , 2015, Scientific Reports.

[31]  Sander Dieleman,et al.  Rotation-invariant convolutional neural networks for galaxy morphology prediction , 2015, ArXiv.

[32]  Diversity in stomatal function is integral to modelling plant carbon and water fluxes , 2017, Nature Ecology & Evolution.

[33]  Paulo S. Martins,et al.  Segmenting High-quality Digital Images of Stomata using the Wavelet Spot Detection and the Watershed Transform , 2017, VISIGRAPP.

[34]  José Paulo Sousa,et al.  Handbook of protocols for standardized measurement of terrestrial invertebrate functional traits , 2017 .

[35]  G. Bonan,et al.  Stomatal Function across Temporal and Spatial Scales: Deep-Time Trends, Land-Atmosphere Coupling and Global Models1[OPEN] , 2017, Plant Physiology.

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[38]  Odemir Martinez Bruno,et al.  Automatic counting of stomata in epidermis microscopic images , 2014 .

[39]  Sofie Meeus,et al.  From leaf to label: A robust automated workflow for stomata detection , 2019, Ecology and evolution.

[40]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[41]  F. I. Woodward,et al.  Stomatal numbers are sensitive to increases in CO2 from pre-industrial levels , 1987, Nature.

[42]  David C. Tank,et al.  An update of the Angiosperm Phylogeny Group classification for the orders and families of flowering plants: , 2009 .

[43]  Paul G. Constantine,et al.  Exploring the Sensitivity of Photosynthesis and Stomatal Resistance Parameters in a Land Surface Model , 2017 .

[44]  Galina Gussarova,et al.  Role of the outer stomatal ledges in the mechanics of guard cell movements , 2017, Trees.

[45]  Scarlett Liu,et al.  A Fast Method to Measure Stomatal Aperture by MSER on Smart Mobile Phone , 2016 .

[46]  J A Raven,et al.  Stomatal vs. genome size in angiosperms: the somatic tail wagging the genomic dog? , 2010, Annals of botany.

[47]  P. Franks,et al.  Smaller, faster stomata: scaling of stomatal size, rate of response, and stomatal conductance , 2013, Journal of experimental botany.

[48]  P. Rudall,et al.  Evolution and development of monocot stomata. , 2017, American journal of botany.

[49]  Eric Hervet,et al.  Applications for deep learning in ecology , 2019, Methods in Ecology and Evolution.

[50]  Christoph Meinel,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.

[51]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[52]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[53]  Scarlett Liu,et al.  Microscope image based fully automated stomata detection and pore measurement method for grapevines , 2017, Plant Methods.

[54]  Tony P. Pridmore,et al.  Deep Machine Learning provides state-of-the-art performance in image-based plant phenotyping , 2016 .

[55]  Ilia J Leitch,et al.  Genome size is a strong predictor of cell size and stomatal density in angiosperms. , 2008, The New phytologist.

[56]  F. Woodward,et al.  Deep-time evidence of a link between elevated CO2 concentrations and perturbations in the hydrological cycle via drop in plant transpiration , 2012 .

[57]  H. Verbeeck,et al.  Century‐long apparent decrease in intrinsic water‐use efficiency with no evidence of progressive nutrient limitation in African tropical forests , 2020, Global change biology.

[58]  P. Reich,et al.  A handbook of protocols for standardised and easy measurement of plant functional traits worldwide , 2003 .

[59]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[60]  D. C. Gitz,et al.  Methods for Creating Stomatal Impressions Directly onto Archivable Slides , 2009 .

[61]  W. Yin,et al.  PdEPF1 regulates water-use efficiency and drought tolerance by modulating stomatal density in poplar. , 2016, Plant biotechnology journal.

[62]  N. Kutsuna,et al.  CARTA-based semi-automatic detection of stomatal regions on an Arabidopsis cotyledon surface , 2014 .

[63]  Taghi M. Khoshgoftaar,et al.  Deep learning applications and challenges in big data analytics , 2015, Journal of Big Data.

[64]  Patrick Mäder,et al.  Machine learning for image based species identification , 2018, Methods in Ecology and Evolution.

[65]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[66]  John E. Eriksson,et al.  Image Quality Ranking Method for Microscopy , 2016, Scientific Reports.

[67]  L. Sack,et al.  Variation of stomatal traits from cold-temperate to tropical forests and association with water use efficiency , 2018 .

[68]  F. Woodward,et al.  The role of stomata in sensing and driving environmental change , 2003, Nature.

[69]  Paolo Napoletano,et al.  Benchmark Analysis of Representative Deep Neural Network Architectures , 2018, IEEE Access.

[70]  P. Reich,et al.  New handbook for standardised measurement of plant functional traits worldwide , 2013 .

[71]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[72]  Sergio Guadarrama,et al.  Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[73]  Using Clear Nail Polish to Make Arabidopsis Epidermal Impressions for Measuring the Change of Stomatal Aperture Size in Immune Response. , 2017, Methods in molecular biology.

[74]  Hong Hu,et al.  Evolutionary Association of Stomatal Traits with Leaf Vein Density in Paphiopedilum, Orchidaceae , 2012, PloS one.

[75]  D. Harris,et al.  Widespread mistaken identity in tropical plant collections , 2015, Current Biology.

[76]  I. C. Prentice,et al.  Optimal stomatal behaviour around the world , 2015 .

[77]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[78]  Jean Ponce,et al.  A Theoretical Analysis of Feature Pooling in Visual Recognition , 2010, ICML.

[79]  Adam A. Scaife,et al.  Towards operational predictions of the near-term climate , 2019, Nature Climate Change.