RootPainter: deep learning segmentation of biological images with corrective annotation

We present RootPainter, a GUI-based software tool for the rapid training of deep neural networks for use in biological image analysis. RootPainter facilitates both fully-automatic and semi-automatic image segmentation. We investigate the effectiveness of RootPainter using three plant image datasets, evaluating its potential for root length extraction from chicory roots in soil, biopore counting and root nodule counting from scanned roots. We also use RootPainter to compare dense annotations to corrective ones which are added during the training based on the weaknesses of the current model.

[1]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Tony P. Pridmore,et al.  RootNav 2.0: Deep learning for automatic navigation of complex plant root architectures , 2019, bioRxiv.

[3]  Emmanuelle Gouillart,et al.  scikit-image: image processing in Python , 2014, PeerJ.

[4]  Mohammad Najafi,et al.  Deep phenotyping: deep learning for temporal phenotype/genotype classification , 2017, Plant Methods.

[5]  K. Thorup-Kristensen,et al.  Does earlier sowing of winter wheat improve root growth and N uptake , 2016 .

[6]  R. Horn,et al.  Root growth dynamics inside and outside of soil biopores as affected by crop sequence determined with the profile wall method , 2015, Biology and Fertility of Soils.

[7]  Astrid Junker,et al.  Semi-automated Root Image Analysis (saRIA) , 2019, Scientific Reports.

[8]  W. Horwath,et al.  Net soil carbon input under ambient and elevated CO2 concentrations: isotopic evidence after 4 years , 2000 .

[9]  Atsuto Maki,et al.  A systematic study of the class imbalance problem in convolutional neural networks , 2017, Neural Networks.

[10]  Shimon Rachmilevitch,et al.  Root taxa identification in plant mixtures – current techniques and future challenges , 2012, Plant and Soil.

[11]  Bastian Leibe,et al.  Iteratively Trained Interactive Segmentation , 2018, BMVC.

[12]  J. Cai,et al.  High-Throughput Field Imaging and Basic Image Analysis in a Wheat Breeding Programme , 2019, Front. Plant Sci..

[13]  Rico Sennrich,et al.  Edinburgh Neural Machine Translation Systems for WMT 16 , 2016, WMT.

[14]  Eduardo Valle,et al.  Data Augmentation for Skin Lesion Analysis , 2018, OR 2.0/CARE/CLIP/ISIC@MICCAI.

[15]  Boaz Barak,et al.  Deep double descent: where bigger models and more data hurt , 2019, ICLR.

[16]  Luís Santos,et al.  Deep Learning Applications in Agriculture: A Short Review , 2019, ROBOT.

[17]  Kristian Thorup-Kristensen,et al.  Root system-based limits to agricultural productivity and efficiency: the farming systems context. , 2016, Annals of botany.

[18]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[19]  Devis Tuia,et al.  Half a Percent of Labels is Enough: Efficient Animal Detection in UAV Imagery Using Deep CNNs and Active Learning , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Yoshua Bengio,et al.  Practical Recommendations for Gradient-Based Training of Deep Architectures , 2012, Neural Networks: Tricks of the Trade.

[21]  Daguang Xu,et al.  Interactive segmentation of medical images through fully convolutional neural networks , 2019, ArXiv.

[22]  Diane Rowland,et al.  Overcoming Small Minirhizotron Datasets Using Transfer Learning , 2020, Comput. Electron. Agric..

[23]  K. Thorup-Kristensen,et al.  Construction of a large-scale semi-field facility to study genotypic differences in deep root growth and resources acquisition , 2019, Plant Methods.

[24]  T. Kautz.  Research on subsoil biopores and their functions in organically managed soils: A review , 2014, Renewable Agriculture and Food Systems.

[25]  Yoshua Bengio,et al.  An Empirical Study of Example Forgetting during Deep Neural Network Learning , 2018, ICLR.

[26]  Hanspeter Pfister,et al.  ICON: An interactive approach to train deep neural networks for segmentation of neuronal structures , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[27]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[28]  J. Olesen,et al.  Exposing Deep Roots: A Rhizobox Laboratory. , 2020, Trends in plant science.

[29]  K. Thorup-Kristensen,et al.  Winter wheat cultivars and nitrogen (N) fertilization—Effects on root growth, N uptake efficiency and N use efficiency , 2015 .

[30]  Mohammadreza Soltaninejad,et al.  Three Dimensional Root CT Segmentation Using Multi-Resolution Encoder-Decoder Networks , 2020, IEEE Transactions on Image Processing.

[31]  Javier Ibáñez,et al.  A new image-based tool for the high throughput phenotyping of pollen viability: evaluation of inter- and intra-cultivar diversity in grapevine , 2018, Plant Methods.

[32]  Rodrigo Benenson,et al.  Large-Scale Interactive Object Segmentation With Human Annotators , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[34]  Jens Petersen,et al.  Segmentation of roots in soil with U-Net , 2019, Plant Methods.

[35]  Nathan Srebro,et al.  The Implicit Bias of Gradient Descent on Separable Data , 2017, J. Mach. Learn. Res..

[36]  K. Thorup-Kristensen,et al.  Uptake of subsoil water below 2 m fails to alleviate drought response in deep-rooted Chicory (Cichorium intybus L.) , 2019, Plant and Soil.

[37]  Vladimir Naumovich Vapni.  The Nature of Statistical Learning Theory , 1995 .

[38]  Xavier Draye,et al.  An online database for plant image analysis software tools , 2013, Plant Methods.

[39]  Jonathan P. Lynch,et al.  Roots of the Second Green Revolution , 2007 .

[40]  Qingjie Liu,et al.  Road Extraction by Deep Residual U-Net , 2017, IEEE Geoscience and Remote Sensing Letters.

[41]  Ian Stavness,et al.  Deep Plant Phenomics: A Deep Learning Platform for Complex Plant Phenotyping Tasks , 2017, Front. Plant Sci..

[42]  K. Thorup-Kristensen.  Effect of deep and shallow root systems on the dynamics of soil inorganic N during 3-year crop rotations , 2006, Plant and Soil.

[43]  Peter König,et al.  Data augmentation instead of explicit regularization , 2018, ArXiv.

[44]  P. Ambus,et al.  Temporal and spatial distribution of roots and competition for nitrogen in pea-barley intercrops – a field study employing 32P technique , 2001, Plant and Soil.

[45]  Tony P. Pridmore,et al.  Deep Learning for Multi-task Plant Phenotyping , 2017, bioRxiv.

[46]  Przemyslaw Prusinkiewicz,et al.  The use of plant models in deep learning: an application to leaf counting in rosette plants , 2018, Plant Methods.

[47]  Kaiming He,et al.  Group Normalization , 2018, ECCV.

[48]  Miriam Athmann,et al.  Optimising Cropping Techniques for Nutrient and Environmental Management in Organic Agriculture , 2015 .

[49]  J. Olesen,et al.  Digging Deeper for Agricultural Resources, the Value of Deep Rooting. , 2020, Trends in plant science.

[50]  Qi Tian,et al.  Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Data , 2019, ArXiv.

[51]  Michael Gygli,et al.  Interactive Video Object Segmentation in the Wild , 2017, ArXiv.

[52]  A. Kamilaris,et al.  A review of the use of convolutional neural networks in agriculture , 2018, The Journal of Agricultural Science.

[53]  M. Dingkuhn,et al.  Combining Image Analysis, Genome Wide Association Studies and Different Field Trials to Reveal Stable Genetic Regions Related to Panicle Architecture and the Number of Spikelets per Panicle in Rice , 2016, Front. Plant Sci..

[54]  Leslie N. Smith,et al.  A disciplined approach to neural network hyper-parameters: Part 1 - learning rate, batch size, momentum, and weight decay , 2018, ArXiv.

[55]  Kristian Thorup-Kristensen,et al.  Crop yield, root growth, and nutrient dynamics in a conventional and three organic cropping systems with different levels of external inputs and N re-cycling through fertility building crops , 2012 .

[56]  Hanno Scharr,et al.  Image Analysis: The New Bottleneck in Plant Phenotyping [Applications Corner] , 2015, IEEE Signal Processing Magazine.

[57]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[58]  Quantification of soil biopore density after perennial fodder cropping , 2015, Plant and Soil.

[59]  Kavita Bala,et al.  Block Annotation: Better Image Annotation for Semantic Segmentation with Sub-Image Decomposition , 2020, ArXiv.

[60]  Zhuwen Li,et al.  Interactive Image Segmentation with Latent Diversity , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[61]  Yang Hu,et al.  A Fully Convolutional Two-Stream Fusion Network for Interactive Image Segmentation , 2018, Neural Networks.

[62]  Tony P. Pridmore,et al.  Three Dimensional Root CT Segmentation using Multi-Resolution Encoder-Decoder Networks , 2019, bioRxiv.

[63]  Samy Bengio,et al.  Links between perceptrons, MLPs and SVMs , 2004, ICML.

[64]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  T. Kautz,et al.  Dynamics of plant nutrient uptake as affected by biopore-associated root growth in arable subsoil , 2017, Plant and Soil.

[66]  K. Thorup-Kristensen.  Are differences in root growth of nitrogen catch crops important for their ability to reduce soil nitrate-N content, and how can this be measured? , 2001, Plant and Soil.

[67]  Michael Gygli,et al.  Continuous Adaptation for Interactive Object Segmentation by Learning from Corrections , 2019, ECCV.

[68]  Changying Li,et al.  Convolutional Neural Networks for Image-Based High-Throughput Plant Phenotyping: A Review , 2020, Plant phenomics.