Close-range hyperspectral imaging of whole plants for digital phenotyping: Recent applications and illumination correction approaches

Digital plant phenotyping is emerging as a key research domain at the interface of information technology and plant science. Digital phenotyping aims to deploy high-end non-destructive sensing techniques and information technology infrastructures to automate the extraction of both structural and physiological traits from plants under phenotyping experiments. One of the promising sensor technologies for plant phenotyping is hyperspectral imaging (HSI). The main benefit of utilising HSI compared to other imaging techniques is the possibility to extract simultaneously structural and physiological information on plants. The use of HSI for analysis of parts of plants, e.g. plucked leaves, has already been demonstrated. However, there are several significant challenges associated with the use of HSI for extraction of information from a whole plant, and hence this is an active area of research. These challenges are related to data processing after image acquisition. The hyperspectral data acquired of a plant suffers from variations in illumination owing to light scattering, shadowing of plant parts, multiple scattering and a complex combination of scattering and shadowing. The extent of these effects depends on the type of plants and their complex geometry. A range of approaches has been introduced to deal with these effects, however, no concrete approach is yet ready. In this article, we provide a comprehensive review of recent studies of close-range HSI of whole plants. Several studies have used HSI for plant analysis but were limited to imaging of leaves, which is considerably more straightforward than imaging of the whole plant, and thus do not relate to digital phenotyping. In this article, we discuss and compare the approaches used to deal with the effects of variation in illumination, which are an issue for imaging of whole plants. Furthermore, future possibilities to deal with these effects are also highlighted.

[1]  Qin Zhang,et al.  A Review of Imaging Techniques for Plant Phenotyping , 2014, Sensors.

[2]  F. Loreto,et al.  Plant Phenotyping Research Trends, a Science Mapping Approach , 2019, Front. Plant Sci..

[3]  H. Martens,et al.  Light scattering and light absorbance separated by extended multiplicative signal correction. application to near-infrared transmission analysis of powder mixtures. , 2003, Analytical chemistry.

[4]  F. Baret,et al.  Estimation of Plant and Canopy Architectural Traits Using the Digital Plant Phenotyping Platform1[OPEN] , 2019, Plant Physiology.

[5]  M. Tester,et al.  Phenomics--technologies to relieve the phenotyping bottleneck. , 2011, Trends in plant science.

[6]  L. Xiong,et al.  Crop Phenomics and High-throughput Phenotyping: Past Decades, Current Challenges and Future Perspectives. , 2020, Molecular plant.

[7]  Jianwei Qin,et al.  Hyperspectral Imaging Instruments , 2010 .

[8]  Douglas N. Rutledge,et al.  SPORT pre-processing can improve near-infrared quality prediction models for fresh fruits and agro-materials , 2020 .

[9]  Jean-Michel Roger,et al.  Pre-processing Methods , 2020 .

[10]  Jie Liu,et al.  Improving High-Throughput Phenotyping Using Fusion of Close-Range Hyperspectral Camera and Low-Cost Depth Sensor , 2018, Sensors.

[11]  J. Féret,et al.  Exploring the potential of PROCOSINE and close-range hyperspectral imaging to study the effects of fungal diseases on leaf physiology , 2018, Scientific Reports.

[12]  Guillaume Lobet,et al.  Image Analysis in Plant Sciences: Publish Then Perish. , 2017, Trends in plant science.

[13]  Chris Brien,et al.  The Development of Hyperspectral Distribution Maps to Predict the Content and Distribution of Nitrogen and Water in Wheat (Triticum aestivum) , 2019, Front. Plant Sci..

[14]  K. Kersting,et al.  Early drought stress detection in cereals: simplex volume maximisation for hyperspectral image analysis. , 2012, Functional plant biology : FPB.

[15]  Gerrit Polder,et al.  Potato Virus Y Detection in Seed Potatoes Using Deep Learning on Hyperspectral Images , 2019, Front. Plant Sci..

[16]  Chika Yinka-Banjo,et al.  A review of generative adversarial networks and its application in cybersecurity , 2019, Artificial Intelligence Review.

[17]  P. Mishra,et al.  Close Range Spectral Imaging for Disease Detection in Plants Using Autonomous Platforms: a Review on Recent Studies , 2020, Current Robotics Reports.

[18]  Lutz Plümer,et al.  Generation and application of hyperspectral 3D plant models: methods and challenges , 2015, Machine Vision and Applications.

[19]  Jun‐Li Xu,et al.  Development of a polarized hyperspectral imaging system for investigation of absorption and scattering properties , 2019, Journal of Near Infrared Spectroscopy.

[20]  F. Baret,et al.  PROSPECT: A model of leaf optical properties spectra , 1990 .

[21]  Da-Wen Sun,et al.  A polarized hyperspectral imaging system for in vivo detection: Multiple applications in sunflower leaf analysis , 2019, Comput. Electron. Agric..

[22]  Yufeng Ge,et al.  Temporal dynamics of maize plant growth, water use, and leaf water content using automated high throughput RGB and hyperspectral imaging , 2016, Comput. Electron. Agric..

[23]  T. Næs,et al.  The Effect of Multiplicative Scatter Correction (MSC) and Linearity Improvement in NIR Spectroscopy , 1988 .

[24]  Ulrich Schurr,et al.  Future scenarios for plant phenotyping. , 2013, Annual review of plant biology.

[25]  Abhiram Das,et al.  Image-Based High-Throughput Field Phenotyping of Crop Roots1[W][OPEN] , 2014, Plant Physiology.

[26]  Jean Michel Roger,et al.  Orthogonal Projections in the Row and the Column Spaces , 2016 .

[27]  Simon Bennertz,et al.  Specim IQ: Evaluation of a New, Miniaturized Handheld Hyperspectral Camera and Its Application for Plant Phenotyping and Disease Detection , 2018, Sensors.

[28]  Alison Nordon,et al.  Homogenising and Segmenting Hyperspectral Images of Plants and Testing Chemicals in a High-Throughput Plant Phenotyping Setup , 2019, 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[29]  L. Plümer,et al.  Detection of early plant stress responses in hyperspectral images , 2014 .

[30]  Yufeng Ge,et al.  High Throughput In vivo Analysis of Plant Leaf Chemical Properties Using Hyperspectral Imaging , 2017, Front. Plant Sci..

[31]  Paul Scheunders,et al.  Close range hyperspectral imaging of plants: A review , 2017 .

[32]  Jean-Michel Roger,et al.  Sequential preprocessing through ORThogonalization (SPORT) and its application to near infrared spectroscopy , 2020 .

[33]  Dong Hwan Kim,et al.  An automated, high-throughput plant phenotyping system using machine learning-based plant segmentation and image analysis , 2018, PloS one.

[34]  Xilin Chen,et al.  Hyperspectral Light Field Stereo Matching , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  D. Inzé,et al.  Cell to whole-plant phenotyping: the best is yet to come. , 2013, Trends in plant science.

[36]  W. Verhoef,et al.  PROSPECT+SAIL models: A review of use for vegetation characterization , 2009 .

[37]  Offer Rozenstein,et al.  A Hyperspectral-Physiological Phenomics System: Measuring Diurnal Transpiration Rates and Diurnal Reflectance , 2020, Remote. Sens..

[38]  Paul Scheunders,et al.  Close-range hyperspectral image analysis for the early detection of stress responses in individual plants in a high-throughput phenotyping platform , 2018 .

[39]  Jose A. Jiménez-Berni,et al.  Review: New sensors and data-driven approaches—A path to next generation phenomics☆ , 2019, Plant science : an international journal of experimental plant biology.

[40]  T. Pridmore,et al.  Plant Phenomics, From Sensors to Knowledge , 2017, Current Biology.

[41]  Paul Scheunders,et al.  Analysis of hyperspectral images for detection of drought stress and recovery in maize plants in a high-throughput phenotyping platform , 2019, Comput. Electron. Agric..

[42]  A. Walter,et al.  Plant phenotyping: from bean weighing to image analysis , 2015, Plant Methods.

[43]  Általános és összehasonlító irodalomtudomány.  Guns, Germs, and Steel , 2010 .

[44]  Chunjiang Zhao,et al.  Crop Phenomics: Current Status and Perspectives , 2019, Front. Plant Sci..

[45]  Alison Nordon,et al.  Early Detection Of Drought Stress in Arabidopsis Thaliana Utilsing a Portable Hyperspectral Imaging Setup , 2019, 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[46]  Malia A. Gehan,et al.  Lights, camera, action: high-throughput plant phenotyping is ready for a close-up. , 2015, Current opinion in plant biology.

[47]  Ming Chen,et al.  Data-mining Techniques for Image-based Plant Phenotypic Traits Identification and Classification , 2019, Scientific Reports.

[48]  R. Barnes,et al.  Standard Normal Variate Transformation and De-Trending of Near-Infrared Diffuse Reflectance Spectra , 1989 .

[49]  Beata Walczak,et al.  VSN: Variable sorting for normalization , 2020 .

[50]  Peter W. B. Phillips,et al.  The adoption of automated phenotyping by plant breeders , 2018, Euphytica.

[51]  Francesco Cellini,et al.  Drought phenotyping in Vitis vinifera using RGB and NIR imaging , 2019, Scientia Horticulturae.

[52]  Man Zhang,et al.  Prediction of Leaf Water Content in Maize Seedlings Based on Hyperspectral Information , 2019, IFAC-PapersOnLine.

[53]  J. Féret,et al.  A physically-based model for retrieving foliar biochemistry and leaf orientation using close-range imaging spectroscopy , 2016 .

[54]  Douglas N. Rutledge,et al.  Utilising variable sorting for normalisation to correct illumination effects in close-range spectral images of potato plants , 2020 .

[55]  Frans van den Berg,et al.  Review of the most common pre-processing techniques for near-infrared spectra , 2009 .

[56]  Lutz Plümer,et al.  Calibration of hyperspectral close-range pushbroom cameras for plant phenotyping , 2015 .

[57]  U. Schurr,et al.  Plant Phenotyping: Past, Present, and Future , 2019, Plant phenomics.

[58]  Raffaele Casa,et al.  High-Throughput Plant Phenotyping for Developing Novel Biostimulants: From Lab to Field or From Field to Lab? , 2018, Front. Plant Sci..

[59]  P. Roumet,et al.  How plant structure impacts the biochemical leaf traits assessment from in-field hyperspectral images: A simulation study based on light propagation modeling in 3D virtual wheat scenes , 2017 .

[60]  Y. Ge,et al.  Application of high-throughput plant phenotyping for assessing biophysical traits and drought response in two oak species under controlled environment , 2020 .

[61]  Ye Sun,et al.  Nondestructive Determination of Nitrogen, Phosphorus and Potassium Contents in Greenhouse Tomato Plants Based on Multispectral Three-Dimensional Imaging , 2019, Sensors.

[62]  Douglas N. Rutledge,et al.  MBA-GUI: A chemometric graphical user interface for multi-block data visualisation, regression, classification, variable selection and automated pre-processing , 2020, Chemometrics and Intelligent Laboratory Systems.

[63]  P. Roumet,et al.  A spectral correction method for multi-scattering effects in close range hyperspectral imagery of vegetation scenes: application to nitrogen content assessment in wheat , 2017, Precision Agriculture.

[64]  Andrew P French,et al.  Hyperspectral image analysis techniques for the detection and classification of the early onset of plant disease and stress , 2017, Plant Methods.