Applying machine learning based on multiscale classifiers to detect remote phenology patterns in Cerrado savanna trees

Abstract Plant phenology is one of the most reliable indicators of species responses to global climate change, motivating the development of new technologies for phenological monitoring. Digital cameras or near remote systems have been efficiently applied as multi-channel imaging sensors, where leaf color information is extracted from the RGB (Red, Green, and Blue) color channels, and the changes in green levels are used to infer leafing patterns of plant species. In this scenario, texture information is a great ally for image analysis that has been little used in phenology studies. We monitored leaf-changing patterns of Cerrado savanna vegetation by taking daily digital images. We extract RGB channels from the digital images and correlate them with phenological changes. Additionally, we benefit from the inclusion of textural metrics for quantifying spatial heterogeneity. Our first goals are: (1) to test if color change information is able to characterize the phenological pattern of a group of species; (2) to test if the temporal variation in image texture is useful to distinguish plant species; and (3) to test if individuals from the same species may be automatically identified using digital images. In this paper, we present a machine learning approach based on multiscale classifiers to detect phenological patterns in the digital images. Our results indicate that: (1) extreme hours (morning and afternoon) are the best for identifying plant species; (2) different plant species present a different behavior with respect to the color change information; and (3) texture variation along temporal images is promising information for capturing phenological patterns. Based on those results, we suggest that individuals from the same species and functional group might be identified using digital images, and introduce a new tool to help phenology experts in the identification of new individuals from the same species in the image and their location on the ground.

[1]  H. Wanner,et al.  Tree phenology and carbon dioxide fluxes - use of digital photography for process-based interpretation at the ecosystem scale , 2009 .

[2]  Peter B. Reich,et al.  PHENOLOGY OF TROPICAL FORESTS : PATTERNS, CAUSES, AND CONSEQUENCES , 1995 .

[3]  R. Grote,et al.  The timing of bud burst and its effect on tree growth , 2004, International journal of biometeorology.

[4]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[5]  Daniel T. Larose,et al.  Discovering Knowledge in Data: An Introduction to Data Mining , 2005 .

[6]  José Alexandre Felizola Diniz-Filho,et al.  The shared influence of phylogeny and ecology on the reproductive patterns of Myrteae (Myrtaceae) , 2010 .

[7]  Gian-Reto Walther,et al.  Plants in a warmer world , 2003 .

[8]  Jurandy Almeida,et al.  Fusion of Local and Global Descriptors for Content-Based Image and Video Retrieval , 2012, CIARP.

[9]  Jing Li Wang,et al.  Color image segmentation: advances and prospects , 2001, Pattern Recognit..

[10]  Andrew D Richardson,et al.  Near-surface remote sensing of spatial and temporal variation in canopy phenology. , 2009, Ecological applications : a publication of the Ecological Society of America.

[11]  S. Running,et al.  The impact of growing-season length variability on carbon assimilation and evapotranspiration over 88 years in the eastern US deciduous forest , 1999, International journal of biometeorology.

[12]  Paolo Remagnino,et al.  Plant species identification using digital morphometrics: A review , 2012, Expert Syst. Appl..

[13]  W. John Kress,et al.  Leafsnap: A Computer Vision System for Automatic Plant Species Identification , 2012, ECCV.

[14]  Philippe Ciais,et al.  Modeling climate change effects on the potential production of French plains forests at the sub-regional level. , 2005, Tree physiology.

[15]  C. Rosenzweig,et al.  Attributing physical and biological impacts to anthropogenic climate change , 2008, Nature.

[16]  Michael Unser,et al.  Sum and Difference Histograms for Texture Classification , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  O. Hoegh‐Guldberg,et al.  Ecological responses to recent climate change , 2002, Nature.

[18]  Sylvie Philipp-Foliguet,et al.  Descriptor correlation analysis for remote sensing image multi-scale classification , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[19]  Jurandy Almeida,et al.  Visual rhythm-based time series analysis for phenology studies , 2013, 2013 IEEE International Conference on Image Processing.

[20]  Jurandy Almeida,et al.  Shape-based time series analysis for remote phenology studies , 2013, 2013 IEEE International Geoscience and Remote Sensing Symposium - IGARSS.

[21]  Hervé Le Men,et al.  Scale-Sets Image Analysis , 2005, International Journal of Computer Vision.

[22]  D. Hollinger,et al.  Use of digital webcam images to track spring green-up in a deciduous broadleaf forest , 2007, Oecologia.

[23]  Ethem Alpaydin,et al.  Introduction to Machine Learning (Adaptive Computation and Machine Learning) , 2004 .

[24]  Jurandy Almeida,et al.  Remote phenology: Applying machine learning to detect phenological patterns in a cerrado savanna , 2012, 2012 IEEE 8th International Conference on E-Science.

[25]  Reiko Ide,et al.  Use of digital cameras for phenological observations , 2010, Ecol. Informatics.

[26]  Ricardo da Silva Torres,et al.  Content-Based Image Retrieval: Theory and Applications , 2006, RITA.

[27]  Robert E. Schapire,et al.  A Brief Introduction to Boosting , 1999, IJCAI.

[28]  Sylvie Philipp-Foliguet,et al.  Multi-Scale Classification of Remote Sensing Images , 2011 .

[29]  G. Negi,et al.  Leaf and bud demography and shoot growth in evergreen and deciduous trees of central Himalaya, India , 2006, Trees.

[30]  Leopoldo Magno Coutinho,et al.  O conceito do cerrado , 1978 .

[31]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[32]  Horst Bischof,et al.  On-line Boosting and Vision , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[33]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[34]  C. D. Keeling,et al.  Increased activity of northern vegetation inferred from atmospheric CO2 measurements , 1996, Nature.

[35]  Kenlo Nishida Nasahara,et al.  Using digital camera images to detect canopy condition of deciduous broad-leaved trees , 2011 .

[36]  Frédéric Precioso,et al.  Boosted kernel for image categorization , 2014, Multimedia Tools and Applications.

[37]  Mark D. Schwartz,et al.  Phenology: An Integrative Environmental Science , 2013, Springer Netherlands.

[38]  G. Yohe,et al.  A globally coherent fingerprint of climate change impacts across natural systems , 2003, Nature.

[39]  Mark D. Schwartz,et al.  Assessing satellite‐derived start‐of‐season measures in the conterminous USA , 2002 .

[40]  Sylvie Philipp-Foliguet,et al.  Multiscale Classification of Remote Sensing Images , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[41]  S. Kurc,et al.  Digital image-derived greenness links deep soil moisture to carbon uptake in a creosotebush-dominated shrubland , 2010 .

[42]  Volker C. Radeloff,et al.  The Impact of Phenological Variation on Texture Measures of Remotely Sensed Imagery , 2009, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.