AI on the Bog: Monitoring and Evaluating Cranberry Crop Risk

Machine vision for precision agriculture has attracted considerable research interest in recent years. The goal of this paper is to develop an end-to-end cranberry health monitoring system to enable and support real time cranberry over-heating assessment to facilitate informed decisions that may sustain the economic viability of the farm. Toward this goal, we propose two main deep learning-based modules for: 1) cranberry fruit segmentation to delineate the exact fruit regions in the cranberry field image that are exposed to sun, 2) prediction of cloud coverage conditions and sun irradiance to estimate the inner temperature of exposed cranberries. We develop drone-based field data and ground-based sky data collection systems to collect video imagery at multiple time points for use in crop health analysis. Extensive evaluation on the data set shows that it is possible to predict exposed fruit's inner temperature with high accuracy (0.02% MAPE). The sun irradiance prediction error was found to be 8.41-20.36% MAPE in the 5-20 minutes time horizon. With 62.54% mIoU for segmentation and 13.46 MAE for counting accuracies in exposed fruit identification, this system is capable of giving informed feedback to growers to take precautionary action (e.g. irrigation) in identified crop field regions with higher risk of sunburn in the near future. Though this novel system is applied for cranberry health monitoring, it represents a pioneering step forward for efficient farming and is useful in precision agriculture beyond the problem of cranberry overheating.

[1]  C. Daughtry,et al.  What good are unmanned aircraft systems for agricultural remote sensing and precision agriculture? , 2018 .

[2]  Terrence Chen,et al.  Short term cloud coverage prediction using ground based all sky imager , 2014, 2014 IEEE International Conference on Smart Grid Communications (SmartGridComm).

[3]  Edward J. Delp,et al.  Locating Objects Without Bounding Boxes , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Linesh Raja,et al.  Agriculture drones: A modern breakthrough in precision agriculture , 2017 .

[6]  Kristin J. Dana,et al.  Finding Berries: Segmentation and Counting of Cranberries using Point Supervision and Shape Priors , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Sundar A. Christopher,et al.  Automatic contrail detection and segmentation , 1998, IEEE Trans. Geosci. Remote. Sens..

[8]  Silas C. Michaelides,et al.  Multifeature texture analysis for the classification of clouds in satellite imagery , 2003, IEEE Trans. Geosci. Remote. Sens..

[9]  Hanan Samet,et al.  A general approach to connected-component labeling for arbitrary image representations , 1992, JACM.

[10]  R. Kerry,et al.  Investigating temporal and spatial patterns of cranberry yield in New Jersey fields , 2017, Precision Agriculture.

[11]  Rafael Rieder,et al.  Computer vision and artificial intelligence in precision agriculture for grain crops: A systematic review , 2018, Comput. Electron. Agric..

[12]  Daniel Rueckert,et al.  Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences , 2018, MICCAI.

[13]  P. J. Huber Robust Estimation of a Location Parameter , 1964 .

[14]  J. Nieto,et al.  Collaborative UAV-UGV Environment Reconstruction in Precision Agriculture , 2018 .

[15]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[16]  Guy Rochard,et al.  Unsupervised segmentation of low clouds from infrared METEOSAT images based on a contextual spatio-temporal labeling approach , 2002, IEEE Trans. Geosci. Remote. Sens..

[17]  Wright-Patterson Afb,et al.  Feature Selection Using a Multilayer Perceptron , 1990 .

[18]  Shawn D. Newsam,et al.  Improving Semantic Segmentation via Video Propagation and Label Relaxation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[20]  Andrew Zisserman,et al.  Learning To Count Objects in Images , 2010, NIPS.

[21]  Ramprasaath R. Selvaraju,et al.  Counting Everyday Objects in Everyday Scenes , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Nima Tajbakhsh,et al.  UNet++: A Nested U-Net Architecture for Medical Image Segmentation , 2018, DLMIA/ML-CDS@MICCAI.

[23]  Ming-Hsuan Yang,et al.  SegFlow: Joint Learning for Video Object Segmentation and Optical Flow , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Slobodan Ilic,et al.  Seeing Beyond Appearance - Mapping Real Images into Geometrical Domains for Unsupervised CAD-based Recognition , 2018, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[25]  Stefan Winkler,et al.  Color-Based Segmentation of Sky/Cloud Images From Ground-Based Cameras , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[26]  Jun Yang,et al.  A Hybrid Thresholding Algorithm for Cloud Detection on Ground-Based Color Images , 2011 .

[27]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[29]  Felipe Gonzalez,et al.  Autonomous UAV with vision based on-board decision making for remote sensing and precision agriculture , 2017, 2017 IEEE Aerospace Conference.

[30]  J. Gallichand,et al.  Reducing cranberry heat stress and midday depression with evaporative cooling , 2016 .

[31]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[32]  A. Heinle,et al.  Automatic cloud classification of whole sky images , 2010 .

[33]  Sanjiv Singh,et al.  Yield estimation in vineyards by visual grape detection , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[35]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[36]  Gaston Lefranc,et al.  Olive Fruits Recognition Using Neural Networks , 2013, ITQM.

[37]  W. Maes,et al.  Perspectives for Remote Sensing with Unmanned Aerial Vehicles in Precision Agriculture. , 2019, Trends in plant science.

[38]  Um Rao Mogili,et al.  Review on Application of Drone Systems in Precision Agriculture , 2018 .

[39]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[40]  Nicolas H. Younan,et al.  On the Use of a Cluster Ensemble Cloud Classification Technique in Satellite Precipitation Estimation , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[41]  Juan Carlos Corrales,et al.  A computer vision system for automatic cherry beans detection on coffee trees , 2020, Pattern Recognit. Lett..

[42]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Mark W. Schmidt,et al.  Where are the Blobs: Counting by Localization with Point Supervision , 2018, ECCV.

[44]  Roland Siegwart,et al.  A Toolbox for Easily Calibrating Omnidirectional Cameras , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[45]  Byron Boots,et al.  4D crop monitoring: Spatio-temporal reconstruction for agriculture , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[46]  Frank Rosenblatt,et al.  PRINCIPLES OF NEURODYNAMICS. PERCEPTRONS AND THE THEORY OF BRAIN MECHANISMS , 1963 .

[47]  P. Croft Field Conditions Associated with Cranberry Scald , 1995 .

[48]  Peter V. Oudemans,et al.  Spatial Analysis of Cranberry Yield at Three Scales , 2005 .

[49]  Fei-Fei Li,et al.  What's the Point: Semantic Segmentation with Point Supervision , 2015, ECCV.

[50]  Roland Siegwart,et al.  AgriColMap: Aerial-Ground Collaborative 3D Mapping for Precision Farming , 2018, IEEE Robotics and Automation Letters.