3-D Imaging Systems for Agricultural Applications—A Review

Efficiency increase of resources through automation of agriculture requires more information about the production process, as well as process and machinery status. Sensors are necessary for monitoring the status and condition of production by recognizing the surrounding structures such as objects, field structures, natural or artificial markers, and obstacles. Currently, three dimensional (3-D) sensors are economically affordable and technologically advanced to a great extent, so a breakthrough is already possible if enough research projects are commercialized. The aim of this review paper is to investigate the state-of-the-art of 3-D vision systems in agriculture, and the role and value that only 3-D data can have to provide information about environmental structures based on the recent progress in optical 3-D sensors. The structure of this research consists of an overview of the different optical 3-D vision techniques, based on the basic principles. Afterwards, their application in agriculture are reviewed. The main focus lays on vehicle navigation, and crop and animal husbandry. The depth dimension brought by 3-D sensors provides key information that greatly facilitates the implementation of automation and robotics in agriculture.

[1]  David A. Forsyth,et al.  Shape from Texture without Boundaries , 2002, International Journal of Computer Vision.

[2]  Ray A. Jarvis,et al.  A Perspective on Range Finding Techniques for Computer Vision , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  E. Barbosa,et al.  Multiwavelength electronic speckle pattern interferometry for surface shape measurement. , 2007, Applied optics.

[4]  Andrea Pezzuolo,et al.  Three-dimensional sensor for dynamic characterization of soil microrelief , 2013 .

[5]  R. D. Tillett,et al.  A stereo imaging system for the metric 3D recovery of porcine surface anatomy , 2004 .

[6]  Joanne C. White,et al.  Lidar sampling for large-area forest characterization: A review , 2012 .

[7]  M.L.I. Wigham,et al.  MARVIN : high speed 3D imaging for seedling classification , 2009 .

[8]  Stefano Soatto,et al.  A geometric approach to shape from defocus , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Satoru Toyooka,et al.  Blooming Processes in Flowers Studied By Dynamic Electronic Speckle Pattern Interferometry (DESPI) , 2003 .

[10]  Hansjörg Kutterer,et al.  Automatic weed detection in grassland , 2009 .

[11]  Arno Ruckelshausen,et al.  Feldtaugliche Multisensorplattform für High-Throughput Getreidephänotypisierung - Aufbau und Datenhandling , 2016, GIL Jahrestagung.

[12]  J. Wei,et al.  Autonomous Guidance of a Corn Harvester using Stereo Vision , 2007 .

[13]  Daniel Moura,et al.  In-field crop row phenotyping from 3D modeling performed using Structure from Motion , 2015, Comput. Electron. Agric..

[14]  François Blais.  Review of 20 years of range sensor development , 2004, J. Electronic Imaging.

[15]  Patrick Wambacq,et al.  Development and application of computer vision systems for use in livestock production , 1991 .

[16]  Andres Hernandez,et al.  Towards the Development of a Smart Flying Sensor: Illustration in the Field of Precision Agriculture , 2015, Sensors.

[17]  Qin Zhang,et al.  Creating a panoramic field image using multi-spectral stereovision system , 2008 .

[18]  Eldert J. van Henten,et al.  IEEE Robotics and Automation Society Technical Committee on Agricultural Robotics and Automation [TC Spotlight] , 2013, IEEE Robotics Autom. Mag..

[19]  Heeyoung Jung,et al.  Optical Sensing Method for Screening Disease in Melon Seeds by Using Optical Coherence Tomography , 2011, Sensors.

[20]  Noboru Noguchi,et al.  Development of a Laser Scanner-based Navigation System for a Combine Harvester , 2013 .

[21]  J. C. Noordam,et al.  Automated rose cutting in greenhouses with 3D vision and robotics: analysis of 3D vision techniques for stem detection , 2005 .

[22]  Lu Jiang,et al.  3D Surface Reconstruction and Analysis in Automated Apple Stem-End/Calyx Identification , 2009 .

[23]  Michel Dhome,et al.  Three-dimensional reconstruction by zooming , 1993, IEEE Trans. Robotics Autom..

[24]  Rasmus Nyholm Jørgensen,et al.  Study on Line Imaging Spectroscopy as a Tool for Nitrogen Diagnostics in Precision Farming , 2002 .

[25]  Takeo Kanade,et al.  A multiple-baseline stereo , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  David Reiser,et al.  Reconstruction of geo-referenced maize plants using a consumer time-of-flight camera in different agricultural environments , 2016, GIL Jahrestagung.

[27]  Mogens Blanke,et al.  Original paper: Stereo vision with texture learning for fault-tolerant automatic baling , 2011 .

[28]  Changming Sun,et al.  Fast optical flow using 3D shortest path techniques , 2002, Image Vis. Comput..

[29]  Thierry Oggier,et al.  CCD / CMOS Lock-In Pixel for Range Imaging : Challenges , Limitations and State-ofthe-Art , 2005 .

[30]  Joaquim Salvi,et al.  A state of the art in structured light patterns for surface profilometry , 2010, Pattern Recognit..

[31]  Wouter Saeys,et al.  Estimation of the crop density of small grains using LiDAR sensors. , 2009 .

[32]  F. van der Heijden,et al.  Weed detection in 3D images , 2011, Precision Agriculture.

[33]  Arno Ruckelshausen,et al.  BreedVision — A Multi-Sensor Platform for Non-Destructive Field-Based Phenotyping in Plant Breeding , 2013, Sensors.

[34]  Hossein Mousazadeh,et al.  A technical review on navigation systems of agricultural autonomous off-road vehicles , 2013 .

[35]  Paul C. Struik,et al.  Functional-Structural Plant Modelling in Crop Production , 2007 .

[36]  Andreas Zell,et al.  Plant Species Classification Using a 3D LIDAR Sensor and Machine Learning , 2010, 2010 Ninth International Conference on Machine Learning and Applications.

[37]  Aymen Ben Azouz,et al.  Development of a teat sensing system for robotic milking by combining thermal imaging and stereovision technique , 2015, Comput. Electron. Agric..

[38]  Steven R. Raine,et al.  Applied machine vision of plants – a review with implications for field deployment in automated farming operations 1 , 2010 .

[39]  Jiahua Wu,et al.  Extracting the three-dimensional shape of live pigs using stereo photogrammetry , 2004 .

[40]  Qin Zhang,et al.  A Stereovision-based Crop Row Detection Method for Tractor-automated Guidance , 2005 .

[41]  H. Macher,et al.  FIRST EXPERIENCES WITH KINECT V2 SENSOR FOR CLOSE RANGE 3D MODELLING , 2015 .

[42]  Mongkol Ekpanyapong,et al.  Fruit detection, tracking, and 3D reconstruction for crop mapping and yield estimation , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[43]  Chris McCarthy,et al.  A review of autonomous navigation systems in agricultural environments , 2013 .

[44]  Lie Tang,et al.  Automatic inter-plant spacing sensing at early growth stages using a 3D vision sensor , 2012 .

[45]  Qin Zhang,et al.  Stereovision-based lateral offset measurement for vehicle navigation in cultivated stubble fields , 2011 .

[46]  Sergi Foix,et al.  Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison , 2014 .

[47]  Daniel Berckmans,et al.  Comparison of a three-dimensional and two-dimensional camera system for automated measurement of back posture in dairy cows , 2014 .

[48]  Oussama Khatib,et al.  Experimental Robotics IV, The 4th International Symposium, Stanford, California, USA, June 30 - July 2, 1995 , 1997, ISER.

[49]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[50]  Hans W. Griepentrog,et al.  Safe and Reliable - Further Development of a Field Robot , 2009 .

[51]  Giulio Reina,et al.  Towards Autonomous Agriculture: Automatic Ground Detection Using Trinocular Stereovision , 2012, Sensors.

[52]  José A. Martínez-Casasnovas,et al.  Precision analysis of the effect of ephemeral gully erosion on vine vigour using NDVI images , 2013 .

[53]  Boaz Zion,et al.  Review: The use of computer vision technologies in aquaculture - A review , 2012 .

[54]  Berthold K. P. Horn.  SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[55]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[56]  Shree K. Nayar,et al.  Shape from Focus , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Leon Hirsch.  Handbook Of Computer Vision And Applications , 2016 .

[58]  Toru Tamaki,et al.  A preliminarily study for predicting body weight and milk properties in lactating Holstein cows using a three-dimensional camera system , 2015, Comput. Electron. Agric..

[59]  S. Vougioukas,et al.  Orchard tree digitization for structural-geometrical modeling , 2015 .

[60]  Qin Zhang,et al.  Creation of Three-dimensional Crop Maps based on Aerial Stereoimages , 2005 .

[61]  Pierre Gouton,et al.  3D Image Acquisition System Based on Shape from Focus Technique , 2013, Sensors.

[62]  John F. Reid,et al.  Obstacle detection using stereo vision to enhance safety of autonomous machines , 2005 .

[63]  Amy Tabb,et al.  Shape from Silhouette Probability Maps: Reconstruction of Thin Objects in the Presence of Silhouette Extraction and Calibration Error , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[64]  Uma Maheswari Rajagopalan,et al.  An optical interferometric technique for assessing ozone induced damage and recovery under cumulative exposures for a Japanese rice cultivar , 2014, SpringerPlus.

[65]  Keiji Hanawa,et al.  Development of a Stereo Vision System to Assist the Operation of Agricultural Tractors , 2012 .

[66]  Y. Edan,et al.  CROPS : intelligent sensing and manipulation for sustainable production and harvesting of high value crops, clever robots for crops : final report sweet-pepper harvesting robot , 2014 .

[67]  Federico Pallottino,et al.  Comparison between manual and stereovision body traits measurements of Lipizzan horses , 2015, Comput. Electron. Agric..

[68]  Glenn Guthrie.  3D Computer vision contact patch measurements inside off-road vehicle tyres , 2016 .

[69]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[70]  F. J. García-Ramos,et al.  Non-destructive technologies for fruit and vegetable size determination - a review , 2009 .

[71]  David Reiser,et al.  3D Maize Plant Reconstruction Based on Georeferenced Overlapping LiDAR Point Clouds , 2015, Remote. Sens..

[72]  Pablo J. Zarco-Tejada,et al.  Tree height quantification using very high resolution imagery acquired from an unmanned aerial vehicle (UAV) and automatic 3D photo-reconstruction methods , 2014 .

[73]  Swapnil Kulkarni,et al.  Automation for Agriculture , 2015 .

[74]  Silvio Savarese,et al.  Shape reconstruction from shadows and reflections , 2005 .

[75]  Takeo Kanade,et al.  Shape-From-Silhouette Across Time Part I: Theory and Algorithms , 2005, International Journal of Computer Vision.

[76]  Jeehyun Kim,et al.  Application of optical coherence tomography to detect Cucumber green mottle mosaic virus (CGMMV) infected cucumber seed , 2012, Horticulture, Environment, and Biotechnology.

[77]  P. Lammers,et al.  Identify Plant Drought Stress by 3D-Based Image , 2012 .

[78]  Federico Pallottino,et al.  A low-cost stereovision system to estimate size and weight of live sheep , 2014 .

[79]  Noboru Noguchi,et al.  Development of a target recognition and following system for a field robot , 2013 .

[80]  Moulay A. Akhloufi.  3D vision system for intelligent milking robot automation , 2013, Electronic Imaging.

[81]  Jacob Goldberger,et al.  Obstacle detection in a greenhouse environment using the Kinect sensor , 2015, Comput. Electron. Agric..

[82]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[83]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision: From Images to Geometric Models , 2003 .

[84]  Bikram Adhikari,et al.  3D Reconstruction of Apple Trees for Mechanical Pruning , 2011 .

[85]  Johanna Link,et al.  Combined Spectral and Spatial Modeling of Corn Yield Based on Aerial Images and Crop Surface Models Acquired with an Unmanned Aircraft System , 2014, Remote. Sens..

[86]  David Hernández-López,et al.  Vineyard yield estimation by automatic 3D bunch modelling in field conditions , 2015, Comput. Electron. Agric..

[87]  Yael Edan,et al.  Harvesting Robots for High‐value Crops: State‐of‐the‐art Review and Challenges Ahead , 2014, J. Field Robotics.

[88]  Giulio Reina,et al.  Ambient awareness for agricultural robotic vehicles , 2016, ArXiv.

[89]  H. S. Wolff,et al.  iRun: Horizontal and Vertical Shape of a Region-Based Graph Compression , 2022, Sensors.

[90]  Qin Zhang,et al.  A Review of Imaging Techniques for Plant Phenotyping , 2014, Sensors.

[91]  Thiago T. Santos,et al.  Image-based 3 D digitizing for plant architecture analysis and phenotyping , 2012 .

[92]  R. Sanz,et al.  A review of methods and applications of the geometric characterization of tree crops in agricultural activities , 2012 .

[93]  K. Shadan,et al.  Available online: , 2012 .

[94]  K. Kawasue,et al.  Three-Dimensional Shape Measurement System for Black Cattle Using KINECT Sensor , .

[95]  Jian Jin,et al.  Corn plant sensing using real‐time stereo vision , 2009, J. Field Robotics.

[96]  Olaf Hellwich,et al.  A Benchmarking Dataset for Performance Evaluation of Automatic Surface Reconstruction Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[97]  J. W. Hofstee,et al.  Phenotyping large tomato plants in the greenhouse using a 3D light-field camera , 2014 .

[98]  Yutaka Kaizu,et al.  Development of a Tractor Navigation System Using Augmented Reality , 2012 .

[99]  F.B.T.F. Golbach,et al.  Reverse Volumetric Intersection (RVI), a method to generate 3D images of plants using multiple views , 2005 .

[100]  A. Lino,et al.  Determinação da topografia de uma fruta pela técnica de moiré de sombra com multiplicação de franjas , 2004 .

[101]  Carme Torras,et al.  Robotic leaf probing via segmentation of range data into surface patches , 2012 .

[102]  M D Fox,et al.  Holographic interferometric measurement of motions in mature plants. , 1977, Plant physiology.

[103]  Paul J. Besl,et al.  Active, optical range imaging sensors , 1988, Machine Vision and Applications.

[104]  C. Glasbey,et al.  Spy - See - Advanced vision system for phenotyping in greenhouses , 2009 .

[105]  Stephan Schraml,et al.  A Novel HDR Depth Camera for Real-Time 3D 360° Panoramic Vision , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[106]  M. Westoby,et al.  ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications , 2012 .

[107]  Xianfeng Li,et al.  Review of Recent Machine-Vision Technologies in Agriculture , 2009, 2009 Second International Symposium on Knowledge Acquisition and Modeling.

[108]  N. Kondo,et al.  A review of automation and robotics for the bio-industry , 2008 .

[109]  Noboru Noguchi,et al.  Human detection for a robot tractor using omni-directional stereo vision , 2012 .

[110]  Peter Biber,et al.  Plant detection and mapping for agricultural robots using a 3D LIDAR sensor , 2011, Robotics Auton. Syst..