Autonomous Identification and Positioning of Trucks during Collaborative Forage Harvesting

In the process of collaborative operation, the unloading automation of the forage harvester is of great significance to improve harvesting efficiency and reduce labor intensity. However, non-standard transport trucks and unstructured field environments make it extremely difficult to identify and properly position loading containers. In this paper, a global model with three coordinate systems is established to describe a collaborative harvesting system. Then, a method based on depth perception is proposed to dynamically identify and position the truck container, including data preprocessing, point cloud pose transformation based on the singular value decomposition (SVD) algorithm, segmentation and projection of the upper edge, edge lines extraction and corner points positioning based on the Random Sample Consensus (RANSAC) algorithm, and fusion and visualization of results on the depth image. Finally, the effectiveness of the proposed method has been verified by field experiments with different trucks. The results demonstrated that the identification accuracy of the container region is about 90%, and the absolute error of center point positioning is less than 100 mm. The proposed method is robust to containers with different appearances and provided a methodological reference for dynamic identification and positioning of containers in forage harvesting.

[1]  Friedrich Fraundorfer,et al.  Visual Odometry Part I: The First 30 Years and Fundamentals , 2022 .

[2]  Bruno Siciliano,et al.  Fast Statistical Outlier Removal Based Method for Large 3D Point Clouds of Outdoor Environments , 2018, SyRoCo.

[3]  Xiangjun Zou,et al.  Fruit detection in natural environment using partial shape matching and probabilistic Hough transform , 2019, Precision Agriculture.

[4]  Silvio Savarese,et al.  Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Won Suk Lee,et al.  DYNAMIC LOAD ESTIMATION IN SILAGE HARVESTING , 2005 .

[6]  Hao Zhou,et al.  Three-dimensional perception of orchard banana central stock enhanced by adaptive multi-vision technology , 2020, Comput. Electron. Agric..

[7]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[8]  L. Tedeschi,et al.  Effects of source and concentration of neutral detergent fiber from roughage in beef cattle diets on feed intake, ingestive behavior, and ruminal kinetics. , 2020, Journal of animal science.

[9]  Michihisa Iida,et al.  Autonomous positioning of the unloading auger of a combine harvester by a laser sensor and GNSS , 2015 .

[10]  Konstantinos G. Derpanis,et al.  Overview of the RANSAC Algorithm , 2005 .

[11]  Hans-Heinrich Harms,et al.  Loading of Agricultural Trailers Using a Model-Based Method , 2009 .

[12]  Michihisa Iida,et al.  Application of Image Processing Technology for Unloading Automation of Robotic Head-Feeding Combine Harvester , 2012 .

[13]  Lining Sun,et al.  RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud Registration , 2019, Sensors.

[14]  Marco Bietresato,et al.  A numerical approach for evaluating and properly setting self-propelled forage harvesters. , 2013 .

[15]  Kevin J. Shinners.  Engineering Principles of Silage Harvesting Equipment , 2015 .

[16]  John David Gaard.  Grain wagon fill detection using ultrasonic sensors , 2012 .

[17]  G. Cozzi,et al.  Feeding behaviour, diet digestibility, rumen fluid and metabolic parameters of beef cattle fed total mixed rations with a stepped substitution of wheat straw with maize silage , 2009 .

[18]  R. Zinn,et al.  Influence of forage NDF level, source and pelletizing on growth performance, dietary energetics, and characteristics of digestive function for feedlot cattle , 2013 .

[19]  Avital Bechar,et al.  Agricultural robots for field operations: Concepts and components , 2016 .

[20]  C. Livesey,et al.  Occurrence and significance of mycotoxins in forage crops and silage: a review , 1998 .

[21]  Xiangjun Zou,et al.  Recognition and Localization Methods for Vision-Based Fruit Picking Robots: A Review , 2020, Frontiers in Plant Science.

[22]  Dan Liu,et al.  Grain Truck Loading Status Detection Based on Machine Vision , 2019, 2019 IEEE 4th International Conference on Image, Vision and Computing (ICIVC).

[23]  K. Shinners,et al.  CONSERVED FORAGE ( SILAGE AND HAY ) : PROGRESS AND PRIORITIES , 2001 .

[24]  Xiangjun Zou,et al.  Color-, depth-, and shape-based 3D fruit detection , 2019, Precision Agriculture.

[25]  Dionysis Bochtis,et al.  Automatic Recognition of Operation Modes of Combines and Transport Units based on GNSS Trajectories , 2013 .