A general method to filter out defective spatial observations from yield mapping datasets

Yield maps are recognized as a valuable tool with regard to managing upcoming crop production but can contain a large amount of defective data that might result in misleading decisions. These anomalies must be removed before further processing to ensure the quality of future decisions. This paper proposes a new holistic methodology to filter out defective observations likely to be present in yield datasets. The notion of spatial neighbourhood has been refined to embrace the specific characteristics of such on-the-go vehicle based datasets. Observations are compared with their newly-defined spatial neighbourhood and the most abnormal ones are classified as defective observations based on a density-based clustering algorithm. The approach was conceived to be as non-parametric and automated as far as possible to pre-process a growing number of datasets without supervision. The proposed approach showed promising results on real yield datasets with the detection of well-known sources of errors such as filling and emptying times, speed changes and non-fully used cutting bar.

[1]  Yanji Wang,et al.  A harvest area measurement system based on ultrasonic sensors and DGPS for yield map correction , 2010, Precision Agriculture.

[2]  Brett Whelan,et al.  Quantification and comparison of wheat yield variation across space and time. , 2009 .

[3]  Hazaël Jones,et al.  Simulating yield datasets: an opportunity to improve data filtering algorithms , 2017 .

[4]  D. F. Heermann,et al.  Frequency Analysis of Yield for Delineating Yield Response Zones , 2004, Precision Agriculture.

[5]  H. D. Kutzbach,et al.  Investigations on a particular yield mapping system for combine harvesters , 1996 .

[6]  Dong-Hoon Lee,et al.  Automated Yield Map Delay Identification Using Phase Correlation Methodology , 2012 .

[7]  Chang-Tien Lu,et al.  On Detecting Spatial Outliers , 2008, GeoInformatica.

[8]  Wang Wei,et al.  Experiment research of impact-based sensor to monitor corn ear yield , 2010, 2010 International Conference on Computer Application and System Modeling (ICCASM 2010).

[9]  Achim Dobermann,et al.  Screening Yield Monitor Data Improves Grain Yield Maps , 2004 .

[10]  W. Tobler.  A Computer Movie Simulating Urban Growth in the Detroit Region , 1970 .

[11]  Brett Whelan,et al.  Establishing Management Classes for Broadacre Agricultural Production , 2007 .

[12]  Zhigang Zhang,et al.  Dynamic Compensation for Impact-Based Grain Flow Sensor , 2011, CCTA.

[13]  B. Ostendorf,et al.  Post-processing methods to eliminate erroneous grain yield measurements: review and directions for future development , 2013, Precision Agriculture.

[14]  Sun-Ok Chung,et al.  Determining yield monitoring system delay time with geostatistical and data segmentation approaches , 2002 .

[15]  Irad Ben-Gal.  Outlier Detection , 2005, The Data Mining and Knowledge Discovery Handbook.

[16]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[17]  Kenneth A. Sudduth,et al.  Yield Editor: Software for Removing Errors from Crop Yield Maps , 2007 .

[18]  Jugal K. Kalita,et al.  A Survey of Outlier Detection Methods in Network Anomaly Identification , 2011, Comput. J..

[19]  Chang-Tien Lu,et al.  Algorithms for spatial outlier detection , 2003, Third IEEE International Conference on Data Mining.

[20]  Craig L. Dobbins,et al.  Spatial analysis of yield monitor data: case studies of on-farm trials and farm management decision making , 2008, Precision Agriculture.

[21]  Kedar Sawant,et al.  Adaptive Methods for Determining DBSCAN Parameters , 2014 .

[22]  Harry Dankowicz,et al.  A dynamic grain flow model for a mass flow yield sensor on a combine , 2010, Precision Agriculture.

[23]  M. Hubert,et al.  Outlier detection for skewed data , 2008 .

[24]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[25]  Lian Duan,et al.  A Local Density Based Spatial Clustering Algorithm with Noise , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[26]  Wei Sun,et al.  An integrated framework for software to provide yield data cleaning and estimation of an opportunity index for site-specific crop management , 2012, Precision Agriculture.

[27]  Clyde W. Fraisse,et al.  COMBINE HARVEST AREA DETERMINATION BY VECTOR PROCESSING OF GPS POSITION DATA , 1999 .

[28]  Selcuk Arslan.  A Grain Flow Model to Simulate Grain Yield Sensor Response , 2008, Sensors.

[29]  M Spekken,et al.  A simple method for filtering spatial data , 2013 .

[30]  Martin Charlton,et al.  Multivariate Spatial Outlier Detection Using Robust Geographically Weighted Methods , 2013, Mathematical Geosciences.

[31]  Thomas S. Colvin,et al.  Grain Yield Mapping: Yield Sensing, Yield Reconstruction, and Errors , 2002, Precision Agriculture.

[32]  Simon Blackmore,et al.  Remedial Correction of Yield Map Data , 2004, Precision Agriculture.

[33]  Brett Whelan,et al.  A preliminary approach to assessing the opportunity for site-specific crop management in a field, using yield monitor data , 2003 .

[34]  Graciela Metternicht,et al.  Comparing the performance of techniques to improve the quality of yield maps , 2005 .

[35]  Peter Filzmoser,et al.  Noname manuscript No. (will be inserted by the editor) Identification of local multivariate outliers , 2022 .

[36]  Douglas M. Hawkins.  Identification of Outliers , 1980, Monographs on Applied Probability and Statistics.