Intelligent Perception-Based Cattle Lameness Detection and Behaviour Recognition: A Review

Simple Summary Cattle lameness detection as well as behaviour recognition are the two main objectives in the applications of precision livestock farming (PLF). Over the last five years, the development of smart sensors, big data, and artificial intelligence has offered more automatic tools. In this review, we discuss over 100 papers that used automated techniques to detect cattle lameness and to recognise animal behaviours. To assist researchers and policy-makers in promoting various livestock technologies for monitoring cattle welfare and productivity, we conducted a comprehensive investigation of intelligent perception for cattle lameness detection and behaviour analysis in the PLF domain. Based on the literature review, we anticipate that PLF will develop in an objective, autonomous, and real-time direction. Additionally, we suggest that further research should be dedicated to improving the data quality, modeling accuracy, and commercial availability. Abstract The growing world population has increased the demand for animal-sourced protein. However, animal farming productivity is faced with challenges from traditional farming practices, socioeconomic status, and climate change. In recent years, smart sensors, big data, and deep learning have been applied to animal welfare measurement and livestock farming applications, including behaviour recognition and health monitoring. In order to facilitate research in this area, this review summarises and analyses some main techniques used in smart livestock farming, focusing on those related to cattle lameness detection and behaviour recognition. In this study, more than 100 relevant papers on cattle lameness detection and behaviour recognition have been evaluated and discussed. Based on a review and a comparison of recent technologies and methods, we anticipate that intelligent perception for cattle behaviour and welfare monitoring will develop towards standardisation, a larger scale, and intelligence, combined with Internet of things (IoT) and deep learning technologies. In addition, the key challenges and opportunities of future research are also highlighted and discussed.

[1]  Phung Cong Phi Khanh,et al.  An IoT-Based Design Using Accelerometers in Animal Behavior Recognition Systems , 2022, IEEE Sensors Journal.

[2]  Salah Sukkarieh,et al.  One-Shot Learning-Based Animal Video Segmentation , 2022, IEEE Transactions on Industrial Informatics.

[3]  Salah Sukkarieh,et al.  Intelligent perception for cattle monitoring: A review for cattle identification, body condition score evaluation, and weight estimation , 2021, Comput. Electron. Agric..

[4]  D. Weary,et al.  Pairwise comparison locomotion scoring for dairy cattle. , 2021, Journal of dairy science.

[5]  Taek Sung Lee,et al.  Deep Learning-Based Cattle Vocal Classification Model and Real-Time Livestock Monitoring System with Noise Filtering , 2021, Animals : an open access journal from MDPI.

[6]  Gang Liu,et al.  A Review: Development of Computer Vision-Based Lameness Detection for Dairy Cows and Discussion of the Practical Applications , 2021, Sensors.

[7]  Suresh Neethirajan,et al.  Measuring Farm Animal Emotions—Sensor-Based Approaches , 2021, Sensors.

[8]  Paul Rodríguez,et al.  A systematic literature review on the use of machine learning in precision livestock farming , 2020, Comput. Electron. Agric..

[9]  Dong Sun Park,et al.  Deep learning-based hierarchical cattle behavior recognition with spatio-temporal information , 2020, Comput. Electron. Agric..

[10]  Malika Belkadi,et al.  Image analysis for individual identification and feeding behaviour monitoring of dairy cows based on Convolutional Neural Networks (CNN) , 2020 .

[11]  I. Veissier,et al.  Animal Welfare Management in a Digital World , 2020, Animals : an open access journal from MDPI.

[12]  X Kang,et al.  Accurate detection of lameness in dairy cattle with computer vision: A new and individualized detection strategy based on the analysis of the supporting phase. , 2020, Journal of dairy science.

[13]  Cheng Fei,et al.  Automatic recognition of ingestive-related behaviors of dairy cows based on triaxial acceleration , 2020 .

[14]  Xuqiang Yin,et al.  Using an EfficientNet-LSTM for the recognition of single Cow's motion behaviours in a complicated environment , 2020, Comput. Electron. Agric..

[15]  Huaibo Song,et al.  Single-stream long-term optical flow convolution network for action recognition of lameness dairy cow , 2020, Comput. Electron. Agric..

[16]  Dongjian He,et al.  A computer vision-based method for spatial-temporal action recognition of tail-biting behaviour in group-housed pigs , 2020 .

[17]  Leonardo L. Giovanini,et al.  An online method for estimating grazing and rumination bouts using acoustic signals in grazing cattle , 2020, Comput. Electron. Agric..

[18]  Alan Davy,et al.  Machine learning based fog computing assisted data-driven approach for early lameness detection in dairy cattle , 2020, Comput. Electron. Agric..

[19]  Guipeng Chen,et al.  Automated cattle counting using Mask R-CNN in quadcopter vision system , 2020, Comput. Electron. Agric..

[20]  N W O'Leary,et al.  Invited review: Cattle lameness detection with accelerometers. , 2020, Journal of dairy science.

[21]  Naoshi Kondo,et al.  Dam behavior patterns in Japanese black beef cattle prior to calving: Automated detection using LSTM-RNN , 2020, Comput. Electron. Agric..

[22]  D. Berckmans,et al.  Individualised automated lameness detection in dairy cows and the impact of historical window length on algorithm performance. , 2020, Animal : an international journal of animal bioscience.

[23]  Anders Ringgaard Kristensen,et al.  Assessment of the value of information of precision livestock farming: A conceptual framework , 2019 .

[24]  Rachida Aoudjit,et al.  Unsupervised automated monitoring of dairy cows' behavior based on Inertial Measurement Unit attached to their back , 2019, Comput. Electron. Agric..

[25]  Lvwen Huang,et al.  Body Dimension Measurements of Qinchuan Cattle with Transfer Learning from LiDAR Sensing , 2019, Sensors.

[26]  L. Riaboff,et al.  Evaluation of pre-processing methods for the prediction of cattle behaviour from accelerometer data , 2019, Comput. Electron. Agric..

[27]  Salah Sukkarieh,et al.  Cattle segmentation and contour extraction based on Mask R-CNN for precision livestock farming , 2019, Comput. Electron. Agric..

[28]  Dongjian He,et al.  Detection of cow mounting behavior using region geometry and optical flow characteristics , 2019, Comput. Electron. Agric..

[29]  Bo Jiang,et al.  Lameness detection of dairy cows based on a double normal background statistical model , 2019, Comput. Electron. Agric..

[30]  Z. Venter,et al.  Cattle don’t care: Animal behaviour is similar regardless of grazing management in grasslands , 2019, Agriculture, Ecosystems & Environment.

[31]  Naoshi Kondo,et al.  Classification of multiple cattle behavior patterns using a recurrent neural network with long short-term memory and inertial measurement units , 2019, Comput. Electron. Agric..

[32]  Maher Alsaaod,et al.  Automatic lameness detection in cattle. , 2019, Veterinary journal.

[33]  M. Alsaaod,et al.  Objective assessment of lameness in cattle after foot surgery , 2018, PloS one.

[34]  W. Büscher,et al.  Using walking speed for lameness detection in lactating dairy cows , 2018, Livestock Science.

[35]  M. Farish,et al.  The use of infrared thermography for detecting digital dermatitis in dairy cattle: What is the best measure of temperature and foot location to use? , 2018, Veterinary journal.

[36]  D. He,et al.  Automatic lameness detection in dairy cattle based on leg swing analysis with an image processing technique , 2018, Comput. Electron. Agric..

[37]  James Patrick Underwood,et al.  Object Detection for Cattle Gait Tracking , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[38]  D. Main,et al.  Optimising lameness detection in dairy cattle by using handheld infrared thermometers , 2018, Veterinary medicine and science.

[39]  C. Wrenzycki,et al.  Vocalization as an indicator of estrus climax in Holstein heifers during natural estrus and superovulation. , 2018, Journal of dairy science.

[40]  S. Ozkaya,et al.  Association between aggressive behaviour and high-energy feeding level in beef cattle , 2018 .

[41]  Javier Bajo,et al.  Combination of Multi-Agent Systems and Wireless Sensor Networks for the Monitoring of Cattle , 2018, Sensors.

[42]  Aaron Ingham,et al.  Cattle behaviour classification from collar, halter, and ear tag sensors , 2017 .

[43]  Michael T. Rose,et al.  Fixed-time data segmentation and behavior classification of pasture-based cattle: Enhancing performance using a hidden Markov model , 2017 .

[44]  Isabelle Veissier,et al.  Image analysis to refine measurements of dairy cow behaviour from a real-time location system , 2017, Biosystems Engineering.

[45]  L. Shalloo,et al.  Evaluation of the RumiWatchSystem for measuring grazing behaviour of cows , 2017, Journal of Neuroscience Methods.

[46]  B. Sturm,et al.  Implementation of machine vision for detecting behaviour of cattle and pigs , 2017 .

[47]  Kang-Sun Choi,et al.  Cow Behavior Recognition Using Motion History Image Feature , 2017, ICIAR.

[48]  John M. Antle,et al.  Toward a new generation of agricultural system data, models, and knowledge products: State of agricultural systems science , 2017, Agricultural systems.

[49]  M. Hayes,et al.  Future Protein Supply and Demand: Strategies and Factors Influencing a Sustainable Equilibrium , 2017, Foods.

[50]  Andriamasinoro Lalaina Herinaina Andriamandroso,et al.  Development of an open-source algorithm based on inertial measurement units (IMU) of a smartphone to detect cattle grass intake and ruminating behaviors , 2017, Comput. Electron. Agric..

[51]  Wang Zhihai,et al.  Cow behavior recognition based on image analysis and activities , 2017 .

[52]  Gao Ronghua,et al.  Cow Behavioral Recognition Using Dynamic Analysis , 2017, 2017 International Conference on Smart Grid and Electrical Automation (ICSGEA).

[53]  Margaret Kosmala,et al.  Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning , 2017, Proceedings of the National Academy of Sciences.

[54]  Alain N. Rousseau,et al.  Rethinking environment control strategy of confined animal housing systems through precision livestock farming , 2017 .

[55]  Md. Sumon Shahriar,et al.  Behavior classification of cows fitted with motion collars: Decomposing multi-class classification into a set of binary problems , 2016, Comput. Electron. Agric..

[56]  Ilias Kyriazakis,et al.  Early detection of health and welfare compromises through automated detection of behavioural changes in pigs , 2016, Veterinary journal.

[57]  D Berckmans,et al.  Lameness detection in dairy cattle: single predictor v. multivariate analysis of image-based posture processing and behaviour and performance sensing. , 2016, Animal : an international journal of animal bioscience.

[58]  Maher Alsaaod,et al.  Use of Extended Characteristics of Locomotion and Feeding Behavior for Automated Identification of Lame Dairy Cows , 2016, PloS one.

[59]  A. Steiner,et al.  Analysis of behavioral changes in dairy cows associated with claw horn lesions. , 2016, Journal of dairy science.

[60]  Pete Smith,et al.  Greenhouse gas mitigation potentials in the livestock sector , 2016 .

[61]  M.L. Williams,et al.  A novel behavioral model of the pasture-based dairy cow from GPS data using data mining and machine learning techniques. , 2016, Journal of dairy science.

[62]  N. Tadich,et al.  Thermographic assessment of hoof temperature in dairy cows with different mobility scores , 2016 .

[63]  L. Munksgaard,et al.  Lameness detection via leg-mounted accelerometers on dairy cows on four commercial farms. , 2015, Animal : an international journal of animal bioscience.

[64]  Lucas P. J. J. Noldus,et al.  Sound analysis in dairy cattle vocalisation as a potential welfare monitor , 2015, Comput. Electron. Agric..

[65]  Claudia Bahr,et al.  Lameness Detection in Dairy Cows: Part 2. Use of Sensors to Automatically Register Changes in Locomotion or Behavior , 2015, Animals : an open access journal from MDPI.

[66]  Bart Sonck,et al.  Variables of gait inconsistency outperform basic gait variables in detecting mildly lame cows , 2015 .

[67]  Claudia Arcidiacono,et al.  The automatic detection of dairy cow feeding and standing behaviours in free-stall barns by a computer vision-based system , 2015 .

[68]  John Moran,et al.  Cow Talk: Understanding Dairy Cow Behaviour to Improve Their Welfare on Asian Farms , 2015 .

[69]  Greg Bishop-Hurley,et al.  Dynamic cattle behavioural classification using supervised ensemble classifiers , 2015, Comput. Electron. Agric..

[70]  T. Knowles,et al.  Infrared thermometry for lesion monitoring in cattle lameness , 2014, Veterinary Record.

[71]  R. Bro,et al.  Lameness detection challenges in automated milking systems addressed with partial least squares discriminant analysis. , 2014, Journal of dairy science.

[72]  Stefano Viazzi,et al.  Manual and automatic locomotion scoring systems in dairy cows: a review. , 2014, Preventive veterinary medicine.

[73]  Uri Tasch,et al.  Predictive models of lameness in dairy cows achieve high sensitivity and specificity with force measurements in three dimensions , 2015, Journal of Dairy Research.

[74]  M Pastell,et al.  Short communication: Lameness impairs feeding behavior of dairy cows. , 2014, Journal of dairy science.

[75]  Du-Ming Tsai,et al.  A motion and image analysis method for automatic detection of estrus and mating behavior in cattle , 2014 .

[76]  D. Berckmans,et al.  Precision livestock farming technologies for welfare management in intensive livestock systems. , 2014, Revue scientifique et technique.

[77]  Ephraim Maltz,et al.  Automatic lameness detection based on consecutive 3D-video recordings , 2014 .

[78]  Elisabetta Riva,et al.  Automated measurement of lying behavior for monitoring the comfort and welfare of lactating dairy cows , 2013 .

[79]  C Kamphuis,et al.  Applying additive logistic regression to data derived from sensors monitoring behavioral and physiological characteristics of dairy cows to detect lameness. , 2013, Journal of dairy science.

[80]  Daniel Berckmans,et al.  Analysis of aggressive behaviours of pigs by automatic video recordings , 2013 .

[81]  Richard S Gates,et al.  Machine vision to identify broiler breeder behavior , 2013 .

[82]  M. Haskell,et al.  Short-term temperament tests in beef cattle relate to long-term measures of behavior recorded in the home pen. , 2013, Journal of animal science.

[83]  Daniel Berckmans,et al.  Development of an early warning system for a broiler house using computer vision , 2013 .

[84]  Joachim Krieter,et al.  Principal component analysis for the early detection of mastitis and lameness in dairy cows. , 2013, The Journal of dairy research.

[85]  R M de Mol,et al.  Applicability of day-to-day variation in behavior for the automated detection of lameness in dairy cows. , 2013, Journal of dairy science.

[86]  W Heuwieser,et al.  Technical note: evaluation of data loggers for measuring lying behavior in dairy calves. , 2013, Journal of dairy science.

[87]  Yukinori Tani,et al.  Automatic recognition and classification of cattle chewing activity by an acoustic monitoring method with a single-axis acceleration sensor , 2013 .

[88]  L. Plümer,et al.  Electronic detection of lameness in dairy cows through measuring pedometric activity and lying behavior , 2012 .

[89]  N. Chapinal,et al.  Validation of an automated method to count steps while cows stand on a weighing platform and its application as a measure to detect lameness. , 2012, Journal of dairy science.

[90]  D Berckmans,et al.  Automatic measurement of touch and release angles of the fetlock joint for lameness detection in dairy cattle using vision techniques. , 2012, Journal of dairy science.

[91]  Daniel Berckmans,et al.  Online lameness detection in dairy cattle using Body Movement Pattern (BMP) , 2011, 2011 11th International Conference on Intelligent Systems Design and Applications.

[92]  R. M. Dyer,et al.  Diversity in the magnitude of hind limb unloading occurs with similar forms of lameness in dairy cows. , 2011, The Journal of dairy research.

[93]  Niels Kjølstad Poulsen,et al.  Original paper: Oestrus detection in dairy cows from activity and lying data using on-line individual models , 2011 .

[94]  C. Winckler,et al.  Evaluation of data loggers, sampling intervals, and editing techniques for measuring the lying behavior of dairy cattle. , 2010, Journal of Dairy Science.

[95]  P. Thornton.  Livestock production: recent trends, future prospects , 2010, Philosophical Transactions of the Royal Society B: Biological Sciences.

[96]  John F Mee,et al.  Estrus detection and estrus characteristics in housed and pastured Holstein-Friesian cows. , 2010, Theriogenology.

[97]  Daniel Berckmans,et al.  Evaluation of Step Overlap as an Automatic Measure in Dairy Cow Locomotion , 2010 .

[98]  N. Chapinal,et al.  Automated methods for detecting lameness and measuring analgesia in dairy cattle. , 2010, Journal of dairy science.

[99]  J. Rushen,et al.  Measures of weight distribution of dairy cows to detect lameness and the presence of hoof lesions. , 2010, Journal of dairy science.

[100]  B. Butt,et al.  Seasonal space-time dynamics of cattle behavior and mobility among Maasai pastoralists in semi-arid Kenya. , 2010 .

[101]  M. Pastell,et al.  A wireless accelerometer system with wavelet analysis for assessing lameness in cattle. , 2009 .

[102]  Joachim Krieter,et al.  Mastitis and lameness detection in dairy cows by application of fuzzy logic , 2009 .

[103]  A. Lawrence,et al.  Consistency of aggressive feeding behaviour in dairy cows , 2009 .

[104]  D. Weary,et al.  Using gait score, walking speed, and lying behavior to detect hoof lesions in dairy cows. , 2009, Journal of dairy science.

[105]  M. Kolehmainen,et al.  Cow behaviour pattern recognition using a three-dimensional accelerometer and support vector machines , 2009 .

[106]  Peter I. Corke,et al.  Monitoring Animal Behaviour and Environmental Interactions Using Wireless Sensor Networks, GPS Collars and Satellite Remote Sensing , 2009, Sensors.

[107]  Daniel Berckmans,et al.  Automatic detection of lameness in dairy cattle-Vision-based trackway analysis in cow's locomotion , 2008 .

[108]  M. Rivera-Ferre,et al.  The future of agriculture , 2008, EMBO reports.

[109]  M. Coffey,et al.  Changes in feeding behavior as possible indicators for the automatic monitoring of health disorders in dairy cows. , 2008, Journal of dairy science.

[110]  Lotta Rydhmer,et al.  Aggressive and sexual behaviour of growing and finishing pigs reared in groups, without castration , 2006 .

[111]  P. Rajala-Schultz,et al.  Association between subjective lameness grade and kinetic gait parameters in horses with experimentally induced forelimb lameness. , 2005, American journal of veterinary research.

[112]  Bas Kemp,et al.  Pedometer readings for estrous detection and as predictor for time of ovulation in dairy cattle. , 2005, Theriogenology.

[113]  Lars Schrader,et al.  A new method to measure behavioural activity levels in dairy cows , 2003 .

[114]  W M Sischo,et al.  Using time-lapse video photography to assess dairy cattle lying behavior in a free-stall barn. , 2002, Journal of dairy science.

[115]  C. Winckler,et al.  The Reliability and Repeatability of a Lameness Scoring System for Use as an Indicator of Welfare in Dairy Cattle , 2001 .

[116]  R. Huirne,et al.  Economic losses due to clinical lameness in dairy cattle , 1997 .

[117]  D. Sprecher,et al.  A lameness scoring system that uses posture and gait to predict dairy cattle reproductive performance. , 1997, Theriogenology.

[118]  Rony Geers,et al.  Electronic Identification, Monitoring and Tracking of Animals , 1997 .

[119]  Huaibo Song,et al.  Using a CNN-LSTM for basic behaviors detection of a single dairy cow in a complex environment , 2021, Comput. Electron. Agric..

[120]  S. Sukkarieh,et al.  BiGRU-Attention Based Cow Behavior Classification Using Video Data for Precision Livestock Farming , 2021, Transactions of the ASABE.

[121]  Yiannis Ampatzidis,et al.  Horse foraging behavior detection using sound recognition techniques and artificial intelligence , 2021, Comput. Electron. Agric..

[122]  Greg Bishop-Hurley,et al.  A sensor-based solution to monitor grazing cattle drinking behaviour and water intake , 2020, Comput. Electron. Agric..

[123]  Dongjian He,et al.  Lameness detection of dairy cows based on the YOLOv3 deep learning algorithm and a relative step size characteristic vector , 2020 .

[124]  Salah Sukkarieh,et al.  Individual Cattle Identification Using a Deep Learning Based Framework , 2019, IFAC-PapersOnLine.

[125]  Mark Trotter,et al.  The role of interoperable data standards in precision livestock farming in extensive livestock systems: A review , 2019, Comput. Electron. Agric..

[126]  Wouter Saeys,et al.  Farm-specific economic value of automatic lameness detection systems in dairy cattle: From concepts to operational simulations. , 2018, Journal of dairy science.

[127]  Ž. Mihaljević,et al.  Influence of an enriched environment on aggressive behaviour in beef cattle. , 2018 .

[128]  K. Abdul Jabbar,et al.  Early and non-intrusive lameness detection in dairy cows using 3-dimensional video , 2017 .

[129]  Thomas Banhazi,et al.  A brief review of the application of machine vision in livestock behaviour analysis. , 2016 .

[130]  Greg Bishop-Hurley,et al.  Behavioral classification of data from collars containing motion sensors in grazing cattle , 2015, Comput. Electron. Agric..

[131]  S. Viazzi,et al.  Comparison of locomotion scoring for dairy cows by experienced and inexperienced raters using live or video observation methods. , 2015 .

[132]  Daniel Berckmans,et al.  Comparison of a three-dimensional and two-dimensional camera system for automated measurement of back posture in dairy cows , 2014 .

[133]  W. McCormick,et al.  Digital Infrared Thermal Imaging and manual lameness scoring as a means for lameness detection in cattle , 2014 .

[134]  Z. E. Barker,et al.  Investigating the value dairy farmers place on a reduction of lameness in their herds using a willingness to pay approach. , 2014, Veterinary journal.

[135]  D Berckmans,et al.  Analysis of individual classification of lameness using automatic measurement of back posture in dairy cattle. , 2013, Journal of dairy science.

[136]  Yin Ling,et al.  Design of system for monitoring dairy cattle's behavioral features based on wireless sensor networks. , 2010 .

[137]  Frede Aakmann Tøgersen,et al.  Evaluation of a lameness scoring system for dairy cows. , 2008, Journal of dairy science.

[138]  J. Jago,et al.  Validation of a technology for objectively measuring behaviour in dairy cows and its application for oestrous detection , 2007 .

[139]  Keith M. Kendrick,et al.  Intelligent perception , 1998 .