Multi-Pig Part Detection and Association with a Fully-Convolutional Network

Computer vision systems have the potential to provide automated, non-invasive monitoring of livestock animals, however, the lack of public datasets with well-defined targets and evaluation metrics presents a significant challenge for researchers. Consequently, existing solutions often focus on achieving task-specific objectives using relatively small, private datasets. This work introduces a new dataset and method for instance-level detection of multiple pigs in group-housed environments. The method uses a single fully-convolutional neural network to detect the location and orientation of each animal, where both body part locations and pairwise associations are represented in the image space. Accompanying this method is a new dataset containing 2000 annotated images with 24,842 individually annotated pigs from 17 different locations. The proposed method achieves over 99% precision and over 96% recall when detecting pigs in environments previously seen by the network during training. To evaluate the robustness of the trained network, it is also tested on environments and lighting conditions unseen in the training set, where it achieves 91% precision and 67% recall. The dataset is publicly available for download.

[1]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Claudia Arcidiacono,et al.  Localisation and identification performances of a real-time location system based on ultra wide band technology for monitoring and tracking dairy cow behaviour in a semi-open free-stall barn , 2014 .

[4]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[6]  G. Rosa,et al.  A novel automated system to acquire biometric and morphological measurements and predict body weight of pigs via 3D computer vision. , 2018, Journal of animal science.

[7]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[8]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  B.M. Cuezva,et al.  UWB MAC and network solutions for low data rate with location and tracking applications , 2005, 2005 IEEE International Conference on Ultra-Wideband.

[10]  Yongwha Chung,et al.  Individual Pig Detection Using Kinect Depth Information , 2016 .

[11]  Wei Li,et al.  Validity of the Microsoft Kinect sensor for assessment of normal walking patterns in pigs , 2015, Comput. Electron. Agric..

[12]  K. J. Ellis,et al.  Cattle health monitoring using wireless sensor networks , 2004 .

[13]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[14]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[15]  R. D’Eath,et al.  Early indicators of tail biting outbreaks in pigs , 2018, Applied Animal Behaviour Science.

[16]  Raymond E. Floyd,et al.  RFID in Animal-Tracking Applications , 2015, IEEE Potentials.

[17]  Eric Psota,et al.  Tracking of group-housed pigs using multi-ellipsoid expectation maximisation , 2018, IET Comput. Vis..

[18]  Håkan Ardö,et al.  Convolutional neural network-based cow interaction watchdog , 2017, IET Comput. Vis..

[19]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[20]  Pilar Barreiro,et al.  A Review of Wireless Sensor Technologies and Applications in Agriculture and Food Industry: State of the Art and Current Trends , 2009, Sensors.

[21]  Christer Bergsten,et al.  Learning Based Image Segmentation of Pigs in a Pen , 2014 .

[22]  Ilias Kyriazakis,et al.  Automated tracking to measure behavioural changes in pigs for health and welfare monitoring , 2017, Scientific Reports.

[23]  Yongwha Chung,et al.  A Kinect-Based Segmentation of Touching-Pigs for Real-Time Monitoring , 2018, Sensors.

[24]  Andrew J. J. MacIntosh,et al.  Fractal measures in activity patterns: Do gastrointestinal parasites affect the complexity of sheep behaviour? , 2018, Applied Animal Behaviour Science.

[25]  Zetian Fu,et al.  Development and evaluation on a RFID-based traceability system for cattle/beef quality safety in China , 2013 .

[26]  Uwe Richter,et al.  Using machine vision for investigation of changes in pig group lying patterns , 2015, Comput. Electron. Agric..

[27]  Torben Gregersen,et al.  Original papers: Development of a real-time computer vision system for tracking loose-housed pigs , 2011 .

[28]  Jonathan Tompson,et al.  PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model , 2018, ECCV.

[29]  Charalampos Z. Patrikakis,et al.  A complete farm management system based on animal identification using RFID technology , 2010 .

[30]  Raquel Urtasun,et al.  Understanding the Effective Receptive Field in Deep Convolutional Neural Networks , 2016, NIPS.

[31]  Michael D. Johnson,et al.  An Advanced, Low-Cost, GPS-Based Animal Tracking System , 2006 .

[32]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Yongwha Chung,et al.  Depth-Based Detection of Standing-Pigs in Moving Noise Environments , 2017, Sensors.

[34]  Cécile Cornou,et al.  Sow-activity classification from acceleration patterns: A machine learning approach , 2013 .

[35]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Marcella Guarino,et al.  Precision livestock farming : an overview of image and sound labelling , 2013 .

[38]  Ilias Kyriazakis,et al.  Early detection of health and welfare compromises through automated detection of behavioural changes in pigs , 2016, Veterinary journal.

[39]  Wenhan Luo,et al.  Multiple object tracking: A literature review , 2014, Artif. Intell..

[40]  Claudia Bahr,et al.  Automatic monitoring of pig locomotion using image analysis , 2014 .

[41]  Joachim Krieter,et al.  Agonistic behaviour after mixing in pigs under commercial farm conditions , 2011 .

[42]  Luigi di Stefano,et al.  A simple and efficient connected components labeling algorithm , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[43]  Suresh Neethirajan,et al.  Recent advances in wearable sensors for animal health management , 2017 .

[44]  F. Tuyttens,et al.  Observer bias in animal behaviour research: can we believe what we score, if we score what we believe? , 2014, Animal Behaviour.

[45]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[47]  Jinchang Ren,et al.  Automatic Animal Detection from Kinect Sensed Images for Livestock Monitoring and Assessment , 2015, 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing.

[48]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Р Ю Чуйков,et al.  Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector , 2017 .

[51]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[52]  Gérard Lachapelle,et al.  Challenges in the design of a GNSS ear tag for feedlot cattle , 2010 .

[53]  Mac Schwager,et al.  Robust classification of animal tracking data , 2007 .

[54]  Andrea Pezzuolo,et al.  On-barn pig weight estimation based on body measurements by a Kinect v1 depth camera , 2018, Comput. Electron. Agric..

[55]  V. H. Oddy,et al.  Using a three-axis accelerometer to identify and classify sheep behaviour at pasture , 2016 .

[56]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Do-Hyeun Kim,et al.  Animal Situation Tracking Service Using RFID, GPS, and Sensors , 2010, 2010 Second International Conference on Computer and Network Technology.

[58]  Lance C. Pérez,et al.  Health Monitoring of Group-Housed Pigs using Depth-Enabled Multi-Object Tracking , 2016 .

[59]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[60]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[62]  Jørgen Kongsro,et al.  Estimation of pig weight using a Microsoft Kinect prototype imaging system , 2014 .

[63]  Jean-Marie Aerts,et al.  Is precision livestock farming an engineer's daydream or nightmare, an animal's friend or foe, and a farmer's panacea or pitfall? , 2008 .

[64]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[65]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[66]  David Kirk,et al.  NVIDIA cuda software and gpu parallel computing architecture , 2007, ISMM '07.

[67]  Yongwha Chung,et al.  Automatic Recognition of Aggressive Behavior in Pigs Using a Kinect Depth Sensor , 2016, Sensors.

[68]  Kai Liu,et al.  Automatic recognition of lactating sow behaviors through depth image processing , 2016, Comput. Electron. Agric..

[69]  D. Berckmans,et al.  Precision Livestock Farming: An international review of scientific and commercial aspects , 2012 .