Three-Dimensional Pose Estimation for Laboratory Mouse From Monocular Images

Video-based activity and behavior analysis of mice has garnered wide attention in biomedical research. Animal facilities hold large numbers of mice housed in “home-cages” densely stored within ventilated racks. Automated analysis of mice activity in their home-cages can provide a new set of sensitive measures for detecting abnormalities and time-resolved deviation from the baseline behavior. Large-scale monitoring in animal facilities requires minimal footprint hardware that integrates seamlessly with the ventilated racks. The compactness of hardware imposes the use of fisheye lenses positioned in close proximity to the cage. In this paper, we propose a systematic approach to accurately estimate the 3D pose of the mouse from single-monocular fisheye-distorted images. Our approach employs a novel adaptation of a structured forest algorithm. We benchmark our algorithm against existing methods. We demonstrate the utility of the pose estimates in predicting mouse behavior in a continuous video.

[1]  L P Noldus,et al.  EthoVision: A versatile video tracking system for automation of behavioral experiments , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[2]  A. Pérez-Escudero,et al.  idTracker: tracking individuals in a group by automatic identification of unmarked animals , 2014, Nature Methods.

[3]  Rowland R. Sillito,et al.  Assessing mouse behaviour throughout the light/dark cycle using automated in-cage analysis tools , 2017, Journal of Neuroscience Methods.

[4]  Monson H. Hayes,et al.  Scalable Vision System for Mouse Homecage Ethology , 2016, ACIVS.

[5]  A. Cressant,et al.  Computerized video analysis of social interactions in mice , 2012, Nature Methods.

[6]  Yaser Sheikh,et al.  Structure from Recurrent Motion: From Rigidity to Recurrency , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Kristin Branson,et al.  Machine vision methods for analyzing social interactions , 2017, Journal of Experimental Biology.

[8]  Serge Belongie,et al.  Three Brown Mice : See How They Run , 2003 .

[9]  Jian Sun,et al.  Face Alignment by Explicit Shape Regression , 2012, International Journal of Computer Vision.

[10]  Adam Claridge‐Chang,et al.  The surveillance state of behavioral automation , 2012, Current Opinion in Neurobiology.

[11]  Pietro Perona,et al.  The Fastest Pedestrian Detector in the West , 2010, BMVC.

[12]  Andrew D. Steele,et al.  The power of automated high-resolution behavior analysis revealed by its application to mouse models of Huntington's and prion diseases , 2007, Proceedings of the National Academy of Sciences.

[13]  C. Lawrence Zitnick,et al.  Fast Edge Detection Using Structured Forests , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Kourosh Zarringhalam,et al.  An open system for automatic home-cage behavioral analysis and its application to male and female mouse models of Huntington's disease , 2012, Behavioural Brain Research.

[15]  Ronald Poppe,et al.  Vision-based human motion analysis: An overview , 2007, Comput. Vis. Image Underst..

[16]  Xavier P. Burgos-Artizzu,et al.  Three dimensional pose estimation of mouse from monocular images in compact systems , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[17]  Claire Richardson,et al.  The power of automated behavioural homecage technologies in characterizing disease progression in laboratory mice: A review , 2015 .

[18]  O. Feinerman,et al.  Automated long-term tracking and social behavioural phenotyping of animal colonies within a semi-natural environment , 2013, Nature Communications.

[19]  P. Perona,et al.  utomated multi-day tracking of marked mice for the analysis of ocial behaviour , 2013 .

[20]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[21]  Kristin Branson,et al.  Computational Analysis of Behavior. , 2016, Annual review of neuroscience.

[22]  Pietro Perona,et al.  Improved Multi-Class Cost-Sensitive Boosting via Estimation of the Minimum-Risk Class , 2016, ArXiv.

[23]  James D. Malley,et al.  SCORHE: A novel and practical approach to video monitoring of laboratory mice housed in vivarium cage racks , 2015, Behavior research methods.

[24]  Guillaume-Alexandre Bilodeau,et al.  Catching a Rat by Its Edglets , 2013, IEEE Transactions on Image Processing.

[25]  Kristin Branson,et al.  JAABA: interactive machine learning for automatic annotation of animal behavior , 2013, Nature Methods.

[26]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[27]  Thomas Serre,et al.  Automated home-cage behavioural phenotyping of mice. , 2010, Nature communications.

[28]  Pietro Perona,et al.  Social behavior recognition in continuous video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[30]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[31]  Pietro Perona,et al.  Cascaded pose regression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Antonio Criminisi,et al.  Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning , 2012, Found. Trends Comput. Graph. Vis..

[33]  Xavier P. Burgos-Artizzu,et al.  Cascaded regression for 3D pose estimation for mouse in fisheye lens distorted monocular images , 2016, 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[34]  Pietro Perona,et al.  Merging Pose Estimates Across Space and Time , 2013, BMVC.

[35]  Pietro Perona,et al.  Robust Face Landmark Estimation under Occlusion , 2013, 2013 IEEE International Conference on Computer Vision.

[36]  Monitoring technologies and genetic engineering are producing a growing array of animal models for psychiatric disorders , .

[37]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[38]  Vittorio Murino,et al.  Automatic Visual Tracking and Social Behaviour Analysis with Multiple Mice , 2013, PloS one.

[39]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[40]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  David J. Anderson,et al.  Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning , 2015, Proceedings of the National Academy of Sciences.

[42]  Pietro Perona,et al.  Automated image-based tracking and its application in ecology. , 2014, Trends in ecology & evolution.

[43]  Wenjun Zeng,et al.  Online Human Action Detection using Joint Classification-Regression Recurrent Neural Networks , 2016, ECCV.

[44]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Serge J. Belongie,et al.  Tracking multiple mouse contours (without too many samples) , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).