Optical Mouse: 3D Mouse Pose From Single-View Video

We present a method to infer the 3D pose of mice, including the limbs and feet, from monocular videos. Many human clinical conditions and their corresponding animal models result in abnormal motion, and accurately measuring 3D motion at scale offers insights into health. The 3D poses improve classification of health-related attributes over 2D representations. The inferred poses are accurate enough to estimate stride length even when the feet are mostly occluded. This method could be applied as part of a continuous monitoring system to non-invasively measure animal health.

[1]  Yu Xu,et al.  Gait Assessment of Pain and Analgesics: Comparison of the DigiGait™ and CatWalk™ Gait Imaging Systems , 2019, Neuroscience Bulletin.

[2]  R. Deacon.  Measuring Motor Coordination in Mice , 2013, Journal of visualized experiments : JoVE.

[3]  Ryan P. Adams,et al.  Mapping Sub-Second Structure in Mouse Behavior , 2015, Neuron.

[4]  橘 敏明,et al.  Open-Field Testの妥当性と信頼性 , 1983 .

[5]  Claire Richardson,et al.  The power of automated behavioural homecage technologies in characterizing disease progression in laboratory mice: A review , 2015 .

[6]  Cordelia Schmid,et al.  Learning from Synthetic Humans , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Jitendra Malik,et al.  Learning 3D Human Dynamics From Video , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Thomas Serre,et al.  Automated home-cage behavioural phenotyping of mice. , 2010, Nature communications.

[10]  L P Noldus,et al.  EthoVision: A versatile video tracking system for automation of behavioral experiments , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[11]  David J. Anderson,et al.  Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning , 2015, Proceedings of the National Academy of Sciences.

[12]  Rowland R. Sillito,et al.  Assessing mouse behaviour throughout the light/dark cycle using automated in-cage analysis tools , 2017, Journal of Neuroscience Methods.

[13]  Kristin Branson,et al.  JAABA: interactive machine learning for automatic annotation of animal behavior , 2013, Nature Methods.

[14]  Ersin Yumer,et al.  Self-supervised Learning of Motion Capture , 2017, NIPS.

[15]  Xiaowei Zhou,et al.  Learning to Estimate 3D Human Pose and Shape from a Single Color Image , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Andrew Zisserman,et al.  Exploiting Temporal Context for 3D Human Pose Estimation in the Wild , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Peter V. Gehler,et al.  Unite the People: Closing the Loop Between 3D and 2D Human Representations , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Roger Zimmermann,et al.  Towards Natural and Accurate Future Motion Prediction of Humans and Animals , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Alison L. Sheets,et al.  Quantitative Evaluation of 3D Mouse Behaviors and Motor Function in the Open-Field after Spinal Cord Injury Using Markerless Motion Tracking , 2013, PloS one.

[20]  Jitendra Malik,et al.  Tracking people with twists and exponential maps , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[21]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[22]  Ryan P. Adams,et al.  Composing graphical models with neural networks for structured representations and fast inference , 2016, NIPS.

[23]  Kevin M. Cury,et al.  DeepLabCut: markerless pose estimation of user-defined body parts with deep learning , 2018, Nature Neuroscience.

[24]  Mikhail Kislin,et al.  Fast animal pose estimation using deep neural networks , 2018, Nature Methods.

[25]  Cristian Sminchisescu,et al.  Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes: The Importance of Multiple Scene Constraints , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Jonathan Krynitsky,et al.  Three-Dimensional Pose Estimation for Laboratory Mouse From Monocular Images , 2019, IEEE Transactions on Image Processing.

[27]  Peter V. Gehler,et al.  Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image , 2016, ECCV.

[28]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[29]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[30]  Varun Ramakrishna,et al.  Convolutional Pose Machines , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Maren L Mahowald,et al.  A comparison of DigiGait™ and TreadScan™ imaging systems: assessment of pain using gait analysis in murine monoarthritis , 2014, Journal of pain research.