Markerless Rat Behavior Quantification With Cascade Neural Network

Quantifying rat behavior through video surveillance is crucial for medicine, neuroscience, and other fields. In this paper, we focus on the challenging problem of estimating landmark points, such as the rat's eyes and joints, only with image processing and quantify the motion behavior of the rat. Firstly, we placed the rat on a special running machine and used a high frame rate camera to capture its motion. Secondly, we designed the cascade convolution network (CCN) and cascade hourglass network (CHN), which are two structures to extract features of the images. Three coordinate calculation methods—fully connected regression (FCR), heatmap maximum position (HMP), and heatmap integral regression (HIR)—were used to locate the coordinates of the landmark points. Thirdly, through a strict normalized evaluation criterion, we analyzed the accuracy of the different structures and coordinate calculation methods for rat landmark point estimation in various feature map sizes. The results demonstrated that the CCN structure with the HIR method achieved the highest estimation accuracy of 75%, which is sufficient to accurately track and quantify rat joint motion.

[1]  Zhen He,et al.  Numerical Coordinate Regression with Convolutional Neural Networks , 2018, ArXiv.

[2]  Cewu Lu,et al.  RMPE: Regional Multi-person Pose Estimation , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[3]  William Singhose,et al.  Comparison of Kinect and Vicon Motion Capture of Upper-Body Joint Angle Tracking , 2018, 2018 IEEE 14th International Conference on Control and Automation (ICCA).

[4]  Xiaogang Wang,et al.  End-to-End Learning of Deformable Mixture of Parts and Deep Convolutional Neural Networks for Human Pose Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  B. Fisher,et al.  Correlates and Analysis of Motor Function in Humans and Animal Models of Parkinson’s Disease , 2011 .

[6]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[7]  O. Feinerman,et al.  Automated long-term tracking and social behavioural phenotyping of animal colonies within a semi-natural environment , 2013, Nature Communications.

[8]  Seungil Ro,et al.  Effect of electrical stimulation on neural regeneration via the p38-RhoA and ERK1/2-Bcl-2 pathways in spinal cord-injured rats , 2018, Neural regeneration research.

[9]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Tong Ran,et al.  Required width of exit to avoid the faster-is-slower effect in highly competitive evacuation , 2017 .

[11]  Varun Ramakrishna,et al.  Convolutional Pose Machines , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Bernt Schiele,et al.  2D Human Pose Estimation: New Benchmark and State of the Art Analysis , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  R. Morris.  Developments of a water-maze procedure for studying spatial learning in the rat , 1984, Journal of Neuroscience Methods.

[14]  Chi Xu,et al.  Mouse Pose Estimation From Depth Images , 2015, ArXiv.

[15]  R N Walsh,et al.  The Open-Field Test: a critical review. , 1976, Psychological bulletin.

[16]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[17]  L. Miller,et al.  Decoding the rat forelimb movement direction from epidural and intracortical field potentials , 2011, Journal of neural engineering.

[18]  Silvestro Micera,et al.  Closed-loop neuromodulation of spinal sensorimotor circuits controls refined locomotion after complete spinal cord injury , 2014, Science Translational Medicine.

[19]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[20]  Kevin M. Cury,et al.  DeepLabCut: markerless pose estimation of user-defined body parts with deep learning , 2018, Nature Neuroscience.

[21]  Jitendra Malik,et al.  Human Pose Estimation with Iterative Error Feedback , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Feng Duan,et al.  Rat Behavior Observation System Based on Transfer Learning , 2019, IEEE Access.

[23]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[24]  Theofano Eftaxiopoulou,et al.  Gait compensations in rats after a temporary nerve palsy quantified using temporo-spatial and kinematic parameters , 2014, Journal of Neuroscience Methods.

[25]  S. Micera,et al.  Decoding bipedal locomotion from the rat sensorimotor cortex , 2015, Journal of neural engineering.

[26]  Omid Haji Maghsoudi,et al.  Superpixels based marker tracking vs. hue thresholding in rodent biomechanics application , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[27]  A. Cressant,et al.  Computerized video analysis of social interactions in mice , 2012, Nature Methods.

[28]  Leonardo Alexandre Peyré-Tartaruga,et al.  Locomotor behaviour assessment in the rat by kinematic analysis , 2006 .

[29]  P. Perona,et al.  utomated multi-day tracking of marked mice for the analysis of ocial behaviour , 2013 .

[30]  S. Micera,et al.  Mechanisms Underlying the Neuromodulation of Spinal Circuits for Correcting Gait and Balance Deficits after Spinal Cord Injury , 2016, Neuron.

[31]  Pietro Perona,et al.  Social behavior recognition in continuous video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Pengyue Zhang,et al.  D1-like dopamine receptor dysfunction in the lateral habenula nucleus increased anxiety-like behavior in rat , 2017, Neuroscience.

[33]  Yichen Wei,et al.  Integral Human Pose Regression , 2017, ECCV.

[34]  Xiaogang Wang,et al.  Structured Feature Learning for Pose Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Guglielmo Foffani,et al.  Spike count, spike timing and temporal information in the cortex of awake, freely moving rats , 2014, Journal of neural engineering.