A novel method for robust markerless tracking of rodent paws in 3D

Studying animal locomotion improves our understanding of motor control and aids in the treatment of motor impairment. Mice are a premier model of human disease and are the model system of choice for much of basic neuroscience. Placement of the tips of appendages, here paws, is typically critical for locomotion. Tracking paws from a video is difficult, however, due to frequent occlusions and collisions. We propose a method and provide software to track the paws of rodents. We use a superpixel-based method to segment the paws, direct linear transform to perform 3D reconstruction, a 3D Kalman filter (KF) to solve the matching problem and label paws across frames, and spline fits through time to resolve common collisions. The automated method was compared to manual tracking. The method had an average of 2.54 mistakes requiring manual correction per 1000 frames with a maximum of 5.29 possible errors while these values were estimates of the expected errors. We present an algorithm and its implementation to track the paws of running rodents. This algorithm can be applied to different animals as long as the tips of the legs can be differentiated from the background and other parts of the body using color features. The presented algorithm provides a robust tool for future studies in multiple fields, where precise quantification of locomotor behavior from a high-speed video is required. We further present a graphical user interface (GUI) to track, visualize, and edit the tracking data.

[1]  Willias Masocha,et al.  Gait analysis of C57BL/6 mice with complete Freund's adjuvant-induced arthritis using the CatWalk system , 2013, BMC Musculoskeletal Disorders.

[2]  Ryan P. Adams,et al.  Mapping Sub-Second Structure in Mouse Behavior , 2015, Neuron.

[3]  Mandyam V. Srinivasan,et al.  FicTrac: A visual method for tracking spherical motion and generating fictive animal paths , 2014, Journal of Neuroscience Methods.

[4]  J. Riddell,et al.  Gait Analysis in a Mecp2 Knockout Mouse Model of Rett Syndrome Reveals Early-Onset and Progressive Motor Deficits , 2014, PloS one.

[5]  Matthias Endres,et al.  Assessment of Paclitaxel Induced Sensory Polyneuropathy with “Catwalk” Automated Gait Analysis in Mice , 2013, PloS one.

[6]  Emmanuelle Gouillart,et al.  scikit-image: image processing in Python , 2014, PeerJ.

[7]  Rozhin Penjweini,et al.  Investigating the effect of poly‐l‐lactic acid nanoparticles carrying hypericin on the flow‐biased diffusive motion of HeLa cell organelles , 2019, The Journal of pharmacy and pharmacology.

[8]  Americo A. Migliaccio,et al.  Characterization of the 3D angular vestibulo-ocular reflex in C57BL6 mice , 2011, Experimental Brain Research.

[9]  Bonaventure Intercontinental,et al.  ON DECISION AND CONTROL , 1985 .

[10]  Omid Haji Maghsoudi,et al.  Matlab software for impedance spectroscopy designed for neuroscience applications , 2018, Journal of Neuroscience Methods.

[11]  Nassir Navab,et al.  3D Pictorial Structures Revisited: Multiple Human Pose Estimation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  O. Haji Maghsoudi,et al.  A Rodent Paw Tracker Using Support Vector Machine , 2016, 2016 IEEE Signal Processing in Medicine and Biology Symposium (SPMB).

[13]  B. Roth DREADDs for Neuroscientists , 2016, Neuron.

[14]  Alireza Talebpour,et al.  Detection of small bowel tumor in wireless capsule endoscopy images using an adaptive neuro-fuzzy inference system , 2017, Journal of biomedical research.

[15]  Andrew A Biewener,et al.  Morphological and kinematic basis of the hummingbird flight stroke: scaling of flight muscle transmission ratio , 2012, Proceedings of the Royal Society B: Biological Sciences.

[16]  Richard A. Brown,et al.  Introduction to random signals and applied kalman filtering (3rd ed , 2012 .

[17]  Qi Xu,et al.  A Video Tracking System for Limb Motion Measurement in Small Animals , 2010, 2010 International Conference on Optoelectronics and Image Processing.

[18]  Tyson L Hedrick,et al.  Software techniques for two- and three-dimensional kinematic measurements of biological and biomimetic systems , 2008, Bioinspiration & biomimetics.

[19]  S. Grillner,et al.  Neuronal Control of Locomotion 'From Mollusc to Man ' , 1999 .

[20]  Ian D. Reid,et al.  Joint Probabilistic Data Association Revisited , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  H. Hatze,et al.  High-precision three-dimensional photogrammetric calibration and object space reconstruction using a modified DLT-approach. , 1988, Journal of biomechanics.

[22]  Andre Kyme,et al.  Markerless Motion Tracking of Awake Animals in Positron Emission Tomography , 2014, IEEE Transactions on Medical Imaging.

[23]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  K. A. Clarke,et al.  Gait Analysis in the Mouse , 1999, Physiology & Behavior.

[25]  R. Nitsch,et al.  High-speed video gait analysis reveals early and characteristic locomotor phenotypes in mouse models of neurodegenerative movement disorders , 2016, Behavioural Brain Research.

[26]  Yongdong Zhang,et al.  STAT: Spatial-Temporal Attention Mechanism for Video Captioning , 2020, IEEE Transactions on Multimedia.

[27]  Pedro Melo-Pinto,et al.  Effect of skin movement on the analysis of hindlimb kinematics during treadmill locomotion in rats , 2006, Journal of Neuroscience Methods.

[28]  H. Soltanian-Zadeh,et al.  Automatic organs' detection in WCE , 2012, The 16th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP 2012).

[29]  Omid Haji Maghsoudi,et al.  Open-source Python software for analysis of 3D kinematics from quadrupedal animals , 2019, Biomed. Signal Process. Control..

[30]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[31]  Paria Mehrani,et al.  Superpixels and Supervoxels in an Energy Optimization Framework , 2010, ECCV.

[32]  S. Grillner,et al.  Neuronal Control of LocomotionFrom Mollusc to Man , 1999 .

[33]  Michael H. Dickinson,et al.  Multi-camera real-time three-dimensional tracking of multiple flying animals , 2010, Journal of The Royal Society Interface.

[34]  J. Guckenheimer,et al.  Finding the dimension of slow dynamics in a rhythmic system , 2012, Journal of The Royal Society Interface.

[35]  Travis E. Oliphant,et al.  Python for Scientific Computing , 2007, Computing in Science & Engineering.

[36]  T. Hedrick,et al.  Three-dimensional flow and lift characteristics of a hovering ruby-throated hummingbird , 2014, Journal of The Royal Society Interface.

[37]  Omid Haji Maghsoudi,et al.  A computer aided method to detect bleeding, tumor, and disease regions in Wireless Capsule Endoscopy , 2016, 2016 IEEE Signal Processing in Medicine and Biology Symposium (SPMB).

[38]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Omid Haji Maghsoudi,et al.  Application of Superpixels to Segment Several Landmarks in Running Rodents , 2018, Pattern Recognition and Image Analysis.

[40]  Cor J. Veenman,et al.  Resolving Motion Correspondence for Densely Moving Points , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Margrit Betke,et al.  A protocol and calibration method for accurate multi-camera field videography , 2014, Journal of Experimental Biology.

[42]  Andrew J. Spence,et al.  Closing the loop in legged neuromechanics: An open-source computer vision controlled treadmill , 2013, Journal of Neuroscience Methods.

[43]  Omid Haji Maghsoudi,et al.  Superpixels based marker tracking vs. hue thresholding in rodent biomechanics application , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[44]  Johann Reger,et al.  Constructing predictive models of human running , 2015, Journal of The Royal Society Interface.

[45]  P. Carlson-Kuhta,et al.  Forms of forward quadrupedal locomotion. II. A comparison of posture, hindlimb kinematics, and motor patterns for upslope and level walking. , 1998, Journal of neurophysiology.

[46]  O. HajiMaghsoudi,et al.  Automatic informative tissue's discriminators in WCE , 2012, 2012 IEEE International Conference on Imaging Systems and Techniques Proceedings.

[47]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  S. Frizelle,et al.  A comparison of DigiGait™ and TreadScan™ imaging systems: assessment of pain using gait analysis in murine monoarthritis , 2013, Journal of pain research.

[49]  H. Okano,et al.  Long-Term Safety Issues of iPSC-Based Cell Therapy in a Spinal Cord Injury Model: Oncogenic Transformation with Epithelial-Mesenchymal Transition , 2015, Stem cell reports.

[50]  Omid Haji Maghsoudi,et al.  Three‐dimensional‐based landmark tracker employing a superpixels method for neuroscience, biomechanics, and biology studies , 2019, Int. J. Imaging Syst. Technol..

[51]  Thomas R Oxland,et al.  Improved RSA accuracy with DLT and balanced calibration marker distributions with an assessment of initial-calibration. , 2003, Journal of biomechanics.

[52]  Joel C. Glover,et al.  A Neonatal Mouse Spinal Cord Injury Model for Assessing Post-Injury Adaptive Plasticity and Human Stem Cell Integration , 2013, PloS one.

[53]  James M. Rehg,et al.  Multiple Hypothesis Tracking Revisited , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[54]  Antoni B. Chan,et al.  Counting People Crossing a Line Using Integer Programming and Local Features , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[55]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[56]  Pavel Zemcík,et al.  Absolute pose estimation from line correspondences using direct linear transformation , 2016, Comput. Vis. Image Underst..

[57]  Tasos Karakostas,et al.  Three-dimensional rodent motion analysis and neurodegenerative disorders , 2014, Journal of Neuroscience Methods.

[58]  W B Veldhuis,et al.  Automated quantitative gait analysis during overground locomotion in the rat: its application to spinal cord contusion and transection injuries. , 2001, Journal of neurotrauma.

[59]  M. Marcus,et al.  The CatWalk gait analysis in assessment of both dynamic and static gait changes after adult rat sciatic nerve resection , 2007, Journal of Neuroscience Methods.