Towards human performance on automatic motion tracking of infant spontaneous movements

Assessment of spontaneous movements can predict the long-term developmental outcomes in high-risk infants. In order to develop algorithms for automated prediction of later function based on early motor repertoire, high-precision tracking of segments and joints are required. Four types of convolutional neural networks were investigated on a novel infant pose dataset, covering the large variation in 1 424 videos from a clinical international community. The precision level of the networks was evaluated as the deviation between the estimated keypoint positions and human expert annotations. The computational efficiency was also assessed to determine the feasibility of the neural networks in clinical practice. The study shows that the precision of the best performing infant motion tracker is similar to the inter-rater error of human experts, while still operating efficiently. In conclusion, the proposed tracking of infant movements can pave the way for early detection of motor disorders in children with perinatal brain injuries by quantifying infant movements from video recordings with human precision.

[1]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  C. Einspieler,et al.  Is the Early Development of Girls with Rett Disorder Really Normal? , 2005, Pediatric Research.

[3]  M. Braga,et al.  Exploratory Data Analysis , 2018, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[4]  L. Doyle,et al.  The Baby Moves smartphone app for General Movements Assessment: Engagement amongst extremely preterm and term‐born infants in a state‐wide geographical study , 2018, Journal of paediatrics and child health.

[5]  Andrea Guzzetta,et al.  The Pooled Diagnostic Accuracy of Neuroimaging, General Movements, and Neurological Examination for Diagnosing Cerebral Palsy Early in High-Risk Infants: A Case Control Study , 2019, Journal of clinical medicine.

[6]  Robert Peharz,et al.  Fidgety movements - tiny in appearance, but huge in impact. , 2016, Jornal de pediatria.

[7]  Giovanni Maria Farinella,et al.  On the Estimation of Children's Poses , 2017, ICIAP.

[8]  Jianbo Shi,et al.  Learning Temporal Pose Estimation from Sparsely-Labeled Videos , 2019, NeurIPS.

[9]  Rich Caruana,et al.  Model compression , 2006, KDD '06.

[10]  Lars Adde,et al.  Approaching human precision on automatic markerless tracking of human movements , 2020 .

[11]  Jeff Sigafoos,et al.  Highlighting the first 5 months of life: General movements in infants later diagnosed with autism spectrum disorder or Rett Syndrome. , 2014, Research in autism spectrum disorders.

[12]  Giovanni Cioni,et al.  Prechtl's Method on the Qualitative Assessment of General Movements in Preterm, Term and Young Infants , 2004 .

[13]  A. Jensenius,et al.  Early prediction of cerebral palsy by computer‐based video analysis of general movements: a feasibility study , 2010, Developmental medicine and child neurology.

[14]  Erich Elsen,et al.  Fast Sparse ConvNets , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Bernt Schiele,et al.  2D Human Pose Estimation: New Benchmark and State of the Art Analysis , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Sarah McIntyre,et al.  Early, Accurate Diagnosis and Early Intervention in Cerebral Palsy: Advances in Diagnosis and Treatment , 2017, JAMA pediatrics.

[17]  Michael Arens,et al.  Computer Vision for Medical Infant Motion Analysis: State of the Art and RGB-D Data Set , 2018, ECCV Workshops.

[18]  Deborah Gaebler-Spira,et al.  The Predictive Accuracy of the General Movement Assessment for Cerebral Palsy: A Prospective, Observational Study of High-Risk Infants in a Clinical Follow-Up Setting , 2019, Journal of clinical medicine.

[19]  Giovanni Cioni,et al.  Cerebral Palsy: Early Markers of Clinical Phenotype and Functional Outcome , 2019, Journal of clinical medicine.

[20]  Bernt Schiele,et al.  DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model , 2016, ECCV.

[21]  Deborah Gaebler-Spira,et al.  Machine Learning of Infant Spontaneous Movements for the Early Prediction of Cerebral Palsy: A Multi-Site Cohort Study , 2019, Journal of clinical medicine.

[22]  Xiaogang Wang,et al.  Learning Feature Pyramids for Human Pose Estimation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  Dong Liu,et al.  Deep High-Resolution Representation Learning for Human Pose Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Pietro Perona,et al.  Benchmarking and Error Diagnosis in Multi-instance Pose Estimation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[25]  Helen Loeb,et al.  Computer vision to automatically assess infant neuromotor risk , 2019, bioRxiv.

[26]  Daniel Groos,et al.  Infant Body Part Tracking in Videos Using Deep Learning - Facilitating Early Detection of Cerebral Palsy , 2018 .

[27]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[28]  Christa Einspieler,et al.  Prechtl's assessment of general movements: a diagnostic tool for the functional assessment of the young nervous system. , 2005, Mental retardation and developmental disabilities research reviews.

[29]  Koji Shimatani,et al.  Evaluation of Fidgety Movements of Infants Based on Gestalt Perception Reflects Differences in Limb Movement Trajectory Curvature. , 2019, Physical therapy.

[30]  Tom Chau,et al.  Detection of Atypical and Typical Infant Movements using Computer-based Video Analysis , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[31]  Heri Ramampiaro,et al.  EfficientPose: Scalable single-person pose estimation , 2020, ArXiv.

[32]  Alexander Refsum Jensenius,et al.  Computer-based video analysis identifies infants with absence of fidgety movements , 2017, Pediatric Research.

[33]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Fabrizio Ferrari,et al.  Motor and Postural Patterns Concomitant with General Movements Are Associated with Cerebral Palsy at Term and Fidgety Age in Preterm Infants , 2019, Journal of clinical medicine.

[35]  Ying Wu,et al.  Deeply Learned Compositional Models for Human Pose Estimation , 2018, ECCV.

[36]  Øyvind Stavdahl,et al.  Weakly supervised motion segmentation with particle matching , 2015, Comput. Vis. Image Underst..

[37]  Greg Mori,et al.  CLIP-Q: Deep Network Compression Learning by In-parallel Pruning-Quantization , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.