InfiniteForm: A synthetic, minimal bias dataset for fitness applications

The growing popularity of remote fitness has increased the demand for highly accurate computer vision models that track human poses. However, the best methods still fail in many real-world fitness scenarios, suggesting that there is a domain gap between current datasets and real-world fitness data. To enable the field to address fitness-specific vision problems, we created InfiniteForm – an opensource synthetic dataset of 60k images with diverse fitness poses (15 categories), both singleand multi-person scenes, and realistic variation in lighting, camera angles, and occlusions. As a synthetic dataset, InfiniteForm offers minimal bias in body shape and skin tone, and provides pixel-perfect labels for standard annotations like 2D keypoints, as well as those that are difficult or impossible for humans to produce like depth and occlusion. In addition, we introduce a novel generative procedure for creating diverse synthetic poses from predefined exercise categories. This generative process can be extended to any application where pose diversity is needed to train robust computer vision models.

[1]  S. Tamang,et al.  Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data , 2018, JAMA internal medicine.

[2]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[3]  Richard R. Yang,et al.  Pose Trainer: Correcting Exercise Posture using Pose Estimation , 2020, ArXiv.

[4]  Bernt Schiele,et al.  2D Human Pose Estimation: New Benchmark and State of the Art Analysis , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Cordelia Schmid,et al.  Learning from Synthetic Humans , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Chen Chen,et al.  Recognizing Exercises and Counting Repetitions in Real Time , 2020, ArXiv.

[7]  Dimitrios Tzionas,et al.  Expressive Body Capture: 3D Hands, Face, and Body From a Single Image , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Brian W. Powers,et al.  Dissecting racial bias in an algorithm used to manage the health of populations , 2019, Science.

[9]  Chandra Kambhamettu,et al.  Estimating Physical Activity Intensity And Energy Expenditure Using Computer Vision On Videos , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[10]  Iasonas Kokkinos,et al.  DensePose: Dense Human Pose Estimation in the Wild , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Michael J. Black,et al.  VIBE: Video Inference for Human Body Pose and Shape Estimation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Salem Alelyani,et al.  Detection and Evaluation of Machine Learning Bias , 2021, Applied Sciences.

[13]  Luís C. Lamb,et al.  Assessing gender bias in machine translation: a case study with Google Translate , 2018, Neural Computing and Applications.

[14]  Yuta Nakashima,et al.  Yoga-82: A New Dataset for Fine-grained Classification of Human Poses , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  T. Fitzpatrick.  The validity and practicality of sun-reactive skin types I through VI. , 1988, Archives of dermatology.

[16]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[17]  Susan Leavy,et al.  Uncovering gender bias in newspaper coverage of Irish politicians using machine learning , 2018, Digit. Scholarsh. Humanit..

[18]  Joachim Tesch,et al.  AGORA: Avatars in Geography Optimized for Regression Analysis , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Mark Everingham,et al.  Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation , 2010, BMVC.

[20]  A. Garbett,et al.  Towards Understanding People’s Experiences of AI Computer Vision Fitness Instructor Apps , 2021, Conference on Designing Interactive Systems.