ExemPoser: Predicting Poses of Experts as Examples for Beginners in Climbing Using a Neural Network

It is important for beginners to imitate poses of experts in various sports; especially in sport climbing, performance depends greatly on the pose that should be taken for given holds. However, it is difficult for beginners to learn the proper poses for all patterns from experts since climbing holds are completely different for each course. Therefore, we propose a system that predict a pose of experts from the positions of the hands and feet of the climber--the positions of holds used by the climber--using a neural network. In other words, our system simulates what pose experts take for the holds the climber is now using. The positions of hands and feet are calculated from a image of the climber captured from behind. To allow users to check what pose is ideal in real time during practice, we have adopted a simple and lightweight network structure with little computational delay. We asked experts to compare the poses predicted by our system with the poses of beginners, and we confirmed that the poses predicted by our system were in most cases better than or as good as those of beginners.

[1]  Kyung-Ryul Chung,et al.  A study on the development of image analysis instrument and estimation of mass, volume and center of gravity using CT image in Korean , 2014 .

[2]  Antonio Krüger,et al.  BouldAR: using augmented reality to support collaborative boulder training , 2013, CHI Extended Abstracts.

[3]  Antonio Krüger,et al.  ClimbVis: Investigating In-situ Visualizations for Understanding Climbing Movements by Demonstration , 2017, ISS.

[4]  Yasutoshi Makino,et al.  Computational Foresight: Forecasting Human Body Motion in Real-time for Reducing Delays in Interactive System , 2017, ISS.

[5]  Florian Daiber,et al.  ClimbSense: Automatic Climbing Route Recognition using Wrist-worn Inertia Measurement Units , 2015, CHI.

[6]  Hiroshi Mizoguchi,et al.  Detecting and modeling play behavior using sensor-embedded rock-climbing equipment , 2010, IDC.

[7]  Hideki Koike,et al.  Real-time human motion forecasting using a RGB camera , 2018, VRST.

[8]  Jan Berg,et al.  Digiwall: an interactive climbing wall , 2005, ACE '05.

[9]  Perttu Hämäläinen,et al.  The Augmented Climbing Wall: High-Exertion Proximity Interaction on a Wall-Sized Interactive Surface , 2016, CHI.

[10]  Paul G. Kry,et al.  Static pose reconstruction with an instrumented bouldering wall , 2012, VRST '12.

[11]  Cassim Ladha,et al.  ClimbAX: skill assessment for climbing enthusiasts , 2013, UbiComp.

[12]  Kourosh Naderi,et al.  Discovering and synthesizing humanoid climbing movements , 2017, ACM Trans. Graph..

[13]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Amin Babadi,et al.  A Reinforcement Learning Approach To Synthesizing Climbing Movements , 2019, 2019 IEEE Conference on Games (CoG).

[15]  Hideki Koike,et al.  FuturePose - Mixed Reality Martial Arts Training Using Real-Time 3D Human Pose Forecasting With a RGB Camera , 2019, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[16]  Antonella De Angeli,et al.  Design Opportunities for Wearable Devices in Learning to Climb , 2016, NordiCHI.

[17]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Antonio Krüger,et al.  ClimbAware: Investigating Perception and Acceptance of Wearables in Rock Climbing , 2016, CHI.

[19]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Susanne Boll,et al.  ClimbingAssist: direct vibro-tactile feedback on climbing technique , 2016, UbiComp Adjunct.

[21]  Takeo Igarashi,et al.  Interactive climbing route design using a simulated virtual climber , 2011, SA '11.