A Multimodal Gamified Platform for Real-Time User Feedback in Sports Performance

In this paper we introduce a novel platform that utilises multi-modal low-cost motion capture technology for the delivery of real-time visual feedback for sports performance. This platform supports the expansion to multi-modal interfaces that utilise haptic and audio feedback, which scales effectively with motor task complexity. We demonstrate an implementation of our platform within the field of sports performance. The platform includes low-cost motion capture through a fusion technique, combining a Microsoft Kinect V2 with two wrist inertial sensors, which make use of the accelerometer and gyroscope sensors, alongside a game-based Graphical User Interface (GUI) for instruction, visual feedback and gamified score tracking.

[1]  Noel E. O'Connor,et al.  Human gait monitoring using body-worn inertial sensors and kinematic modelling , 2015, 2015 IEEE SENSORS.

[2]  Chris Lonsdale,et al.  Patient motivation and adherence to postsurgery rehabilitation exercise recommendations: the influence of physiotherapists' autonomy-supportive behaviors. , 2009, Archives of physical medicine and rehabilitation.

[3]  Noel E. O'Connor,et al.  Low-cost accurate skeleton tracking based on fusion of kinect and wearable inertial sensors , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[4]  Markus Windolf,et al.  Systematic accuracy and precision analysis of video motion capturing systems--exemplified on the Vicon-460 system. , 2008, Journal of biomechanics.

[5]  Beatrix Vereijken,et al.  Exercise and rehabilitation delivered through exergames in older adults: An integrative review of technologies, safety and efficacy , 2016, Int. J. Medical Informatics.

[6]  David J. Reinkensmeyer,et al.  "If I can't do it once, why do it a hundred times?": Connecting volition to movement success in a virtual environment motivates people to exercise the arm after stroke , 2007, 2007 Virtual Rehabilitation.

[7]  Zhengyou Zhang,et al.  Microsoft Kinect Sensor and Its Effect , 2012, IEEE Multim..

[8]  Noel E. O'Connor,et al.  A Multi-Modal 3D Capturing Platform for Learning and Preservation of Traditional Sports and Games , 2015, ACM Multimedia.

[9]  Noel E. O'Connor,et al.  An advanced virtual dance performance evaluator , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Noel E. O'Connor,et al.  Towards Automatic Activity Classification and Movement Assessment during a Sports Training Session , 2022 .

[11]  D. Roetenberg,et al.  Xsens MVN: Full 6DOF Human Motion Tracking Using Miniature Inertial Sensors , 2009 .

[12]  Taehyun Rhee,et al.  Realtime human motion control with a small number of inertial sensors , 2011, SI3D.

[13]  J. Deutsch,et al.  Use of a Low-Cost, Commercially Available Gaming Console (Wii) for Rehabilitation of an Adolescent With Cerebral Palsy , 2008, Physical Therapy.

[14]  Luca Benini,et al.  MOCA: A Low-Power, Low-Cost Motion Capture System Based on Integrated Accelerometers , 2007, Adv. Multim..

[15]  Greg Welch,et al.  Motion Tracking: No Silver Bullet, but a Respectable Arsenal , 2002, IEEE Computer Graphics and Applications.