An Augmented Reality-Assisted Therapeutic Healthcare Exercise System Based on Bare-Hand Interaction

ABSTRACT Augmented Reality (AR)-assisted exercises for the enhancement of finger functions have been developed for many years and have proven to be effective. Most applications tracked the motion of upper-extremity using handheld trackers, which are not convenient to put on and take off. Bare-hand tracking algorithms have been utilized to address this limitation; however, only few AR-assisted healthcare approaches have been designed to train upper-extremity skills using bare-hand tracking, and even fewer approaches detect the Range of Motion (ROM) of fingers with low-cost devices. This paper presents a low-cost and multi-modal residential-based AR-assisted therapeutic healthcare exercise system. A computer vision-based bare-hand interaction method is proposed in this system. This method is designed to estimate finger bending degrees through their projection lengths. With an accurate feature detection strategy, this method is able to detect the full ROM using two web cameras. This system also incorporates a vibration wrist band to stimulate the users with tactile feedback, and an assessment module to evaluate user performance.

[1]  A. Buryanov,et al.  Proportions of Hand Segments , 2010 .

[2]  B. C. Harmeling-van der Wel,et al.  Hierarchical Properties of the Motor Function Sections of the Fugl-Meyer Assessment Scale for People After Stroke: A Retrospective Study , 2008, Physical Therapy.

[3]  Anne Bruton,et al.  A study to compare the reliability of composite finger flexion with goniometry for measurement of range of motion in the hand , 2002, Clinical rehabilitation.

[4]  Chris Harris,et al.  Markerless Motion Capture and Measurement of Hand Kinematics: Validation and Application to Home-Based Upper Limb Rehabilitation , 2013, IEEE Transactions on Biomedical Engineering.

[5]  D. Hose,et al.  A Goniometric Glove for Clinical Hand Assessment , 2000, Journal of hand surgery.

[6]  Maria del Carmen Juan Lizandra,et al.  An Augmented Reality System for the Treatment of Phobia to Small Animals Viewed Via an Optical See-Through HMD: Comparison With a Similar System Viewed Via a Video See-Through HMD , 2011, Int. J. Hum. Comput. Interact..

[7]  Xun Luo,et al.  Integration of Augmented Reality and Assistive Devices for Post-Stroke Hand Opening Rehabilitation , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[8]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Gordon Cheng,et al.  Unconstrained Real-time Markerless Hand Tracking for Humanoid Interaction , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[10]  R. Wagenaar,et al.  The functional recovery of stroke: a comparison between neuro-developmental treatment and the Brunnstrom method. , 2020, Scandinavian journal of rehabilitation medicine.

[11]  C. Parry.  Rehabilitation of the hand , 1973 .

[12]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[13]  M. Ishikawa,et al.  High-speed estimation of multi-finger position and pose for input interface of the mobile devices , 2012, The 1st IEEE Global Conference on Consumer Electronics 2012.

[14]  Andrew Y. C. Nee,et al.  Augmented reality applications in manufacturing: a survey , 2008 .

[15]  Soh-Khim Ong,et al.  Augmented Reality in Assistive Technology and Rehabilitation Engineering , 2011, Handbook of Augmented Reality.

[16]  J R NAPIER,et al.  Functional recovery. , 1951, The British journal of physical medicine : including its application to industry.

[17]  Stacy L Fritz,et al.  Active Finger Extension Predicts Outcomes After Constraint-Induced Movement Therapy for Individuals With Hemiparesis After Stroke , 2005, Stroke.

[18]  Gilda Aparecida de Assis,et al.  A Markeless Augmented Reality Tracking for Enhancing the User Interaction during Virtual Rehabilitation , 2013, 2013 XV Symposium on Virtual and Augmented Reality.

[19]  Zhigeng Pan,et al.  A real-time bimanual 3D interaction method based on bare-hand tracking , 2011, MM '11.

[20]  S. Subramanian,et al.  Virtual Reality Environments for Rehabilitation of the Upper Limb after Stroke , 2006, 2006 International Workshop on Virtual Rehabilitation.

[21]  Soh-Khim Ong,et al.  A novel approach in rehabilitation of hand-eye coordination and finger dexterity , 2011, Virtual Reality.

[22]  Ying Wu,et al.  Modeling the constraints of human hand motion , 2000, Proceedings Workshop on Human Motion.

[23]  Soh-Khim Ong,et al.  Vision-Based Hand Interaction in Augmented Reality Environment , 2011, Int. J. Hum. Comput. Interact..

[24]  D. Evans,et al.  Rehabilitation of the hand , 1973 .

[25]  Archana S. Ghotkar,et al.  Hand Segmentation Techniques to Hand Gesture Recognition for Natural Human Computer Interaction , 2012 .

[26]  Sheng Quan Xie,et al.  Exoskeleton robots for upper-limb rehabilitation: state of the art and future prospects. , 2012, Medical engineering & physics.

[27]  Stéphane Natkin,et al.  How to Analyse Therapeutic Games: The Player / Game / Therapy Model , 2012, ICEC.

[28]  C. Peota.  Novel approach. , 2011, Minnesota medicine.

[29]  Albert A. Rizzo,et al.  Development and evaluation of low cost game-based balance rehabilitation tool using the microsoft kinect sensor , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[30]  Tobias Höllerer,et al.  Handy AR: Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[31]  James M. Laffey,et al.  Understanding Usability and User Experience of Web-Based 3D Graphics Technology , 2008, Int. J. Hum. Comput. Interact..