Time-of-Flight Camera Based Virtual Reality Interaction for Balance Rehabilitation Purposes

The 3D Human Body Models (3D HBMs) and the 3D Virtual Reality Environments (3D VREs) enable users to interact with simulated scenarios in an engaging and natural way. The Computer Vision (CV) based Motion Capture (MoCap) systems allow us to obtain user models (i.e., self-avatars) without using cumbersome and uncomfortable physical tools (e.g., sensor suites) which could adversely affect user experience. This last point is of great importance in developing interactive applications for balance rehabilitation purposes where the recovery of lost skills is related to different factors (e.g., patient motivation) including spontaneity of the interaction during the virtual rehabilitative exercises. This paper presents an overview of the Customized Rehabilitation Framework (CRF), a single range imaging sensor based system oriented to patients who experienced with brain strokes, head traumas or neurodegenerative disorders. In particular, the paper is focused on the implementation of two new ad-hoc virtual exercises (i.e., Surfboard and Swing) supporting patients in recovering physical and functional balance. Observations on accuracy of user body models and their real-time interaction ability within rehabilitative simulated environments are presented. In addition, basic experiments concerning usefulness of the proposed exercises to support balance rehabilitation purposes are also reported.

[1]  Javier Ruiz Hidalgo,et al.  Detecting end-effectors on 2.5D data using geometric deformable models: Application to human pose estimation , 2013, Comput. Vis. Image Underst..

[2]  Jose-Juan Hernandez-Lopez,et al.  Detecting objects using color and depth segmentation with Kinect sensor , 2012 .

[3]  Sergio Escalera,et al.  Automatic digital biometry analysis based on depth maps , 2013, Comput. Ind..

[4]  Chris Harris,et al.  Markerless Motion Capture and Measurement of Hand Kinematics: Validation and Application to Home-Based Upper Limb Rehabilitation , 2013, IEEE Transactions on Biomedical Engineering.

[5]  Oliver Bringmann,et al.  Automatic techniques for 3D reconstruction of critical workplace body postures from range imaging data , 2013 .

[6]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[7]  Hubert P. H. Shum,et al.  Real-Time Posture Reconstruction for Microsoft Kinect , 2013, IEEE Transactions on Cybernetics.

[8]  Helman Stern,et al.  Sensors for Gesture Recognition Systems , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[9]  Giovanni De Gasperis,et al.  Movement analysis based on virtual reality and 3D depth sensing camera for whole body rehabilitation , 2012, CompIMAGE.

[10]  Miriam Vollenbroek-Hutten,et al.  Chronic pain rehabilitation with a serious game using multimodal input , 2011, 2011 International Conference on Virtual Rehabilitation.

[11]  Ruzena Bajcsy,et al.  Packing Volumes by Spheres , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Pietro Siciliano,et al.  In-home hierarchical posture classification with a time-of-flight 3D sensor. , 2014, Gait & posture.

[13]  Luigi Cinque,et al.  Overall design and implementation of the virtual glove , 2013, Comput. Biol. Medicine.

[14]  Marco Ferrari,et al.  Prefrontal Cortex Activated Bilaterally by a Tilt Board Balance Task: A Functional Near-Infrared Spectroscopy Study in a Semi-Immersive Virtual Reality Environment , 2013, Brain Topography.

[15]  Yao-Jen Chang,et al.  A Kinect-based upper limb rehabilitation system to assist people with cerebral palsy. , 2013, Research in developmental disabilities.

[16]  Corey J. Bohil,et al.  Virtual reality in neuroscience research and therapy , 2011, Nature Reviews Neuroscience.

[17]  Luis Salgado,et al.  Background foreground segmentation with RGB-D Kinect data: An efficient combination of classifiers , 2014, J. Vis. Commun. Image Represent..

[18]  Hong Wei,et al.  A survey of human motion analysis using depth imagery , 2013, Pattern Recognit. Lett..

[19]  Albert A. Rizzo,et al.  Virtual Reality in Brain Damage Rehabilitation: Review , 2005, Cyberpsychology Behav. Soc. Netw..

[20]  Dong Pyo Jang,et al.  Development of virtual reality proprioceptive rehabilitation system for stroke patients , 2014, Comput. Methods Programs Biomed..

[21]  Danilo Avola,et al.  Design of an efficient framework for fast prototyping of customized human-computer interfaces and virtual environments for rehabilitation , 2013, Comput. Methods Programs Biomed..

[22]  Qingmin Liao,et al.  Part template: 3D representation for multiview human pose estimation , 2013, Pattern Recognit..

[23]  Marco Ferrari,et al.  A semi-immersive virtual reality incremental swing balance task activates prefrontal cortex: A functional near-infrared spectroscopy study , 2014, NeuroImage.

[24]  Linda Denehy,et al.  Validity of the Microsoft Kinect for assessment of postural control. , 2012, Gait & posture.

[25]  Yao-Jen Chang,et al.  A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. , 2011, Research in developmental disabilities.

[26]  Behzad Dariush,et al.  Kinematic self retargeting: A framework for human pose estimation , 2010, Comput. Vis. Image Underst..

[27]  J. O'Rourke,et al.  A spherical representation of a human body for visualizing movement , 1979, Proceedings of the IEEE.

[28]  Weihai Chen,et al.  Fast 3D modeling in complex environments using a single Kinect sensor , 2014 .

[29]  Juan Manuel Ibarra Zannatha,et al.  Development of a system based on 3D vision, interactive virtual environments, ergonometric signals and a humanoid for stroke rehabilitation , 2013, Comput. Methods Programs Biomed..

[30]  Giuseppe Placidi,et al.  A smart virtual glove for the hand telerehabilitation , 2007, Comput. Biol. Medicine.