Kinect in neurorehabilitation: Computer vision system for real time hand and object detection and distance estimation

This paper presents image processing and scene analysis methods that can provide artificial vision that is of interest for automatic selection of hand trajectory and prehension. The new algorithm, which uses data from the Kinect sensor, allows real-time detection of the hand of the person grasping an object at working table in front of that person. The outputs are real world coordinates of the hand and the object. The image processing is done in Matlab over the depth image stream taken from the Microsoft Kinect as a sensory input. Results show that in the presented system setup our program is capable of tracking hand movements in the transverse plane and estimating hand and object position in real-time with tolerable estimation error for the selection of stimulation paradigm that could control hand trajectory.

[1]  Strahinja Došen,et al.  Transradial prosthesis: artificial vision for control of prehension. , 2011, Artificial organs.

[2]  Blake Hannaford,et al.  Using Kinect TM and a Haptic Interface for Implementation of Real-Time Virtual Fixtures , 2011 .

[3]  Yao-Jen Chang,et al.  A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. , 2011, Research in developmental disabilities.

[4]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[5]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[6]  Hui-mei Justina Hsu.  The Potential of Kinect in Education , 2011 .

[7]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[8]  Jiri Matas,et al.  Optimal Randomized RANSAC , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Strahinja Dosen,et al.  Control of Prehension for the Transradial Prosthesis: Natural-like Image Recognition System , 2009 .

[10]  Philippe Poignet,et al.  Joint angle estimation in rehabilitation with inertial sensors and its integration with Kinect , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[11]  Jiri Matas,et al.  Randomized RANSAC with Td, d test , 2004, Image Vis. Comput..

[12]  Dejan B Popovic,et al.  Restitution of reaching and grasping promoted by functional electrical therapy. , 2002, Artificial organs.

[13]  Jiri Matas,et al.  Randomized RANSAC with T(d, d) test , 2002, BMVC.

[14]  T. Sinkjaer,et al.  Clinical evaluation of Functional Electrical Therapy in acute hemiplegic subjects. , 2003, Journal of rehabilitation research and development.

[15]  Mario Ciampi,et al.  Controller-free exploration of medical image data: Experiencing the Kinect , 2011, 2011 24th International Symposium on Computer-Based Medical Systems (CBMS).

[16]  Marko Markovic,et al.  Stereovision system for estimation of the grasp type for electrotherapy , 2011 .

[17]  Janis Fehr,et al.  Computing Range Flow from Multi-modal Kinect Data , 2011, ISVC.

[18]  Blake Hannaford,et al.  Using Kinect and a Haptic Interface for Implementation of Real-Time Virtual Fixture , 2011 .

[19]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[20]  David Nistér,et al.  Preemptive RANSAC for live structure and motion estimation , 2005, Machine Vision and Applications.