A Multi-Sensory Gesture-Based Occupational Therapy Environment for Controlling Home Appliances

The proliferation of networked home appliances, coupled with the popularity of low cost gesture detection sensors has made it possible to create smart home environments where users can manipulate devices of daily use through gestures. In this paper, we present a multi-sensory environment that allows a disabled person to control actual devices around the house that are presented to her in a mixed reality environment, as part of occupational therapy exercises. An analytical engine processes the motion data collected by gesture recognition sensors, measures recovery metrics, such as joint range of motion and speed of movement, etc. and presents the results to the therapist or the patient in the form of graphs and exercise statistics. We have incorporated intuitive 2D and 3D user interfaces into the environment to make the therapy experience more engaging and immersive for the disabled people. From the multimedia occupational therapy exercise data analysis, a therapist can get determine the ability of a patient to perform daily life activities without the help of others.