A Mixed-Reality Training Environment for Upper Limb Prosthesis Control

Adjusting to an amputation can often times be difficult for the body. Post-surgery, amputees not only have to incur expensive rehabilitation treatment costs, but also have to wait for up to several months before receiving a properly fitted prosthesis. We developed a mixed-reality training environment where amputees can train, at their own time and convenience, and interact with holographic objects, while also receiving tactile and proprioceptive feedback. We incorporate positional information through inertial sensors, touch and proprioception information through vibrational feedback, all integrated into an augmented-reality (AR) environment viewed through the Microsoft HoloLens TM. Training tasks were designed to account for limb rotation and object relocation in a three-dimensional space with a correct palm orientation essential for an intuitive grasp and release of objects. Our results showed an improved performance in training time, overshoot and completion rate with vibratory feedback (of both touch and proprioception) over without feedback. Furthermore, EMG activity was analyzed to estimate the muscular effort during each task.