Multicamera Summarization of Rehabilitation Sessions in Home Environment

In this paper we present a cyber-physiotherapy system (CyPhy) that brings daily rehabilitation to patient's home with supervision from trained therapist. CyPhy is able to capture and record RGB-D, skeleton, and physiotherapy-related medical sensing data streams from patient's exercises using multiple cameras and body sensors. With hours of exercises from every patient, that are captured every day from multiple cameras, therapists spend huge amount of their time watching videos to monitor the correctness of patients' moves. This becomes even more challenging in the presence of multiple cameras where the therapist might not know which camera stream shows the incorrect motion. In this paper, we explore the multicamera summarization problem from various aspects: (1) We first explore the types of exercises that benefit the most from using multiple cameras; (2) We propose a method to detect incorrect motion from multiple cameras in rehabilitation exercises; (3) We show how the analysis of incorrect motion is used to summarize the video and recommend the camera view that best visualizes the mistake. Our method for detecting incorrect motion achieves more than 92% accuracy at wide range of thresholds with significant improvement of 20% over single camera and 10% over the closest approach that uses multiple cameras.