Human gait pattern changes detection system: A multimodal vision-based and novelty detection learning approach

Abstract This paper proposes a novel gait rehabilitation analysis system, based on an innovative multimodal vision-based sensor setup, focused on detecting gait pattern changes over time. The proposed setup is based on inexpensive technologies, without compromising performance, and was designed to be deployed on walkers, which are a typical assistive aid used in gait rehabilitation. In the medical field the evaluation of a patient's rehabilitation progress is typically performed by a medical professional through subjective techniques based on the professional's visual perception and experience. In this context, we are proposing an automatic system to detect the progress of patients undergoing rehabilitation therapy. Our approach is able to perform novelty detection for gait pattern classification based on One-Class Support Vector Machines (OC-SVM). Using point-cloud and RGB-D data, we detect the lower limbs (waist, legs and feet) by using Weighted Kernel-Density Estimation and Weighted Least-Squares to segment the legs into several parts (thighs and shins), and by fitting 3D ellipsoids to model them. Feet are detected using k-means clustering and a Circular Hough Transform. A temporal analysis of the feet's depth is also performed to detect heel strike events. Spatiotemporal and kinematic features are extracted from the lower limbs’ model and fed to a classifier based on the fusion of several OC-SVMs. Experiments with volunteers using the robotic walker platform ISR-AIWALKER, where the proposed system was deployed, revealed a lower limbs tracking accuracy of 3° and that our novelty detection approach successfully identified novel gait patterns, evidencing an overall 97.89% sensitivity.