A Gesture Recognition Framework for Cognitive Assessment

17(Suppl.): . Purpose: Most developed countries have been confronted with an acute growth in their elderly population. This situation is accompanied by increased prevalence of several health issues decreasing their quality of life. Alzheimer's disease is one of the most common cognitive disorders associated with aging. This has resulted in an urge to take advantage of technological tools to help doctors by providing early diagnosis which is vital for treatment of the elderly. These patients are not able to correctly emulate forelimb gestures and commit spatial/temporal errors, e.g. pretend to strike a nail with a hammer. 1 We developed a gesture recognition and evaluation tool to assist doctors in early detection of cognitive disorders (Figure 1). The framework performs precise gesture recognition based on state-of-the-art computer vision algorithms and evaluates subjects' performances to obtain a diagnostic report for clinicians. Method: The gesture recognition part of the framework uses four different methods. The first method employs local descriptors such as HOG/HOF 2 using shape and motion cues of a person performing a gesture. In the second method, skeleton information of the subjects is used to benefit from the body pose information. The third method uses depth information to detect exact hand patches with body part trajectories. We also trained a deep learning based recognition model 3 which empowered the framework to perform reliable recognitions. Results and Discussion: The framework is trained to recognize correct/incorrect performances. To have an objective decision on correctness of the gesture performances, clinicians annotated the data used for training the recognition algorithms. The utilized dataset was collected at the Institute Claude Pompidou (ICP) at Nice consisting of 14 dynamic and 15 static gestures selected by clinicians. The framework achieved 90% average accuracy on classifying gestures which is a reliable rate for diagnosis. Additionally, reaction and movement time of a subject are also detected. These two motor reflex criteria which measures responsive level of a subject are important factors in cognitive diagnosis. Motion descriptors are utilized to detect those quantities. All these types of information are illustrated in the user interface which provides clinicians with a detailed assessment of individual gesture and overall performance (Figure 1). Conclusion: We developed a user-friendly tool to help doctors in the diagnosis of cognitive disorders by providing a complete assessment of gestures performed by subjects. Future Work: We will conduct extensive evaluations on a larger population of patients with different levels of cognitive disorders. To improve gesture recognition task, more accurate deep models will be investigated.