A system for automated hand pose estimation in neurophysiological research

The goal of this Master’s thesis has been to develop an algorithm that can track the spatial location of the rat forepaw as well as detailed movements around multiple joints when the animals engage in a skilled reaching task. The kinematic data obtained from the behavioural analysis is subsequently matched to neuronal activity patterns recorded in different motor related brain structures during reaching in order to investigate how the nervous system learns to control skilled movements. The method uses an articulated 3D model whose projection is fitted to the video by maximizing a function that estimates matching quality using segmented silhouettes and edges in the images. The matching function is maximized by a mixture of iterative methods and database search which take advantage of the stereotypical aspects of paw movements to build a computationally efficient method. Physical constraints are added to make sure the results are meaningful and to decrease the size of the pose space. Part of the work has also been devoted to adapting the experimental setup to allow for automatic tracking of specific motor behaviors. The result is an experimental setup that features two cameras acquiring images at 200 frames per second and, in addition to the cameras, three mirrors producing a total of six views used in tracking. Promising tracking results are shown as well as preliminary neurophysiological data recorded from the first animal.