Visual Tracking of Multiple Humans with Machine Learning based Robustness Enhancement applied to Real-World Robotic Systems

This thesis presents a robust real-time vision-based 3D multiple human tracker capable of automatically identifying, labelling and tracking multiple humans. The primary contribution is a methodology to improve robustness and integration in real-world scenarios. The system consists of 2 stages, 1. a vision based human tracking system using multiple visual cues and a robust occlusion handler, 2. a machine learning based module for intelligent multi-modal fusion and robust to drastic changes in lighting. The fusion module performs an on line analysis of image parameters influencing the performance of the tracker. Thereafter, optimal weights are generated for each visual modality for the current scene. The thesis also proposes a novel approach to validate the 3D multiple human tracking system through zero-error ground truth data. It shows integration into variety of distributed robotic systems being used in real world applications.