Combined Thermal Infrared and Visible Cameras for Pedestrian Detection and Tracking

In order to enhance the accuracy and real-time performance of mobile robot in complex environment, a pedestrian detection and tracking method combined with infrared and visible image features is proposed. Firstly, a 4 degree of freedom multispectral vision platform is designed. Secondly, a coarse-fine pedestrian detection and tracking is implemented by using census transform histogram (CENTRIST) descriptor and kernel correlation filtering (KCF). Then, a head-eye coordination control strategy for multispectral vision platform is established. Finally, the accuracy of pedestrian detection and tracking algorithm as well as the effectiveness of the active visual tracking system are verified. The experimental results clearly demonstrate the effectiveness of our multispectral visual tracking system compared to that based on visible or infrared spectrum singly, and the success rate of pedestrian tracking based on multispectral images is increased by 28.9% and 10.1% respectively.