Thermal and 3D Kinect Sensor Fusion for Robust People Detection using Evolutionary Selection of Supervised Classifiers

In this paper we propose a novel approach for combining information from low cost multiple sensors for people detection on a mobile robot. Robustly detecting people is a key capability needed for robots that operate in populated environments. Several works show the advantages of fusing data coming from complementary sensors. Kinect sensor offers a rich data set at a significantly low cost, however, there are some limitations using it in a mobile platform, mainly that Kinect relies on images captured by a static camera. To cope with these limitations, this work is based on the fusion of Kinect and thermopile array sensor mounted on top of a mobile platform. We propose the implementation of evolutionary selection of people detection supervised classifiers built using several computer vision transformation. Experimental results carried out with a mobile platform in a manufacturing shop floor show that the percentage of wrong classified using only Kinect is drastically reduced with the classification algorithms and with the combination of the three information sources.

[1]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[2]  Grzegorz Cielniak,et al.  Active people recognition using thermal and grey images on a mobile security robot , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Rafael Arnay,et al.  Real-time adaptive obstacle detection based on an image database , 2011, Comput. Vis. Image Underst..

[4]  Jean-Marc Odobez,et al.  Fast human detection from joint appearance and foreground feature subset covariances , 2011, Comput. Vis. Image Underst..

[5]  Kurt Hornik,et al.  The support vector machine under test , 2003, Neurocomputing.

[6]  Basilio Sierra,et al.  Histogram distance-based Bayesian Network structure learning: A supervised classification specific approach , 2009, Decis. Support Syst..

[7]  Bernt Schiele,et al.  Visual People Detection - Different Models, Comparison and Discussion , 2009, ICRA 2009.

[8]  Shuzhi Sam Ge,et al.  Robust Human Detection and Identification by Using Stereo and thermal Images in Human Robot Interaction , 2007, Int. J. Inf. Acquis..

[9]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[10]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[11]  Martin Hofmann,et al.  Fusion of Multi-Modal Sensors in a Voxel Occupancy Grid for Tracking and Behaviour Analysis , 2011, WIAMIS 2011.

[12]  Peter Bajcsy,et al.  Integration of thermal and visible imagery for robust foreground detection in tele-immersive spaces , 2008, 2008 11th International Conference on Information Fusion.

[13]  Hideya Takahashi,et al.  Fusion of Infrared and Visible Images for Robust Person Detection , 2011 .

[14]  Paulo Peixoto,et al.  On Exploration of Classifier Ensemble Synergism in Pedestrian Detection , 2010, IEEE Transactions on Intelligent Transportation Systems.

[15]  P. Larra,et al.  Feature Subset Selection by Bayesian Networks Based Optimization Abstract|a New Method for Feature Subset Selection in Machine Learning, Fss-ebna , 1999 .

[16]  Huosheng Hu,et al.  A Bank of Unscented Kalman Filters for Multimodal Human Perception with Mobile Service Robots , 2010, Int. J. Soc. Robotics.

[17]  X. Maldague,et al.  Thermal imaging for enhanced foreground background segmentation , 2006 .

[18]  Wolfram Burgard,et al.  Using Boosted Features for the Detection of People in 2D Range Data , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[19]  Pedro Larrañaga,et al.  Using Bayesian networks in the construction of a bi-level multi-classifier. A case study using intensive care unit patients data , 2001, Artif. Intell. Medicine.

[20]  Ryo Kurazume,et al.  Multi-Part People Detection Using 2D Range Data , 2010, Int. J. Soc. Robotics.

[21]  Vijayan K. Asari,et al.  Face recognition in multi-sensor images based on a novel modular feature selection technique , 2010, Inf. Fusion.

[22]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[23]  Bojan Cestnik,et al.  Estimating Probabilities: A Crucial Task in Machine Learning , 1990, ECAI.