Human Detection and Identification by Robots Using Thermal and Visual Information in Domestic Environments

In this paper a robust system for enabling robots to detect and identify humans in domestic environments is proposed. Robust human detection is achieved through the use of thermal and visual information sources that are integrated to detect human-candidate objects, which are further processed in order to verify the presence of humans and their identity using face information in the thermal and visual spectrums. Face detection is used to verify the presence of humans, and face recognition to identify them. Active vision mechanisms are employed in order to improve the relative pose of a candidate object/person in case direct identification is not possible. The response of the different modules is characterized, and the proposed system is validated using image databases of real domestic environments, and human detection and identification benchmarks of the RoboCup@Home research community.

[1]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[2]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Josef Kittler,et al.  Face Recognition with LWIR Imagery Using Local Binary Patterns , 2009, ICB.

[5]  Massimo Bertozzi,et al.  Pedestrian detection for driver assistance using multiresolution infrared vision , 2004, IEEE Transactions on Vehicular Technology.

[6]  Larry S. Davis,et al.  Probabilistic template based pedestrian detection in infrared videos , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[7]  Lijun Jiang,et al.  Infrared Face Recognition by Using Blood Perfusion Data , 2005, AVBPA.

[8]  Gérard G. Medioni,et al.  Robust real-time vision for a personal service robot , 2007, Comput. Vis. Image Underst..

[9]  Javier Ruiz-del-Solar,et al.  Face recognition using thermal infrared images for Human-Robot Interaction applications: A comparative study , 2009, 2009 6th Latin American Robotics Symposium (LARS 2009).

[10]  Huosheng Hu,et al.  Multisensor-Based Human Detection and Tracking for Mobile Service Robots , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  S. S. Mudaly.  Novel computer-based infrared pedestrian data-acquisition system , 1979 .

[12]  Dariu Gavrila,et al.  Monocular Pedestrian Detection: Survey and Experiments , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Erik Hjelmås,et al.  Face Detection: A Survey , 2001, Comput. Vis. Image Underst..

[14]  Seong G. Kong,et al.  Recent advances in visual and infrared face recognition - a review , 2005, Comput. Vis. Image Underst..

[15]  Mei-Chen Yeh,et al.  Fast Human Detection Using a Cascade of Histograms of Oriented Gradients , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[16]  Javier Ruiz-del-Solar,et al.  Robust skin segmentation using neighborhood information , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[17]  Fatih Murat Porikli,et al.  Pedestrian Detection via Classification on Riemannian Manifolds , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  A. Broggi,et al.  A modular tracking system for far infrared pedestrian recognition , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[19]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.

[20]  Pawan Sinha,et al.  Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About , 2006, Proceedings of the IEEE.

[21]  J. Miura,et al.  Robust Stereo-Based Person Detection and Tracking for a Person Following Robot , 2009 .

[22]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[23]  Cordelia Schmid,et al.  Human Detection Using Oriented Histograms of Flow and Appearance , 2006, ECCV.

[24]  Horst-Michael Groß,et al.  An approach to multi-modal human-machine interaction for intelligent service robots , 2003, Robotics Auton. Syst..

[25]  Pietro Perona,et al.  Pedestrian detection: A benchmark , 2009, CVPR.

[26]  Jian Yao,et al.  Fast human detection from videos using covariance features , 2008, ECCV 2008.

[27]  Ran He,et al.  Highly Accurate and Fast Face Recognition Using Near Infrared Images , 2006, ICB.

[28]  Paul A. Viola,et al.  Detecting Pedestrians Using Patterns of Motion and Appearance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[29]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[30]  Jian Zhang,et al.  Fast Pedestrian Detection Using a Cascade of Boosted Covariance Features , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Javier Ruiz-del-Solar,et al.  Face Recognition for Human-Robot Interaction Applications: A Comparative Study , 2009, RoboCup.

[32]  Shuzhi Sam Ge,et al.  Stereo-based human detection for mobile service robots , 2004, ICARCV 2004 8th Control, Automation, Robotics and Vision Conference, 2004..

[33]  Javier Ruiz-del-Solar,et al.  A unified learning framework for object detection and classification using nested cascades of boosted classifiers , 2008, Machine Vision and Applications.

[34]  Javier Ruiz-del-Solar,et al.  Recognition of Faces in Unconstrained Environments: A Comparative Study , 2009, EURASIP J. Adv. Signal Process..

[35]  Shengcai Liao,et al.  Illumination Invariant Face Recognition Using Near-Infrared Images , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  W. Ritter,et al.  Reinforcing the reliability of pedestrian detection in far-infrared sensing , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[37]  Zhi-Hua Zhou,et al.  Face recognition from a single image per person: A survey , 2006, Pattern Recognit..

[38]  Dariu Gavrila,et al.  Multi-cue Pedestrian Detection and Tracking from a Moving Vehicle , 2007, International Journal of Computer Vision.

[39]  Javier Ruiz-del-Solar,et al.  Thermal Face Recognition Using Local Interest Points and Descriptors for HRI Applications , 2010, RoboCup.

[40]  Javier Ruiz-del-Solar,et al.  UChile HomeBreakers 2010 Team Description Paper , 2010 .

[41]  Bernt Schiele,et al.  A Performance Evaluation of Single and Multi-feature People Detection , 2008, DAGM-Symposium.

[42]  Dariu Gavrila,et al.  An Experimental Study on Pedestrian Classification , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  David A. McAllester,et al.  Cascade object detection with deformable part models , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Horst-Michael Groß,et al.  A multi-modal system for tracking and analyzing faces on a mobile robot , 2004, Robotics Auton. Syst..

[45]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[46]  Luca Iocchi,et al.  RoboCup@Home: Scientific Competition and Benchmarking for Domestic Service Robots , 2009 .

[47]  Joseph Wilder,et al.  Comparison of visible and infra-red imagery for face recognition , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.