Enhancing biometric recognition with spatio-temporal reasoning in smart environments

We discuss smart environments that identify and track their occupants using unobtrusive recognition modalities such as face, gait, and voice. In order to alleviate the inherent limitations of recognition, we propose spatio-temporal reasoning techniques based upon an analysis of the occupant tracks. The key idea underlying our approach is to determine the identity of a person based upon information from a track of events rather than a single event. We abstract a smart environment by a probabilistic state transition system in which each state records a set of individuals who are present in various zones of the smart environment. An event abstracts a recognition step, and the transition function defines the mapping between states upon the occurrence of an event. We express two forms of spatio-temporal reasoning in the form of transition functions: a track-based transition function and an error-correcting transition function. We also define the concepts of ‘precision’ and ‘recall’ to quantify the performance of the smart environment and provide experimental results to clarify the performance improvements from spatio-temporal reasoning. Our conclusion is that the state transition system is an effective abstraction of a smart environment and the application of spatial-temporal reasoning enhances the overall performance of a biometric recognition system.

[1]  Jonathan G. Fiscus,et al.  Multimodal Technologies for Perception of Humans, International Evaluation Workshops CLEAR 2007 and RT 2007, Baltimore, MD, USA, May 8-11, 2007, Revised Selected Papers , 2008, CLEAR.

[2]  Marco Gori,et al.  Adaptive Processing of Sequences and Data Structures , 1998, Lecture Notes in Computer Science.

[3]  C. J. van Rijsbergen,et al.  Information Retrieval , 1979, Encyclopedia of GIS.

[4]  J. Krumm,et al.  Multi-camera multi-person tracking for EasyLiving , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[5]  Martial Michel,et al.  The CLEAR 2007 Evaluation , 2007, CLEAR.

[6]  Venu Govindaraju,et al.  Multimodal identification and tracking in smart environments , 2010, Personal and Ubiquitous Computing.

[7]  Venu Govindaraju,et al.  Spatio-Temporal Reasoning in Biometrics Based Smart Environments , 2011, ANT/MobiWIS.

[8]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[9]  Alex Pentland,et al.  Face Recognition for Smart Environments , 2000, Computer.

[10]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[11]  Zoubin Ghahramani,et al.  Learning Dynamic Bayesian Networks , 1997, Summer School on Neural Networks.

[12]  Wonchan Lee,et al.  Real Time Face Detection and Recognition System Using Haar-Like Feature/HMM in Ubiquitous Network Environments , 2005, ICCSA.

[13]  Venu Govindaraju,et al.  Three R ’ s of Cyber-Physical Spaces , 2011 .

[14]  Rainer Stiefelhagen,et al.  Multimodal identity tracking in a smart room , 2007, Personal and Ubiquitous Computing.

[15]  Massimo Tistarelli,et al.  Handbook of Remote Biometrics: for Surveillance and Security , 2009 .

[16]  Venu Govindaraju,et al.  The Three Rs of Cyberphysical Spaces , 2011, Computer.

[17]  Ilias Maglogiannis,et al.  Face detection and recognition of natural human emotion using Markov random fields , 2007, Personal and Ubiquitous Computing.

[18]  Venu Govindaraju,et al.  Vector Model Based Indexing and Retrieval of Handwritten Medical Forms , 2007 .

[19]  Djamel Bouchaffra,et al.  A Methodology for Mapping Scores to Probabilities , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Miguel A. Patricio,et al.  Context-based scene recognition from visual data in smart homes: an Information Fusion approach , 2012, Personal and Ubiquitous Computing.

[21]  Svetha Venkatesh,et al.  Tracking and Surveillance in Wide-Area Spatial Environments Using the Abstract Hidden Markov Model , 2001, Int. J. Pattern Recognit. Artif. Intell..

[22]  Bernhard Nebel,et al.  Qualitative Spatio-Temporal Reasoning with RCC-8 and Allen's Interval Calculus: Computational Complexity , 2002, ECAI.

[23]  Mireia Farrús,et al.  Audio, Video and Multimodal Person Identification in a Smart Room , 2006, CLEAR.

[24]  Qin Jin,et al.  Multi-modal Person Identification in a Smart Environment , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Venu Govindaraju,et al.  Integrating recognition and reasoning in smart environments , 2008 .