Person localization using a wearable camera towards enhancing social interactions for individuals with visual impairment

Individuals with visual impairments are at a loss when it comes to everyday social interactions as majority (65%) of these interactions happen through visual non-verbal media. Recently,efforts have been made towards development of an assistive technology called the Social Interaction Assistant [14] which enables access to such useful cues so as to compensate for the lack of vision and other visual impairments. There have been studies which enumerate the important needs of such individuals when they interact in social situations. Along with feedback about their own social behavior, these studies indicate that individuals with visual disabilities are interested in a number of cues related to the people in their surroundings. In this paper, we discuss the importance of person localization while building a human-centric assistive technology which addresses the essential needs of the visually impaired users. Next, we describe the challenges that arise when a wearable camera setup is used as an input source in order to perform person localization. Finally, we present a computer vision based algorithm adapted to handle the issues that are inherent when such a wearable camera setup is used and demonstrate its performance on a number of example sequences.

[1]  Mei-Chen Yeh,et al.  Fast Human Detection Using a Cascade of Histograms of Oriented Gradients , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[2]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[3]  Liang Li,et al.  Directional entropy feature for human detection , 2008, 2008 19th International Conference on Pattern Recognition.

[4]  Yuan Li,et al.  Tracking in Low Frame Rate Video: A Cascade Particle Filter with Discriminative Observers of Different Lifespans , 2007, CVPR.

[5]  Larry S. Davis,et al.  Fast multiple object tracking via a hierarchical particle filter , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Robert C. Bolles,et al.  Parametric Correspondence and Chamfer Matching: Two New Techniques for Image Matching , 1977, IJCAI.

[7]  Baoxin Li,et al.  Head Tracking Using Particle Filter with Intensity Gradient and Color Histogram , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[8]  Sethuraman Panchanathan,et al.  Using tactile rhythm to convey interpersonal distances to individuals who are blind , 2009, CHI Extended Abstracts.

[9]  Ying Wu,et al.  Robust Visual Tracking by Integrating Multiple Cues Based on Co-Inference Learning , 2004, International Journal of Computer Vision.

[10]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[11]  Jiaheng Cao,et al.  Scale Space Histogram of Oriented Gradients for Human Detection , 2008, 2008 International Symposium on Information Science and Engineering.

[12]  Junseok Kwon,et al.  Tracking of Abrupt Motion Using Wang-Landau Monte Carlo Estimation , 2008, ECCV.

[13]  Fatih Murat Porikli,et al.  Integral histogram: a fast way to extract histograms in Cartesian spaces , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  V. Balasubramanian,et al.  Human Centered Multimedia Computing: A New Paradigm for the Design of Assistive and Rehabilitative Environments , 2008, 2008 International Conference on Signal Processing, Communications and Networking.

[15]  Fatih Murat Porikli,et al.  Covariance Tracking using Model Update Based on Lie Algebra , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[16]  B. Schiele,et al.  Combined Object Categorization and Segmentation With an Implicit Shape Model , 2004 .

[17]  Stanley T. Birchfield,et al.  Elliptical head tracking using intensity gradients and color histograms , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[18]  Ramakant Nevatia,et al.  Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[19]  Sethuraman Panchanathan,et al.  A Systematic Requirements Analysis and Development of an Assistive Device to Enhance the Social Interaction of People Who are Blind or Visually Impaired , 2008 .

[20]  Fatih Porikli,et al.  Performance Evaluation of Object Detection and Tracking Systems , 2006 .

[21]  Sethuraman Panchanathan,et al.  Enriched human-centered multimedia computing through inspirations from disabilities and deficit-centered computing solutions , 2008, HCC '08.

[22]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Sethuraman Panchanathan,et al.  iCARE interaction assistant: a wearable face recognition system for individuals with visual impairments , 2005, Assets '05.

[24]  Neeti A. Ogale,et al.  A survey of techniques for human detection from video , 2006 .

[25]  Pietro Perona,et al.  Object class recognition by unsupervised scale-invariant learning , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[26]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[27]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[28]  Greg Welch,et al.  Welch & Bishop , An Introduction to the Kalman Filter 2 1 The Discrete Kalman Filter In 1960 , 1994 .

[29]  BlakeAndrew,et al.  C ONDENSATION Conditional Density Propagation forVisual Tracking , 1998 .

[30]  Robert T. Collins,et al.  Mean-shift blob tracking through scale space , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[31]  Larry S. Davis,et al.  Closely coupled object detection and segmentation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[32]  V. Balasubramanian,et al.  Using a haptic belt to convey non-verbal communication cues during social interactions to individuals who are blind , 2008, 2008 IEEE International Workshop on Haptic Audio visual Environments and Games.

[33]  Bernt Schiele,et al.  Pedestrian detection in crowded scenes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[34]  P. Meer,et al.  Covariance Tracking using Model Update Based on Means on Riemannian Manifolds , 2005 .

[35]  R. Chapuis,et al.  Shape-based pedestrian detection and localization , 2003, Proceedings of the 2003 IEEE International Conference on Intelligent Transportation Systems.

[36]  Fatih Murat Porikli,et al.  Object tracking in low-frame-rate video , 2005, IS&T/SPIE Electronic Imaging.

[37]  James J. Little,et al.  A Boosted Particle Filter: Multitarget Detection and Tracking , 2004, ECCV.

[38]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[39]  Larry S. Davis,et al.  Quasi-Random Sampling for Condensation , 2000, ECCV.

[40]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[41]  Fatih Murat Porikli,et al.  Human Detection via Classification on Riemannian Manifolds , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Luc Van Gool,et al.  Object Tracking with an Adaptive Color-Based Particle Filter , 2002, DAGM-Symposium.

[43]  Jorge Batista,et al.  A region covariance embedded in a particle filter for multi-objects tracking , 2008 .