An appearance based approach for human and object tracking

A system for tracking humans and detecting human-object interactions in indoor environments is described. A combination of correlogram and histogram information is used to model object and human color distributions. Humans and objects are detected using a background subtraction algorithm. The models are built on the fly and used to track them on a frame by frame basis. The system is able to detect when people merge into groups and segment them during occlusion. Identities are preserved during the sequence, even if a person enters and leaves the scene. The system is also able to detect when a person deposits or removes an object from the scene. In the first case the models are used to track the object retroactively in time. In the second case the objects are tracked for the rest of the sequence. Experimental results using indoor video sequences are presented.

[1]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[2]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Larry S. Davis,et al.  M2Tracker: A Multi-view Approach to Segmenting and Tracking People in a Cluttered Scene Using Region-Based Stereo , 2002, ECCV.

[4]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[5]  Massimiliano Pontil,et al.  People Recognition in Image Sequences by Supervised Learning , 2000 .

[6]  Rohini K. Srihari,et al.  Geometric histogram: a distribution of geometric configurations of color subsets , 1999, Electronic Imaging.

[7]  Larry S. Davis,et al.  Tracking humans from a moving platform , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[8]  Larry S. Davis,et al.  Probabilistic framework for segmenting people under occlusion , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[9]  Teuvo Kohonen,et al.  Learning vector quantization , 1998 .

[10]  J. Krumm,et al.  Multi-camera multi-person tracking for EasyLiving , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[11]  Azriel Rosenfeld,et al.  Tracking Groups of People , 2000, Comput. Vis. Image Underst..

[12]  Jiang Li,et al.  Color based multiple people tracking , 2002, 7th International Conference on Control, Automation, Robotics and Vision, 2002. ICARCV 2002..

[13]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[15]  Jing Huang,et al.  Spatial Color Indexing and Applications , 2004, International Journal of Computer Vision.

[16]  Azriel Rosenfeld,et al.  3D object tracking using shape-encoded particle propagation , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[17]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[18]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[19]  Larry S. Davis,et al.  A Robust Background Subtraction and Shadow Detection , 1999 .

[20]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[21]  Maria Petrou,et al.  Multidimensional Co-occurrence Matrices for Object Recognition and Matching , 1996, CVGIP Graph. Model. Image Process..

[22]  Shaogang Gong,et al.  Segmentation and Tracking Using Color Mixture Models , 1998, ACCV.

[23]  Stephan Volmer,et al.  Color co-occurrence descriptors for querying-by-example , 1998, Proceedings 1998 MultiMedia Modeling. MMM'98 (Cat. No.98EX200).

[24]  Ramin Zabih,et al.  Histogram refinement for content-based image retrieval , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.