Detecting Carried Objects in Short Video Sequences

We propose a new method for detecting objects such as bags carried by pedestrians depicted in short video sequences. In common with earlier work [1,2] on the same problem, the method starts by averaging aligned foreground regions of a walking pedestrian to produce a representation of motion and shape (known as a temporal template) that has some immunity to noise in foreground segmentations and phase of the walking cycle. Our key novelty is for carried objects to be revealed by comparing the temporal templates against view-specific exemplars generated offline for unencumbered pedestrians. A likelihood map obtained from this match is combined in a Markov random field with a map of prior probabilities for carried objects and a spatial continuity assumption, from which we obtain a segmentation of carried objects using the MAP solution. We have re-implemented the earlier state of the art method [1] and demonstrate a substantial improvement in performance for the new method on the challenging PETS2006 dataset [3]. Although developed for a specific problem, the method could be applied to the detection of irregularities in appearance for other categories of object that move in a periodic fashion.

[1]  P. Rousseeuw.  Least Median of Squares Regression , 1984 .

[2]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[3]  Larry S. Davis,et al.  Hydra: multiple people detection and tracking using silhouettes , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[4]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Larry S. Davis,et al.  Backpack: Detection of People Carrying Objects Using Silhouettes , 2001, Comput. Vis. Image Underst..

[7]  Larry S. Davis,et al.  Detection of people carrying objects : a motion-based recognition approach , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[8]  Giovanni Attolico,et al.  Detection of objects carried by people , 2002, Proceedings. International Conference on Image Processing.

[9]  Larry Davis,et al.  Modelling pedestrian shapes for outlier detection: a neural net based approach , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[10]  Derek R. Magee,et al.  Tracking multiple vehicles using foreground, background and motion models , 2004, Image Vis. Comput..

[11]  M. Everingham.  The PASCAL Visual Object Classes Challenge 2005 Development Kit , 2005 .

[12]  Xuelong Li,et al.  Human Carrying Status in Visual Surveillance , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[13]  Tieniu Tan,et al.  Principal axis-based correspondence between multiple cameras for people tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Vincent Lepetit,et al.  Human body pose detection using Bayesian spatio-temporal templates , 2006, Comput. Vis. Image Underst..

[15]  Larry S. Davis,et al.  Human Appearance Change Detection , 2007, 14th International Conference on Image Analysis and Processing (ICIAP 2007).

[16]  Vincent Lepetit,et al.  Bridging the Gap between Detection and Tracking for 3D Monocular Video-Based Motion Capture , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[18]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.