Temporal mapping of surveillance video for indexing and summarization

A comprehensive scheme of temporal profile of surveillance video is proposed.It provides the sampling selection, analyzes properties of shape and motion of the temporal profile, develop the algorithms to detect dynamic targets, and measure their motion direction.Finally, the visualization of targets with multiple temporal slices with transparency is provided.Further, robust object detection based on background updating, and real time data transmission of is given.A comparison of our method with other video indexing methods is also given. This work converts the surveillance video to a temporal domain image called temporal profile that is scrollable and scalable for quick searching of long surveillance video by human operators. Such a profile is sampled with linear pixel lines located at critical locations in the video frames. It has precise time stamp on the target passing events through those locations in the field of view, shows target shapes for identification, and facilitates the target search in long videos. In this paper, we first study the projection and shape properties of dynamic scenes in the temporal profile so as to set sampling lines. Then, we design methods to capture target motion and preserve target shapes for target recognition in the temporal profile. It also provides the uniformed resolution of large crowds passing through so that it is powerful in target counting and flow measuring. We also align multiple sampling lines to visualize the spatial information missed in a single line temporal profile. Finally, we achieve real time adaptive background removal and robust target extraction to ensure long-term surveillance. Compared to the original video or the shortened video, this temporal profile reduced data by one dimension while keeping the majority of information for further video investigation. As an intermediate indexing image, the profile image can be transmitted via network much faster than video for online video searching task by multiple operators. Because the temporal profile can abstract passing targets with efficient computation, an even more compact digest of the surveillance video can be created.

[1]  Gabriel Taubin,et al.  Real-Time Median Filtering for Embedded Smart Cameras , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[2]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Suchendra M. Bhandarkar,et al.  Fast and Robust Background Updating for Real-time Traffic Surveillance and Monitoring , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[5]  Leonard McMillan,et al.  Computational time-lapse video , 2007, SIGGRAPH 2007.

[6]  Dezhen Song,et al.  The co-opticon: shared access to a robotic streaming video camera , 2003, MULTIMEDIA '03.

[7]  Yael Pritch,et al.  Webcam Synopsis: Peeking Around the World , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[8]  Rajiv Gupta,et al.  Linear Pushbroom Cameras , 1994, ECCV.

[9]  Tohru Yoshioka,et al.  Development of detection algorithm for vehicles using multi-line CCD sensor , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[10]  Kwan-Liu Ma,et al.  Dynamic video narratives , 2010, SIGGRAPH 2010.

[11]  FengWu-Chi,et al.  Panoptes: scalable low-power video sensor networking technologies , 2005 .

[12]  Paul Rademacher,et al.  Multiple-center-of-projection images , 1998, SIGGRAPH.

[13]  Kai Yang,et al.  Automatic categorization-based multi-stage pedestrian detection , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[14]  Jiang Yu Zheng,et al.  Line cameras for monitoring and surveillance sensor networks , 2007, ACM Multimedia.

[15]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[16]  Yu Zhou,et al.  Scanning scene tunnel for city traversing , 2006, IEEE Transactions on Visualization and Computer Graphics.

[17]  Min Chen,et al.  Video visualization , 2003, IEEE Visualization, 2003. VIS 2003..

[18]  Xiaogang Wang,et al.  Scene-Independent Group Profiling in Crowd , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Sung-Jea Ko,et al.  New autofocusing technique using the frequency selective weighted median filter for video cameras , 1999, 1999 Digest of Technical Papers. International Conference on Consumer Electronics (Cat. No.99CH36277).

[20]  Yael Pritch,et al.  Clustered Synopsis of Surveillance Video , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[21]  Srinivasan Seshan,et al.  IrisNet: an internet-scale architecture for multimedia sensors , 2005, MULTIMEDIA '05.

[22]  Yael Pritch,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2008 1 Non-Chronological Video , 2022 .

[23]  Baining Guo,et al.  Context-aware textures , 2007, TOGS.

[24]  Qi Tian,et al.  Foreground object detection from videos containing complex background , 2003, MULTIMEDIA '03.

[25]  Jiang Yu Zheng.  Digital Route Panoramas , 2003, IEEE Multim..

[26]  Adam Finkelstein,et al.  Video tapestries with continuous temporal zoom , 2010, SIGGRAPH 2010.

[27]  Jiang Yu Zheng,et al.  Temporal Mapping of Surveillance Video , 2014, 2014 22nd International Conference on Pattern Recognition.

[28]  Stan Z. Li,et al.  Online content-aware video condensation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  K. Ishii,et al.  Automatic vehicle image extraction based on spatio-temporal image analysis , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[30]  Jiang Yu Zheng,et al.  Automatic heterogeneous video summarization in temporal profile , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[31]  Jean-Marc Odobez,et al.  A Sequential Topic Model for Mining Recurrent Activities from Long Term Video Logs , 2013, International Journal of Computer Vision.

[32]  Jiang Yu Zheng,et al.  Detecting walking pedestrians from leg motion in driving video , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[33]  Jiang Yu Zheng,et al.  Removing Temporal Stationary Blur in Route Panoramas , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[34]  Saburo Tsuji,et al.  Generating Dynamic Projection Images for Scene Representation and Understanding , 1998, Comput. Vis. Image Underst..

[35]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[36]  N. Pettersson,et al.  A new pedestrian dataset for supervised learning , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[37]  Jiang Yu Zheng,et al.  Localized temporal profile of surveillance video , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[38]  Leonard McMillan,et al.  General Linear Cameras , 2004, ECCV.

[39]  Slimane Larabi,et al.  Background Subtraction Algorithms with Post-processing: A Review , 2014, 2014 22nd International Conference on Pattern Recognition.

[40]  Yuzhen Niu,et al.  Direct manipulation video navigation in 3D , 2013, CHI.

[41]  Adrian Schischmanow,et al.  A Traffic Object Detection System for Road Traffic Measurement and Management , 2003 .

[42]  Osama Masoud,et al.  Detection and classification of vehicles , 2002, IEEE Trans. Intell. Transp. Syst..

[43]  Bo Yang,et al.  VISATRAM: a real-time vision system for automatic traffic monitoring , 2000, Image Vis. Comput..

[44]  Hiroshi Ishiguro,et al.  Memory-Based Attention Control for Activity Recognition at a Subway Station , 2007, IEEE MultiMedia.

[45]  Min Chen,et al.  Video visualization , 2003 .

[46]  Yuzhen Niu,et al.  Video summagator: an interface for video summarization and navigation , 2012, CHI.

[47]  Thomas S. Huang,et al.  A fast two-dimensional median filtering algorithm , 1979 .

[48]  Mongi A. Abidi,et al.  SAFER under vehicle inspection through video mosaic building , 2004, Ind. Robot.

[49]  Saburo Tsuji,et al.  From anorthoscope perception to dynamic vision , 1990, Proceedings., IEEE International Conference on Robotics and Automation.