Action-based Multi-field Video Visualization

One challenge in video processing is to detect actions and events, known or unknown, in video streams dynamically. This paper proposes a visualization solution, where a video stream is depicted as a series of snapshots at a relatively sparse interval, and detected actions are highlighted with continuous abstract illustrations. The combined imagery and illustrative visualization conveys multi-field information in a manner similar to electrocardiograms (ECG) and seismographs. We thus name this type of video visualization as VideoPerpetuoGram (VPG). In this paper, we describe a system that handles the raw and processed information of the video stream in a multi-field visualization pipeline. As examples, we consider the needs for highlighting several types of processed information, including detected actions in video streams, and estimated relationship between recognized objects. We examine the effective means for depicting multi-field information in VPG, and support our choice of visual mappings through a survey. Our GPU implementation facilitates the VPGspecific viewing specification through a sheared object space, as well as volume bricking and combinational rendering of volume data and glyphs.

[1]  Anthony L Bertapelle.  Spectral Analysis of Time Series. , 1979 .

[2]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[3]  Colin Ware,et al.  Color sequences for univariate maps: theory, experiments and principles , 1988, IEEE Computer Graphics and Applications.

[4]  Giacomo Mauro DAriano.  Statistical Analysis of Behavioural Data: An Approach Based on Time-Structured Models. , 1994 .

[5]  Brian Cabral,et al.  Accelerated volume rendering and tomographic reconstruction using texture mapping hardware , 1994, VVS '94.

[6]  Kazuo Kyuma,et al.  Computer vision for computer games , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[7]  Christopher G. Healey,et al.  Choosing effective colours for data visualization , 1996, Proceedings of Seventh Annual IEEE Visualization '96.

[8]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[9]  Sidney Fels,et al.  Techniques for Interactive Video Cubism , 2000 .

[10]  Colin Ware,et al.  Information Visualization: Perception for Design , 2000 .

[11]  Ken Perlin,et al.  Painterly rendering for video and interaction , 2000, NPAR '00.

[12]  Peter-Pike J. Sloan,et al.  Video Cubism , 2001 .

[13]  Mubarak Shah,et al.  View-invariance in action recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[14]  M. Irani,et al.  Event-Based Video Analysis, , 2001 .

[15]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[16]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Lihi Zelnik-Manor,et al.  Event-based analysis of video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[18]  Erik Reinhard,et al.  Face-based luminance matching for perceptual colormap generation , 2002, IEEE Visualization, 2002. VIS 2002..

[19]  G. Daniel,et al.  Video visualization , 2003, IEEE Visualization, 2003. VIS 2003..

[20]  Yang Song,et al.  Unsupervised Learning of Human Motion , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Jitendra Malik,et al.  Recognizing action at a distance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[22]  Kiyoharu Aizawa,et al.  A Solid-State, Simultaneous Wide Angle - Detailed View Video Surveillance Camera , 2003, VMV.

[23]  Suya You,et al.  3D video surveillance with Augmented Virtual Environments , 2003, IWVS '03.

[24]  Mubarak Shah,et al.  View-Invariant Representation and Recognition of Actions , 2002, International Journal of Computer Vision.

[25]  James J. Little,et al.  A Boosted Particle Filter: Multitarget Detection and Tracking , 2004, ECCV.

[26]  Robert B. Fisher,et al.  The PETS04 Surveillance Ground-Truth Data Sets , 2004 .

[27]  Eli Shechtman,et al.  Space-time behavior based correlation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  Gordon Erlebacher,et al.  Overview of Flow Visualization , 2005, The Visualization Handbook.

[29]  Thomas Ertl,et al.  A Generic Software Framework for the GPU Volume Rendering Pipeline , 2005 .

[30]  Timo Ropinski,et al.  Illustrating Dynamics of Time-Varying Volume Datasets in Static Images , 2006 .

[31]  Min Chen,et al.  Visual Signatures in Video Visualization , 2006, IEEE Transactions on Visualization and Computer Graphics.

[32]  Min Chen,et al.  GPU-assisted Multi-field Video Volume Visualization , 2006, VG@SIGGRAPH.

[33]  Yasuyuki Matsushita,et al.  Dynamic stills and clip trailers , 2006, The Visual Computer.

[34]  Yi Wang,et al.  Contextualized Videos: Combining Videos with Environment Models to Support Situational Understanding , 2007, IEEE Transactions on Visualization and Computer Graphics.