Two-dimensional mesh-based visual-object representation for interactive synthetic/natural digital video

This paper first provides an overview of two-dimensional (2-D) and three-dimensional mesh models for digital video processing. It then introduces 2-D mesh-based modeling of video objects as a compact representation of motion and shape for interactive, synthetic/natural video manipulation, compression, and indexing. The 2-D mesh representation and the mesh geometry and motion compression have been included in the visual tools of the upcoming MPEG-4 standard. Functionalities enabled by 2-D mesh-based visual-object representation include animation of still texture maps, transfiguration of video overlays, video morphing, and shape-and motion-based retrieval of video objects.

[1]  Yao Wang,et al.  Active mesh-a feature seeking and tracking image sequence representation scheme , 1994, IEEE Trans. Image Process..

[2]  Leonardo Chiariglione.  MPEG and multimedia communications , 1997, IEEE Trans. Circuits Syst. Video Technol..

[3]  A. Murat Tekalp,et al.  Hierarchical temporal video segmentation and content characterization , 1997, Other Conferences.

[4]  Petros Maragos,et al.  Affine models for image matching and motion detection , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[5]  Jörn Ostermann,et al.  Object-oriented analysis-synthesis coding of moving images , 1989, Signal Process. Image Commun..

[6]  A. Murat Tekalp,et al.  Object-based video indexing for virtual-studio productions , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Minerva M. Yeung,et al.  Efficient matching and clustering of video shots , 1995, Proceedings., International Conference on Image Processing.

[8]  Fernando Pereira,et al.  MPEG-4: Context and objectives , 1997, Signal Process. Image Commun..

[9]  M. Kunt,et al.  Second-generation image-coding techniques , 1985, Proceedings of the IEEE.

[10]  Gary J. Sullivan,et al.  Motion compensation for video compression using control grid interpolation , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[11]  Shmuel Peleg,et al.  A Three-Frame Algorithm for Estimating Two-Component Image Motion , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  John E. Howland,et al.  Computer graphics , 1990, IEEE Potentials.

[13]  Jonathan D. Courtney.  Automatic video indexing via object motion analysis , 1997, Pattern Recognit..

[14]  Arun N. Netravali,et al.  Digital Pictures: Representation and Compression , 1988 .

[15]  A. Murat Tekalp,et al.  Video indexing through integration of syntactic and semantic features , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[16]  J. Nieweglowski,et al.  A novel video coding scheme based on temporal prediction using digital image warping , 1993, IEEE 1993 International Conference on Consumer Electronics Digest of Technical Papers.

[17]  A. Murat Tekalp,et al.  Object-based video manipulation and composition using 2D meshes in VRML , 1997, Proceedings of First Signal Processing Society Workshop on Multimedia Signal Processing.

[18]  Michal Irani,et al.  Video indexing based on mosaic representations , 1998, Proc. IEEE.

[19]  A. Murat Tekalp,et al.  Tracking Motion and Intensity Variations Using Hierarchical 2-D Mesh Modeling for Synthetic Object Transfiguration , 1996, CVGIP Graph. Model. Image Process..

[20]  Mohammad Ghanbari,et al.  General approach to block-matching motion estimation , 1993 .

[21]  Stephen W. Smoliar,et al.  Content-based video browsing tools , 1995, Electronic Imaging.

[22]  George Wolberg,et al.  Digital image warping , 1990 .

[23]  Gabriel Taubin,et al.  Geometric compression through topological surgery , 1998, TOGS.

[24]  A. Murat Tekalp,et al.  Two-dimensional object-based coding using a content-based mesh and affine motion parameterization , 1995, Proceedings., International Conference on Image Processing.

[25]  D. Legall,et al.  MPEG : A video compression standard for multimedia applications , 1991 .

[26]  A. Murat Tekalp,et al.  Simultaneous alpha map generation and 2-D mesh tracking for multimedia applications , 1997, Proceedings of International Conference on Image Processing.

[27]  A. Murat Tekalp,et al.  Object-based video coding using forward-tracking 2D mesh layers , 1997, Electronic Imaging.

[28]  오승준.  [서평]「Digital Video Processing」 , 1996 .

[29]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[30]  Haibo Li,et al.  Image sequence coding at very low bit rates: a review , 1994, IEEE Trans. Image Process..

[31]  A. Murat Tekalp,et al.  Special effects authoring using 2-D mesh models , 1997, Proceedings of International Conference on Image Processing.

[32]  Didier Le Gall,et al.  MPEG: a video compression standard for multimedia applications , 1991, CACM.

[33]  Behzad Shahraray,et al.  Automatic generation of pictorial transcripts of video programs , 1995, Electronic Imaging.

[34]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[35]  Michael Deering,et al.  Geometry compression , 1995, SIGGRAPH.

[36]  H. Brusewitz,et al.  Motion compensation with triangles , 1990 .

[37]  Yao Wang,et al.  Use of two-dimensional deformable mesh structures for video coding .I. The synthesis problem: mesh-based function approximation and mapping , 1996, IEEE Trans. Circuits Syst. Video Technol..

[38]  Hiroshi Harashima,et al.  Motion compensation based on spatial transformations , 1994, IEEE Trans. Circuits Syst. Video Technol..

[39]  Tsuhan Chen,et al.  A new frame interpolation scheme for talking head sequences , 1995, Proceedings., International Conference on Image Processing.

[40]  A. Murat Tekalp,et al.  Two-dimensional triangular mesh-based mosaicking for object tracking in the presence of occlusion , 1997, Electronic Imaging.

[41]  Anthony Vetro,et al.  Use of two-dimensional deformable mesh structures for video coding. II. The analysis problem and a region-based coder employing an active mesh representation , 1996, IEEE Trans. Circuits Syst. Video Technol..

[42]  A. Murat Tekalp,et al.  Occlusion-adaptive, content-based mesh design and forward tracking , 1997, IEEE Trans. Image Process..

[43]  A. Murat Tekalp,et al.  2-D mesh-based tracking of deformable objects with occlusion , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[44]  Yao Wang,et al.  Motion-Compensated Prediction Using Nodal-Based Deformable Block Matching , 1995, J. Vis. Commun. Image Represent..

[45]  Shih-Fu Chang,et al.  Scene change detection in an MPEG-compressed video sequence , 1995, Electronic Imaging.

[46]  R. Urtasun.  Image Warping , 2000 .

[47]  A. Murat Tekalp,et al.  2-D mesh-based synthetic transfiguration of an object with occlusion , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[48]  John Beidler,et al.  Data Structures and Algorithms , 1996, Wiley Encyclopedia of Computer Science and Engineering.

[49]  A. Murat Tekalp,et al.  Closed-form connectivity-preserving solutions for motion compensation using 2-D meshes , 1997, IEEE Trans. Image Process..

[50]  Michael T. Orchard,et al.  Overlapped block motion compensation: an estimation-theoretic approach , 1994, IEEE Trans. Image Process..

[51]  Mark de Berg,et al.  Computational geometry: algorithms and applications , 1997 .

[52]  Hideo Hashimoto,et al.  Video indexing using motion vectors , 1992, Other Conferences.

[53]  Michael T. Orchard,et al.  Optimal unified approach to warping and overlapped block estimation in video coding , 1996, Other Conferences.

[54]  Michael T. OrchardDepartment.  Optimal Uniied Approach to Warping and Overlapped Block Motion Estimation in Video Coding , 1996 .

[55]  Jonathan Richard Shewchuk,et al.  Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator , 1996, WACG.

[56]  Chung-Lin Huang,et al.  A new motion compensation method for image sequence coding using hierarchical grid interpolation , 1994, IEEE Trans. Circuits Syst. Video Technol..

[57]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Ramesh C. Jain,et al.  Knowledge-guided parsing in video databases , 1993, Electronic Imaging.

[59]  Thomas Sikora,et al.  The MPEG-4 video standard verification model , 1997, IEEE Trans. Circuits Syst. Video Technol..

[60]  Hugues Hoppe,et al.  Progressive meshes , 1996, SIGGRAPH.

[61]  A. Murat Tekalp,et al.  2-D mesh geometry and motion compression for efficient object-based video representation , 1997, Proceedings of International Conference on Image Processing.

[62]  Boon-Lock Yeo,et al.  Video content characterization and compaction for digital library applications , 1997, Electronic Imaging.

[63]  Boon-Lock Yeo,et al.  Rapid scene analysis on compressed video , 1995, IEEE Trans. Circuits Syst. Video Technol..