Representing moving images with layers

We describe a system for representing moving images with sets of overlapping layers. Each layer contains an intensity map that defines the additive values of each pixel, along with an alpha map that serves as a mask indicating the transparency. The layers are ordered in depth and they occlude each other in accord with the rules of compositing. Velocity maps define how the layers are to be warped over time. The layered representation is more flexible than standard image transforms and can capture many important properties of natural image sequences. We describe some methods for decomposing image sequences into layers using motion analysis, and we discuss how the representation may be used for image coding and other applications.

[1]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[2]  T. S. Huang,et al.  Image Sequence Enhancement , 1981 .

[3]  R. Lenz,et al.  Image Sequence Coding Using Scene Analysis and Spatio-Temporal Interpolation , 1983 .

[4]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[6]  A. Dale Magoun,et al.  Decision, estimation and classification , 1989 .

[7]  Jörn Ostermann,et al.  Object-oriented analysis-synthesis coding of moving images , 1989, Signal Process. Image Commun..

[8]  Shmuel Peleg,et al.  Computing two motions from three frames , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[9]  Gary J. Sullivan,et al.  Motion compensation for video compression using control grid interpolation , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[10]  Patrick Campbell McLean,et al.  Structured video coding , 1991 .

[11]  A. Pentland,et al.  Robust estimation of a multi-layered motion representation , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[12]  Hiroshi Harashima,et al.  Iterative motion estimation method using triangular patches for motion compensation , 1991, Other Conferences.

[13]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[14]  Kenji Mase,et al.  Unified computational theory for motion transparency and motion boundaries based on eigenenergy analysis , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Kiyoharu Aizawa,et al.  Human facial motion modeling, analysis, and synthesis for video compression , 1991, Other Conferences.

[16]  Michael J. Black,et al.  Robust dynamic motion estimation over time , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Michal Irani,et al.  Image sequence enhancement using multiple motions analysis , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Ichiro Matsuda,et al.  Adaptive transform image coding based on variable-shape block segmentation with smoothing filter , 1992, Other Conferences.

[19]  Demetri Terzopoulos,et al.  Adaptive meshes and shells: irregular triangulation, discontinuities, and hierarchical subdivision , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Eric Dubois,et al.  Motion estimation with detection of occlusion areas , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[21]  Edward H. Adelson,et al.  Layered representation for motion analysis , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Edward H. Adelson,et al.  Layered representation for image sequence coding , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[23]  Demetri Terzopoulos,et al.  Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Ujjaval Yogesh Desai,et al.  Coding of segmented image sequences , 1994 .

[25]  John Wang,et al.  Applying mid-level vision techniques for video data compression and manipulation , 1994, Electronic Imaging.