3D structure extraction coding of image sequences

Abstract This paper presents a 3D structure extraction coding scheme that first computes the 3D structural properties such as 3D shape, motion, and location of objects and then codes image sequences by utilizing such 3D information. The goal is to achieve efficient and flexible coding while still avoiding the visual distortions through the use of 3D scene characteristics inherent in image sequences. To accomplish this, we present two multiframe algorithms for the robust estimation of such 3D structural properties, one from motion and one from stereo. The approach taken in these algorithms is to successively estimate 3D information from a longer sequence for a significant reduction in error. Three variations of 3D structure extraction coding are then presented — 3D motion interpolative coding, 3D motion compensation coding, and “viewpoint” compensation stereo image coding — to suggest that the approach can be viable for high-quality visual communications.

[1]  Kiyoharu Aizawa,et al.  Model-based analysis synthesis image coding (MBASIC) system for a person's face , 1989, Signal Process. Image Commun..

[2]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[3]  Demetri Terzopoulos,et al.  Multilevel computational processes for visual surface reconstruction , 1983, Comput. Vis. Graph. Image Process..

[4]  Hiroyuki Morikawa,et al.  Rigid and Nonrigid Motion Analysis: Robust Recovery of 3-D Structure and Motion , 1990, MVA.

[5]  Jörn Ostermann,et al.  Object-oriented analysis-synthesis coding of moving images , 1989, Signal Process. Image Commun..

[6]  M. Lukacs,et al.  Predictive coding of multi-viewpoint image sets , 1986, ICASSP '86. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[7]  R. Franke.  Scattered data interpolation: tests of some methods , 1982 .

[8]  Don E. Pearson,et al.  Texture mapping in model-based image coding , 1990, Signal Process. Image Commun..

[9]  Y. Aloimonos,et al.  Visual shape computation , 1988, Proc. IEEE.

[10]  Joachim Heel,et al.  Temporally integrated surface reconstruction , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[11]  James D. Foley,et al.  Fundamentals of interactive computer graphics , 1982 .

[12]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[13]  H. Morikawa,et al.  Structure and motion of deformable objects from image sequences , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[14]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[15]  Eric L. W. Grimson,et al.  From Images to Surfaces: A Computational Study of the Human Early Visual System , 1981 .

[16]  D. Pearson.  Model-based image coding , 1989, IEEE Global Telecommunications Conference, 1989, and Exhibition. 'Communications Technology for the 1990s and Beyond.

[17]  Joseph K. Kearney,et al.  Optical Flow Estimation: An Error Analysis of Gradient-Based Methods with Local Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Robert Forchheimer,et al.  Image coding-from waveforms in animation , 1989, IEEE Trans. Acoust. Speech Signal Process..

[19]  H. Morikawa,et al.  3-D structure extraction coding of image sequences , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[20]  J. Yan,et al.  Encoding of Images Based on a Two-Component Source Model , 1977, IEEE Trans. Commun..