A Hierarchical Markov Modeling Approach for the Segmentation and Tracking of Deformable Shapes

In many applications of dynamic scene analysis, the objects or structures to be analyzed undergo deformations that have to be modeled. In this paper, we develop a hierarchical statistical modeling framework for the representation, segmentation, and tracking of 2D deformable structures in image sequences. The model relies on the specification of a template, on which global as well as local deformations are defined. Global deformations are modeled using a statistical modal analysis of the deformations observed on a representative population. Local deformations are represented by a (first-order) Markov random process. A model-based segmentation of the scene is obtained by a joint bayesian estimation of global deformation parameters and local deformation variables. Spatial or spatio-temporal observations are considered in this estimation procedure, yielding an edge-based or a motion-based segmentation of the scene. The segmentation procedure is combined with a temporal tracking of the deformable structure over long image sequences, using a Kalman filtering approach. This combined segmentation-tracking procedure has produced reliable extraction of deformable parts from long image sequences in adverse situations such as low signal-to-noise ratio, nongaussian noise, partial occlusions, or random initialization. The approach is demonstrated on a variety of synthetic as well as real-world image sequences featuring different classes of deformable objects.

[1]  Alex Pentland,et al.  Closed-form solutions for physically-based shape modeling and recognition , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[3]  Sridhar Lakshmanan,et al.  Simultaneous Parameter Estimation and Segmentation of Gibbs Random Fields Using Simulated Annealing , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[5]  ISAAC COHEN,et al.  Using deformable surfaces to segment 3-D images and infer differential structures , 1992, CVGIP Image Underst..

[6]  Yiannis Aloimonos,et al.  Active vision , 2004, International Journal of Computer Vision.

[7]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Alex Pentland,et al.  Shape analysis of brain structures using physical and experimental modes , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Ali Mohammad-Djafari,et al.  On the estimation of hyperparameters in Bayesian approach of solving inverse problems , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[10]  Patrick Pérez,et al.  Restriction of a Markov random field on a graph and multiresolution statistical image modeling , 1996, IEEE Trans. Inf. Theory.

[11]  Christopher J. Taylor,et al.  Model-based image interpretation using genetic algorithms , 1992, Image Vis. Comput..

[12]  Nicholas Ayache,et al.  Fast segmentation, tracking, and analysis of deformable objects , 1993, 1993 (4th) International Conference on Computer Vision.

[13]  Guy Demoment,et al.  Image reconstruction and restoration: overview of common estimation structures and problems , 1989, IEEE Trans. Acoust. Speech Signal Process..

[14]  Alex Pentland,et al.  Recovery of Nonrigid Motion and Structure , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Michael Isard,et al.  Learning to Track the Visual Motion of Contours , 1995, Artif. Intell..

[16]  David C. Hogg,et al.  Learning Flexible Models from Image Sequences , 1994, ECCV.

[17]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[18]  Charles Kervrann,et al.  Robust tracking of stochastic deformable models in long image sequences , 1994, Proceedings of 1st International Conference on Image Processing.

[19]  D. Adam,et al.  Automatic ventricular cavity boundary detection from sequential ultrasound images using simulated annealing. , 1989, IEEE transactions on medical imaging.

[20]  D. Hinkley.  Inference about the change-point from cumulative sum tests , 1971 .

[21]  Charles Kervrann,et al.  Statistical model-based segmentation of deformable motion , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[22]  MARIETTE MAURIZOT,et al.  891 LOCATING SINGULAR POINTS AND CHARACTERIZING DEFORMABLE FLOW FIELDS IN AN IMAGE SEQUENCE , 1994 .

[23]  Andrew Blake,et al.  Dynamic contours: real-time active splines , 1993 .

[24]  Timothy F. Cootes,et al.  The Use of Active Shape Models for Locating Structures in Medical Images , 1993, IPMI.

[25]  Richard Szeliski,et al.  Tracking with Kalman snakes , 1993 .

[26]  Thomas S. Huang,et al.  Image processing , 1971 .

[27]  Rachid Deriche,et al.  Tracking complex primitives in an image sequence , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[28]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[29]  D. M. Keenan,et al.  Towards automated image understanding , 1989 .

[30]  C.-C. Jay Kuo,et al.  Wavelet descriptor of planar curves: theory and applications , 1996, IEEE Trans. Image Process..

[31]  Don R. Hush,et al.  Change detection for target detection and classification in video sequences , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[32]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[33]  James S. Duncan,et al.  Boundary Finding with Parametrically Deformable Models , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Dimitris N. Metaxas,et al.  Dynamic 3D models with local and global deformations: deformable superquadrics , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[35]  Patrick Bouthemy,et al.  Modeling of Atmospheric Disturbances in Meteorological Pictures , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  U. Grenander,et al.  Structural Image Restoration through Deformable Templates , 1991 .

[37]  Alex Pentland,et al.  Recursive estimation of structure and motion using relative orientation constraints , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[38]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[39]  Rachid Deriche,et al.  Tracking line segments , 1990, Image Vis. Comput..

[40]  Stephen J. Riederer,et al.  Medical imaging [6] , 1995 .

[41]  Alex Pentland,et al.  Recovery of non-rigid motion and structure , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[42]  Charles Kervrann,et al.  A hierarchical statistical framework for the segmentation of deformable objects in image sequences , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Demetri Terzopoulos,et al.  Topologically adaptable snakes , 1995, Proceedings of IEEE International Conference on Computer Vision.

[44]  Karl Rohr,et al.  Incremental recognition of pedestrians from image sequences , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Thomas S. Huang,et al.  Salient structure analysis of fluid flow , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Charles Kervrann,et al.  A Markov random field model-based approach to unsupervised texture segmentation using local and global spatial statistics , 1995, IEEE Trans. Image Process..

[47]  Ulf Grenander,et al.  Hands: A Pattern Theoretic Study of Biological Shapes , 1990 .

[48]  Patrick Bouthemy,et al.  Multimodal Estimation of Discontinuous Optical Flow using Markov Random Fields , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Baba C. Vemuri,et al.  Shape Modeling with Front Propagation: A Level Set Approach , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Kanti V. Mardia,et al.  Deformable templates in image sequences , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[51]  Patrick Bouthemy,et al.  Region-Based Tracking Using Affine Motion Models in Long Image Sequences , 1994 .

[52]  Hiroshi Murase,et al.  Learning and recognition of 3D objects from appearance , 1993, [1993] Proceedings IEEE Workshop on Qualitative Vision.

[53]  Timothy F. Cootes,et al.  Use of active shape models for locating structures in medical images , 1994, Image Vis. Comput..

[54]  Roland T. Chin,et al.  Deformable contours: modeling and extraction , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.