Ieee Transactions on Image Processing 1 Figure–ground Segmentation from Occlusion Ieee Transactions on Image Processing

Layered video representations are increasingly popular; see for a recent review. Segmentation of moving objects is a key step for automating such representations. Current motion segmentation methods either fail to segment moving objects in low-textured regions or are computationally very expensive. This paper presents a computationally simple algorithm that segments moving objects, even in low-texture/low-contrast scenes. Our method infers the moving object templates directly from the image intensity values, rather than computing the motion field as an intermediate step. Our model takes into account the rigidity of the moving object and the occlusion of the background by the moving object. We formulate the segmentation problem as the minimization of a penalized likelihood cost function and present an algorithm to estimate all the unknown parameters: the motions, the template of the moving object, and the intensity levels of the object and of the background pixels. The cost function combines a maximum likelihood estimation term with a term that penalizes large templates. The minimization algorithm performs two alternate steps for which we derive closed-form solutions. Relaxation improves the convergence even when low texture makes it very challenging to segment the moving object from the background. Experiments demonstrate the good performance of our method.

[1]  Michal Irani,et al.  Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency , 1993, J. Vis. Commun. Image Represent..

[2]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[3]  José M. F. Moura,et al.  Content-based video sequence representation , 1995, Proceedings., International Conference on Image Processing.

[4]  José M. F. Moura,et al.  Detecting and solving template ambiguities in motion segmentation , 1997, Proceedings of International Conference on Image Processing.

[5]  Ming-Ting Sun,et al.  A robust video object segmentation scheme with prestored background information , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[6]  José M. F. Moura,et al.  Cardiac MR image segmentation: quality assessment of STACS , 2004, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[7]  Anil K. Jain,et al.  Contour extraction of moving objects in complex outdoor scenes , 1995, International Journal of Computer Vision.

[8]  Baba C. Vemuri,et al.  Shape Modeling with Front Propagation: A Level Set Approach , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Hai Tao,et al.  A background layer model for object tracking through occlusion , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[11]  Brendan J. Frey,et al.  Learning flexible sprites in video layers , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[13]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[14]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[15]  L. Ambrosio,et al.  Approximation of functional depending on jumps by elliptic functional via t-convergence , 1990 .

[16]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Michael J. Black,et al.  On the unification of line processes, outlier rejection, and robust statistics with applications in early vision , 1996, International Journal of Computer Vision.

[18]  Baoxin Li,et al.  Adaptive video background replacement , 2001, IEEE International Conference on Multimedia and Expo, 2001. ICME 2001..

[19]  José M. F. Moura,et al.  Video compression via constructs , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[20]  Hai Tao,et al.  Dynamic layer representation with applications to tracking , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[21]  Ming-Chieh Lee,et al.  Semiautomatic segmentation and tracking of semantic video objects , 1998, IEEE Trans. Circuits Syst. Video Technol..

[22]  Lam Kun Ma New approaches to fractal image and very low bit-rate video compression , 1996 .

[23]  Touradj Ebrahimi,et al.  Video segmentation based on multiple features for interactive multimedia applications , 1998, IEEE Trans. Circuits Syst. Video Technol..

[24]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[25]  José M. F. Moura,et al.  Three-dimensional modeling from two-dimensional video , 2001, IEEE Trans. Image Process..

[26]  Maria Huhtala,et al.  Random Variables and Stochastic Processes , 2021, Matrix and Tensor Decompositions in Signal Processing.

[27]  Harpreet S. Sawhney,et al.  Compact Representations of Videos Through Dominant and Multiple Motion Estimation , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Sergio VerdÂ,et al.  The Minimum Description Length Principle in Coding and Modeling , 2000 .

[29]  José M. F. Moura,et al.  Rank 1 Weighted Factorization for 3D Structure Recovery: Algorithms and Performance Analysis , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Jean-Michel Morel,et al.  Variational methods in image segmentation , 1995 .

[31]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[32]  James S. Duncan,et al.  Game-Theoretic Integration for Image Segmentation , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  José M. F. Moura,et al.  Maximum Likelihood Estimation of the Template of a Rigid Moving Object , 2001, EMMCVPR.

[34]  David Mumford,et al.  Communications on Pure and Applied Mathematics , 1989 .

[35]  Patrick Bouthemy,et al.  Motion segmentation and qualitative dynamic scene analysis from an image sequence , 1993, International Journal of Computer Vision.

[36]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[37]  Manuela M. Veloso,et al.  A layered approach for an autonomous robotic soccer system , 1997, AGENTS '97.

[38]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[39]  J. Berger Statistical Decision Theory and Bayesian Analysis , 1988 .

[40]  Edward H. Adelson,et al.  Representing moving images with layers , 1994, IEEE Trans. Image Process..

[41]  Haibo Li,et al.  Image sequence coding at very low bit rates: a review , 1994, IEEE Trans. Image Process..

[42]  José M. F. Moura,et al.  Stochastic active contour for cardiac MR image segmentation , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[43]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[44]  Michal Irani,et al.  Computing occluding and transparent motions , 1994, International Journal of Computer Vision.

[45]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  I. Miller Probability, Random Variables, and Stochastic Processes , 1966 .

[47]  Brendan J. Frey,et al.  Transformed hidden Markov models: estimating mixture models of images and inferring spatial transformations in video sequences , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[48]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[49]  Jorma Rissanen,et al.  The Minimum Description Length Principle in Coding and Modeling , 1998, IEEE Trans. Inf. Theory.

[50]  G. Sapiro,et al.  Geometric partial differential equations and image analysis [Book Reviews] , 2001, IEEE Transactions on Medical Imaging.

[51]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[52]  Anil K. Jain Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[53]  Pedro M. Q. Aguiar,et al.  Rigid Structure from Video , 2005 .

[54]  José M. F. Moura,et al.  Content-based Image Sequence Representation ? , 2004 .

[55]  Norbert Diehl,et al.  Object-oriented motion estimation and segmentation in image sequences , 1991, Signal Process. Image Commun..