Motion segmentation and qualitative dynamic scene analysis from an image sequence

This article deals with analysis of the dynamic content of a scene from an image sequence irrespective of the static or dynamic nature of the camera. The tasks involved can be the detection of moving objects in a scene observed by a mobile camera, or the identification of the movements of some relevant components of the scene relatively to the camera. This problem basically requires a motion-based segmentation step. We present a motion-based segmentation method relying on 2-D affine motion models and a statistical regularization approach which ensures stable motion-based partitions. This can be done without the explicit estimation of optic flow fields. Besides these partitions are linked in time. Therefore, the motion interpretation process can be performed on more than two successive frames. The ability to follow a given coherently moving region within an interval of several images of the sequence makes the interpretation process more robust and more comprehensive. Identification of the kinematic components of the scene is induced from an intermediate layer accomplishing a generic qualitative motion labeling. No 3-D measurements are required. Results obtained on several real-image sequences corresponding to complex outdoor situations are reported.

[1]  Patrick Bouthemy,et al.  Multimodal motion estimation and segmentation using Markov random fields , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[2]  Peter J. Burt,et al.  Object tracking with a moving camera , 1989, [1989] Proceedings. Workshop on Visual Motion.

[3]  Tzay Y. Young,et al.  COMPUTATION OF IMAGE VELOCITY FIELD USING LINEAR AND NONLINEAR OBJECTIVE FUNCTIONS. , 1987 .

[4]  Andrea J. van Doorn,et al.  Invariant Properties of the Motion Parallax Field due to the Movement of Rigid Bodies Relative to an Observer , 1975 .

[5]  N. Ahuja,et al.  Segmentation, matching and estimation of structure and motion of textured piecewise planar surfaces , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[6]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[7]  Tomaso Poggio,et al.  Probabilistic Solution of Ill-Posed Problems in Computational Vision , 1987 .

[8]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Tomaso A. Poggio,et al.  Motion Field and Optical Flow: Qualitative Properties , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Christopher M. Brown,et al.  The theory and practice of Bayesian image labeling , 1990, International Journal of Computer Vision.

[11]  Jin Luo,et al.  Computing motion using analog and binary resistive networks , 1988, Computer.

[12]  Michael J. Black,et al.  Robust dynamic motion estimation over time , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[14]  Patrick Bouthemy,et al.  Detection and Tracking of Moving Objects Based on a Statistical Regularization Method in Space and Time , 1990, European Conference on Computer Vision.

[15]  Anil K. Jain,et al.  Random field models in image analysis , 1989 .

[16]  David W. Murray,et al.  Scene Segmentation from Visual Motion Using Global Optimization , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Valdis Berzins,et al.  ANALYZING OBJECT MOTION BASED ON OPTICAL FLOW. , 1984 .

[18]  Petros Maragos,et al.  Affine models for image matching and motion detection , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[19]  William B. Thompson,et al.  Detecting moving objects , 1989, International Journal of Computer Vision.

[20]  Randal C. Nelson,et al.  Using Flow Field Divergence For Obstacle Avoidance: Towards Qualitative Vision , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[21]  P. Simard,et al.  Restoration of the velocity field of the heart from two-dimensional echocardiograms. , 1989, IEEE transactions on medical imaging.

[22]  Azriel Rosenfeld,et al.  Robust regression methods for computer vision: A review , 1991, International Journal of Computer Vision.

[23]  Alan L. Yuille,et al.  A common framework for image segmentation , 1990, International Journal of Computer Vision.

[24]  Patrick Bouthemy,et al.  Constrained multiscale Markov random fields and the analysis of visual motion , 1992 .

[25]  Charles R. Dyer,et al.  Long-range spatiotemporal motion understanding using spatiotemporal flow curves , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Yiannis Aloimonos,et al.  Purposive and qualitative active vision , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[27]  B. G. Schunck.  The image flow constraint equation , 1986 .

[28]  Patrick Bouthemy,et al.  Multiframe-based identification of mobile components of a scene with a moving camera , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  S. Negahdaripour,et al.  Motion recovery from image sequences using First-order optical flow information , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[30]  Robert J. Schalkoff,et al.  A Model and Tracking Algorithm for a Class of Video Targets , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Patrick Bouthemy,et al.  Region-Based Tracking in an Image Sequence , 1992, ECCV.

[32]  Randal C. Nelson.  Qualitative detection of motion by a moving observer , 2004, International Journal of Computer Vision.

[33]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Shmuel Peleg,et al.  Motion based segmentation , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[35]  Donald Geman,et al.  Boundary Detection by Constrained Optimization , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Allen M. Waxman,et al.  The Analytic Structure of Image Flows: Deformation and Segmentation , 1990, Comput. Vis. Graph. Image Process..

[37]  Bir Bhanu,et al.  Qualitative Motion Understanding , 1987, IJCAI.

[38]  Ralph Hartley,et al.  Segmentation of optical flow fields by pyramid linking , 1985, Pattern Recognit. Lett..

[39]  Patrick Bouthemy,et al.  Derivation of qualitative information in motion analysis , 1990, Image Vis. Comput..

[40]  Anselm Spoerri,et al.  The early detection of motion boundaries , 1990, ICCV 1987.

[41]  Muralidhara Subbarao.  Interpretation of Image Flow: A Spatio-Temporal Approach , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Muralidhara Subbarao.  Bounds on time-to-collision and rotational component from first-order derivatives of image flow Stony Brook, New York 11794-2350 , 1990 .