An EM/E-MRF algorithm for adaptive model based tracking in extremely poor visibility

This paper addresses the problems of visual tracking in conditions of extremely poor visibility. The human visual system can often correctly interpret images that are of such poor quality that they contain insufficient explicit information to do so. We assert that such systems must therefore make use of prior knowledge in several forms. A tracking algorithm is presented which combines observed data (the current image) with predicted data derived from prior knowledge of the object being viewed and an estimate of the camera's motion. During image segmentation, a predicted image is used to estimate class conditional distribution models and an Extended-Markov Random Field technique is used to combine observed image data with expectations of that data within a probabilistic framework. Interpretations of scene content and camera position are then mutually improved using Expectation Maximisation. Models of background and tracked object are continually relearned and adapt iteratively with each new image frame. The algorithm is tested using real video sequences, filmed in poor visibility conditions with complete pre-measured ground-truth data.

[1]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Christopher M. Brown,et al.  The theory and practice of Bayesian image labeling , 1990, International Journal of Computer Vision.

[3]  Anil K. Jain,et al.  MRF model-based algorithms for image segmentation , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[4]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  P. Fieguth,et al.  4.3 – Random Field Models , 2005 .

[6]  Donald B. Gennery,et al.  Visual tracking of known three-dimensional objects , 1992, International Journal of Computer Vision.

[7]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[8]  Wolfram Burgard,et al.  Monte Carlo Localization: Efficient Position Estimation for Mobile Robots , 1999, AAAI/IAAI.

[9]  Gian Luca Foresti,et al.  Visual inspection of sea bottom structures by an autonomous underwater vehicle , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[10]  R. E. Allsop,et al.  Bayesian analysis for fusion of data from disparate imaging systems for surveillance , 2003, Image Vis. Comput..

[11]  David G. Lowe,et al.  Robust model-based motion tracking through the integration of search and estimation , 1992, International Journal of Computer Vision.

[12]  S. Ullman.  High-Level Vision: Object Recognition and Visual Cognition , 1996 .

[13]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[14]  King Ngi Ngan,et al.  A robust Markovian segmentation based on highest confidence first (HCF) , 1997, Proceedings of International Conference on Image Processing.

[15]  Ramesh C. Jain,et al.  Three-dimensional object recognition , 1985, CSUR.

[16]  Timothy F. Cootes,et al.  A mixture model for representing shape variation , 1999, Image Vis. Comput..

[17]  Gary R. Bradski,et al.  Real time face and object tracking as a component of a perceptual user interface , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[18]  Rustam Stolkin,et al.  Measuring complete ground-truth data and error estimates for real video , 2005 .

[19]  M. Hebert,et al.  The Representation, Recognition, and Locating of 3-D Objects , 1986 .

[20]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[21]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[22]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[23]  Takeo Kanade,et al.  Advances in Cooperative Multi-Sensor Video Surveillance , 1999 .

[24]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[25]  Tomaso Poggio,et al.  Probabilistic Solution of Ill-Posed Problems in Computational Vision , 1987 .

[26]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Roberto Cipolla,et al.  Real-time tracking of complex structures with on-line camera calibration , 2002, Image Vis. Comput..

[28]  P. Kohli,et al.  Efficiently solving dynamic Markov random fields using graph cuts , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[29]  Shree K. Nayar,et al.  Vision and the Atmosphere , 2002, International Journal of Computer Vision.

[30]  William H. Press,et al.  Numerical recipes in C , 2002 .

[31]  Rustam Stolkin,et al.  An EM / E-MRF Strategy for Underwater Navigation , 2000, BMVC.

[32]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[33]  Rustam Stolkin,et al.  A calibration system for measuring 3D ground truth for validation and error analysis of robot vision algorithms , 2006 .

[34]  W. Eric L. Grimson,et al.  Using adaptive tracking to classify and monitor activities in a site , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[35]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[36]  William J. Christmas,et al.  Error Propagation for 2D-to-3D Matching with Application to Underwater Navigation , 1996, BMVC.

[37]  Neil A. Thacker,et al.  TINA 2001: The Closed Loop 3D Model Matcher , 2001, BMVC.

[38]  Patrick Bouthemy,et al.  Motion detection in an image sequence using Gibbs distributions , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[39]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[40]  John K. Tsotsos,et al.  Real-time Model-based Tracking Using Perspective Alignment: Parallel Implementation and Stability Analysis , 1995, Research in Computer and Robot Vision.

[41]  Rustam Stolkin,et al.  Video with ground-truth for validation of visual registration, tracking and navigation algorithms , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[42]  Wolfgang Spohn,et al.  The Representation of , 1986 .

[43]  Wei-Liem Loh,et al.  Estimating structured correlation matrices in smooth Gaussian random field models , 2000 .

[44]  V. Grayson CuQlock-Knopp,et al.  Navigation through fog using stereoscopic active imaging , 2000, Defense, Security, and Sensing.

[45]  Gerd Hirzinger,et al.  Registration of CAD-models to images by iterative inverse perspective matching , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[46]  Rustam Stolkin,et al.  An Adaptive Background Model for Camshift Tracking with a Moving Camera , 2006 .

[47]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[49]  Jerry D. Gibson,et al.  Handbook of Image and Video Processing , 2000 .