Modeling Self-Occlusions/Disocclusions in Dynamic Shape and Appearance Tracking for Obtaining Precise Shape

Modeling Self-Occlusions/Disocclusions in Dynamic Shape and Appearance Tracking for Obtaining Precise Shape We present a method to determine the precise shape of a dynamic object from video. This problem is fundamental to computer vision, and has a number of applications, for example, 3D video/cinema post-production, activity recognition and augmented reality. Current tracking algorithms that determine precise shape can be roughly divided into two categories: 1) Global statistics partitioning methods, where the shape of the object is determined by discriminating global image statistics, and 2) Joint shape and appearance matching methods, where a template of the object from the previous frame is matched to the next image. The former is limited in cases of complex object appearance and cluttered background, where global statistics cannot distinguish between the object and background. The latter is able to cope with complex appearance and a cluttered background, but is limited in cases of camera viewpoint change and object articulation, which induce self-occlusions and self-disocclusions of the object of interest. The purpose of this thesis is to model self-occlusion/disocclusion phenomena in a joint shape and appearance tracking framework. We derive a non-linear dynamic model of the object shape and appearance taking into account occlusion phenomena, which is then used to infer self-occlusions/disocclusions, shape and appearance of the object in a variational optimization framework. To ensure robustness to other unmodeled phenomena 5 that are present in real-video sequences, the Kalman filter is used for appearance updating. Experiments show that our method, which incorporates the modeling of self-occlusion/disocclusion, increases the accuracy of shape estimation in situations of viewpoint change and articulation, and out-performs current state-of-the-art methods for shape tracking.

[1]  Stefano Soatto,et al.  A New Geometric Metric in the Space of Curves, and Applications to Tracking Deforming Objects by Prediction and Filtering , 2011, SIAM J. Imaging Sci..

[2]  Gregory D. Hager,et al.  Dynamic Template Tracking and Recognition , 2013, International Journal of Computer Vision.

[3]  Guillermo Sapiro,et al.  A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[4]  Guillermo Sapiro,et al.  Dynamic Color Flow: A Motion-Adaptive Color Model for Object Segmentation in Video , 2010, ECCV.

[5]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[6]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[7]  Anthony J. Yezzi,et al.  Gradient flows and geometric active contour models , 1995, Proceedings of IEEE International Conference on Computer Vision.

[8]  Peter Kauff,et al.  Three-Dimensional Video Postproduction and Processing , 2011, Proceedings of the IEEE.

[9]  Gregory D. Hager,et al.  Efficient Region Tracking With Parametric Models of Geometry and Illumination , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Ying Wu,et al.  Scribble Tracker: A Matting-Based Approach for Robust Tracking , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Alain Trouvé,et al.  Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms , 2005, International Journal of Computer Vision.

[12]  Stefano Soatto,et al.  Detachable Object Detection with Efficient Model Selection , 2011, EMMCVPR.

[13]  David A. Forsyth,et al.  Tracking People by Learning Their Appearance , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[15]  Rachid Deriche,et al.  Symmetrical Dense Optical Flow Estimation with Occlusions Detection , 2002, ECCV.

[16]  Rachid Deriche,et al.  Geodesic Active Regions: A New Framework to Deal with Frame Partition Problems in Computer Vision , 2002, J. Vis. Commun. Image Represent..

[17]  Rachid Deriche,et al.  Geodesic Active Contours and Level Sets for the Detection and Tracking of Moving Objects , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[19]  Stefano Soatto,et al.  Sparse Occlusion Detection with Optical Flow , 2012, International Journal of Computer Vision.

[20]  J. Troutman.  Variational Calculus and Optimal Control: Optimization with Elementary Convexity , 1995 .

[21]  Stefan Roth,et al.  People-tracking-by-detection and people-detection-by-tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Stefano Soatto,et al.  Deformotion: Deforming Motion, Shape Average and the Joint Registration and Approximation of Structures in Images , 2003, International Journal of Computer Vision.

[23]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[24]  Aggelos K. Katsaggelos,et al.  Rate-Distortion Based Video Compression: Optimal Video Frame Compression and Object Boundary Encoding , 1996 .

[25]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[26]  Daniel Cremers,et al.  Dynamical statistical shape priors for level set-based tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[28]  J. Sethian,et al.  Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations , 1988 .

[29]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[30]  Stefano Soatto,et al.  Tracking deformable moving objects under severe occlusions , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[31]  Michael I. Miller,et al.  Large deformation diffeomorphic metric mapping of vector fields , 2005, IEEE Transactions on Medical Imaging.

[32]  Nir A. Sochen,et al.  Variational Stereo Vision with Sharp Discontinuities and Occlusion Handling , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[33]  A. Elgammal,et al.  Inferring 3D body pose from silhouettes using activity manifold learning , 2004, CVPR 2004.

[34]  Guillermo Sapiro,et al.  Morphing Active Contours , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Tao Zhang,et al.  Active contours for tracking distributions , 2004, IEEE Transactions on Image Processing.

[36]  Stefano Soatto,et al.  Dynamic Shape and Appearance Modeling Via Moving and Deforming Layers , 2005, EMMCVPR.

[37]  Luc Van Gool,et al.  A Probabilistic Approach to Large Displacement Optical Flow and Occlusion Detection , 2004, ECCV Workshop SMVP.

[38]  Roman Goldenberg,et al.  Fast Geodesic Active Contours , 1999, Scale-Space.

[39]  Jitendra Malik,et al.  Occlusion boundary detection and figure/ground assignment from optical flow , 2011, CVPR 2011.

[40]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[41]  J A Sethian,et al.  A fast marching level set method for monotonically advancing fronts. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[42]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[43]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[44]  Guillermo Sapiro,et al.  Video SnapCut: robust video object cutout using localized classifiers , 2009, ACM Trans. Graph..

[45]  Namrata Vaswani,et al.  Tracking Deforming Objects Using Particle Filtering for Geometric Active Contours , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Patricio A. Vela,et al.  Geometric Observers for Dynamically Evolving Curves , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Hui Cheng,et al.  Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection , 2006, ECCV.