Interactive rotoscoping: Extracting and tracking object sketch

In this paper, we discuss an integrated system for video rotoscoping extracting and tracking object sketch (structural shape) across video sequence. This system consists of two key components: object sketch computing and graph-based object tracking. Given a video clip, we first use a primal sketch algorithm to search bottom-up sketch proposals and additively pursue sketch strokes in each frame. User is allowed to edit the sketch in the beginning frame, such as adding strokes and removing the cluttered edges, and the refined sketch is saved as the template. A graph-based tracking method is then proposed for sequentially matching the template to following frames, and the template is kept update by geometric transformation. Once the matching is unsatisfied at one frame, the system is allowed the user interaction for correction. In the experiments, we apply this system on several videos and present the performance evaluation with comparison.

[1]  A. Yezzi,et al.  A variational framework for joint segmentation and registration , 2001, Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis (MMBIA 2001).

[2]  Tony Lindeberg,et al.  Edge Detection and Ridge Detection with Automatic Scale Selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Song-Chun Zhu,et al.  Towards a mathematical theory of primal sketch and sketchability , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[4]  Barry Horwitz,et al.  An Automatic Threshold-Based Scaling Method for Enhancing the Usefulness of Tc-HMPAO SPECT in the Diagnosis of Alzheimer's Disease , 1998, MICCAI.

[5]  Andrew Blake,et al.  A Probabilistic Background Model for Tracking , 2000, ECCV.

[6]  K. Mardia,et al.  Statistical Shape Analysis , 1998 .

[7]  Abdol-Reza Mansouri,et al.  Region Tracking via Level Set PDEs without Motion Computation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[9]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Zhuowen Tu,et al.  Supervised Learning of Edges and Object Boundaries , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Song-Chun Zhu,et al.  Deformable Template As Active Basis , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[13]  Liang Lin,et al.  Layered graph matching by composite cluster sampling with collaborative and competitive interactions , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Hai Jin,et al.  Layered shape matching and registration: Stochastic sampling with hierarchical graph representation , 2008, 2008 19th International Conference on Pattern Recognition.

[15]  Xin Li,et al.  Contour-based object tracking with occlusion handling in video acquired using mobile cameras , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, ACM Trans. Graph..