Lazy texture selection based on active learning

Interactive selection of desired textures and textured objects from a video is a challenging problem in video editing. In this paper, we present a scalable framework that accurately selects textured objects with only moderate user interaction. Our method applies the active learning methodology, and the user only needs to label minimal initial training data and subsequent query data. An active learning algorithm uses these labeled data to obtain an initial classifier and iteratively improves it until its performance becomes satisfactory. A revised graph-cut algorithm based on the trained classifier has also been developed to improve the spatial coherence of selected texture regions. We show that our system is responsive even with videos of a large number of frames, and it frees the user from extensive labeling work. A variety of operations, such as color editing, compositing, and texture cloning, can be then applied to the selected textures to achieve interesting editing effects.

[1]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[2]  Naoki Abe,et al.  Query Learning Strategies Using Boosting and Bagging , 1998, ICML.

[3]  Harry Shum,et al.  Natural Image Colorization , 2007, Rendering Techniques.

[4]  Tong Zhang,et al.  Active learning using adaptive resampling , 2000, KDD '00.

[5]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.

[6]  Jitendra Malik,et al.  Contour and Texture Analysis for Image Segmentation , 2001, International Journal of Computer Vision.

[7]  Antonio Criminisi,et al.  TextonBoost: Joint Appearance, Shape and Context Modeling for Multi-class Object Recognition and Segmentation , 2006, ECCV.

[8]  Lior Wolf,et al.  One-sided object cutout using principal-channels , 2009, 2009 11th IEEE International Conference on Computer-Aided Design and Computer Graphics.

[9]  Harry Shum,et al.  Video object cut and paste , 2005, ACM Trans. Graph..

[10]  Neil A. Dodgson,et al.  Self-similarity based texture editing , 2002, SIGGRAPH.

[11]  Guillermo Sapiro,et al.  Interactive Image Segmentation via Adaptive Weighted Distances , 2007, IEEE Transactions on Image Processing.

[12]  David Salesin,et al.  Interactive digital photomontage , 2004, ACM Trans. Graph..

[13]  Michael Gleicher,et al.  This document was created with FrameMaker 4.0.4 Image Snapping , 2022 .

[14]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[16]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[17]  Jitendra Malik,et al.  Learning to Detect Natural Image Boundaries Using Brightness and Texture , 2002, NIPS.

[18]  C.-C. Jay Kuo,et al.  Texture analysis and classification with tree-structured wavelet transform , 1993, IEEE Trans. Image Process..

[19]  William A. Barrett,et al.  Intelligent scissors for image composition , 1995, SIGGRAPH.

[20]  Harry Shum,et al.  Lazy snapping , 2004, ACM Trans. Graph..

[21]  Jessica K. Hodgins,et al.  Flow-based video synthesis and editing , 2004, ACM Trans. Graph..

[22]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[23]  William E. Higgins,et al.  Texture Segmentation using 2-D Gabor Elementary Functions , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Edward H. Adelson,et al.  Eurographics Symposium on Rendering 2008 Scribbleboost: Adding Classification to Edge-aware Interpolation of Local Image and Video Adjustments , 2022 .

[25]  Alexei A. Efros,et al.  Automatic photo pop-up , 2005, ACM Trans. Graph..

[26]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[27]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[28]  Lior Wolf,et al.  Principal-channels for One-sided Object Cutout , 2008 .

[29]  Shai Avidan SpatialBoost: Adding Spatial Reasoning to AdaBoost , 2006, ECCV.

[30]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, ACM Trans. Graph..

[31]  Jue Wang Discriminative Gaussian Mixtures for Interactive Image Segmentation , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[32]  David D. Denison,et al.  Nonlinear estimation and classification , 2003 .

[33]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[34]  Tien-Tsin Wong,et al.  Manga colorization , 2006, ACM Trans. Graph..

[35]  Dani Lischinski,et al.  Colorization by example , 2005, EGSR '05.

[36]  David Salesin,et al.  Animating pictures with stochastic motion textures , 2005, ACM Trans. Graph..