Unsupervised Texture Flow Estimation Using Appearance-Space Clustering and Correspondence

This paper presents a texture flow estimation method that uses an appearance-space clustering and a correspondence search in the space of deformed exemplars. To estimate the underlying texture flow, such as scale, orientation, and texture label, most existing approaches require a certain amount of user interactions. Strict assumptions on a geometric model further limit the flow estimation to such a near-regular texture as a gradient-like pattern. We address these problems by extracting distinct texture exemplars in an unsupervised way and using an efficient search strategy on a deformation parameter space. This enables estimating a coherent flow in a fully automatic manner, even when an input image contains multiple textures of different categories. A set of texture exemplars that describes the input texture image is first extracted via a medoid-based clustering in appearance space. The texture exemplars are then matched with the input image to infer deformation parameters. In particular, we define a distance function for measuring a similarity between the texture exemplar and a deformed target patch centered at each pixel from the input image, and then propose to use a randomized search strategy to estimate these parameters efficiently. The deformation flow field is further refined by adaptively smoothing the flow field under guidance of a matching confidence score. We show that a local visual similarity, directly measured from appearance space, explains local behaviors of the flow very well, and the flow field can be estimated very efficiently when the matching criterion meets the randomized search strategy. Experimental results on synthetic and natural images show that the proposed method outperforms existing methods.

[1]  Yanxi Liu,et al.  Deformed Lattice Detection in Real-World Images Using Mean-Shift Belief Propagation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Kwanghoon Sohn,et al.  Hole filling with random walks using occlusion constraints in view synthesis , 2011, 2011 18th IEEE International Conference on Image Processing.

[3]  Horst Bischof,et al.  ROI-SEG: Unsupervised Color Segmentation by Combining Differently Focused Sub Results , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Kwanghoon Sohn,et al.  Normalized tone-mapping operators for color quality improvement in 3DTV , 2014, 2014 9th IEEE Conference on Industrial Electronics and Applications.

[5]  Sylvain Lefebvre,et al.  Appearance-space texture synthesis , 2006, ACM Trans. Graph..

[6]  Michael S. Brown,et al.  Robust Estimation of Texture Flow via Dense Feature Sampling , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Eitan Grinspun,et al.  Multiscale texture synthesis , 2008, SIGGRAPH 2008.

[8]  Sylvain Paris,et al.  Capture of hair geometry from multiple images , 2004, ACM Trans. Graph..

[9]  John W. Fisher,et al.  Analysis of orientation and scale in smoothly varying textures , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[11]  Kwanghoon Sohn,et al.  Visual fatigue evaluation and enhancement for 2D-plus-depth video , 2010, 2010 IEEE International Conference on Image Processing.

[12]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[13]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[14]  Kwanghoon Sohn,et al.  Cost aggregation with anisotropic diffusion in feature space for hybrid stereo matching , 2011, 2011 18th IEEE International Conference on Image Processing.

[15]  Seungryong Kim,et al.  Local self-similarity frequency descriptor for multispectral feature matching , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[16]  Kun Zhou,et al.  Inverse texture synthesis , 2008, ACM Trans. Graph..

[17]  Andrew W. Fitzgibbon,et al.  PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation , 2014, International Journal of Computer Vision.

[18]  Narendra Ahuja,et al.  Extracting Texels in 2.1D Natural Textures , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  Matthias Hein,et al.  Hilbertian Metrics and Positive Definite Kernels on Probability Measures , 2005, AISTATS.

[20]  Alexei A. Efros,et al.  Discovering Texture Regularity as a Higher-Order Correspondence Problem , 2006, ECCV.

[21]  Trevor Darrell,et al.  Unsupervised Learning of Categories from Sets of Partially Matching Image Features , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[22]  Lei Jiang,et al.  Statistical Invariance for Texture Synthesis , 2012, IEEE Transactions on Visualization and Computer Graphics.

[23]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[24]  Takeo Kanade,et al.  Mode-seeking by Medoidshifts , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[25]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[26]  손광훈,et al.  Contents retargeting method and apparatus , 2014 .

[27]  Yanxi Liu,et al.  Near-regular texture analysis and manipulation , 2004, SIGGRAPH 2004.

[28]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[29]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[30]  Brian Cabral,et al.  Imaging vector fields using line integral convolution , 1993, SIGGRAPH.

[31]  Ohad Ben-Shahar,et al.  The Perceptual Organization of Texture Flow: A Contextual Inference Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Nipun Kwatra,et al.  Texture optimization for example-based synthesis , 2005, ACM Trans. Graph..

[33]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Kwanghoon Sohn,et al.  Hybrid approach for accurate depth acquisition with structured light and stereo camera , 2012, IEEE international Symposium on Broadband Multimedia Systems and Broadcasting.

[35]  Seungyong Lee,et al.  Flow-Based Image Abstraction , 2009, IEEE Transactions on Visualization and Computer Graphics.

[36]  Carsten Rother,et al.  PatchMatch Stereo - Stereo Matching with Slanted Support Windows , 2011, BMVC.