Modifying Non-local Variations Across Multiple Views

We present an algorithm for modifying small non-local variations between repeating structures and patterns in multiple images of the same scene. The modification is consistent across views, even-though the images could have been photographed from different view points and under different lighting conditions. We show that when modifying each image independently the correspondence between them breaks and the geometric structure of the scene gets distorted. Our approach modifies the views while maintaining correspondence, hence, we succeed in modifying appearance and structure variations consistently. We demonstrate our methods on a number of challenging examples, photographed in different lighting, scales and view points.

[1]  Sylvain Paris,et al.  Learning photographic global tonal adjustment with a database of input / output image pairs , 2011, CVPR 2011.

[2]  Andrew Blake,et al.  Cosegmentation of Image Pairs by Histogram Matching - Incorporating a Global Constraint into MRFs , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  Yongwei Nie,et al.  Efficient image/video retexturing using parallel bilateral grids , 2011, VRCAI.

[4]  Tal Hassner,et al.  Dense Correspondences across Scenes and Scales , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Vladimir Kolmogorov,et al.  Cosegmentation Revisited: Models and Optimization , 2010, ECCV.

[6]  Vikas Singh,et al.  An efficient algorithm for Co-segmentation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[7]  Daniel Cohen-Or,et al.  Time-varying weathering in texture space , 2016, ACM Trans. Graph..

[8]  Yanxi Liu,et al.  Near-regular texture analysis and manipulation , 2004, SIGGRAPH 2004.

[9]  Ce Liu,et al.  Exploring new representations and applications for motion analysis , 2009 .

[10]  Eli Shechtman,et al.  Cosaliency: where people look when comparing images , 2010, UIST.

[11]  Tal Hassner,et al.  Effective face frontalization in unconstrained images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Xueqing Li,et al.  Leveraging stereopsis for saliency analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Qiang Wu,et al.  Stereoscopic visual saliency prediction based on stereo contrast and stereo focus , 2017, EURASIP J. Image Video Process..

[14]  W. Heidrich,et al.  Texture Replacement of Garments in Monocular Video Sequences , 2022 .

[15]  Olga Sorkine-Hornung,et al.  Transfusive image manipulation , 2012, ACM Trans. Graph..

[16]  Kalyan Sunkavalli,et al.  Nautilus , 2017, ACM Trans. Graph..

[17]  Sylvain Lefebvre,et al.  Texture Synthesis From Photographs , 2008, Comput. Graph. Forum.

[18]  A. Torralba,et al.  Motion magnification , 2005, SIGGRAPH 2005.

[19]  King Ngi Ngan,et al.  Co-Salient Object Detection From Multiple Images , 2013, IEEE Transactions on Multimedia.

[20]  Qingming Huang,et al.  Co-Saliency Detection for RGBD Images Based on Multi-Constraint Feature Matching and Cross Label Propagation , 2017, IEEE Transactions on Image Processing.

[21]  Xiaochun Cao,et al.  Cluster-Based Co-Saliency Detection , 2013, IEEE Transactions on Image Processing.

[22]  Kaihao Liang,et al.  Iterative re-weighted least squares algorithm for l-minimization with tight frame and 0 < p ≤ 1 , 2019, Linear Algebra and its Applications.

[23]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[24]  Vikas Singh,et al.  Scale invariant cosegmentation for image groups , 2011, CVPR 2011.

[25]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Different Scenes , 2008, ECCV.

[26]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[27]  H. Hirschmüller.  Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information , 2005, CVPR.

[28]  William T. Freeman,et al.  Revealing and modifying non-local variations in a single image , 2015, ACM Trans. Graph..

[29]  Vladimir G. Kim,et al.  Symmetry-guided texture synthesis and manipulation , 2012, TOGS.

[30]  Jitendra Malik,et al.  Computing Local Surface Orientation and Shape from Texture for Curved Surfaces , 1997, International Journal of Computer Vision.

[31]  King Ngi Ngan,et al.  A Co-Saliency Model of Image Pairs , 2011, IEEE Transactions on Image Processing.

[32]  Nikos Paragios,et al.  Unsupervised co-segmentation through region matching , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Nam Ik Cho,et al.  Co-Salient Object Detection Based on Deep Saliency Networks and Seed Propagation Over an Integrated Graph , 2017, IEEE Transactions on Image Processing.

[34]  Stéphane Mallat,et al.  The Texture Gradient Equation for Recovering Shape from Texture , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Frédo Durand,et al.  Deviation magnification , 2015, ACM Trans. Graph..

[36]  David A. Forsyth,et al.  Capturing and animating occluded cloth , 2007, ACM Trans. Graph..

[37]  Ilan Shimshoni,et al.  Image-based texture replacement using multiview images , 2008, VRST '08.

[38]  Lei Jiang,et al.  Statistical Invariance for Texture Synthesis , 2012, IEEE Transactions on Visualization and Computer Graphics.

[39]  Xiaojun Qi,et al.  Unsupervised cosegmentation based on superpixel matching and Fastgrabcut , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[40]  Frédo Durand,et al.  Phase-based video motion processing , 2013, ACM Trans. Graph..

[41]  C. Dyer,et al.  Half-integrality based algorithms for cosegmentation of images , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Andrea Fusiello,et al.  Quasi-Euclidean uncalibrated epipolar rectification , 2008, 2008 19th International Conference on Pattern Recognition.