Repeated Pattern Detection Using CNN Activations

We propose a new approach for detecting repeated patterns on a grid in a single image. To do so, we detect repetitions in the space of pre-trained deep CNN filter responses at all layer levels. These encode features at several conceptual levels (from low-level patches to high-level semantics) as well as scales (from local to global). As a result, our repeated pattern detector is robust to challenging cases where repeated tiles show strong variation in visual appearance due to occlusions, lighting or background clutter. Our method contrasts with previous approaches that rely on keypoint extraction, description and clustering or on patch correlation. These generally only detect low-level feature clusters that do not handle variations in visual appearance of the patterns very well. Our method is simpler, yet incorporates high level features implicitly. As such, we can demonstrate detections of repetitions with strong appearance variations, organized on a nearly-regular axis-aligned grid Results show robustness and consistency throughout a varied database of more than 150 images.

[1]  Jan-Michael Frahm,et al.  Repetition-based dense single-view reconstruction , 2011, CVPR 2011.

[2]  Yanxi Liu,et al.  Detecting and matching repeated patterns for automatic geo-tagging in urban environments , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Yanxi Liu,et al.  Deformed Lattice Detection in Real-World Images Using Mean-Shift Belief Propagation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Luc Van Gool,et al.  Grouping Based on Coupled Diffusion Maps , 1999, Shape, Contour and Grouping in Computer Vision.

[5]  Yanxi Liu,et al.  A computational model for periodic pattern perception based on frieze and wallpaper groups , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Jaakko Lehtinen,et al.  Reflectance modeling by neural texture synthesis , 2016, ACM Trans. Graph..

[7]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[8]  Yanxi Liu,et al.  Quantitative Evaluation of Near Regular Texture Synthesis Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[9]  Tao Wang,et al.  End-to-end text recognition with convolutional neural networks , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Masatoshi Okutomi,et al.  Visual Place Recognition with Repetitive Structures , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[13]  Jiri Matas,et al.  Image Matching and Retrieval by Repetitive Patterns , 2010, 2010 20th International Conference on Pattern Recognition.

[14]  B. Schiele,et al.  Combined Object Categorization and Segmentation With an Implicit Shape Model , 2004 .

[15]  Dumitru Erhan,et al.  Deep Neural Networks for Object Detection , 2013, NIPS.

[16]  Jan-Michael Frahm,et al.  Detecting Large Repetitive Structures with Salient Boundaries , 2010, ECCV.

[17]  Yanxi Liu,et al.  Near-regular texture analysis and manipulation , 2004, SIGGRAPH 2004.

[18]  Long Quan,et al.  Translation symmetry detection in a fronto-parallel view , 2011, CVPR 2011.

[19]  Yanxi Liu,et al.  GRASP Recurring Patterns from a Single View , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Jiri Matas,et al.  Rectification, and Segmentation of Coplanar Repeated Patterns , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Nikos Paragios,et al.  Segmentation of building facades using procedural shape priors , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Trevor Darrell,et al.  Do Convnets Learn Correspondence? , 2014, NIPS.

[23]  Luc Van Gool,et al.  Noncombinatorial Detection of Regular Repetitions under Perspective Skew , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[25]  Yanxi Liu,et al.  Translation-Symmetry-Based Perceptual Grouping with Applications to Urban Scenes , 2010, ACCV.