Learning to Discover Reflection Symmetry via Polar Matching Convolution

The task of reflection symmetry detection remains challenging due to significant variations and ambiguities of symmetry patterns in the wild. Furthermore, since the local regions are required to match in reflection for detecting a symmetry pattern, it is hard for standard convolutional networks, which are not equivariant to rotation and reflection, to learn the task. To address the issue, we introduce a new convolutional technique, dubbed the polar matching convolution, which leverages a polar feature pooling, a selfsimilarity encoding, and a systematic kernel design for axes of different angles. The proposed high-dimensional kernel convolution network effectively learns to discover symmetry patterns from real-world images, overcoming the limitations of standard convolution. In addition, we present a new dataset and introduce a self-supervised learning strategy by augmenting the dataset with synthesizing images. Experiments demonstrate that our method outperforms state-ofthe-art methods in terms of accuracy and robustness.

[1]  Iasonas Kokkinos,et al.  Learning-Based Symmetry Detection in Natural Images , 2012, ECCV.

[2]  Yanxi Liu,et al.  Symmetry Detection from RealWorld Images Competition 2013: Summary and Results , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[3]  Shanmuganathan Raman,et al.  SymmMap: Estimation of the 2-D Reflection Symmetry Map and Its Applications , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[4]  J. Wagemans,et al.  Detection of visual symmetries. , 1995, Spatial vision.

[5]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[6]  Zesheng Tang,et al.  Reflection Symmetry Detection Using Locally Affine Invariant Edge Correspondence , 2015, IEEE Transactions on Image Processing.

[7]  Maks Ovsjanikov,et al.  Detection of Mirror-Symmetric Image Patches , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Michael Werman,et al.  A convolutional approach to reflection symmetry , 2016, Pattern Recognit. Lett..

[9]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[10]  Cécile Barat,et al.  Multiple Reflection Symmetry Detection via Linear-Directional Kernel Density Estimation , 2017, CAIP.

[11]  Yan Wang,et al.  Object Skeleton Extraction in Natural Images by Fusing Scale-Associated Deep Side Outputs , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Luc Van Gool,et al.  Computational Symmetry in Computer Vision and Computer Graphics , 2010, Found. Trends Comput. Graph. Vis..

[13]  Yanxi Liu,et al.  Beyond Planar Symmetry: Modeling Human Perception of Reflection and Rotation Symmetries in the Wild , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Peter Norvig,et al.  The Unreasonable Effectiveness of Data , 2009, IEEE Intelligent Systems.

[15]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[18]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[19]  Richard Szeliski,et al.  Detecting and Reconstructing 3D Mirror Symmetric Objects , 2012, ECCV.

[20]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[21]  Gareth Loy,et al.  Detecting Bilateral Symmetry in Perspective , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[22]  David Grant Colburn Hildebrand,et al.  Finding Mirror Symmetry via Registration and Optimal Symmetric Pairwise Assignment of Curves , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[23]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[24]  Michael Werman,et al.  Mirror Symmetry Histograms for Capturing Geometric Properties in Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Sven J. Dickinson,et al.  2017 ICCV Challenge: Detecting Symmetry in the Wild , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[27]  Minsu Cho,et al.  Bilateral Symmetry Detection via Symmetry-Growing , 2009, BMVC.

[28]  Jan-Olof Eklundh,et al.  Detecting Symmetry and Symmetric Constellations of Features , 2006, ECCV.

[29]  Risi Kondor,et al.  On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups , 2018, ICML.

[30]  Kunihiko Fukushima,et al.  Use of non-uniform spatial blur for image comparison: symmetry axis extraction , 2005, Neural Networks.

[31]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Max Welling,et al.  Group Equivariant Convolutional Networks , 2016, ICML.

[33]  Jiri Matas,et al.  Efficient Symmetry Detection Using Local Affine Frames , 2007, SCIA.

[34]  Masayuki Kikuchi,et al.  Symmetry axis extraction by a neural network , 2006, Neurocomputing.

[35]  Yiannis Aloimonos,et al.  Detection and Segmentation of 2D Curved Reflection Symmetric Structures , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[36]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Jitendra Malik,et al.  Learning to detect natural image boundaries using local brightness, color, and texture cues , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.