A Spherical Approach to Planar Semantic Segmentation

We investigate a geometrically motivated modification to semantic segmentation. In particular, we reformulate typical planar CNN as a projected spherical CNN where image distortions are reduced, and thus generalisation increased. Since prior formulations of spherical CNNs require computation on full spheres, fair comparison between planar and spherical methods have not been previously presented. In this work, we first extend spherical deep learning to support high-resolution images by exploiting the reduced field of view of classical images. Then, we employ our spherical representation to reduce distortion effects of standard deep learning systems. On typical benchmarks, we apply our spherical representation and consistently outperform the classical representation of multiple existing architectures. Additionally, we introduce direct spherical pretraining from planar datasets to further improve results. Finally, we compare our method on nonplanar datasets, where we improve accuracy, and outperform running time of spherical state of the art for non-complete input spheres.

[1]  Jan-Michael Frahm,et al.  Convolutions on Spherical Images , 2019, CVPR Workshops.

[2]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[3]  Roberto Cipolla,et al.  Orientation-Aware Semantic Segmentation on Icosahedron Spheres , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[4]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Matthias Nießner,et al.  Spherical CNNs on Unstructured Grids , 2019, ICLR.

[6]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Max Welling,et al.  Convolutional Networks for Spherical Signals , 2017, ArXiv.

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Andreas Geiger,et al.  SphereNet: Learning Spherical Representations for Detection and Classification in Omnidirectional Images , 2018, ECCV.

[12]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Xiaoxiao Li,et al.  Semantic Image Segmentation via Deep Parsing Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Bastian Leibe,et al.  Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Max Welling,et al.  Gauge Equivariant Convolutional Networks and the Icosahedral CNN 1 , 2019 .

[17]  M. Tomasi,et al.  Convolutional neural networks on the HEALPix sphere: a pixel-based algorithm and its application to CMB data analysis , 2019, Astronomy & Astrophysics.

[18]  Kuk-Jin Yoon,et al.  SpherePHD: Applying CNNs on a Spherical PolyHeDron Representation of 360° Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Antonio M. López,et al.  The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).