Subspace Capsule Network

Convolutional neural networks (CNNs) have become a key asset to most of fields in AI. Despite their successful performance, CNNs suffer from a major drawback. They fail to capture the hierarchy of spatial relation among different parts of an entity. As a remedy to this problem, the idea of capsules was proposed by Hinton. In this paper, we propose the SubSpace Capsule Network (SCN) that exploits the idea of capsule networks to model possible variations in the appearance or implicitly-defined properties of an entity through a group of capsule subspaces instead of simply grouping neurons to create capsules. A capsule is created by projecting an input feature vector from a lower layer onto the capsule subspace using a learnable transformation. This transformation finds the degree of alignment of the input with the properties modeled by the capsule subspace.We show that SCN is a general capsule network that can successfully be applied to both discriminative and generative models without incurring computational overhead compared to CNN during test time. Effectiveness of SCN is evaluated through a comprehensive set of experiments on supervised image classification, semi-supervised image classification and high-resolution image generation tasks using the generative adversarial network (GAN) framework. SCN significantly improves the performance of the baseline models in all 3 tasks.

[1]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[2]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[3]  Guo-Jun Qi,et al.  Generalized Loss-Sensitive Adversarial Learning with Manifold Margins , 2018, ECCV.

[4]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[5]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[6]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[7]  Guo-Jun Qi,et al.  CapProNet: Deep Feature Learning via Orthogonal Projections onto Capsule Subspaces , 2018, NeurIPS.

[8]  Claudia Clopath,et al.  Image Synthesis with a Convolutional Capsule Generative Adversarial Network , 2018, MIDL.

[9]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[10]  Aaron C. Courville,et al.  Adversarially Learned Inference , 2016, ICLR.

[11]  Yi-Hsuan Yang,et al.  MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment , 2017, AAAI.

[12]  Mubarak Shah,et al.  VideoCapsuleNet: A Simplified Network for Action Detection , 2018, NeurIPS.

[13]  Mohammad Taha Bahadori,et al.  Spectral Capsule Networks , 2018 .

[14]  Mubarak Shah,et al.  Visual Text Correction , 2018, ECCV.

[15]  Geoffrey E. Hinton,et al.  Transforming Auto-Encoders , 2011, ICANN.

[16]  Ulas Bagci,et al.  Capsules for Object Segmentation , 2018, ArXiv.

[17]  Min Yang,et al.  Investigating Capsule Networks with Dynamic Routing for Text Classification , 2018, EMNLP.

[18]  Jeff Donahue,et al.  Large Scale GAN Training for High Fidelity Natural Image Synthesis , 2018, ICLR.

[20]  Yu-Gang Jiang,et al.  Learning Fashion Compatibility with Bidirectional LSTMs , 2017, ACM Multimedia.

[21]  Geoffrey E. Hinton,et al.  Matrix capsules with EM routing , 2018, ICLR.

[22]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[23]  Mayank Vatsa,et al.  Dual Directed Capsule Network for Very Low Resolution Image Recognition , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[24]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[25]  Premkumar Natarajan,et al.  CapsuleGAN: Generative Adversarial Capsule Network , 2018, ECCV Workshops.

[26]  E. Denman,et al.  The matrix sign function and computations in systems , 1976 .

[27]  Shin Ishii,et al.  Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[29]  Nicholas J. Higham,et al.  Stable iterations for the matrix square root , 1997, Numerical Algorithms.

[30]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[31]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[32]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[33]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.