Deep Hybrid Real-Complex-Valued Convolutional Neural Networks for Image Classification

Shallow complex-valued convolutional neural networks (CVCNNs) have displayed better performance than their real-valued counterparts (RVCNNs). This paper presents a deep CVCNN architecture inspired by the well known VGG architecture. The different structure of learning in the complex domain compared with the real domain means that CVCNNs make systematically different errors than RVCNNs. This led to the idea of a hybrid real-complex-valued ensemble of the two types of networks, which combines the advantages of both. Experiments done on the SVHN, CIFAR-10, and CIFAR- 100 datasets show better results of the CVCNNs compared with RVCNNs, and significantly better results of the hybrid real-complex-valued ensemble compared with both types of networks.

[1]  Mujdat Cetin,et al.  Sparse Signal Representation for Complex-Valued Imaging , 2009, 2009 IEEE 13th Digital Signal Processing Workshop and 5th IEEE Signal Processing Education Workshop.

[2]  Vince D. Calhoun,et al.  ICA of full complex-valued fMRI data using phase information of spatial maps , 2015, Journal of Neuroscience Methods.

[3]  Yoshua Bengio,et al.  Unitary Evolution Recurrent Neural Networks , 2015, ICML.

[4]  Stella X. Yu,et al.  Better than real: Complex-valued neural nets for MRI fingerprinting , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[5]  Anupama Pande,et al.  Complex-Valued Neural Network in Image Recognition: A Study on the Effectiveness of Radial Basis Function , 2007 .

[6]  Luis Hernandez-Garcia,et al.  Complex‐valued analysis of arterial spin labeling–based functional magnetic resonance imaging signals , 2009, Magnetic resonance in medicine.

[7]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[8]  Sanjay Shitole,et al.  Adaptive parametric estimator for complex valued images , 2015, 2015 International Conference on Technologies for Sustainable Development (ICTSD).

[9]  S. Mallat,et al.  Invariant Scattering Convolution Networks , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Robert W. Ives,et al.  Compression of complex-valued SAR images , 1999, IEEE Trans. Image Process..

[11]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[12]  Y. Hui,et al.  MRI reconstruction from truncated data using a complex domain backpropagation neural network , 1995, IEEE Pacific Rim Conference on Communications, Computers, and Signal Processing. Proceedings.

[13]  Michael A. Casey,et al.  Learning Representations Using Complex-Valued Nets , 2015, ArXiv.

[14]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[15]  Les E. Atlas,et al.  Full-Capacity Unitary Recurrent Neural Networks , 2016, NIPS.

[16]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[17]  Jasper Snoek,et al.  Spectral Representations for Convolutional Neural Networks , 2015, NIPS.

[18]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[19]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[20]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[21]  Alfred O. Hero,et al.  Complex input convolutional neural networks for wide angle SAR ATR , 2016, 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[22]  Calin-Adrian Popa,et al.  Complex-valued convolutional neural networks for real-valued image classification , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[23]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[24]  Olaf Hellwich,et al.  Complex-Valued Convolutional Neural Networks for Object Detection in PolSAR data , 2010 .

[25]  Nitzan Guberman,et al.  On Complex Valued Convolutional Neural Networks , 2016, ArXiv.

[26]  Mark Tygert,et al.  A Mathematical Motivation for Complex-Valued Convolutional Networks , 2015, Neural Computation.

[27]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[28]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[29]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[30]  Yann LeCun,et al.  Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs , 2016, ICML.