ODMTCNet: An Interpretable Multi-view Deep Neural Network Architecture for Image Feature Representation

Recently, deep cascade architecture-based algorithms have attracted wide interest and have been applied to various application domains successfully. However, the longstanding challenge of interpretability, is still considered as an Achilles’ heel of such algorithms. Moreover, due to its data-driven nature, the deep cascade architecture likely causes over-fitting problems when there is no sufficient data available. To address these pressing issues, this work proposes an interpretable multi-view deep neural network architecture, namely optimal discriminant multi-view tensor convolutional network (ODMTCNet), by integrating statistical machine learning (SML) principles with the deep neural network (DNN) architecture. Benefiting from the joint strength of SML and DNN, we demonstrate that ODMTCNet is analytically interpretable for multi-view image feature representation. Specifically, a discriminant multi-view tensor convolution strategy is proposed and integrated with the desired deep cascade architecture to generate high quality feature representations. Different from the traditional DNN models, the parameters of the convolutional layers in ODMTCNet are determined by analytically solving a SML–based optimization problem in each convolutional layer independently. This work demonstrates that, in ODMTCNet, the relation between the optimal performance and parameters (e.g., the number of convolutional filters) can be predicted, with each layer generating justified knowledge representations, leading to an interpretable multi-view based convolutional network. In addition, an information theoretic based descriptor, information quality (IQ), is utilized for feature representation of the given multi-view data sets. Because of its unique design, ODMTCNet is able to handle image data sets of different scales, large or small, effectively addressing the data hungry nature of DNN in image representation and forming a generic platform for multi-view image feature representation. To validate the effectiveness and the generic nature of the proposed ODMTCNet, we conducted experiments on four image data sets of different scales: The Olivetti Research Lab (ORL) database, Facial Recognition Technology (FERET) database, ETH–80 database and Caltech 256 database. The results show the superiority of the proposed solution compared to state-of-the-art.