Discovering Internal Representations from Object-CNNs Using Population Encoding

In this paper, we provide a method for understanding the internal representations of Convolutional Neural Networks (CNNs) trained on objects. We hypothesize that the information is distributed across multiple neuronal responses and propose a simple clustering technique to extract this information, which we call \emph{population encoding}. The population encoding technique looks into the entrails of an object-CNN at multiple layers of the network and shows the implicit presence of mid-level object part semantics distributed in the neuronal responses. Our qualitative visualizations show that population encoding can extract mid-level image patches that are visually tighter than the patches that produce high single-filter activations. Moreover, our comprehensive quantitative experiments using the object key point annotations from the PASCAL3D+ dataset corroborate the visualizations by demonstrating the superiority of population encoding over single-filter detectors, in the task of object-part detection. We also perform some preliminary experiments where we uncover the compositional relations between the adjacent layers using the parts detected by population encoding clusters. Finally, based on the insights gained from this work, we point to various new directions which will enable us to have a better understanding of the CNN's internal representations.