Multiple Object Scene Description for the Visually Impaired Using Pre-trained Convolutional Neural Networks

This paper introduces a new method for multiple object scene description as part of a system to guide the visually impaired in an indoor environment. Here we are interested in a coarse scene description, where only the presence of certain objects is indicated regardless of its position in the scene. The proposed method is based on the extraction of powerful features using pre-trained convolutional neural networks (CNN), then training a Neural Network regression to predict the content of any unknown scene based on its CNN feature. We have found the CNN feature to be highly descriptive, even though it is trained on auxiliary data from a completely different domain.