Toward an assisted indoor scene perception for blind people with image multilabeling strategies

A novel coarse indoor scene description framework for blind people is introduced.Three image multilabeling implementation strategies are proposed.Experimental validation was conducted on different public indoor environments.Results qualify the approach for a near real time blind assistance technology. In this work, we present novel strategies to coarsely describe indoor scenes by listing the objects surrounding a blind person equipped with a portable digital camera. They rely on a new multilabeling approach which consists in computing the similarity between a query image and a set of multilabeled images stored in a library in order to pick up the most similar images. Since each image of the library conveys its own list of objects, the co-occurrence of objects between the most similar images is exploited to "multilabel" the query image. The multilabeling approach is implemented by means of three different strategies. They are respectively based on the scale invariant feature transform (SIFT), the notion of bag of words, and principal component analysis (PCA). The proposed methods were tested on datasets corresponding to two different public indoor sites. Promising results have been obtained and suggest that near real-time implementation can be envisioned for describing public indoor environments with numerous predefined objects and with a good accuracy.