Leveraging Activity Indexing for Egocentric Image Retrieval

Wearable cameras can daily gather large amounts of image data that require powerful image indexing and retrieval techniques in order to find the information of interest. In this work, we address the indexing problem of egocentric data by exploring the relevance of different information sources provided by Convolutional Neural Networks (CNN) combined with image metadata. The proposed method was tested on a public egocentric dataset of 45.000 images and gave encouraging results.

[1]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[2]  Petia Radeva,et al.  Egocentric Image Retrieval with Convolutional Neural Networks , 2016, CCIA.

[3]  Alan F. Smeaton,et al.  LifeLogging: Personal Big Data , 2014, Found. Trends Inf. Retr..

[4]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[5]  Petia Radeva,et al.  LEMoRe: A Lifelog Engine for Moments Retrieval at the NTCIR-Lifelog LSAT Task , 2016, NTCIR.

[6]  Petia Radeva,et al.  Toward Storytelling From Visual Lifelogging: An Overview , 2015, IEEE Transactions on Human-Machine Systems.

[7]  Rami Albatal,et al.  NTCIR Lifelog: The First Test Collection for Lifelog Research , 2016, SIGIR.

[8]  Noel E. O'Connor,et al.  Where is my Phone?: Personal Object Retrieval from Egocentric Images , 2016, LTA@MM.

[9]  Alan F. Smeaton,et al.  Combining image descriptors to effectively retrieve events from visual lifelogs , 2008, MIR '08.

[10]  Yuichi Ohta,et al.  Image Retrieval of First-Person Vision for Pedestrian Navigation in Urban Area , 2010, 2010 20th International Conference on Pattern Recognition.

[11]  Trevor Darrell,et al.  LSDA: Large Scale Detection through Adaptation , 2014, NIPS.

[12]  Tiziana Catarci,et al.  Digital memories in an era of ubiquitous computing and abundant storage , 2006, CACM.

[13]  Steve E Hodges,et al.  Wearable cameras in health: the state of the art and future possibilities. , 2013, American journal of preventive medicine.

[14]  Yong Jae Lee,et al.  Discovering important people and objects for egocentric video summarization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.