Multimedia Lab @ ImageCLEF 2018 Lifelog Moment Retrieval Task

This paper describes the participation of the Multimedia Lab team at the ImageCLEF 2018 Lifelog Moment Retrieval Task. Our method makes use of visual information, text information and metadata. Our approach consists of the following steps: we reduce the number of images to analyze by eliminating the ones that are blurry or do not meet certain metadata criteria, extract relevant concepts with several Convolutional Neural Networks, perform K-means clustering on the Oriented Gradients and Color Histograms features and rerank the remaining images according to a relevance score computed between each image concept and the queried topic.

[1]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[2]  Chokri Ben Amar,et al.  Regim Lab Team at ImageCLEF Lifelog Moment Retrieval Task 2018 , 2018, CLEF.

[3]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[4]  Michael Riegler,et al.  Overview of ImageCLEF 2018: Challenges, Datasets and Evaluation , 2018, CLEF.

[5]  Michael Riegler,et al.  Overview of ImageCLEFlifelog 2019: Solve My Life Puzzle and Lifelog Moment Retrieval , 2019, CLEF.

[6]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[7]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[8]  Rami Albatal,et al.  Overview of NTCIR-13 Lifelog-2 Task , 2017, NTCIR.

[9]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Michael Riegler,et al.  Overview of ImageCLEFlifelog 2017: Lifelog Retrieval and Summarization , 2017, CLEF.

[11]  Matthieu Guillaumin,et al.  Food-101 - Mining Discriminative Components with Random Forests , 2014, ECCV.

[12]  Carlos R. del-Blanco,et al.  ImageCLEF 2019: Multimedia Retrieval in Medicine, Lifelogging, Security and Nature , 2019, CLEF.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[16]  Carlos R. del-Blanco,et al.  Retrieving Events in Life Logging , 2018, CLEF.

[17]  Michael Riegler,et al.  Overview of ImageCLEFlifelog 2018: Daily Living Understanding and Lifelog Moment Retrieval , 2018, CLEF.

[18]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[19]  Rami Albatal,et al.  Overview of NTCIR-12 Lifelog Task , 2016, NTCIR.

[20]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Vigneshwaran Subbaraju,et al.  VCI2R at the NTCIR-13 Lifelog-2 Lifelog Semantic Access Task , 2017, NTCIR Conference on Evaluation of Information Access Technologies.

[22]  Vinh-Tiep Nguyen,et al.  Lifelog Moment Retrieval with Visual Concept Fusion and Text-based Query Expansion , 2018, CLEF.

[23]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Bogdan Ionescu,et al.  A Textual Filtering of HOG-Based Hierarchical Clustering of Lifelog Data , 2017, CLEF.

[25]  Hsin-Hsi Chen,et al.  Visual Concept Selection with Textual Knowledge for Understanding Activities of Daily Living and Life Moment Retrieval , 2018, CLEF.

[26]  Hiroyuki Toda,et al.  PBG at the NTCIR-13 Lifelog-2 LAT , LSAT , and LEST Tasks , 2017 .

[27]  Vigneshwaran Subbaraju,et al.  VC-I2R@ImageCLEF2017: Ensemble of Deep Learned Features for Lifelog Video Summarization , 2017, CLEF.

[28]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.