LifeSeeker 3.0: An Interactive Lifelog Search Engine for LSC'21

In this paper, we present the interactive lifelog retrieval engine developed for the LSC'21 comparative benchmarking challenge. The LifeSeeker 3.0 interactive lifelog retrieval engine is an enhanced version of our previous system participating in LSC'20 - LifeSeeker 2.0. The system is developed by both Dublin City University and the Ho Chi Minh City University of Science. The implementation of LifeSeeker 3.0 focuses on searching and filtering by text query using a weighted Bag-of-Words model with visual concept augmentation and three weighted vocabularies. The visual similarity search is improved using a bag of local convolutional features; while improving the previous version's performance, enhancing query processing time, result displaying, and browsing support.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Minh-Triet Tran,et al.  LifeSeeker 2.0: Interactive Lifelog Search Engine at LSC 2020 , 2020, LSC@ICMR.

[3]  Dongyoon Han,et al.  Character Region Awareness for Text Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Gregor Kovalčík,et al.  VIRET Tool with Advanced Visual Browsing and Feedback , 2020, LSC@ICMR.

[5]  Sachin R. Gengaje,et al.  A novel scale and rotation invariant texture image retrieval method using fuzzy logic classifier , 2014, Comput. Electr. Eng..

[6]  Vannevar Bush,et al.  As we may think , 1945, INTR.

[7]  Klaus Schöffmann,et al.  Video Browser Showdown 2012-2019: A Review , 2019, CBMI.

[8]  Rami Albatal,et al.  Overview of NTCIR-13 Lifelog-2 Task , 2017, NTCIR.

[9]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[10]  Michael Riegler,et al.  Overview of ImageCLEFlifelog 2019: Solve My Life Puzzle and Lifelog Moment Retrieval , 2019, CLEF.

[11]  Gordon Bell,et al.  MyLifeBits: a personal database for everything , 2006, CACM.

[12]  Rami Albatal,et al.  A Test Collection for Interactive Lifelog Retrieval , 2019, MMM.

[13]  Michael S. Bernstein,et al.  Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations , 2016, International Journal of Computer Vision.

[14]  Minh-Triet Tran,et al.  [Invited papers] Comparing Approaches to Interactive Lifelog Search at the Lifelog Search Challenge (LSC2018) , 2019, ITE Transactions on Media Technology and Applications.

[15]  Sudipta Mukhopadhyay,et al.  Content-based texture image retrieval using fuzzy class membership , 2013, Pattern Recognit. Lett..

[16]  Minh-Triet Tran,et al.  LifeSeeker: Interactive Lifelog Search Engine at LSC 2019 , 2019, LSC@ICMR.

[17]  Jakub Lokoč,et al.  SOMHunter for Lifelog Search , 2020, LSC@ICMR.

[18]  Minh-Triet Tran,et al.  Overview of ImageCLEF Lifelog 2020: Lifelog Moment Retrieval and Sport Performance Lifelog , 2020, CLEF.

[19]  Rainer Lienhart,et al.  Image retrieval on large-scale image databases , 2007, CIVR '07.

[20]  Rami Albatal,et al.  Overview of the NTCIR-14 Lifelog-3 task , 2019 .

[21]  Bens Pardamean,et al.  Quantified Self-Using Consumer Wearable Device: Predicting Physical and Mental Health , 2020, Healthcare informatics research.

[22]  Michael Riegler,et al.  Overview of ImageCLEFlifelog 2017: Lifelog Retrieval and Summarization , 2017, CLEF.

[23]  Rami Albatal,et al.  Overview of NTCIR-12 Lifelog Task , 2016, NTCIR.

[24]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  David Harrison,et al.  Review on Wearable Technology Sensors Used in Consumer Sport Applications , 2019, Sensors.

[26]  Lei Zhang,et al.  Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[27]  Prakash S. Mohod,et al.  Content-based image retrieval from videos using CBIR and ABIR algorithm , 2015, 2015 Global Conference on Communication Technologies (GCCT).

[28]  Nguyen Thanh Binh,et al.  Myscéal: An Experimental Interactive Lifelog Retrieval System for LSC'20 , 2020, LSC@ICMR.

[29]  Minh-Triet Tran,et al.  Introduction to the Fourth Annual Lifelog Search Challenge, LSC'21 , 2021, ICMR.

[30]  Swati Goel,et al.  A Survey on Recent Image Indexing and Retrieval Techniques for Low-Level Feature Extraction in CBIR Systems , 2015, 2015 IEEE International Conference on Computational Intelligence & Communication Technology.

[31]  Noel E. O'Connor,et al.  Bags of Local Convolutional Features for Scalable Instance Search , 2016, ICMR.

[32]  Michael Riegler,et al.  Overview of ImageCLEFlifelog 2018: Daily Living Understanding and Lifelog Moment Retrieval , 2018, CLEF.