Impact of Blind Image Quality Assessment on the Retrieval of Lifelog Images

The use of personal lifelogs can be beneficial to improve the quality of our life, as they can serve as tools for memory augmentation or for providing support to people with memory issues. In visual lifelogs, data are captured by cameras in the form of images or videos. However, a considerable amount of these images or videos are affected by different types of distortions or noise due to the non-controlled acquisition process. This article addresses the use of Blind Image Quality Assessment algorithms as a pre-processing approach in the retrieval of lifelogging images. As the amount of lifelog images has increased over the last few years, it is fundamental to find solutions to filter images in a lifelog data collection. We evaluate the impact of a Blind Image Quality Assessment algorithm by performing different retrieval experiments through a lifelogging system named MEMORIA. The results are promising and show that our approach can reduce the amount of images to process and retrieve in a lifelog data collection without losing valuable information, and provide to the user the most valuable images. By excluding a considerable amount of images in the pre-processing stage of a lifelogging system, its performance can be increased by saving time and resources.

[1]  C. Gurrin,et al.  Introduction to the Fifth Annual Lifelog Search Challenge, LSC'22 , 2022, ICMR.

[2]  A. Neves,et al.  MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2022 , 2022, LSC@ICMR.

[3]  C. Gurrin,et al.  E-Myscéal: Embedding-based Interactive Lifelog Retrieval System for LSC'22 , 2022, LSC@ICMR.

[4]  C. Gurrin,et al.  Voxento 3.0: A Prototype Voice-Controlled Interactive Search Engine for Lifelog , 2022, LSC@ICMR.

[5]  C. Gurrin,et al.  Flexible Interactive Retrieval SysTem 3.0 for Visual Lifelog Exploration at LSC 2022 , 2022, LSC@ICMR.

[6]  Minh-Triet Tran,et al.  Introduction to the Fourth Annual Lifelog Search Challenge, LSC'21 , 2021, ICMR.

[7]  A. Neves,et al.  Lifelog Retrieval From Daily Digital Data: Narrative Review , 2021, JMIR mHealth and uHealth.

[8]  Alessandro Rozza,et al.  No Reference, Opinion Unaware Image Quality Assessment by Anomaly Detection , 2021, Sensors.

[9]  António J. R. Neves,et al.  Image selection based on low level properties for lifelog moment retrieval , 2020, International Conference on Machine Vision.

[10]  Dietmar Saupe,et al.  KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment , 2019, IEEE Transactions on Image Processing.

[11]  Kan Chen,et al.  Billion-scale semi-supervised learning for image classification , 2019, ArXiv.

[12]  David J. Crandall,et al.  Deepdiary: Lifelogging image captioning and summarization , 2018, J. Vis. Commun. Image Represent..

[13]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Alan F. Smeaton,et al.  Image Aesthetics and Content in Selecting Memorable Keyframes from Lifelogs , 2018, MMM.

[15]  Weidong Min,et al.  No-reference/Blind Image Quality Assessment: A Survey , 2017 .

[16]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[17]  Alan C. Bovik,et al.  Massive Online Crowdsourced Study of Subjective and Objective Picture Quality , 2015, IEEE Transactions on Image Processing.

[18]  Kazuto Kamiyama,et al.  Keyframe Selection Framework Based on Visual and Excitement Features for Lifelog Image Sequences , 2015, Int. J. Soc. Robotics.

[19]  Joemon M. Jose,et al.  "My Day in Review": Visually Summarising Noisy Lifelog Data , 2015, ICMR.

[20]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[21]  Alan F. Smeaton,et al.  LifeLogging: Personal Big Data , 2014, Found. Trends Inf. Retr..

[22]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[23]  Alan F. Smeaton,et al.  An Examination of a Large Visual Lifelog , 2008, AIRS.

[24]  Jana Machajdik,et al.  A Keyframe Selection of Lifelog Image Sequences , 2013, MVA.