Considering Manual Annotations in Dynamic Segmentation of Multimodal Lifelog Data

Multimodal lifelog data consists of continual streams of multimodal sensor data about the life experience of an individual. In order to be effective, any lifelog retrieval system needs to segment continual lifelog data into manageable units. In this paper, we explore the effect of incorporating manual annotations into the lifelog event segmentation process, and we present a study into the effect of high-quality manual annotations on a query-time document segmentation process for lifelog data and evaluate the approach using an open and available test collection. We show that activity based manual annotations enhance the understanding of information retrieval and we highlight a number of potential topics of interest for the community.

[1]  Rob Kitchin,et al.  ‘Outlines of a World Coming into Existence’: Pervasive Computing and the Ethics of Forgetting , 2007 .

[2]  Petia Radeva,et al.  Video Segmentation of Life-Logging Videos , 2014, AMDO.

[3]  Yong Jae Lee,et al.  Predicting Important Objects for Egocentric Video Summarization , 2015, International Journal of Computer Vision.

[4]  Joo-Hwee Lim,et al.  Organizing and retrieving episodic memories from first person view , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[5]  Hiroyuki Toda,et al.  PBG at the NTCIR-13 Lifelog-2 LAT , LSAT , and LEST Tasks , 2017 .

[6]  Abigail Sellen,et al.  Beyond total capture , 2010, Commun. ACM.

[7]  Petia Radeva,et al.  R-Clustering for Egocentric Video Segmentation , 2015, IbPRIA.

[8]  Alan F. Smeaton,et al.  Automatically Segmenting LifeLog Data into Events , 2008, 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services.

[9]  Rashmi Gupta Considering Documents in Lifelog Information Retrieval , 2018, ICMR.

[10]  Aiden R Doherty,et al.  Automatically assisting human memory: A SenseCam browser , 2011, Memory.

[11]  Joo-Hwee Lim,et al.  Predicting Visual Context for Unsupervised Event Segmentation in Continuous Photo-streams , 2018, ACM Multimedia.

[12]  Petia Radeva,et al.  SR-clustering: Semantic regularized clustering for egocentric photo streams segmentation , 2015, Comput. Vis. Image Underst..

[13]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[14]  Jenny Benois-Pineau,et al.  Hierarchical Hidden Markov Model in detecting activities of daily living in wearable videos for studies of dementia , 2011, Multimedia Tools and Applications.

[15]  Cathal Gurrin,et al.  Approaches for Event Segmentation of Visual Lifelog Data , 2018, MMM.

[16]  Jeffrey M. Zacks,et al.  Human brain activity time-locked to perceptual event boundaries , 2001, Nature Neuroscience.

[17]  Alan F. Smeaton,et al.  Using bluetooth and GPS metadata to measure event similarity in SenseCam Images , 2007 .

[18]  Klaus Schöffmann,et al.  LSE2018 Panel - Challenges of Lifelog Search and Access , 2018, LSC@ICMR.

[19]  Geoffrey Zweig,et al.  From captions to visual concepts and back , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Rami Albatal,et al.  Overview of NTCIR-13 Lifelog-2 Task , 2017, NTCIR.

[21]  Paul Clough,et al.  ImageCLEF: Experimental Evaluation in Visual Information Retrieval , 2010 .