Experiments in Lifelog Organisation and Retrieval at NTCIR

Lifelogging can be described as the process by which individuals use various software and hardware devices to gather large archives of multimodal personal data from multiple sources and store them in a personal data archive, called a lifelog. The Lifelog task at NTCIR was a comparative benchmarking exercise with the aim of encouraging research into the organisation and retrieval of data from multimodal lifelogs. The Lifelog task ran for over 4 years from NTCIR-12 until NTCIR-14 (2015.02–2019.06); it supported participants to submit to five subtasks, each tackling a different challenge related to lifelog retrieval. In this chapter, a motivation is given for the Lifelog task and a review of progress since NTCIR-12 is presented. Finally, the lessons learned and challenges within the domain of lifelog retrieval are presented.

[1]  Georges Quénot,et al.  LIG-MRIM at NTCIR-12 Lifelog Semantic Access Task , 2016, NTCIR.

[2]  Cathal Gurrin,et al.  Visual Insights from Personal Lifelogs: Insight at the NTCIR-12 Lifelog LIT Task , 2016, NTCIR.

[3]  Marc Langheinrich,et al.  Remembering through lifelogging: A survey of human memory augmentation , 2016, Pervasive Mob. Comput..

[4]  Rami Albatal,et al.  A privacy by design approach to lifelogging , 2014 .

[5]  Rami Albatal,et al.  NTCIR Lifelog: The First Test Collection for Lifelog Research , 2016, SIGIR.

[6]  Vigneshwaran Subbaraju,et al.  VCI2R at the NTCIR-13 Lifelog-2 Lifelog Semantic Access Task , 2017, NTCIR Conference on Evaluation of Information Access Technologies.

[7]  Jean-Christophe Nebel,et al.  Recognition of Activities of Daily Living with Egocentric Vision: A Review , 2016, Sensors.

[8]  Steve Hodges,et al.  Neuropsychological Rehabilitation , 2013 .

[9]  Minh-Triet Tran,et al.  [Invited papers] Comparing Approaches to Interactive Lifelog Search at the Lifelog Search Challenge (LSC2018) , 2019, ITE Transactions on Media Technology and Applications.

[10]  Frank Hopfgartner,et al.  Rethinking the Test Collection Methodology for Personal Self-tracking Data , 2019, MMM.

[11]  Abdelmalek Benattayallah,et al.  An fMRI study of long-term everyday memory using SenseCam , 2011, Memory.

[12]  Guido Zuccon,et al.  QUT at the NTCIR Lifelog Semantic Access Task , 2016, NTCIR.

[13]  Cathal Gurrin,et al.  DCU at the NTCIR-13 Lifelog-2 Task , 2017 .

[14]  Liang-Pu Chen,et al.  Image Searching by Events with Deep Learning for NTCIR-12 Lifelog , 2016, NTCIR.

[15]  David J. Crandall,et al.  Privacy behaviors of lifeloggers using wearable cameras , 2014, UbiComp.

[16]  Alan F. Smeaton,et al.  LifeLogging: Personal Big Data , 2014, Found. Trends Inf. Retr..

[17]  Joemon M. Jose,et al.  A user-study examining visualization of lifelogs , 2016, 2016 14th International Workshop on Content-Based Multimedia Indexing (CBMI).

[18]  Rami Albatal,et al.  Overview of NTCIR-13 Lifelog-2 Task , 2017, NTCIR.

[19]  Rob Kitchin,et al.  ‘Outlines of a World Coming into Existence’: Pervasive Computing and the Ethics of Forgetting , 2007 .

[20]  Abigail Sellen,et al.  Beyond total capture , 2010, Commun. ACM.

[21]  Alan F. Smeaton,et al.  Constructing a SenseCam visual diary as a media process , 2008, Multimedia Systems.

[22]  Tomoyosi Akiba,et al.  Repeated Event Discovery from Image Sequences by Using Segmental Dynamic Time Warping: Experiment at the NTCIR-12 Lifelog Task , 2016, NTCIR.

[23]  Petia Radeva,et al.  LEMoRe: A Lifelog Engine for Moments Retrieval at the NTCIR-Lifelog LSAT Task , 2016, NTCIR.

[24]  Jiang Zhou,et al.  The use of wearable cameras in assessing children's dietary intake and behaviours in China , 2019, Appetite.

[25]  Alan F. Smeaton,et al.  Experiences of Aiding Autobiographical Memory Using the SenseCam , 2012, Hum. Comput. Interact..

[26]  Aiden R Doherty,et al.  Automatically assisting human memory: A SenseCam browser , 2011, Memory.

[27]  Michael Riegler,et al.  Building a Disclosed Lifelog Dataset: Challenges, Principles and Processes , 2017, CBMI.

[28]  C. Gurrin,et al.  Children’s everyday exposure to food marketing: an objective analysis using wearable cameras , 2017, International Journal of Behavioral Nutrition and Physical Activity.

[29]  Jiro Tanaka,et al.  Ensuring Privacy during Pervasive Logging by a Passerby , 2014, J. Inf. Process..

[30]  P. Barnard,et al.  Exploring the basis and boundary conditions of SenseCam-facilitated recollection , 2011, Memory.

[31]  George Awad,et al.  On Influential Trends in Interactive Video Retrieval: Video Browser Showdown 2015–2017 , 2018, IEEE Transactions on Multimedia.

[32]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[33]  Michael Riegler,et al.  Overview of ImageCLEF 2018: Challenges, Datasets and Evaluation , 2018, CLEF.

[34]  D. Kahneman,et al.  A Survey Method for Characterizing Daily Life Experience: The Day Reconstruction Method , 2004, Science.

[35]  Kelly A. Mackintosh,et al.  Can Wearable Cameras Be Used to Validate School-Aged Children’s Lifestyle Behaviours? , 2019, Children.

[36]  Vinh-Tiep Nguyen,et al.  Smart Lifelog Retrieval System with Habit-based Concepts and Moment Visualization , 2019, LSC '19.

[37]  Tetsuya Sakai,et al.  SLLL at the NTCIR-12 Lifelog Task: Sleepflower and the LIT Subtask , 2016, NTCIR.

[38]  Minh-Triet Tran,et al.  A Baseline Interactive Retrieval Engine for Visual Lifelogs at the NTCIR-14 Lifelog-3 Task , 2019, NTCIR.

[39]  Rami Albatal,et al.  Overview of NTCIR-12 Lifelog Task , 2016, NTCIR.

[40]  Gordon Bell,et al.  MyLifeBits: fulfilling the Memex vision , 2002, MULTIMEDIA '02.

[41]  Weiguo Fan,et al.  VTIR at the NTCIR-12 2016 Lifelog Semantic Access Task , 2016, NTCIR.

[42]  P. Schofield,et al.  The use of a wearable camera to explore daily functioning of older adults living with persistent pain: Methodological reflections and recommendations , 2018, Journal of rehabilitation and assistive technologies engineering.