Organizing Egocentric Videos for Daily Living Monitoring

Egocentric videos are becoming popular since the possibility to observe the scene flow from the user's point of view (First Person Vision). Among the different assistive applications in this context there is the daily living monitoring of a user that is wearing the camera. In this paper we propose a system devoted to automatically organize videos acquired by the user over different days. By employing an unsupervised segmentation, each egocentric video is divided in chapters by considering the visual content. The video segments related to the different days are hence linked to produce graphs which are coherent with respect to the context in which the user acts. Experiments on two different datasets demonstrate the effectiveness of the proposed approach which outperforms the state of the art, both in accuracy and computational time with a good margin.

[1]  Jenny Benois-Pineau,et al.  Hierarchical Hidden Markov Model in detecting activities of daily living in wearable videos for studies of dementia , 2011, Multimedia Tools and Applications.

[2]  Noah Snavely,et al.  Material recognition in the wild with the Materials in Context Database , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Bernt Schiele,et al.  Taking a deeper look at pedestrians , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Takeo Kanade,et al.  Quality of Life Technology , 2012, Proc. IEEE.

[5]  Sebastiano Battiato,et al.  RECfusion: Automatic Scene Clustering and Tracking in Videos from Multiple Sources , 2016 .

[6]  A. Baddeley,et al.  The essential handbook of memory disorders for clinicians , 2004 .

[7]  Jitendra Malik,et al.  Analyzing the Performance of Multilayer Neural Networks for Object Recognition , 2014, ECCV.

[8]  Alan F. Smeaton,et al.  LifeLogging: Personal Big Data , 2014, Found. Trends Inf. Retr..

[9]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Giovanni Maria Farinella,et al.  Representing scenes for real-time context classification on mobile devices , 2015, Pattern Recognit..

[11]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[12]  Jenny Benois-Pineau,et al.  Recognition of Activities of Daily Living in natural “at home” scenario for assessment of Alzheimer's disease patients , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[13]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[14]  Dima Damen,et al.  You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video , 2014, BMVC.

[15]  Gerald Schaefer,et al.  Illuminant and device invariant colour using histogram equalisation , 2005, Pattern Recognit..

[16]  Li Deng,et al.  A tutorial survey of architectures, algorithms, and applications for deep learning , 2014, APSIPA Transactions on Signal and Information Processing.

[17]  Giovanni Maria Farinella,et al.  RECfusion: Automatic Video Curation Driven by Visual Content Popularity , 2015, ACM Multimedia.

[18]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[19]  Jean-François Dartigues,et al.  Autonomy at home and early diagnosis in Alzheimer's Disease: Utility of video indexing applied to clinical issues, the IMMED project , 2011 .

[20]  Barbara A. Wilson,et al.  External memory aids and computers in memory rehabilitation , 2002 .

[21]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[22]  Kristen Grauman,et al.  Story-Driven Summarization for Egocentric Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Shmuel Peleg,et al.  Temporal Segmentation of Egocentric Videos , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Antonino Furnari,et al.  Segmenting Egocentric Videos to Highlight Personal Locations of Interest , 2016 .

[25]  Bolei Zhou,et al.  Learning Deep Features for Scene Recognition using Places Database , 2014, NIPS.

[26]  Jenny Benois-Pineau,et al.  Strategies for multiple feature fusion with Hierarchical HMM: Application to activity recognition from wearable audiovisual sensors , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[27]  Ali Farhadi,et al.  Understanding egocentric activities , 2011, 2011 International Conference on Computer Vision.

[28]  Anind K. Dey,et al.  Lifelogging memory appliance for people with episodic memory impairment , 2008, UbiComp.

[29]  Giovanni Maria Farinella,et al.  Scene classification in compressed and constrained domain , 2011 .

[30]  Giovanni Maria Farinella,et al.  Recognizing Personal Contexts from Egocentric Images , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[31]  Ali Farhadi,et al.  Generating Notifications for Missing Actions: Don't Forget to Turn the Lights Off! , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[32]  Yiannis Aloimonos,et al.  Deformation and Viewpoint Invariant Color Histograms , 2006, BMVC.

[33]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[34]  Shmuel Peleg,et al.  Wisdom of the Crowd in Egocentric Video Curation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.