Ubiquitous Home: Retrieval of Experiences in a Home Environment

Automated capture and retrieval of experiences at home is interesting due to the wide variety and personal significance of such experiences. We present a system for retrieval and summarization of continuously captured multimedia data from Ubiquitous Home, a two-room house consisting of a large number of cameras and microphones. Data from pressure based sensors on the floor are analyzed to segment footsteps of different persons. Video and audio handover are implemented to retrieve continuous video streams corresponding to moving persons. An adaptive algorithm based on the rate of footsteps summarizes these video streams. A novel method for audio segmentation using multiple microphones is used for video retrieval based on sounds with high accuracy. An experiment, in which a family lived in this house for twelve days, was conducted. The system was evaluated by the residents who used the system for retrieving their own experiences; we report and discuss the results.

[1]  Alexander H. Waibel CHIL - Computers in the Human Interaction Loop , 2005, MVA.

[2]  Mingchun Liu,et al.  A study on content-based classification and retrieval of audio database , 2001, Proceedings 2001 International Database Engineering and Applications Symposium.

[3]  Kiyoharu Aizawa,et al.  Person Tracking and Multicamera Video Retrieval Using Floor Sensors in a Ubiquitous Environment , 2005, CIVR.

[4]  Tatsuya Yamazaki,et al.  Ubiquitous home: real-life testbed for home context-aware service , 2005, First International Conference on Testbeds and Research Infrastructures for the DEvelopment of NeTworks and COMmunities.

[5]  Kiyoharu Aizawa,et al.  Efficient retrieval of life log based on context and content , 2004, CARPE'04.

[6]  Kiyoharu Aizawa,et al.  Evaluation of video summarization for a large number of cameras in ubiquitous home , 2005, MULTIMEDIA '05.

[7]  Wei-Ying Ma,et al.  Multimedia information retrieval: what is it, and why isn't anyone using it? , 2005, MIR '05.

[8]  Kazutaka Hirata,et al.  Memory cues for meeting video retrieval , 2004, CARPE'04.

[9]  Simon King,et al.  From context to content: leveraging context to infer media metadata , 2004, MULTIMEDIA '04.

[10]  Ramesh C. Jain,et al.  Electronic Chronicles: Empowering Individuals, Groups, and Organizations , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[11]  Kiyoharu Aizawa,et al.  Video handover for retrieval in a ubiquitous environment using floor sensor data , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[12]  Kent L. Norman,et al.  Development of an instrument measuring user satisfaction of the human-computer interface , 1988, CHI '88.